Re: 2 Qs: size of tables & worksheets? graphed output to html?



Leonard,

Thanks for the great tips.  But what's R?


On 10/02/2009 03:23 PM Leonard Mada wrote:
Dear Ken,

I came across a similar problem a few month ago.

I always thought that the chances of hitting a rows limit are minimal,
but then one day I got an error by Excel 2007 when I tried to
import a csv-file.

The csv file had slightly more than 2 million rows.

Of course, the remaining spreadsheet applications failed
miserably, too.

The biggest problem was that even Excel - while loading
1 million rows, it still fails to graph that data.
It will display only ~32,000 data-points, which was a
very big disappointment.

Fortunately, R solved very fast the problem.

If you have a csv-file, load it into R:
my.data <- read.csv("path/to/csv/csv_file.csv")

Then plot the relevant variable:
plot(my.data[[column_number]]

where column_number is an integer equivalent to the column number.

And by the way, R opened and processed the csv file orders
of magnitude faster than equivalent spreadsheet programs.

Sincerely,

Leonard


-------- Original-Nachricht --------
Datum: Fri, 02 Oct 2009 04:35:10 -0400
Von: ken <gebser mousecar com>
An: gnumeric-list gnome org
Betreff: 2 Qs: size of tables & worksheets?  graphed output to html?

I have two questions. Both pertain to a systems monitoring project I'm
developing for a college I work for.  We want to monitor two linux
servers.  Data (on mem, disk, cpu, etc. usage) will be collected every
ten seconds.  This will then be graphed to show what's been going on
with the systems for various periods of time... like up to three weeks.
 So this will be a lot of rows and columns.  A piece of data every ten
seconds over three weeks comes close to 181,440 pieces of data.  Let's
call this 200,000 rows.  Multiply this by, say, ten data items each for
memory, swap, disk, and cpu and we have 8M data points, or cells.  The
best info I got from google (thus far) was 2005 info which stated,
"GNUmeric could accept more than 256 columns and more of 65000 rows
(needs to be re-compiled)."  Does anyone have better, more recent info
about gnumeric which would indicate that it could or could not handle
200,000x40 cells?  Better yet, what are gnumeric's data-holding
capacities?

Secondly, users will query the database via a web form, which, once
submitted, produce a variety of graphs.  Can gnumeric produce its graphs
as a separate image file suitable for insertion into a web page...?
like a jpeg or png file?  And, if so, could those image files contain
borders noting, e.g., the days and hours along the x-axis and
percentages on the y-axis?


Of course the functionality I'm searching for would likely be more
readily found in a database app, like MySQL, but I've been using
gnumeric for a long time (though not its graphing capabilities), so am
more comfortable with it.  If it provides the functionality I'm looking
for, then I can feel okay about bending the theoreticals.


TIA for your expert replies.

-- 
War is a failure of the imagination.
        --William Blake

_______________________________________________
gnumeric-list mailing list
gnumeric-list gnome org
http://mail.gnome.org/mailman/listinfo/gnumeric-list




[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]