Re: 2 Qs: size of tables & worksheets? graphed output to html?

[ken <gebser mousecar com>]

Leonard,

Thanks for the great tips.  But what's R?


On 10/02/2009 03:23 PM Leonard Mada wrote:
> Dear Ken,
>
> I came across a similar problem a few month ago.
>
> I always thought that the chances of hitting a rows limit are minimal,
> but then one day I got an error by Excel 2007 when I tried to
> import a csv-file.
>
> The csv file had slightly more than 2 million rows.
>
> Of course, the remaining spreadsheet applications failed
> miserably, too.
>
> The biggest problem was that even Excel - while loading
> 1 million rows, it still fails to graph that data.
> It will display only ~32,000 data-points, which was a
> very big disappointment.
>
> Fortunately, R solved very fast the problem.
>
> If you have a csv-file, load it into R:
> my.data <- read.csv("path/to/csv/csv_file.csv")
>
> Then plot the relevant variable:
> plot(my.data[[column_number]]
>
> where column_number is an integer equivalent to the column number.
>
> And by the way, R opened and processed the csv file orders
> of magnitude faster than equivalent spreadsheet programs.
>
> Sincerely,
>
> Leonard
>
>
> -------- Original-Nachricht --------
> > Datum: Fri, 02 Oct 2009 04:35:10 -0400
> > Von: ken <gebser mousecar com>
> > An: gnumeric-list gnome org
> > Betreff: 2 Qs: size of tables & worksheets?  graphed output to html?
>
> > I have two questions. Both pertain to a systems monitoring project I'm
> > developing for a college I work for.  We want to monitor two linux
> > servers.  Data (on mem, disk, cpu, etc. usage) will be collected every
> > ten seconds.  This will then be graphed to show what's been going on
> > with the systems for various periods of time... like up to three weeks.
> >  So this will be a lot of rows and columns.  A piece of data every ten
> > seconds over three weeks comes close to 181,440 pieces of data.  Let's
> > call this 200,000 rows.  Multiply this by, say, ten data items each for
> > memory, swap, disk, and cpu and we have 8M data points, or cells.  The
> > best info I got from google (thus far) was 2005 info which stated,
> > "GNUmeric could accept more than 256 columns and more of 65000 rows
> > (needs to be re-compiled)."  Does anyone have better, more recent info
> > about gnumeric which would indicate that it could or could not handle
> > 200,000x40 cells?  Better yet, what are gnumeric's data-holding
> > capacities?
> >
> > Secondly, users will query the database via a web form, which, once
> > submitted, produce a variety of graphs.  Can gnumeric produce its graphs
> > as a separate image file suitable for insertion into a web page...?
> > like a jpeg or png file?  And, if so, could those image files contain
> > borders noting, e.g., the days and hours along the x-axis and
> > percentages on the y-axis?
> >
> >
> > Of course the functionality I'm searching for would likely be more
> > readily found in a database app, like MySQL, but I've been using
> > gnumeric for a long time (though not its graphing capabilities), so am
> > more comfortable with it.  If it provides the functionality I'm looking
> > for, then I can feel okay about bending the theoreticals.
> >
> >
> > TIA for your expert replies.
> >
> > -- 
> > War is a failure of the imagination.
> >         --William Blake
> >
> > _______________________________________________
> > gnumeric-list mailing list
> > gnumeric-list gnome org
> > http://mail.gnome.org/mailman/listinfo/gnumeric-list