Re: CSV files : huge performance issue...

[Vincent Trouilliez <vincent trouilliez wanadoo fr>]

Hi Emmanuel,

I see that you are a fellow Frog so to speak, if your e-mail address is anything to go by, and you do live near Poitiers, then, as I live in Angers (only 100kms up north !), you are more than welcome to stop by my flat and see my problem with Gnuméric first hand, should you ever happen to drive by my neck of the wood ! :o)

I will be most happy to do anything I can to help you solve this huge performance issue, and if I need to make a bug report in Bugzilla, then I will take the time to see what the procedure is, as it seems quite complex apparently, need to set up an account or something, then read the "how to" file a bug etc...
I wil gladly send you my file, assuming Bugzilla can handle 1MB files ?? My biggest CSV file is actually 2.5 MB not 1MB...
If bugzilla can' thandle files that big, maybe I can upload my file to my server and give you the address so you can pick it up ?
Or I can just send it by e-mail this evening if you like, just tell me what's best.

I made another attempt at creating an XY plot, and paid closer attention to what my machine's was doing. It seems that you are right, moving the graph is fast. the problem is that it seems Gnumeric "sleeps" ofr say 4 seconds, then "wakes up" for half a second, during which moving the graph IS fast/normal, but then goes back to sleep and so on. It's cyclic then, and gives the overall impression that Gnuméric is slow, but maybe it's just that something makes it "hang" a bit every other second, and not "responsive" at all.  Maybe this is due to my particular machine/set up (what could cause this ???) althought this would be the first time I have such a problem on my machine, despite using many apps all day long.

I just installed Ubuntu 4.10 (new Debian based distro) which came with Gnuméric 1.2.13, which it seems, is the very latest stable release ?

I also have Mandrake 9.2 on my machine which runs qutie well as well, so I will try Gnuméric there, see if it's any better/different.  Well, that's assuming I can find a Mandrake RPM for GNuméric, as I am not a Computer guru and compiling things from source never works for me !! :o(((


Thanks for your reply,

Regards,

Vince, from Angers :-)



Le vendredi 12 novembre 2004 à 19:24 +0100, Emmanuel Pacaud a écrit :
> Hi,
>
> > First post here, not sure if this is the right place, if it's for
> > users or developpers, but I couldn't find a Gnumeric specific web
> > forum so here I am.
>
> It's the right place for discussion about gnumeric, for users and
> developers.
>
> > I ran into a huge problem today when (trying to) working on a 'CSV'
> > file. No problem to import it, works like a charm. However doing
> > anything with the data proves EXTREMELY slow. So slow that it's
> > actually not useable. My file is not big, only 1 MB, and contains
> > about 800,000 cells (40 columns and 20,000 lines), but creating a
> > graph takes quite some time, simply getting a preview of the graph
> > when clicking on the "preview" button takes 3 to 5 seconds I would
> > say. 
>
> If it takes only 3 or 5 seconds, I guess it's a xy plot. Line plots are
> slower.
>
> We have a serious performance problem that lies in the use of libart for
> rendering. We get nice outputs with it, but it's very slow. We plan to
> drop libart for cairo, expecting speed improvement, but currently
> performance with a lot of data is rather poor.
>
> > This is fair enough, you only do it once. However once the graph is
> > created and put on top of the cells/spreadsheet, if I want to "drag"
> > it somewhere with the mouse, it takes about 15 seconds to move by 3 or
> > 4 cells...
>
> That's not expected. Each graph are rendered and cached in a pixbuf,
> moving them is expected to be quite fast.
>
> Which version do you use ?
>
> What would be great is that you provide us a sample file. Could you open
> a new bug against gnumeric/charting in http://bugzilla.gnome.org, and
> attach there the sample file ?
>
> > It's as is whatever I am trying to do with the mouse, gnumeric
> > processess the entire 800,000 cells, instead of affecting only the 100
> > or so cells taht are atually visible at a given time, in the document
> > window. Am I right in this assumption ?   Also, saving the file into
> > Gnumeric format took about 5 to 10 minutes of hard work for the hard
> > drive... :-/
> > What puzzles me is that last time I worked on such files, it was with
> > MS Excel, and it was lightening fast, opening, creating graphs,
> > manipulating data, moving graphs around, saving the file, everytinhg
> > seemed fast and effortless. So the machine is not at fault. Why is
> > Gnuméric so slow ? Considering Gnuméric aims (I think ?) to be as good
> > as Excel, can I hope that this major issue will be investigated, or is
> > already under investigation ? 
>
> This issues will surely be investigated...
>
> 	Emmanuel.