Re: transpose on bigger datasets?

[Morten Welinder <mortenw gnome org>]

> Why a max at all?

There are issues with the language used for expressions.  When the column
number gets high enough, there will be ambiguities such as whether "TRUE"
is a column name or a constant; "LOG2" might be a function or it might be a
cell name; etc.  I am not sure all of these are well understood.

There is no simular problem with row numbers, but we do need to keep area
sizes from overflowing the "int" type, typically 2^31.  There is no problem with
increasing the number of rows, but right now it would have to come with a
limiting of the number of columns.

I don't think it is impossible to lift these restrictions.  It is
unknown, however,
whether the existing data structures will exhibit unpleasant performance
complexity behaviour for large sheets.  And messing with the low-level data
structures used for spreadsheets is very difficult.  Correctness is not the
hardest thing, performance tradeoff is.

Memory usage is also going to be an issue.  You may assume something
like 200 bytes per non-blank cell.  10^8 * 10^8 * 200 is a mighty big number!

Morten


On Wed, Oct 29, 2014 at 9:22 AM, Berntsson, Martin
<martin berntsson eon se> wrote:
> Dear Morten and all,
>
> Thanks for the file and the advice.
>
> Thoughts about Gnumeric :
> Why are there different max rows and max column?
> Why a max at all? And not limited by machine mem?
> I need a lot of both, and seen from a technical perspective, today
> there are numerous applications that produce many variables (columns) (like 10^8).
>
> Perhaps that could be a suggestion for the next version? - I think there is a general need for that,
> and for me the 1.6x10^7 rows are about to run out.
> I don’t think there are any other software out there that can be used for this.
>
> Thanks anyhow for a great software!
> (probably the best for bigger datasets!)
>
>
> Med vänliga hälsningar / With kind regards / Freundliche Grüße,
>
> Martin Berntsson  DS.
>
> _______________________________________________
> gnumeric-list mailing list
> gnumeric-list gnome org
> https://mail.gnome.org/mailman/listinfo/gnumeric-list