[Shotwell] Slow Import with many duplicates...

[Lucas Beeler]

Hi Norbert,

> BTW, where is the duplication
> code (in the source code), I can
> look into the actual code, too?!

The best place to start examining the import code is by reading through
BatchImport.vala. Now, I freely admit that this source file is complicated
and a little convoluted, but Jim wrote very clear and informative comments
that explain what's going on. If you spend some time reading through the
file, I think you'll be able to pick up the basics of how import works
rather quickly.

Lucas


On Mon, Apr 22, 2013 at 1:31 AM, Norbert Preining <preining at logic.at> wrote:

> Hi Lucas,
>
> sorry for the late reply, I was somewhere between Europe and Japan ...
>
> On Do, 11 Apr 2013, Lucas Beeler wrote:
> > Is your import from devices slow even when there are no duplicates
> present?
>
> I moved away my shotwell folders (in .cache and .local/share) and
> restarted shotwell, importing from the card, and that worked without
> any problem, very quick.
>
> Importing direct into the full library of 8 photos worked till all
> the 8 photos flashed by on the screen, and then shotwell got stuck
> badly at 100% CPU and no reaction. I had to kill it.
> (30000+ photos in shotwell)
>
> > We know of an issue involving duplicates (
>
> It seems that this is the reason ... but not only.
>
> Why would an import from folder into the full library be so blazingly
> fast ... is there no duplicate check done I assume. If this is the
> case, then yes.
>
> > Curiously, does import speed seem in any way proportional to the number
> of
> > photos you have in your library? If you've got an empty library, is
> import
> > faster than in your personal photo library with (I'm guessing) thousands
> of
> > photos? I ask because a slowdown proportional to library size suggests
> that
> > the duplicate detection code path might be involved.
>
> Yes, I have this feeling.
>
> I will also play around with the profiling code.
>
> BTW, where is the duplication code (in the source code), I can look
> into the actual code, too?!
>
> Norbert
>
> ------------------------------------------------------------------------
> PREINING, Norbert                               http://www.preining.info
> JAIST, Japan                                 TeX Live & Debian Developer
> DSA: 0x09C5B094   fp: 14DF 2E6C 0307 BE6D AD76  A9C0 D2BF 4AA3 09C5 B094
> ------------------------------------------------------------------------
>