Re: Photo import problems using python

On 02/01/2010 04:42 AM, Alexander Larsson wrote:
On Sat, 2010-01-30 at 13:56 -0600, Damon Lynch wrote:

After modifying my code to use gio commands like copy() and
enumerate_children(), and hopefully thereby bypassing any potential
FUSE issues, I've discovered that my problems still persist with my
test hardware (a high end camera, the Canon 5D Mk II). So it seems the
problem is not FUSE. These problems include the GUI locking up
momentarily, and the download speed being approximately 1/5th that of
downloading from a python test file (without a GUI) or from Nautilus.
The very same code, when downloading from a memory card reader, has no

You could try the gvfs-copy (will use the gio ops, not fuse) command to
copy a file and see what kind of performance it gets. If thats still
slow then this seems to not be related to your code but rather some gvfs
internal issue.

That's interesting. The problem occurs regardless of whether gio.File.copy() is called by my code, or whether the python standard library executes the copy via FUSE. The problem only seems to occur when a UI is involved. Below I've pasted in some test code which works with pretty much exactly the same performance as Nautilus, but this code has no UI. I include it to give an indication of the gvfs commands I'm using.

As an aside, I have just found a possibly unrelated bug when more than one device is plugged in at the same time. To reproduce:

1. plugin in a memory card into a memory card reader
2. plugin in a camera with a memory card in it
3. browse the contents of both devices, and they're identical, because the contents of the memory card are shown as being in the camera, instead of the camera's content. The memory card's content is displayed ok (they're not swapped).

The order is important. If the camera is plugged in first, and it's contents browsed, it seems to work ok.

If helpful I'll file a bug / check for duplicates.

How exactly are you doing the copy? Are you doing a sync or an async
copy? If sync, are you doing it in a thread? If you're doing a sync copy
on the main thread (i.e. the ui thread), then the kind of UI lockups
you're experiencing is expected.

I am using a sync copy in a thread. All UI updates are handled within the main program thread. All downloads are handled in their own thread (it can download from more than one device at once). All downloader threads send their UI updates to the main thread using a queue.

If it's relevant this is the code in question:

(Of course I'm hoping that the problem can be identified without anyone needing to examine this code, but I include this link in the off-chance it might be helpful).


This very basic test code works just great:

import gio
import os
from threading import Thread

RAW_FILE_EXTENSIONS = ['arw', 'dcr', 'cr2', 'crw', 'dng', 'mef', 'mos', 'mrw',
                        'nef', 'orf', 'pef', 'raf', 'raw', 'sr2']

NON_RAW_IMAGE_FILE_EXTENSIONS = ['jpg', 'jpe', 'jpeg', 'tif', 'tiff']

def isImage(fileName):
    ext = os.path.splitext(fileName)[1].lower()[1:]

def gio_scan(path, images, imageSizeSum):
children = path.enumerate_children('standard::name,standard::type,standard::size,time::modified')
    for child in children:
        if child.get_file_type() == gio.FILE_TYPE_DIRECTORY:
imageSizeSum = gio_scan(path.get_child(child.get_name()), images, imageSizeSum)
        elif child.get_file_type() == gio.FILE_TYPE_REGULAR:
            name = child.get_name()
            if isImage(name):
                size = child.get_size()
images.append((name, path.get_path(), size, child.get_modification_time()),)
                imageSizeSum += size
    return imageSizeSum

def progress_callback(self, v):

class CopyPhotos(Thread):
    def run(self):
        mounts = gio.volume_monitor_get().get_mounts()
        for m in mounts:
            if 'Canon' in m.get_name():

                images = []
                imageSizeSum = 0

                imageSizeSum = gio_scan(m.get_root(), images, imageSizeSum)

                for image, path, size, mtime in images:
                    print image
g_dest = gio.File(path=os.path.join('/home/damon/tmp', image))
                    g_src = gio.File(path=os.path.join(path, image))
if not g_src.copy(g_dest, progress_callback, cancellable=gio.Cancellable()):
                        print "copy of %s failed!!!!!" % image

                        print "Copied: %s" % image

c = CopyPhotos()


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]