Re: Discouraging use of sync APIs

On Tue, 2015-02-10 at 10:30 -0800, Jasper St. Pierre wrote:
One quick example: calling g_file_read_async on a GResourceFile spawns
a new thread and does a synchronous stream read from a block already
in memory.

It should just be a single g_bytes_ref, but we have three different
classes created, a thread spawned, and a large amount of locks to do
the equivalent of memcpy.

That’s not good at all, and seems like it should be easy to fix by
overriding the read_async vfunc for GResourceFile. Is there a bug filed
about it?


On Tue, Feb 10, 2015 at 8:49 AM, Jasper St. Pierre
<jstpierre mecheye net> wrote:
        Right now the way g_file_read_async works is by scheduling a
        task on a worker thread, having the worker thread do the async
        read, and then returning a result.
        As such, it's impossible to have two async reads done at the
        same time, which is really unfortunate from my understanding.
        If I'm reading a large file, and then a small file, the large
        file needs to fully complete before the small file is
        dispatched from the async queue.
        Additionally, when profiling GNOME on ARM, I've been seeing a
        lot of cases of users using g_file_read_async() "just in case"
        for no particular reason, which causes several locks to be
        taken, severely slowing performance.
        We need to seriously improve our async performance before
        asking people to use it.
        On Tue, Feb 10, 2015 at 6:48 AM, Lennart Poettering
        <mztabzr 0pointer de> wrote:
                On Tue, 10.02.15 13:59, Philip Withnall
                (philip tecnocode co uk) wrote:
                > > I am pretty sure if you do async IO like gio does
                for every single
                > > file access you'll just complicate your program
                and make it
                > > substantially slower. For small files normal,
                synchronous disk access
                > > is a ton faster than dispatching things to
                background threads, and
                > > back...
                > The problem is that GIO can’t know which accesses
                are to small, local
                > files, and which aren’t. It already optimises reads
                from pollable
                > streams (sockets) by keeping them in the main thread
                and adding them
                > into the main poll() call.
                Well, but the developer frequently knows that. He
                knows that the
                config file in ~/.config is not going to be more than
                a few K. And
                that it hence is fine to access it synchronously...
                > > Also, glib has wrappers for making mmaping
                available to programs, to
                > > improve seldom-accessed sparse databases
                efficient, do you want to
                > > prohibit that too?
                > No, mmap() is clearly a tool for a different kind of
                problem. If you’re
                > accessing an mmap()ed file, you need to be sure it’s
                local anyway, I
                > think? GMappedFile doesn’t have async versions of
                its methods,
                > presumably for this reason.
                mmap() works pretty Ok these days over NFS. Concurrent
                doesn't. But as long as you just want to access
                something, it's
                That said it's probably not a good idea to use mmap()
                for stuff below
                > As above, how about making that line the distinction
                between calling
                > functions from gstdio.h and using GIO? In the former
                case, you know
                > you’re operating on local files. In the latter, you
                could be operating
                > on files from the moon.
                I'd always leave some freedom for the developers. It
                is certainly good
                to document things and push people into the right
                directions, but I
                think there are many cases where the developer should
                have every right
                to prefer sync access for valid reasons, even from the
                main loop...
                Lennart Poettering, Red Hat
                desktop-devel-list mailing list
                desktop-devel-list gnome org


Attachment: signature.asc
Description: This is a digitally signed message part

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]