Re: Discouraging use of sync APIs

On Tue, 2015-02-10 at 08:49 -0800, Jasper St. Pierre wrote:
Right now the way g_file_read_async works is by scheduling a task on a
worker thread, having the worker thread do the async read, and then
returning a result.

As such, it's impossible to have two async reads done at the same
time, which is really unfortunate from my understanding. If I'm
reading a large file, and then a small file, the large file needs to
fully complete before the small file is dispatched from the async

From my reading of the GTask code, it should support 10 parallel reads
(or other operations) since it uses a thread pool with maximum 10
threads. See g_task_thread_pool_init(). Is this not the case?

Additionally, when profiling GNOME on ARM, I've been seeing a lot of
cases of users using g_file_read_async() "just in case" for no
particular reason, which causes several locks to be taken, severely
slowing performance.

I guess the particular reason depends on what’s being read, as discussed
in the rest of the thread.

We need to seriously improve our async performance before asking
people to use it.

That’s important, but I wouldn’t say it’s a blocker. The original idea
of this thread was to try and address the situation where non-expert
developers are using sync APIs when they should be using async ones, and
then tying themselves in all sorts of knots. I suspect that hitting
these sorts of performance problems is not going to be an issue for them
— but freezing up the main loop is.

I could be wrong, but that’s how I see it.


On Tue, Feb 10, 2015 at 6:48 AM, Lennart Poettering
<mztabzr 0pointer de> wrote:
        On Tue, 10.02.15 13:59, Philip Withnall
        (philip tecnocode co uk) wrote:
        > > I am pretty sure if you do async IO like gio does for
        every single
        > > file access you'll just complicate your program and make
        > > substantially slower. For small files normal, synchronous
        disk access
        > > is a ton faster than dispatching things to background
        threads, and
        > > back...
        > The problem is that GIO can’t know which accesses are to
        small, local
        > files, and which aren’t. It already optimises reads from
        > streams (sockets) by keeping them in the main thread and
        adding them
        > into the main poll() call.
        Well, but the developer frequently knows that. He knows that
        config file in ~/.config is not going to be more than a few K.
        that it hence is fine to access it synchronously...
        > > Also, glib has wrappers for making mmaping available to
        programs, to
        > > improve seldom-accessed sparse databases efficient, do you
        want to
        > > prohibit that too?
        > No, mmap() is clearly a tool for a different kind of
        problem. If you’re
        > accessing an mmap()ed file, you need to be sure it’s local
        anyway, I
        > think? GMappedFile doesn’t have async versions of its
        > presumably for this reason.
        mmap() works pretty Ok these days over NFS. Concurrent access
        doesn't. But as long as you just want to access something,
        That said it's probably not a good idea to use mmap() for
        stuff below
        > As above, how about making that line the distinction between
        > functions from gstdio.h and using GIO? In the former case,
        you know
        > you’re operating on local files. In the latter, you could be
        > on files from the moon.
        I'd always leave some freedom for the developers. It is
        certainly good
        to document things and push people into the right directions,
        but I
        think there are many cases where the developer should have
        every right
        to prefer sync access for valid reasons, even from the main
        Lennart Poettering, Red Hat
        desktop-devel-list mailing list
        desktop-devel-list gnome org


Attachment: signature.asc
Description: This is a digitally signed message part

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]