Re: Discouraging use of sync APIs

["Jasper St. Pierre" <jstpierre mecheye net>]

That's one example of low-hanging fruit. Calling query_info_async is the same thing. Creating a GFileMonitor for a GResourceFile (admittedly nonsense, but arbitrary code that tries to watch any GFile passed into it might hurt) will call query_info_async once every 5 seconds, creating a new thread for each.

If we want to clean this up by playing whack-a-mole and writing a lot of special-case code, we can certainly do that. I'm widely convinced doing so is a waste of time.

Another example: for some odd reason, GLocalFileInputStream isn't a pollable output stream, so calling write_async on it spawns a thread, even when it should be using a write poll integrated with the mainloop. Why was this done? I don't know. Should this be fixed? Yes.

On Wed, Feb 11, 2015 at 4:10 AM, Philip Withnall <philip tecnocode co uk> wrote:

On Tue, 2015-02-10 at 10:30 -0800, Jasper St. Pierre wrote:
> One quick example: calling g_file_read_async on a GResourceFile spawns
> a new thread and does a synchronous stream read from a block already
> in memory.
>
>
> It should just be a single g_bytes_ref, but we have three different
> classes created, a thread spawned, and a large amount of locks to do
> the equivalent of memcpy.

That’s not good at all, and seems like it should be easy to fix by
overriding the read_async vfunc for GResourceFile. Is there a bug filed
about it?

Philip

> On Tue, Feb 10, 2015 at 8:49 AM, Jasper St. Pierre
> <jstpierre mecheye net> wrote:
>         Right now the way g_file_read_async works is by scheduling a
>         task on a worker thread, having the worker thread do the async
>         read, and then returning a result.
>
>         As such, it's impossible to have two async reads done at the
>         same time, which is really unfortunate from my understanding.
>         If I'm reading a large file, and then a small file, the large
>         file needs to fully complete before the small file is
>         dispatched from the async queue.
>
>         Additionally, when profiling GNOME on ARM, I've been seeing a
>         lot of cases of users using g_file_read_async() "just in case"
>         for no particular reason, which causes several locks to be
>         taken, severely slowing performance.
>
>
>         We need to seriously improve our async performance before
>         asking people to use it.
>
>
>         On Tue, Feb 10, 2015 at 6:48 AM, Lennart Poettering
>         <mztabzr 0pointer de> wrote:
>                 On Tue, 10.02.15 13:59, Philip Withnall
>                 (philip tecnocode co uk) wrote:
>
>                 > > I am pretty sure if you do async IO like gio does
>                 for every single
>                 > > file access you'll just complicate your program
>                 and make it
>                 > > substantially slower. For small files normal,
>                 synchronous disk access
>                 > > is a ton faster than dispatching things to
>                 background threads, and
>                 > > back...
>                 >
>                 > The problem is that GIO can’t know which accesses
>                 are to small, local
>                 > files, and which aren’t. It already optimises reads
>                 from pollable
>                 > streams (sockets) by keeping them in the main thread
>                 and adding them
>                 > into the main poll() call.
>
>                 Well, but the developer frequently knows that. He
>                 knows that the
>                 config file in ~/.config is not going to be more than
>                 a few K. And
>                 that it hence is fine to access it synchronously...
>
>                 > > Also, glib has wrappers for making mmaping
>                 available to programs, to
>                 > > improve seldom-accessed sparse databases
>                 efficient, do you want to
>                 > > prohibit that too?
>                 >
>                 > No, mmap() is clearly a tool for a different kind of
>                 problem. If you’re
>                 > accessing an mmap()ed file, you need to be sure it’s
>                 local anyway, I
>                 > think? GMappedFile doesn’t have async versions of
>                 its methods,
>                 > presumably for this reason.
>
>                 mmap() works pretty Ok these days over NFS. Concurrent
>                 access
>                 doesn't. But as long as you just want to access
>                 something, it's
>                 fine...
>
>                 That said it's probably not a good idea to use mmap()
>                 for stuff below
>                 $HOME...
>
>                 > As above, how about making that line the distinction
>                 between calling
>                 > functions from gstdio.h and using GIO? In the former
>                 case, you know
>                 > you’re operating on local files. In the latter, you
>                 could be operating
>                 > on files from the moon.
>
>                 I'd always leave some freedom for the developers. It
>                 is certainly good
>                 to document things and push people into the right
>                 directions, but I
>                 think there are many cases where the developer should
>                 have every right
>                 to prefer sync access for valid reasons, even from the
>                 main loop...
>
>                 Lennart
>
>                 --
>                 Lennart Poettering, Red Hat
>                 _______________________________________________
>                 desktop-devel-list mailing list
>                 desktop-devel-list gnome org
>                 https://mail.gnome.org/mailman/listinfo/desktop-devel-list
>
>
>
>
>         --
>           Jasper
>
>
>
>
> --
>   Jasper
>

--

  Jasper