On Wed, 2015-02-11 at 13:10 -0800, Jasper St. Pierre wrote:
That's one example of low-hanging fruit. Calling query_info_async is the same thing. Creating a GFileMonitor for a GResourceFile (admittedly nonsense, but arbitrary code that tries to watch any GFile passed into it might hurt) will call query_info_async once every 5 seconds, creating a new thread for each.
I’ve filed a bug so this doesn’t get forgotten: https://bugzilla.gnome.org/show_bug.cgi?id=744392 (For the record, Jasper has already reported the file monitoring bug here:) https://bugzilla.gnome.org/show_bug.cgi?id=744010
If we want to clean this up by playing whack-a-mole and writing a lot of special-case code, we can certainly do that. I'm widely convinced doing so is a waste of time.
What would you suggest instead? I think these problems can be solved by creating one or two new interfaces, like the GMemoryBased I just proposed in bug #744392. Anyway, this is getting a little off-topic. We should defer discussion of these performance problems to relevant bug reports, since they’re fixable bugs. Philip
Another example: for some odd reason, GLocalFileInputStream isn't a
pollable output stream, so calling write_async on it spawns a thread,
even when it should be using a write poll integrated with the
mainloop. Why was this done? I don't know. Should this be fixed? Yes.
On Wed, Feb 11, 2015 at 4:10 AM, Philip Withnall
<philip tecnocode co uk> wrote:
On Tue, 2015-02-10 at 10:30 -0800, Jasper St. Pierre wrote:
> One quick example: calling g_file_read_async on a
GResourceFile spawns
> a new thread and does a synchronous stream read from a block
already
> in memory.
>
>
> It should just be a single g_bytes_ref, but we have three
different
> classes created, a thread spawned, and a large amount of
locks to do
> the equivalent of memcpy.
That’s not good at all, and seems like it should be easy to
fix by
overriding the read_async vfunc for GResourceFile. Is there a
bug filed
about it?
Philip
> On Tue, Feb 10, 2015 at 8:49 AM, Jasper St. Pierre
> <jstpierre mecheye net> wrote:
> Right now the way g_file_read_async works is by
scheduling a
> task on a worker thread, having the worker thread do
the async
> read, and then returning a result.
>
> As such, it's impossible to have two async reads
done at the
> same time, which is really unfortunate from my
understanding.
> If I'm reading a large file, and then a small file,
the large
> file needs to fully complete before the small file
is
> dispatched from the async queue.
>
> Additionally, when profiling GNOME on ARM, I've been
seeing a
> lot of cases of users using g_file_read_async()
"just in case"
> for no particular reason, which causes several locks
to be
> taken, severely slowing performance.
>
>
> We need to seriously improve our async performance
before
> asking people to use it.
>
>
> On Tue, Feb 10, 2015 at 6:48 AM, Lennart Poettering
> <mztabzr 0pointer de> wrote:
> On Tue, 10.02.15 13:59, Philip Withnall
> (philip tecnocode co uk) wrote:
>
> > > I am pretty sure if you do async IO like
gio does
> for every single
> > > file access you'll just complicate your
program
> and make it
> > > substantially slower. For small files
normal,
> synchronous disk access
> > > is a ton faster than dispatching things
to
> background threads, and
> > > back...
> >
> > The problem is that GIO can’t know which
accesses
> are to small, local
> > files, and which aren’t. It already
optimises reads
> from pollable
> > streams (sockets) by keeping them in the
main thread
> and adding them
> > into the main poll() call.
>
> Well, but the developer frequently knows
that. He
> knows that the
> config file in ~/.config is not going to be
more than
> a few K. And
> that it hence is fine to access it
synchronously...
>
> > > Also, glib has wrappers for making
mmaping
> available to programs, to
> > > improve seldom-accessed sparse databases
> efficient, do you want to
> > > prohibit that too?
> >
> > No, mmap() is clearly a tool for a
different kind of
> problem. If you’re
> > accessing an mmap()ed file, you need to be
sure it’s
> local anyway, I
> > think? GMappedFile doesn’t have async
versions of
> its methods,
> > presumably for this reason.
>
> mmap() works pretty Ok these days over NFS.
Concurrent
> access
> doesn't. But as long as you just want to
access
> something, it's
> fine...
>
> That said it's probably not a good idea to
use mmap()
> for stuff below
> $HOME...
>
> > As above, how about making that line the
distinction
> between calling
> > functions from gstdio.h and using GIO? In
the former
> case, you know
> > you’re operating on local files. In the
latter, you
> could be operating
> > on files from the moon.
>
> I'd always leave some freedom for the
developers. It
> is certainly good
> to document things and push people into the
right
> directions, but I
> think there are many cases where the
developer should
> have every right
> to prefer sync access for valid reasons,
even from the
> main loop...
>
> Lennart
>
> --
> Lennart Poettering, Red Hat
>
_______________________________________________
> desktop-devel-list mailing list
> desktop-devel-list gnome org
>
https://mail.gnome.org/mailman/listinfo/desktop-devel-list
>
>
>
>
> --
> Jasper
>
>
>
>
> --
> Jasper
>
--
Jasper
Attachment:
signature.asc
Description: This is a digitally signed message part