Re: fsync in glib/gio

[Michael Meeks <michael meeks novell com>]

Hi Alex,

On Fri, 2009-03-13 at 08:38 +0100, Alexander Larsson wrote:
> If you want to you can make all i/o sync by mounting it as such. But
> thats of course really slow. Generally the gio file write operations are
> used for saving files, and people sort of expect that when save returns
> the file is ok on disk. 

	Sure - which is basically what ext3 provides with it's default ordered
mode right ? metadata hits the disk after data - ok, so all pending data
writes were flushed as well as a side-effect(?) but ... ;-)

> And to make matters worse, its perfectly ok for a filesystem (ext4, xfs
> and btrfs do this atm) to reorder the metadata and the data writes such
> that writing to a temp file, renaming over target file and then crashing
> can end up with an empty file. This happens if metadata was saved but
> not the new file contents, and the window for this is about a minute, so
> its not a small race condition.

	Sure, sure - and I guess this is why they had to make 'fsync' not suck
for these filing systems.

> So, you save and the system hangs 10 seconds later. What do you expect?
> Ideally the new file, less problematic the old file. But, if you're left
> with *no* version of the file I'd be pretty pissed off.

	Yep; of course this is not good.

> With the data=ordered setting in ext3 (the default), any fsync will
> result in all dirty data being flushed, not just the data in that file.
> This can be pretty expensive if there is a lot of outstanding I/O.

	Sure - on ext3 it's a 'sync()' call effectively, with the added bonus
that this has a terrible effect on other applications trying to use the
I/O subsystem to eg. read audio to play your mp3, or swap, or allocate
memory, or ... ;-)

	Calling 'fsync' regularly on ext3 will bring your system to a grinding
halt, quite regularly, cf. my comments on hair-loss etc. Lots of our
users will be on these systems, and last I checked [ did you poke your
kernel guys on this ? ] this was not recommended.

> However, this is only a problem if such an operation happens often, and
> file saving is just not that common.

	Sure; although amusingly, if it happens often ~enough (say every few
hundred ms) - then we almost move the OS into a semi-synchronous writes
mode, and have somewhat less to write each time, and so don't suffer
multi-second glitches in the I/O subsystem ;-)

>  And if something constantly is saving something that is a problem
> for multiple other reasons too and should be fixed.

	Oh - well, of course there is regular autosave, and setting of gconf
settings [ we set ~200+ keys on login amazingly ;-], then of course the
IM infrastructure will want to make sure your IM logs are *really*
on-disk each time you get a message, the E-mail client for every mail
message ;-) pretty soon if everyone calls 'fsync' we end up in a fairly
bad place (IMHO).

> Of course, not all file writes are saves. For example, it could be
> nautilus copying 10000 files. This is why I added the ASYNC_WRITE flag
> and used it in the file copy case.

	I guess.

> It occurs often enought that there were several people in the ubuntu
> ext4-eats-my-data bug that had it happen to them multiple times.

	Nasty indeed, and this is the only solution. I suppose the nutshell of
my concern is this:

	* can we in some (evil or otherwise) way avoid hurting
	  desktop performance, interactivity and playback for
	  all ext2/3 systems, and still keep ext4 users happy ? :-)

	Of course, perhaps I'm just way off / out of date wrt. my dislike of
fsync, let me hunt down some kernely people for a sane 2nd opinion.

	Regards,

		Michael.

-- 
 michael meeks novell com  <><, Pseudo Engineer, itinerant idiot