Re: Performance issue when trashing files (backtraced to fsync)

[jcupitt gmail com]

2009/8/11 Alexander Larsson <alexl redhat com>:
> Clearly we should do at least 3, which will fix this case (and other
> similar tempfile cases). However, given the extremely bad performance
> here we should maybe add the extra API in 2 allowing apps to avoid the
> cost when needed? Its kinda ugly to expose that to users, but the
> performance cost is pretty ugly too...

I'm probably being stupid here, but how about putting the fsync in a
timeout? Instead of calling fsync() directly, add a new thing called
g_fsync_queue() which queues up an fsync 'soon'.

Inside g_fsync_queue(), start (or reset) a 1s (perhaps) timeout before
fsync() is called. Maybe plus some login to fsync at least every 5s to
stop starvation.

The danger with not writing metadata in ext3 is (as I remember the
discussions) you have a potentially large window after a disc
operation before the metadata reaches disc (I think it was up to two
minutes, can't remember) and crashes in that danger window could lose
data. fsync() in a timeout would bring the window back, but keep it
rather small. Perhaps this is a good compromise?

John