Performance issue when trashing files (backtraced to fsync)



Hi,

A few days ago i deleted thousands of files with just nautilus. That
went fine but horribly slow. doing the same with the rm command was
way faster.
I made a bug report about it yesterday with my results from that
moment: http://bugzilla.gnome.org/show_bug.cgi?id=591363

So i tried to trace this issue down. First i added dozens of debug
messages to the function: g_local_file_trash in the file glocalfile.c
and that resulted in one function that sucked up time.
The function was: g_file_set_contents (infofile, data, -1, NULL);

Now thankfully Alexander pointed me to where that function is going to
(saved me probably a lot of time backtracing that). the function was
going back to: write_to_temp_file in the file gfileutils.c
Then he gave a valuable suggestion: "can you try disabling the whole
#ifdef HAVE_FSYNC bloc" and i did that. Sure enough that solved the
performance hit but this might have other unexpected side effects you
rather avoid like loss of data.

Anyway here are benchmarks with and without that single fsync block.

With #ifdef HAVE_FSYNC
----------------------------------------------
...
snip // file info
...
Number of files: 1927

real    1m50.061s
user    0m0.344s
sys     0m0.596s


Without #ifdef HAVE_FSYNC
----------------------------------------------
...
snip // file info
...
Number of files: 1927

real	0m0.902s
user	0m0.180s
sys	0m0.392s

With fsync file trashing is ~120x slower then without fsync! way to
big difference if you ask me.

And to be perfectly clear about the lines that i removed in the
"Without #ifdef HAVE_FSYNC" benchmark.
This is all of it:

#ifdef HAVE_FSYNC
  errno = 0;
  /* If the final destination exists, we want to sync the newly written
   * file to ensure the data is on disk when we rename over the destination.
   * otherwise if we get a system crash we can lose both the new and the
   * old file on some filesystems. (I.E. those that don't guarantee the
   * data is written to the disk before the metadata.)
   */
  if (g_file_test (dest_file, G_FILE_TEST_EXISTS) &&
      fsync (fileno (file)) != 0)
    {
      save_errno = errno;

      g_set_error (err,
		   G_FILE_ERROR,
		   g_file_error_from_errno (save_errno),
		   _("Failed to write file '%s': fsync() failed: %s"),
		   display_name,
		   g_strerror (save_errno));

      g_unlink (tmp_name);

      goto out;
    }
#endif

So, the problem is outlined here. Now what would be a solution for this issue?

Thanx,
Mark.


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]