Re: File timestamps lost during commit/deploy

[Colin Walters <walters verbum org>]

Hi,

On Wed, 2013-12-04 at 09:50 -0600, Daniel Drake wrote:

> Somewhere in the process of commiting a series of files to an ostree,
> creating a new empty fs, pulling & deploying from the ostree, the
> original timestamps on the files are lost. They are replaced with the
> timestamp of when the ostree gets deployed. Is that by design, is
> there any way to maintain original timestamps?

Currently the OSTree data format does not contain timestamps, and this
is intentional.  It operates in the same way git does; having timestamps
would really decrease the value of the hardlink content-addressed model.
Admittedly, to a lesser degree than it would for git, but there's still
value in transparently sharing objects of identical physical content
that were generated by different build systems (e.g. gnome-continuous
and rpm).

The problem is this is in some cases the polar opposite of what dpkg/rpm
people want; for things like the gtk+ icon cache, gtk+ considers it out
of date if the file is older than the timestamp on the directory.  This
is to help work around people who do "sudo make install" on apps which
install icons but don't also have gtk-update-icon-cache bits in their
Makefiles.

For OSTree, it still works OK, because the hard link inside the
subdirectory is made after the directory is created.

> This is causing problems for guile-based apps, which install both
> source and compiled libraries. As ostree deploys the compiled
> libraries before the source, the timestamp on the source files is
> newer than the binary. This causes guile to recompile the source,
> thinking that the binary is outdated. Compilation is slow, so the apps
> take a long time to start, for no good reason.

Ah.  So another similar case is Python, which will even try to write
updated files (but silently fail because /usr is read-only).  In the
case of Python though it doesn't really matter as the language is pretty
slow anyways =) and the bytecode compiler is super basic and quite fast
to execute.

> If maintaining original timestamps isn't possible, perhaps ostree
> could at least deploy all files with exactly the exact same timestamp?

Are we talking about mtime or ctime here (or both)?  This matters
because the simple act of creating a hard link will bump the ctime (but
not the mtime).  And that means if say you do another deployment, it's
possible we bump the ctime for some files but not others.

Ok, it looks like Python at least only cares about mtime, which makes
sense.  Hopefully Guile does too.

I don't have a great answer for you - I'd like to have a standard way
for userspace code like Python and Guile to detect that the system is
deployed with a tool like OSTree for which Unix timestamps are not used.

The timestamp comparison is certainly useful for development purposes,
but even with current RPM/dpkg, it's just a waste of a stat() call on
every startup.  (Yes, there is "sudo make install", but those people are
wrong).

I'd be open to a patch though which caused OSTree to set the mtime of
every object to 0.