Re: Improving the JSON cache created by ftp-admin install

[Olav Vitters <olav vitters nl>]

On Mon, Dec 05, 2011 at 06:57:22PM +0100, David King wrote:
> I changed the update script to use this metadata, and it simplified
> the process significantly. However, there are some relatively simple
> improvements that I would like to make to the cache.

I don't think they're simple.

> I propose a more structured format, something similar to the following:
[..]
> I am happy to implement the changes in the ftp-admin script:
> 
> http://git.gnome.org/browse/sysadmin-bin/tree/ftpadmin
> 
> and welcome any comments, especially from anyone who uses the
> current data, or who might otherwise find more detailed metadata
> useful. This is my first experience with Perl and JSON, so please go
> easy on the newbie!

The purpose of those cache.json files are to speed up the execution of
the ftpadmin script. That is the primary purpose of these files.
Secondary purpose is allowing people to reuse these files.

The ftpadmin script is very generic. There are classes which main
purpose is to index directories.

What you're proposing would require pretty massive changes to the entire
script and how the classes are setup. E.g. adding checksums means you'll
have to teach the generic DirectoryInfo class about md5sum and
sha256sum.

The contents of those md5sum and sha256sum files are parsed in a bad way
by ftpadmin. It assumes there are no funny characters in the filename.
If you want to provide these checksums in the JSON file, I think these
files should be properly parsed, slowing things down.

After installing a new tarball, it actually parses all the files again.
The checksum data should be cached, unless the timestamp of a file
changes.

Class structure is something like:
1.  object
2.    BasicInfo
3.      TarInfo
3.      DirectoryInfo
4.        SuiteInfo
4.        ModuleInfo
3.      InstallModule
3.      InstallSuites


Suggest taking a close look at the script. If you can modify it so the
additional IO is minimal and the class structure is still pretty sane,
then I'm ok with it. Just think it might be way more complicated than
you initially assume.


-- 
Regards,
Olav