Re: What are .commitmeta files and why is delta update so huge?

[Leandro Santiago <leandrosansilva gmail com>]

Well, they are quite small (a few kb). Is there any tool to analyse
the content of those files? (I bet not, but asking does not hurt :-))

There might be some other changes, as I create a new tree from the
scratch using rpm-ostree tree compose. But the files are supposed to
be almost the same, as I am using centos and I don't think so many
updates were made in so short interval of time (I built the two trees
with a short interval between them).

I'll try what you suggested, even though I am not sure if it will help
as what I need is reducing the amount of downloaded data.

Thanks

On 25 June 2015 at 16:10, Giuseppe Scrivano <gscrivan redhat com> wrote:
> Leandro Santiago <leandrosansilva gmail com> writes:
>
> > Hi, when running upgrade in the client using delta files, I noticed it
> > tries to request a .commitmeta file, which does not exist and I don't
> > know why and if it has any influence in the process.
>
> .commitmeta (aka detached commit metadata) files are used to add
> metadata to a commit object, like for example GPG signatures.
>
>
> > To summarize, I have generated delta files from commit
> > 2435da4b6874efcfeaae9e342842676fe8a590f89dc80b3b2c991e4f6c8af506 to
> > 0c5135c92c25fd4c90c1d55b710480f5f87de79e141cdfe3b9370ce9aeb57a2e
> >
> > $ ostree static-delta list
> > 2435da4b6874efcfeaae9e342842676fe8a590f89dc80b3b2c991e4f6c8af506-0c5135c92c25fd4c90c1d55b710480f5f87de79e141cdfe3b9370ce9aeb57a2e
> >
> > The client requests:
> > config -> OK
> > refs/heads/centos-atomic-egym/7/x86_64/dev -> OK
> > deltas/JD/XaS2h078_qrp40KEJnb+ilkPidyAs7LJkeT2yK9QY-DFE1ySwl_UyQwdVbcQSA9fh9554UHN_juTcM6a61ei4/superblock
> > -> OK (980KB)
> > deltas/JD/XaS2h078_qrp40KEJnb+ilkPidyAs7LJkeT2yK9QY-DFE1ySwl_UyQwdVbcQSA9fh9554UHN_juTcM6a61ei4/0
> > -> OK (200KB)
> > objects/0c/5135c92c25fd4c90c1d55b710480f5f87de79e141cdfe3b9370ce9aeb57a2e.commitmeta
> > -> NOT FOUND
> > objects/aa/70d6d08d6dd0e8e186131896a7ab150a102b938538cb7ef9ac83c21953c48f.filez
> > -> OK (17MB)
> > objects/a4/92a68fdba07c976ba35c8977e8b4bca243ed7b3b15c1506213c82ee75ba518.filez
> > -> OK (17MB)
> >
> > The difference between the two versions is very small, just some
> > modifications on some txt files (this is intentional). Before using
> > deltas, those updates were 40MB in size, so I was expecting a drastic
> > reduce in size when using delta, but it reduced to 34MB, which I think
> > is still not compatible the delta update idea.
> >
> > So I wonder why the .filez (are they part of delta stuff, each one is
> > 17MB in size?) were downloaded, if the superblock and 0 file were
> > found. I suspect it's related to the lack of the .commitmeta file, but
> > I am not sure.
> >
> > Yes, I know the delta system is not finished yet and you are working
> > on it (I've been checking the source code and I might be able to
> > contribute with something...), but I for now I'd like to know what is
> > the behavior I should expect so far.
>
> big files are not added to the delta files but instead only referenced
> in the superblock file.  You can change this value, the default is 10M,
> using --min-fallback-size to "static-delta generate", can you please try
> with a higher value, like 30?
>
> How big are the original text files that you modified?
>
> Regards,
> Giuseppe



-- 
Sent from my mind