Re: Is delta between different branchs broken on Ostree or I am doing something wrong?

[Leandro Santiago <leandrosansilva gmail com>]

I am discarding the possibility of this problem being caused by the
ostree version difference between the client and server. I tested
version 2015.47 on the server and the behavior was the same. I wonder
it it makes sense to the client to try to reach the file
deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/superblock, which
has no information about the final (to) branch, only the origin
(from). The only identification of the final one is the prefix NI.

By reading README-deltas.md, I realized it's a bit outdated and seems
to not correspond to the current behavior anymore.

Should I open a bug report for my problem or I am doing something
wrong? I suspect this latter one :-(

On 12 June 2015 at 17:34, Leandro Santiago <leandrosansilva gmail com> wrote:
> I realized that when changing from one branch to another one, even
> when I create the delta between them, the client does not find the
> delta file, downloading so a lot of other files. Looking at the http
> accesses, I guess there might be something wrong with it.
>
> In my system each different version is put on a different branch, due
> the fact each version is generated as a new repository and after some
> validation process, merged to the main one using pull-local.
>
> For example, I have this repo with two different branches and no delta files.
>
> So I manually create a repo and initially it looks like:
>
> $ ostree --repo=repo refs
> centos-atomic-egym/7/x86_64/build-00014
> centos-atomic-egym/7/x86_64/build-00015
>
> $ ostree --repo=repo static-delta list
> (No static deltas)
>
> $ ostree --repo=repo static-delta generate
> --from=centos-atomic-egym/7/x86_64/build-00014
> --to=centos-atomic-egym/7/x86_64/build-00015
> Generating static delta:
>   From: e18fb96a1284cc32d4763d7492d7c93527f62e87acc98cf6624026b49fb5f7e6
>   To:   348b7364bde6aa7180ecf496233d162a8a4b50386c46ca1c8255e7142c04776d
> modified: 61
> new reachable: metadata=72 content regular=240 symlink=6
> rollsum for 0/61 modified
> fallback for 20f04854d2e215b8348e9a0308b61f97e3bc5eb72d187dbcd658cf99f3f77e26
> (17.4 MB)
> fallback for 979de06c7206fb8351cb09d0768d2ac8283d5c98e2db77ff76095034de24d2c0
> (17.4 MB)
> part 0 n:315 compressed:1766161 uncompressed:23996840
> uncompressed=23996840 compressed=1766161 loose=1655278
> rollsum=0 objects, 0 bytes
> bsdiff=61 objects
>
> $ ostree --repo=repo static-delta list
> e18fb96a1284cc32d4763d7492d7c93527f62e87acc98cf6624026b49fb5f7e6-348b7364bde6aa7180ecf496233d162a8a4b50386c46ca1c8255e7142c04776d
>
> $ find repo/deltas/
> repo/deltas/
> repo/deltas/4Y
> repo/deltas/4Y/+5ahKEzDLUdj10ktfJNSf2LoesyYz2YkAmtJ+19+Y-NItzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20
> repo/deltas/4Y/+5ahKEzDLUdj10ktfJNSf2LoesyYz2YkAmtJ+19+Y-NItzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/0
> repo/deltas/4Y/+5ahKEzDLUdj10ktfJNSf2LoesyYz2YkAmtJ+19+Y-NItzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/superblock
>
> $ du -sh repo/deltas/4Y/*
> 1.8M    repo/deltas/4Y/+5ahKEzDLUdj10ktfJNSf2LoesyYz2YkAmtJ+19+Y-NItzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20
>
> Then in my client, which is using
> centos-atomic-egym/7/x86_64/build-00014, I change to another branch,
> which is theory must download about 1.8M:
> # rpm-ostree rebase centos-atomic-egym/7/x86_64/build-00015
>
> It indeed download about 42M, and, looking at my HTTP logs, I have:
>
> ::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET /repo/config
> HTTP/1.1" 200 252
> ::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
> /repo/refs/heads/centos-atomic-egym/7/x86_64/build-00015 HTTP/1.1" 200
> 531
> ::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
> /repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/superblock
> HTTP/1.1" 404 734
> ::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
> /repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/superblock
> HTTP/1.1" 404 734
> ::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
> /repo/objects/34/8b7364bde6aa7180ecf496233d162a8a4b50386c46ca1c8255e7142c04776d.commitmeta
> HTTP/1.1" 404 203
> ::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
> /repo/objects/34/8b7364bde6aa7180ecf496233d162a8a4b50386c46ca1c8255e7142c04776d.commitmeta
> HTTP/1.1" 404 203
> ::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
> /repo/objects/34/8b7364bde6aa7180ecf496233d162a8a4b50386c46ca1c8255e7142c04776d.commit
> HTTP/1.1" 200 317
> ::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
> /repo/objects/34/8b7364bde6aa7180ecf496233d162a8a4b50386c46ca1c8255e7142c04776d.commitmeta
> HTTP/1.1" 404 520
> ::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
> /repo/objects/34/8b7364bde6aa7180ecf496233d162a8a4b50386c46ca1c8255e7142c04776d.commitmeta
> HTTP/1.1" 404 520
> ::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
> /repo/objects/d4/490f8f1a328e13d932169f7320faa6e31c39c3f05280fa75e5abcb863eb3e5.dirtree
> HTTP/1.1" 200 1278
> ::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
> /repo/objects/3c/6772a54670e2582cb2fd2efde55d3168e63dda76584451e09354644b68b266.dirtree
> HTTP/1.1" 200 2352
> ::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
> /repo/objects/e8/cc21b43b9aa3b8a5897d5d2acc4e1e4a4ef6b7110230e29875a4fd7108ac87.dirtree
> HTTP/1.1" 200 1030
> (...more requests here...)
>
> The first lines are some misses, indicating the client did try to
> download the delta files, but could not, falling back to the download
> of the complete file set.
> The filenames in the request do not match with the delta files available.
>
> Ostree version on the server: 2015.4 and on the client 2015.7 (centos-7 atomic).
>
> Thank you in advance.
>
> --
> Sent from my mind



-- 
Sent from my mind