GNOME.org
 

Re: Is delta between different branchs broken on Ostree or I am doing something wrong?

I guess I got what has happened here.... I generated a static delta
from empty to the branch my client tries to update to...
$ ostree --repo=repo static-delta generate
--to=centos-atomic-egym/7/x86_64/build-00015 --empty

... and it generated exactly the files the "rebase" command tried to request:

repo/deltas/
repo/deltas/NI
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/14
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/12
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/1
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/4
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/10
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/7
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/16
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/13
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/5
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/3
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/8
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/11
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/0
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/18
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/superblock
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/19
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/17
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/6
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/15
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/2
repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/9
(...)

It means my client was indeed looking for a delta from scratch to the
version it wants to update instead of from the current version. Once
again, it it a bug or part of ostree or rpm-ostree design? Oh, and the
problem with this last delta creation is that it is 178M of size,
which is much more than the non-optimal 42M I had before and the 2M I
was expecting :-(

On 15 June 2015 at 11:54, Leandro Santiago <leandrosansilva gmail com> wrote:

I am discarding the possibility of this problem being caused by the
ostree version difference between the client and server. I tested
version 2015.47 on the server and the behavior was the same. I wonder
it it makes sense to the client to try to reach the file
deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/superblock, which
has no information about the final (to) branch, only the origin
(from). The only identification of the final one is the prefix NI.

By reading README-deltas.md, I realized it's a bit outdated and seems
to not correspond to the current behavior anymore.

Should I open a bug report for my problem or I am doing something
wrong? I suspect this latter one :-(

On 12 June 2015 at 17:34, Leandro Santiago <leandrosansilva gmail com> wrote:

I realized that when changing from one branch to another one, even
when I create the delta between them, the client does not find the
delta file, downloading so a lot of other files. Looking at the http
accesses, I guess there might be something wrong with it.

In my system each different version is put on a different branch, due
the fact each version is generated as a new repository and after some
validation process, merged to the main one using pull-local.

For example, I have this repo with two different branches and no delta files.

So I manually create a repo and initially it looks like:

$ ostree --repo=repo refs
centos-atomic-egym/7/x86_64/build-00014
centos-atomic-egym/7/x86_64/build-00015

$ ostree --repo=repo static-delta list
(No static deltas)

$ ostree --repo=repo static-delta generate
--from=centos-atomic-egym/7/x86_64/build-00014
--to=centos-atomic-egym/7/x86_64/build-00015
Generating static delta:
  From: e18fb96a1284cc32d4763d7492d7c93527f62e87acc98cf6624026b49fb5f7e6
  To:   348b7364bde6aa7180ecf496233d162a8a4b50386c46ca1c8255e7142c04776d
modified: 61
new reachable: metadata=72 content regular=240 symlink=6
rollsum for 0/61 modified
fallback for 20f04854d2e215b8348e9a0308b61f97e3bc5eb72d187dbcd658cf99f3f77e26
(17.4 MB)
fallback for 979de06c7206fb8351cb09d0768d2ac8283d5c98e2db77ff76095034de24d2c0
(17.4 MB)
part 0 n:315 compressed:1766161 uncompressed:23996840
uncompressed=23996840 compressed=1766161 loose=1655278
rollsum=0 objects, 0 bytes
bsdiff=61 objects

$ ostree --repo=repo static-delta list
e18fb96a1284cc32d4763d7492d7c93527f62e87acc98cf6624026b49fb5f7e6-348b7364bde6aa7180ecf496233d162a8a4b50386c46ca1c8255e7142c04776d

$ find repo/deltas/
repo/deltas/
repo/deltas/4Y
repo/deltas/4Y/+5ahKEzDLUdj10ktfJNSf2LoesyYz2YkAmtJ+19+Y-NItzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20
repo/deltas/4Y/+5ahKEzDLUdj10ktfJNSf2LoesyYz2YkAmtJ+19+Y-NItzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/0
repo/deltas/4Y/+5ahKEzDLUdj10ktfJNSf2LoesyYz2YkAmtJ+19+Y-NItzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/superblock

$ du -sh repo/deltas/4Y/*
1.8M    
repo/deltas/4Y/+5ahKEzDLUdj10ktfJNSf2LoesyYz2YkAmtJ+19+Y-NItzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20

Then in my client, which is using
centos-atomic-egym/7/x86_64/build-00014, I change to another branch,
which is theory must download about 1.8M:
# rpm-ostree rebase centos-atomic-egym/7/x86_64/build-00015

It indeed download about 42M, and, looking at my HTTP logs, I have:

::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET /repo/config
HTTP/1.1" 200 252
::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
/repo/refs/heads/centos-atomic-egym/7/x86_64/build-00015 HTTP/1.1" 200
531
::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
/repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/superblock
HTTP/1.1" 404 734
::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
/repo/deltas/NI/tzZL3mqnGA7PSWIz0WKopLUDhsRsocglXnFCwEd20/superblock
HTTP/1.1" 404 734
::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
/repo/objects/34/8b7364bde6aa7180ecf496233d162a8a4b50386c46ca1c8255e7142c04776d.commitmeta
HTTP/1.1" 404 203
::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
/repo/objects/34/8b7364bde6aa7180ecf496233d162a8a4b50386c46ca1c8255e7142c04776d.commitmeta
HTTP/1.1" 404 203
::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
/repo/objects/34/8b7364bde6aa7180ecf496233d162a8a4b50386c46ca1c8255e7142c04776d.commit
HTTP/1.1" 200 317
::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
/repo/objects/34/8b7364bde6aa7180ecf496233d162a8a4b50386c46ca1c8255e7142c04776d.commitmeta
HTTP/1.1" 404 520
::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
/repo/objects/34/8b7364bde6aa7180ecf496233d162a8a4b50386c46ca1c8255e7142c04776d.commitmeta
HTTP/1.1" 404 520
::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
/repo/objects/d4/490f8f1a328e13d932169f7320faa6e31c39c3f05280fa75e5abcb863eb3e5.dirtree
HTTP/1.1" 200 1278
::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
/repo/objects/3c/6772a54670e2582cb2fd2efde55d3168e63dda76584451e09354644b68b266.dirtree
HTTP/1.1" 200 2352
::ffff:127.0.0.1 - - [12/Jun/2015:16:48:35 +0000] "GET
/repo/objects/e8/cc21b43b9aa3b8a5897d5d2acc4e1e4a4ef6b7110230e29875a4fd7108ac87.dirtree
HTTP/1.1" 200 1030
(...more requests here...)

The first lines are some misses, indicating the client did try to
download the delta files, but could not, falling back to the download
of the complete file set.
The filenames in the request do not match with the delta files available.

Ostree version on the server: 2015.4 and on the client 2015.7 (centos-7 atomic).

Thank you in advance.

--
Sent from my mind




--
Sent from my mind




-- 
Sent from my mind
[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]