GNOME.org
 

Reproducible builds in BuildStream

I've been looking into bit-for-bit reproducible builds (as per the https://reproducible-builds.org/ initiative) for BuildStream this week. The good news is that most of the builds I've tried so far are reproducible in BuildStream; that is, the 'files/' directory of the built artifact is identical across multiple builds.
At the moment I'm driving bst with an external script. I originally intended to modify bst so it'd take a '--force-rebuild' option or something similar, but it's not too difficult to do externally so there didn't seem much point modifying the interface to bst. https://gitlab.com/jmacarthur/buildstream/tree/jmac/reproducible contains a script called 'contrib/repro.py', which is very much a prototype, and assumes OSTree, but you can try it if you like.
The next steps will be to try and incorporate reprotest (https://anonscm.debian.org/git/reproducible/reprotest.git), which deliberately alters the build environment to test reproducibility. We'll need to either get the variations reprotest generates applied when setting up our build sandbox, or put reprotest inside the sandbox and make it a wrapper for the build process. I think the second approach will be less work at the moment.
I'm unsure how to treat dependencies at the moment. It's perfectly possible to have a reproducible package whose build-time dependencies are not reproducible, but you've lost your chain of trust at that point so the reproducibility of the final package is not as useful. Should we stop looking after finding one dependency that is unreproducible, for example? Any comments would be welcome.
Here's the results of running my script on "gnome-moduleset", with this command line: 

python3 ../buildstream/contrib/repro.py core/gedit.bst gnome

This is the contents of "reprotestlog":
Diff complete - diff reports 0; gnome/base-linker-priority with cache key gnome/base-linker-priority/6b425d6ca3c0c84597da08e84fa1c932781a9f8945fa796239882e36711134f9 is reproducible. Diff complete - diff reports 0; gnome/base-ninja with cache key gnome/base-ninja/2734712775ea1f98a41653565a690c3b4e3ab0d8a35668a21a2f968d33bed1d3 is reproducible. Diff complete - diff reports 0; gnome/base with cache key gnome/base/1f0eba2c684ab71588c710c3a787c83b1c454049d44034dfc56af886fa31183b is reproducible. Diff complete - diff reports 0; gnome/core-deps-yelp-xsl with cache key gnome/core-deps-yelp-xsl/f4e416f629d8355458e7440dc641f7f9518a74074d9fb31be43ccc3392bd5cc7 is reproducible. Diff complete - diff reports 0; gnome/core-deps-yelp-tools with cache key gnome/core-deps-yelp-tools/1868eb57ec800a6c16fdaa70c024b88dc7efa1e6686602296c1da48372bc0250 is reproducible. Diff complete - diff reports 0; gnome/core-deps-gettext with cache key gnome/core-deps-gettext/576e2ac213ea845634d0ba719c5c0c20380d4e85be9ed0994d93af6757704df4 is reproducible. Diff complete - diff reports 0; gnome/core-deps-gtk-doc with cache key gnome/core-deps-gtk-doc/688abc960879b335b6f551ac3889503dad55c06102cd84022316fa44d24305d0 is reproducible. Diff complete - diff reports 1; gnome/core-deps-glib with cache key gnome/core-deps-glib/f3c7d6a8a1e2d16ef66168e6fcf171c5f3614e018f6b0f2fd6ca07b115bc08db is NOT reproducible. Diff complete - diff reports 0; gnome/core-deps-enchant-2 with cache key gnome/core-deps-enchant-2/a13aa5043ffb5b06710f47b86d3158faa58483b8bd5b4c3306addf6080c0fd49 is reproducible. Diff complete - diff reports 1; gnome/core-deps-gobject-introspection with cache key gnome/core-deps-gobject-introspection/ecbc9523d591eddcdd6dbb9d22e4c03a5ddd851a7c040f64f36c4898cd14bb57 is NOT reproducible. Diff complete - diff reports 0; gnome/core-deps-meson with cache key gnome/core-deps-meson/eadfb60f3c067e5c58c1a982737c14628477246ce8441c567f47739180ad4ff5 is reproducible. Diff complete - diff reports 0; gnome/core-deps-atk with cache key gnome/core-deps-atk/ca91ece11083c1e4e8d223a7d5890a01d596330b01ebe27e8965d0ec73cdd101 is reproducible. Diff complete - diff reports 0; gnome/core-deps-gdk-pixbuf with cache key gnome/core-deps-gdk-pixbuf/e81f85b1ce056bc8780faac307d1ad02a32b94e11bdaa91625c43f820d21dd1f is reproducible. Diff complete - diff reports 0; gnome/core-deps-harfbuzz with cache key gnome/core-deps-harfbuzz/87b5059df4817737f2ec58f87a38c5b3ec8c8bb8393a4ce386fb9dbda9d4a38b is reproducible. Diff complete - diff reports 0; gnome/core-deps-pango with cache key gnome/core-deps-pango/5e408e4e6f0b6613883b79027fa0b308d3ee61eb63afec52c77f41d078528fb9 is reproducible. Diff complete - diff reports 0; gnome/core-deps-wayland with cache key gnome/core-deps-wayland/4035d517f6a9545b6759aed7e35b997dba397fb98d005a2dede09160f07b1387 is reproducible. Diff complete - diff reports 0; gnome/core-deps-wayland-protocols with cache key gnome/core-deps-wayland-protocols/51e0d1ad5fc4e0dd71067b766e88086c31380a11930ac80efc36833a9ff70e2f is reproducible. Diff complete - diff reports 0; gnome/core-at-spi2-core with cache key gnome/core-at-spi2-core/1815cae5ca5a63e03b830a57801c1dc8afb5daf3f777f405152cce20591646e0 is reproducible. Diff complete - diff reports 0; gnome/core-at-spi2-atk with cache key gnome/core-at-spi2-atk/de2a4837cd58c092f9f274cf5f9d4b2d753fde526bdd4b7b649903012e5344c0 is reproducible. Diff complete - diff reports 0; gnome/core-deps-gtk_-3 with cache key gnome/core-deps-gtk_-3/8afec47769853d15c7178567c4b97344bb339a8e0f473cecc702aeadf72cfbbe is reproducible. Diff complete - diff reports 0; gnome/core-deps-iso-codes with cache key gnome/core-deps-iso-codes/9e4221d02ddc98154604240222d4d46a4cc0aa7363e420f0162c484e87318e60 is reproducible. Diff complete - diff reports 0; gnome/core-deps-m4-common with cache key gnome/core-deps-m4-common/628d688543f21dbe34be30b0eb207a26b44f11a89467cc5e99febe58efe85c66 is reproducible. Diff complete - diff reports 0; gnome/core-deps-gspell with cache key gnome/core-deps-gspell/adfe1dc58fd761d96f9b0a5fdd1ead21f270dcb2fcb4b4336b7893092b3f91d5 is reproducible. Diff complete - diff reports 0; gnome/core-deps-gtksourceview-3 with cache key gnome/core-deps-gtksourceview-3/1448bc10029c8d8ceecb83315dc078cf6acc338f7764ecaa804987a186654f15 is reproducible. Diff complete - diff reports 0; gnome/core-deps-py3cairo with cache key gnome/core-deps-py3cairo/6668e2fc0efc74f4eaba887bc62270063640858b20add46445f808201ae1c3bb is reproducible. Diff complete - diff reports 1; gnome/core-deps-pygobject with cache key gnome/core-deps-pygobject/f4fc9b87f411d8285e952c5b4cd34088bcd8ba77771d418de3ac5b77bf77f05a is NOT reproducible. Diff complete - diff reports 1; gnome/core-deps-libpeas with cache key gnome/core-deps-libpeas/c7d9c100e4a99163861d266a588830d2046a921120083747a5f3ce4a5e06431e is NOT reproducible. Diff complete - diff reports 0; gnome/core-deps-gnome-common with cache key gnome/core-deps-gnome-common/5778ada004f4fb101b5169a72102016019a2dd219b6000f0106fcaffcf2f684c is reproducible. Diff complete - diff reports 0; gnome/core-gsettings-desktop-schemas with cache key gnome/core-gsettings-desktop-schemas/44ad4a8076c664b5d5c937515a1bde5292bec998669c85b49db1ac3a63fb2e9c is reproducible.
[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]