[Notes] [Git][BuildStream/buildstream][jonathan/cache-cache-size] 31 commits: Convert uses of external_directory to get_underlying_directory()

[Jonathan Maw <gitlab mg gitlab com>]

Title: GitLab

Jonathan Maw pushed to branch jonathan/cache-cache-size at BuildStream / buildstream

Commits:

• 88ccff6e
  by Jim MacArthur at 2018-08-15T14:33:05Z

  Convert uses of external_directory to get_underlying_directory()
  

• b84218a4
  by Jim MacArthur at 2018-08-15T14:33:05Z

  _filebaseddirectory/directory.py: Move VirtualDirectoryError to Directory
  

• 3341948a
  by Jim MacArthur at 2018-08-15T14:33:05Z

  Move KeyStrength enum out to element_enums.py
  

• 70f35b6c
  by Jim MacArthur at 2018-08-15T14:33:05Z

  Initial implementation of _casbaseddirectory.py
  

• ef8525c2
  by Jim MacArthur at 2018-08-15T14:33:05Z

  Directory API: add mark_changed
  

• 00632108
  by Jim MacArthur at 2018-08-15T14:33:05Z

  filebaseddirectory: Update index after descend
  

• 0213f68d
  by Jim MacArthur at 2018-08-15T14:33:05Z

  Sandbox: Return a CasBasedDirectory when an environment variable is set
  

• 136deb2e
  by Jim MacArthur at 2018-08-15T14:33:05Z

  Add basic storage test 'storage-test.py'
  

• 12920046
  by Jim MacArthur at 2018-08-15T17:03:26Z

  Merge branch 'jmac/cas_virtual_directory' into 'master'
  
  CAS-backed virtual directory implementation
  
  See merge request BuildStream/buildstream!481

• 9d2442b2
  by Chandan Singh at 2018-08-15T17:29:07Z

  Allow source plugins to access previous sources
  
  Source plugin implementations can now specify that they need access to
  previously staged sources by specifying
  `BST_REQUIRES_PREVIOUS_SOURCES_TRACK` and/or
  `BST_REQUIRES_PREVIOUS_SOURCES_FETCH`, corresponding to access at `track`
  and `fetch` times respectively.
  
  Fixes #381.
  
  Replaces !505. For relevant discussion, see this discussion:
  https://gitlab.com/BuildStream/buildstream/merge_requests/505#note_83780747
  

• a7a82650
  by Chandan Singh at 2018-08-15T18:23:04Z

  Add pip source plugin
  
  `pip` source plugin can stage python packages that are either specified
  directly in the element definition or picked up from `requirements.txt`
  from previous sources. In order to support the latter use-case
  (which is also the primary motivation for this plugin), this plugin
  requires access to previous sources and hence is an example of a
  Source Transform source.
  
  Also, bump `BST_FORMAT_VERSION` as this patch adds a new core plugin.
  

• 4a2dd6af
  by Chandan Singh at 2018-08-15T18:58:58Z

  Add NEWS entry for Source Transform and pip source
  

• d4706096
  by Chandan Singh at 2018-08-15T20:21:01Z

  Merge branch 'chandan/sourcetransform' into 'master'
  
  Allow source plugins to access previous sources
  
  Closes #381
  
  See merge request BuildStream/buildstream!568

• c0431f15
  by Valentin David at 2018-08-16T07:43:07Z

  Fix ostree repository mirroring
  
  Ostree mirrors were not sharing the same local repository, so it was
  impossible the request refs from the right local repository when data
  was fetched from a mirror rather than upstream.
  
  Instead of having several repository with one remote each, we now
  have one repository with several remotes.
  
  This fixes #538.
  

• 04f83679
  by Tristan Van Berkom at 2018-08-16T08:54:27Z

  Merge branch 'valentindavid/fallback_mirror_ostree' into 'master'
  
  Fix ostree repository mirroring
  
  Closes #538
  
  See merge request BuildStream/buildstream!658

• 177c4264
  by William Salmon at 2018-08-16T09:52:38Z

  Trying to mitigate a mtime granularity braking the cache tests
  
  This patch mitigates the granularity of the mtimes used on the default
  gitlab runners, Allowing the test suite to pass on these runners.
  

• 01c4ac57
  by Tom Pollard at 2018-08-16T11:01:45Z

  Merge branch 'willsalmon/CacheExpiryTest' into 'master'
  
  Trying to mitigate a file system issue
  
  See merge request BuildStream/buildstream!595

• 0066d701
  by Tiago Gomes at 2018-08-16T12:33:01Z

  Use http instead of https for the gnu ftp mirror
  
  https seems broken on the Debian image:
  
  START   autotools/hello/42930621-fetch.499.log
  START   Fetching https://ftpmirror.gnu.org/gnu/automake/automake-1.16.tar.gz
  FAILURE Fetching https://ftpmirror.gnu.org/gnu/automake/automake-1.16.tar.gz
  FAILURE tar source at hello.bst [line 16 column 2]: Error mirroring https://ftpmirror.gnu.org/gnu/automake/automake-1.16.tar.gz: <urlopen error [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:720)>
  

• 898aa8a0
  by Tiago Gomes at 2018-08-16T13:43:06Z

  Merge branch 'tiagogomes/disable-https-gnu-repo' into 'master'
  
  Use http instead of https for the gnu ftp mirror
  
  See merge request BuildStream/buildstream!676

• b62c361c
  by Tiago Gomes at 2018-08-16T13:54:54Z

  cascache: use errno module
  
  os.errno does no longer work with Python 3.7
  
  Closes #577.
  

• 1e3e2a93
  by Tiago Gomes at 2018-08-16T15:00:12Z

  Merge branch 'tiagogomes/issue-577' into 'master'
  
  cascache: use errno module
  
  Closes #577
  
  See merge request BuildStream/buildstream!674

• 10f27f3f
  by Tristan Van Berkom at 2018-08-17T06:20:40Z

  _frontend/app.py: Notify session completions
  
  Use the optionally implemented desktop notification method
  to notify when a session completes.
  
  Previously, this used to only notify when an interactive prompt
  appears.
  
  This is an attempt to fix #385
  

• c102b92f
  by Tristan Van Berkom at 2018-08-17T08:52:07Z

  Merge branch 'tristan/notifications' into 'master'
  
  _frontend/app.py: Notify session completions
  
  Closes #385
  
  See merge request BuildStream/buildstream!672

• 9b6fac5a
  by William Salmon at 2018-08-17T09:48:01Z

  Add Error to git and ostree sources configure
  
  Raise a error at configure time if the track and ref properties are
  not present in the sources.
  
  This is to address https://gitlab.com/BuildStream/buildstream/issues/471
  that documented unhelpful behaviour when tracking git sources. However
  the issue was also identified in ostree.
  

• d5ed4f45
  by William Salmon at 2018-08-17T09:49:23Z

  Improve error message for build if there are refs missing
  

• c43af4eb
  by Will Salmon at 2018-08-17T10:51:38Z

  Merge branch 'willsalmon/APIFix' into 'master'
  
  Updating the no track or ref fix to match the buildstream style
  
  See merge request BuildStream/buildstream!628

• 7263ea8c
  by Josh Smith at 2018-08-17T13:19:23Z

  job.py: Prevent terminated jobs retrying
  
  Fixes #531: Jobs were retrying when terminated, this lead to the process
  being spawned again and starting up fresh.
  

• 30f696f2
  by Qinusty at 2018-08-17T14:29:45Z

  Merge branch 'Qinusty/531-fetch-retries-on-terminate' into 'master'
  
  Prevent jobs retrying on terminate
  
  Closes #531
  
  See merge request BuildStream/buildstream!662

• d77062a2
  by Jonathan Maw at 2018-08-17T14:50:27Z

  Use ArtifactCache's get_cache_size when calculating the quota
  
  It would have been more convenient to store the quota size in the
  ArtifactCache, except that the scheduler needs the quota size and
  doesn't have access to the ArtifactCache.
  

• c1d85204
  by Jonathan Maw at 2018-08-17T14:50:27Z

  Fix get_cache_size to store it in the ArtifactCache, not the context
  

• 3039e691
  by Jonathan Maw at 2018-08-17T14:50:27Z

  artifactcache: Read and write cache size to disk
  

30 changed files:

• NEWS
• buildstream/__init__.py
• buildstream/_artifactcache/artifactcache.py
• buildstream/_artifactcache/cascache.py
• buildstream/_context.py
• buildstream/_frontend/app.py
• buildstream/_loader/loader.py
• buildstream/_pipeline.py
• buildstream/_scheduler/jobs/job.py
• buildstream/_scheduler/queues/buildqueue.py
• buildstream/_scheduler/queues/fetchqueue.py
• buildstream/_scheduler/scheduler.py
• buildstream/_versions.py
• buildstream/element.py
• + buildstream/element_enums.py
• buildstream/plugins/sources/git.py
• buildstream/plugins/sources/ostree.py
• + buildstream/plugins/sources/pip.py
• buildstream/sandbox/_mount.py
• buildstream/sandbox/_sandboxbwrap.py
• buildstream/sandbox/_sandboxchroot.py
• buildstream/sandbox/sandbox.py
• buildstream/source.py
• + buildstream/storage/_casbaseddirectory.py
• buildstream/storage/_filebaseddirectory.py
• buildstream/storage/directory.py
• doc/examples/autotools/project.conf
• doc/source/core_plugins.rst
• tests/artifactcache/expiry.py
• + tests/cachekey/project/sources/pip1.bst

The diff was not included because it is too large.

—
View it on GitLab.
You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.