-
463698ec
by Francisco Redondo Marchena
at 2018-06-28T14:18:10Z
Add error message when running commands on directories
Issue #446
-
48848f4e
by James Ennis
at 2018-06-29T08:45:12Z
loader.py: Check whether the dir specified is also a <dir>.bst file
- Slight modification to the wording of existing error message.
- Closes #446
-
aaf52d03
by Tristan Maat
at 2018-06-29T15:56:20Z
cli.py: Make buildstream check element paths instead of click
This is to avoid inconsistencies when dealing with paths inside an
elements directory
-
a85aaee6
by Tiago Gomes
at 2018-07-02T14:45:04Z
Fix documentation typo
-
9067e269
by Tiago Gomes
at 2018-07-02T14:45:04Z
Provide better error message on missing commands
Before running a command in the sandbox, check its existence and fail
early if it does not.
This fixes issue #289.
-
c5fe9465
by Jürg Billeter
at 2018-07-08T12:33:46Z
_signals.py: Guard against use in non-main thread
-
3d1e23d6
by Jürg Billeter
at 2018-07-09T13:44:49Z
tests: Remove unneeded ArtifactShare.update_summary() method
The OSTree summary file is no longer used.
-
5f4b9377
by Jürg Billeter
at 2018-07-09T13:44:49Z
_artifactcache/artifactcache.py: Update remote init error message
initialize_remotes() no longer fetches remote refs.
-
9defbb62
by Jürg Billeter
at 2018-07-09T13:44:49Z
tests/frontend/push.py: Add missing skip annotation for non-Linux
-
bc5a40e3
by Phillip Smyth
at 2018-07-11T11:06:40Z
Move _list_dir_contents to __init__.py
-
ddeac63a
by Jonathan Maw
at 2018-07-16T12:19:14Z
tests: Fix filter tests not checking whether files should be missing.
They weren't actually catching it if you checked-out the entire depended
element, instead of just the specified split domains
-
4c6512d6
by Jonathan Maw
at 2018-07-16T16:12:08Z
Only redirect elements for workspace and track operations
It was redirecting for checkout operations
-
fa2eaba2
by Jürg Billeter
at 2018-07-17T05:56:40Z
tests: Use context manager for ArtifactShare
-
ba4581f8
by Jürg Billeter
at 2018-07-17T05:56:40Z
tests/testutils/artifactshare.py: Add support for statvfs mocking
-
8cd81636
by Jürg Billeter
at 2018-07-17T05:56:40Z
tests/frontend/push.py: Use ArtifactShare statvfs mocking
This makes it unnecessary to update the free space in the mock object in
the middle of tests.
-
9b63f6f2
by Jürg Billeter
at 2018-07-17T05:56:40Z
.gitlab-ci.yml: Update images to include protobuf and grpcio
-
1a2ee26e
by Jürg Billeter
at 2018-07-17T05:56:40Z
setup.py: Add grpcio dependency and support for code generation
This allows code generation with ./setup.py build_grpc
-
b585388e
by Jürg Billeter
at 2018-07-17T05:56:40Z
HACKING.rst: Add note about protocol buffers and code generation
-
383e0586
by Jürg Billeter
at 2018-07-17T05:56:40Z
Import Google proto dependencies for the Remote Execution API
This imports protos from https://github.com/googleapis/googleapis/ and
the modules generated with ./setup.py build_grpc.
-
73fe9af1
by Jürg Billeter
at 2018-07-17T05:56:40Z
Import protos for the Bazel Remote Execution API
This imports protos from https://github.com/bazelbuild/remote-apis/ and
the modules generated with ./setup.py build_grpc.
-
df9a4949
by Jürg Billeter
at 2018-07-17T05:56:40Z
remote_execution.proto: Add proposed symlink support
https://github.com/bazelbuild/remote-apis/pull/18/
-
f13bef56
by Jürg Billeter
at 2018-07-17T05:56:40Z
_artifactcache: Add CAS artifact cache
Content Addressable Storage specified in the Remote Execution API.
Fixes #387.
-
a3bdfc18
by Jürg Billeter
at 2018-07-17T05:56:40Z
_artifactcache/cascache.py: Add prune() method
-
687b9a8b
by Jürg Billeter
at 2018-07-17T05:56:40Z
_artifactcache/cascache.py: Add remove() method
-
bed6c800
by Jürg Billeter
at 2018-07-17T05:56:40Z
_artifactcache/cascache.py: Add list_artifacts() method
-
8a0fe273
by Jürg Billeter
at 2018-07-17T05:56:40Z
Add proto for BuildStream reference storage service
-
72284b59
by Jürg Billeter
at 2018-07-17T05:56:40Z
_artifactcache/cascache.py: Add remote cache support
-
ff8703c9
by Jürg Billeter
at 2018-07-17T05:56:40Z
_artifactcache: Add CAS artifact server
-
971606ae
by Jürg Billeter
at 2018-07-17T05:56:40Z
_artifactcache/casserver.py: Add cache cleanup based on pushreceive
-
4b4c5268
by Jürg Billeter
at 2018-07-17T05:56:40Z
_artifactcache/casserver.py: Add update_mtime parameter to resolve_ref()
-
9b879015
by Jürg Billeter
at 2018-07-17T05:56:40Z
_artifactcache/casserver.py: Update artifact ref mtime on access
This makes cache cleanup LRU.
-
8de8ef1f
by Jürg Billeter
at 2018-07-17T05:56:40Z
_platform: Use CAS artifact cache
-
a161c746
by Jürg Billeter
at 2018-07-17T05:56:40Z
tests/testutils/artifactshare.py: Use CAS artifact server
-
d0346e89
by Jürg Billeter
at 2018-07-17T05:56:40Z
tests/cachekey: Update expected cache keys after switch to CAS
-
05a1a25d
by Jürg Billeter
at 2018-07-17T05:56:40Z
tests: Test push/pull on all platforms
-
6fd3f523
by Jürg Billeter
at 2018-07-17T05:56:40Z
tests/frontend/push.py: Remove xfail mark from LRU cache test
The CAS server now supports LRU.
-
b87c696f
by Jürg Billeter
at 2018-07-17T05:56:40Z
tests/frontend/pull.py: Add pull test for missing blobs on the server
-
99846137
by Jürg Billeter
at 2018-07-17T05:56:40Z
tests/integration/workspace.py: Test incremental build on all platforms
-
e35098d3
by Jürg Billeter
at 2018-07-17T05:56:40Z
doc/source/artifacts.rst: Update documentation for CAS artifact server
-
5a7d49dc
by Jürg Billeter
at 2018-07-17T05:56:40Z
Remove tar artifact cache
No longer used.
-
1f8b4aa2
by Jürg Billeter
at 2018-07-17T05:56:40Z
Remove OSTree artifact cache
No longer used.
Fixes #134, #138, #148, #217, #268, #276, #443, #460.
-
d87df668
by Jürg Billeter
at 2018-07-17T05:56:40Z
NEWS: Add news entry for CAS-based artifact cache
-
7acc64e4
by Jürg Billeter
at 2018-07-17T05:56:40Z
element.py: Do not use ArtifactCache.can_diff()
The only existing artifact cache implementation always returns True.
-
f2484fe4
by Jürg Billeter
at 2018-07-17T05:56:40Z
_artifactcache: Remove unused method can_diff()
-
f770ab1c
by Jürg Billeter
at 2018-07-17T08:40:16Z
element.py: Include fail-on-overlap setting in cache key
Fixes #473.
-
7e9282c9
by Phillip Smyth
at 2018-07-17T12:53:55Z
buildstream/_frontend/cli.py: Added a `--deps` flag to `bst checkout`
buildstream/_stream.py: Added deps param to _prepare_sandbox function call
buildstream/element.py: Added deps param and logic to _prepare_sandbox function
-
2797d1fd
by Phillip Smyth
at 2018-07-17T12:53:55Z
tests/frontend/buildcheckout.py: Added tests for "Add a `--deps` flag to `bst checkout`"
-
70a83ef8
by William Salmon
at 2018-07-17T14:04:16Z
Adding a helpful link to the example
-
e4f8fb07
by devcurmudgeon
at 2018-07-17T15:07:05Z
Fix 'main install' to be explicit that it is for Linux distros only
-
87c103d8
by Jürg Billeter
at 2018-07-17T15:07:56Z
_frontend/cli.py: Fix help text for `bst checkout --deps`
-
4c09b686
by Jürg Billeter
at 2018-07-17T17:31:32Z
_artifactcache/cascache.py: Fix prune()
_reachable_refs_dir() failed to add directory digests to the set of
reachable objects.
-
adedd867
by Tristan Maat
at 2018-07-18T05:11:03Z
Bump required python version to 3.5
-
b8543f8a
by Tristan Maat
at 2018-07-18T05:11:03Z
_exceptions.py: Add `detail` to ArtifactErrors
-
fc79ad2c
by Tristan Maat
at 2018-07-18T05:11:03Z
utils.py: Allow `list_relative_paths` to list directories
-
4f9da15d
by Tristan Maat
at 2018-07-18T05:11:03Z
_scheduler/*queue.py: Move queues to a subdirectory
-
24925634
by Tristan Maat
at 2018-07-18T05:45:59Z
Make Jobs abstract and element-independent
-
33984448
by Tristan Maat
at 2018-07-18T05:45:59Z
Add cache_quota to user config
-
1ec5c7b1
by Tristan Maat
at 2018-07-18T05:45:59Z
Make elements keep track of their built artifact size
-
7229d2e5
by Tristan Maat
at 2018-07-18T05:45:59Z
Compute the artifact cache size after each build/pull
-
dc17de38
by Tristan Maat
at 2018-07-18T06:07:24Z
Clean the artifact cache when we hit the cache quota
When the cache quota is hit, we will remove any elements not required
for the current build, until our cache is only filled halfway.
-
6ff12e5e
by Tristan Maat
at 2018-07-18T06:07:24Z
tests/artifactcache/expiry.py: Add expiry tests
-
308d59be
by Tristan Maat
at 2018-07-18T06:07:24Z
NEWS: Add news about cache expiry
-
a950a985
by Tristan Van Berkom
at 2018-07-18T09:59:03Z
_context.py: Adding apis for message recording and task logging.
Added the new Context.recorded_messages() context manager, this
causes messages to be logged in a dedicated log file before being
propagated to the log handler, and also provides an open log handle
for the capturing and logging of stdout/stderr from child processes.
This comes with the additional accessors:
Context.get_log_handle()
Context.get_log_filename()
-
d835c37f
by Tristan Van Berkom
at 2018-07-18T09:59:03Z
Refactor: Use new logging mechanism from Context
o element.py / plugin.py: Removed supporting logging code, and derive
the log handle from Context.
o _scheduler/scheduler.py, _scheduler/queues/queue.py: Adapt to new Job initializer API for the logfile
o _scheduler/jobs/job.py: Run job activities within the new context manager
which turns on logging for a job. Also removed a lot
of custom logging abstract methods which are unneeded.
o _scheduler/jobs: Job implementations need not implement so much custom logging.
-
26b9f6fa
by Tristan Van Berkom
at 2018-07-18T11:52:43Z
_stream.py: Clarifying ambiguous FIXME comment about required artifacts
-
3814905a
by Daniel Playle
at 2018-07-18T12:08:24Z
Reorder app.py imports
When testing locally on some systems, it appears that pylint fails on
the order of imports for `buildstream/_frontend/app.py`. This commit
changes the order of these imports.
This commit addresses issue !453.
-
48f66a3c
by Chandan Singh
at 2018-07-18T18:03:31Z
.gitlab-ci.yml: Use testsuite images for running tests
Instead of re-using the user-facing `buildstream/buildstream-fedora`
image for tests (that comes with BuildStream pre-installed), use
`buildstream/testsuite-fedora` image that is supposed to be used
exclusively for these tests.
Similarly, use the dedicated `buildstream/testsuite-debian` image
instead of `buildstream/buildstream-debian`.
Fixes #458.
For related discussion, see
https://gitlab.com/BuildStream/buildstream-docker-images/issues/8.
-
1016dcb4
by Josh Smith
at 2018-07-19T12:52:21Z
scheduler.py: Correct some anomalies within the docstrings
-
709b9bca
by Francisco Redondo Marchena
at 2018-07-19T14:03:12Z
source-bundle: Enable --except option
Before this option was listed in help but was ignored when
creating the source-bundle.
Issue #468
-
96d07153
by Jordan Petridis
at 2018-07-19T23:54:39Z
Docs: Update the required build packages for fedora based systems.
`ruamel.yaml` seems to require `Python.h` header file to build.
`python3-devel` is what provides it for Fedora.
-
93a77c1d
by Javier Jardón
at 2018-07-19T23:58:28Z
Docs: Update the required build packages for the rest of the systems
Also remove the recomendation to install psutil as we need to build
other python modules anyway (like ruamel)
Completes 96d07153b7817cdaeda57dd163eed52b2b1b31e8
-
57e0d1c5
by William Salmon
at 2018-07-20T00:03:06Z
Added dependency to the Docs
These dependencies are needed to use the plugins that are installed
as part of this guild.
As the dependencies added are not dependencies of the core package
they have been added to a separate section.
The lzip package is needed to build gnome.
https://wiki.gnome.org/Newcomers/BuildSystemComponent
-
1b6688eb
by Javier Jardón
at 2018-07-20T00:07:23Z
install_linux_distro.rst: document plugins installed by default
-
fc63b972
by Javier Jardón
at 2018-07-20T00:12:50Z
install_linux_distro.rst: Document plugins deps for the rest of the systems
-
790fb40b
by Tiago Gomes
at 2018-07-20T09:07:01Z
Update checkout man page for new --deps option
-
9c1f024b
by Tiago Gomes
at 2018-07-20T09:07:01Z
Add support for creating a tarball on bst checkout
One of the tests added is configured to be skipped for now, as dumping
binary data is causing a bad descriptor exception when using the pytest
capture module.
Closes #263.
-
0ddbf0e0
by Tristan Van Berkom
at 2018-07-20T11:52:00Z
setup.py: Specify minimum required version of pytest-cov plugin
This causes the new artifact tests to pass (unless you happened
to already have a recent enough version of pytest-cov, in which case
you didn't notice the breakage).
-
6158ee5a
by Sam Thursfield
at 2018-07-20T13:17:14Z
Fix crash when --debug is passed
I hit the following backtrace running `bst --debug push` and `bst
--debug build`:
pid:16736 id:000[--:--:--][][] START Push
pid:16736 id:000[--:--:--][][] START Loading pipeline
pid:16736 id:000[00:00:00][][] SUCCESS Loading pipeline
pid:16736 id:000[--:--:--][][] START Resolving pipeline
pid:16736 id:000[--:--:--][][] BUG 'MesonElement' object has no attribute '_Element__cache_key'
Traceback (most recent call last):
File "/home/sam/.local/bin/bst", line 8, in <module>
sys.exit(cli())
File "/home/sam/.local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_frontend/cli.py", line 162, in override_main
standalone_mode=standalone_mode, **extra)
File "/home/sam/.local/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/sam/.local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/sam/.local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/sam/.local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/sam/.local/lib/python3.6/site-packages/click/decorators.py", line 27, in new_func
return f(get_current_context().obj, *args, **kwargs)
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_frontend/cli.py", line 471, in push
app.stream.push(elements, selection=deps, remote=remote)
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_stream.py", line 341, in push
fetch_subprojects=True)
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_stream.py", line 814, in _load
fetch_subprojects=fetch_subprojects)
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_pipeline.py", line 119, in load
for meta in meta_elements
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_pipeline.py", line 119, in <listcomp>
for meta in meta_elements
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/element.py", line 894, in _new_from_meta
element = project.create_element(artifacts, meta)
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_project.py", line 184, in create_element
element = self._element_factory.create(self._context, self, artifacts, meta)
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_elementfactory.py", line 57, in create
return element_type(context, project, artifacts, meta, default_config)
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/element.py", line 196, in __init__
super().__init__(meta.name, context, project, meta.provenance, "element")
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/plugin.py", line 171, in __init__
self.debug("Created: {}".format(self))
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/plugin.py", line 407, in debug
self.__message(MessageType.DEBUG, brief, detail=detail)
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/plugin.py", line 652, in __message
self.__context.message(message)
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_context.py", line 338, in message
self._message_handler(message, context=self)
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_frontend/app.py", line 665, in _message_handler
text = self.logger.render(message)
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_frontend/widget.py", line 585, in render
return self._render(message)
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_frontend/widget.py", line 620, in _render
text += widget.render(message)
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_frontend/widget.py", line 228, in render
_, key, missing = plugin._get_display_key()
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/element.py", line 1138, in _get_display_key
cache_key = self._get_cache_key()
File "/home/sam/.local/lib/python3.6/site-packages/buildstream/element.py", line 991, in _get_cache_key
return self.__cache_key
AttributeError: 'MesonElement' object has no attribute '_Element__cache_key'
The issue here is that when `--debug` is enabled, Plugin.__init__()
tries to log a message like this:
pid:16929 id:001[--:--:--][????????][ main:tracker.bst ] DEBUG Created: meson element at tracker.bst [line 1 column 0]
The log formatter is trying to get the cache key of the element in
order to show it in the log, but the attribute hasn't yet been set
because we've not got to the Element() constructor.
-
f484ab4f
by Phillip Smyth
at 2018-07-20T14:29:44Z
Updating .bzr plugin to omit the .bzr dir
Fixes issue #455
-
b662f175
by Tristan Van Berkom
at 2018-07-20T14:29:44Z
git source plugin: Omit the .git directory completely.
This will break builds which use `git describe` to determine
their version number until a more elegant solution is implemented,
and will unblock work on caching of build trees in the artifacts.
Fixes issue #455
-
80effb5c
by Tristan Van Berkom
at 2018-07-22T08:30:40Z
HACKING.rst: No longer requiring issue numbers in commit messages.
Instead, we are enabling the merge commit feature in gitlab and
hopefully reducing some of the friction in review this way.
-
c7124c3d
by Tristan Van Berkom
at 2018-07-22T09:46:05Z
HACKING.rst: Added convention for naming of branches in merge requests.
-
282b1092
by Tristan Van Berkom
at 2018-07-23T08:47:36Z
plugins/sources/git.py: Use --shared instead of --hardlinks
This improves staging performance by avoiding making any
copies of the objects in the cloned repository in the staging
area before removing the .git directory.
Fixes #488
-
fe0df616
by Tristan Van Berkom
at 2018-07-23T09:20:44Z
Merge branch 'tristan/git-stage-with-shared-clone' into 'master'
plugins/sources/git.py: Use --shared instead of --hardlinks
Closes #488
See merge request BuildStream/buildstream!557
-
6559db4c
by Josh Smith
at 2018-07-23T09:22:39Z
element.py: Make Element._get_brief_display_key() available to Internal API
This change will allow for more detailed info messages throughout
the CASCache.
-
6b82e476
by Josh Smith
at 2018-07-23T09:22:39Z
cascache.py: Add push info detailing which remote is being pushed to
Work towards fixing issue #275
-
1d36df25
by Josh Smith
at 2018-07-23T09:22:39Z
cascache.py: Add pull info detailing which remote is being pulled from
Work towards fixing issue #275
-
364706f3
by Tristan Van Berkom
at 2018-07-23T09:57:02Z
Merge branch 'Qinusty/275' into 'master'
Indicate where artifacts are going to and coming from in the log
Closes #275
See merge request BuildStream/buildstream!553
-
e70237a5
by Phillip Smyth
at 2018-07-23T13:26:47Z
Adding caching build trees
buildstream/element.py: Adding build tree to cache
buildstream/_versions.py: Bumping BST_CORE_ARTIFACT_VERSION
-
335afb5b
by Tristan Van Berkom
at 2018-07-23T14:07:07Z
Merge branch 'caching_build_trees_limited' into 'master'
Adding caching build trees (limited)
See merge request BuildStream/buildstream!560
-
60290f31
by Javier Jardón
at 2018-07-23T15:29:37Z
doc/source/install_linux_distro.rst: buildstream doesn't depend on ostree or pygobject anymore
-
de162e43
by Javier Jardón
at 2018-07-24T11:58:16Z
Merge branch 'jjardon/ostree_repo' into 'master'
doc/source/install_linux_distro.rst: buildstream doesn't hard depend on ostree or pygobject anymore
Closes #492
See merge request BuildStream/buildstream!558
-
6106d657
by Josh Smith
at 2018-07-25T10:50:14Z
cascache.py: Remove misleading info message
This addresses issue #501.
-
4a637d63
by Jonathan Maw
at 2018-07-25T12:05:19Z
Merge branch 'Qinusty/501' into 'master'
Remove misleading info message
See merge request BuildStream/buildstream!567
-
a0ad985f
by Phil Dawson
at 2018-07-25T13:03:23Z
tests/frontend/show.py: Add test case for maximum recursion depth being exceeded
Add test to ensure gracefull handling of exception thrown while loading
a pipeline due the python's max recursion depth being exceeded.
This is part of the work for issue #203
-
95b4eae4
by Phil Dawson
at 2018-07-25T13:03:23Z
app.py: Handle exception thrown when recursion limit is exceeded
Because the RecursionError exception was introduced in Python 3.5, until we
drop support for for Python 3.4, we must use RuntimeError.
-
889bf238
by Jim MacArthur
at 2018-07-25T13:43:21Z
Merge branch 'phil/203-BuildStream-crashes-when-dependency-tree-too-deep' into 'master'
Phil/203 BuildStream crashes when dependency tree too deep
See merge request BuildStream/buildstream!512
-
54fee6c8
by Phillip Smyth
at 2018-07-25T14:15:36Z
_stream.py: Added functionality for workspace open -f
tests/frontend/workspace.py: Added tests
-
f62b6cb7
by Jonathan Maw
at 2018-07-25T15:01:33Z
Merge branch 'bst_workspace_open_force_does_nothing' into 'master'
_stream.py: Added functionality for workspace open -f
See merge request BuildStream/buildstream!549
-
c3ea3853
by Phil Dawson
at 2018-07-26T11:44:28Z
.gitlab-ci-yml: Add ubuntu 18 test
