-
4688b2c9
by Tristan Van Berkom
at 2018-12-06T07:35:59Z
tests/sources/git.py: Refactor ref-not-in-track test to use parameterization
Instead of duplicating the whole test body, run it twice while
checking for a warning or an error depending on the parameter.
-
03028c7f
by Tristan Van Berkom
at 2018-12-06T07:35:59Z
types.py: Moving CoreWarnings to the types.py module
This also ensures it is exposed via the main buildstream __init__.py
file, fixes the imports of CoreWarnings, and adjusts the documentation
links to point to the right place.
-
3b7f8df8
by Tristan Van Berkom
at 2018-12-06T07:35:59Z
Source: Fix typo in documentation, and make it a link to `bst track`
Now the commands have documentation linkability so we can link to
them when referring to them anywhere in the docs. This reference
was previously referring to a bogus non-existent `build-stream track`,
maybe this dates way back to the days before we named the frontend `bst`.
-
e5c0f067
by Tristan Van Berkom
at 2018-12-06T07:35:59Z
git source plugin: Fixing documentation linking and typo
Fixes the configurable warning part of this to properly link
to the project.conf documentation describing what configurable
warnings actually are.
Also fix the plugin raise the actual configurable warning which
it advertizes that it raises; which is 'inconsistent-submodule',
not 'inconsistent-submodules'.
-
205c27d8
by Tristan Van Berkom
at 2018-12-06T07:35:59Z
git source plugin: Rename symbol for consistency and readability
Change 'INCONSISTENT_SUBMODULE' for 'WARN_INCONSISTENT_SUBMODULE' to
make things a bit more readable and comprehensive.
-
bea4d4f5
by Tristan Van Berkom
at 2018-12-06T09:37:50Z
Merge branch 'tristan/misc-cleanup' into 'master'
Misc cleanups
See merge request BuildStream/buildstream!993
-
b1d2f001
by Tristan Van Berkom
at 2018-12-06T14:18:31Z
source.py: Add new delegate method validate_cache()
This is guaranteed to be called only once for a given session once the
sources are known to be Consistency.CACHED, if source tracking is enabled
in the session for this source, then this will only be called if the
sources become cached after tracking completes.
-
ee7fc47f
by Tristan Van Berkom
at 2018-12-06T14:18:31Z
git source plugin: Implementing submodule warnings
o Unlisted submodule warning
Now the git plugin will issue a configurable warning if a submodule
exists and is used (checking out the submodule is not disabled),
but is not specified in the source configuration.
o Invalid submodule warning
Now the git source plugin will issue a warning if the configuration
specified a submodule which does not exist in the underlying git repository.
As a side effect, this patch also changes the flow control of the git
plugin such that submodules which are explicitly set to not be
checked out, are also not fetched but instead ignored completely.
-
5fc9a1da
by Tristan Van Berkom
at 2018-12-06T14:18:31Z
git source plugin: Emmit the ref-not-in-track warning from validate_cache()
Now that we have Source.validate_cache(), this is a better place to emmit
the ref-not-in-track warning, since it will be emmitted at the earliest
opportunity and not only at Source.stage() or Source.init_workspace().
This also allows us to remove the `self.tracked` local state, and cleanup
some convoluted calling paths, removing some unnecessary parameters from
the usual codepaths and making the plugin overall more readable.
-
50cb2706
by Tristan Van Berkom
at 2018-12-06T14:18:31Z
tests/testutils/repo/git.py: Adding remove_path() helper
A function for removing paths from a git repo, can also
be used to remove submodules.
-
f139ca8f
by Tristan Van Berkom
at 2018-12-06T14:18:31Z
tests/sources/git.py: Testing the git:unlisted-submodule warning
o Test that it is not triggered in show before fetch, because we
don't know about the unlisted submodules yet
o Test that it is triggered by a fetch command
o Test that it is triggered by `show` after having completed a
fetch command, since now we have the repository and know about
the unlisted submodule
o Test all of this under warning or error conditions (parameterized
for fatal-warnings)
-
afe1aa9f
by Tristan Van Berkom
at 2018-12-06T14:18:31Z
tests/sources/git.py: Testing the git:invalid-submodule warning
o Test that it is not triggered in show before fetch, because we
don't know the submodules yet so we cannot know if they are
valid or not.
o Test that it is triggered by a fetch command
o Test that it is triggered by `show` after having completed a
fetch command, since now we have the repository and know which
specified submodules are invalid
o Test all of this under warning or error conditions (parameterized
for fatal-warnings)
-
57300201
by Tristan Van Berkom
at 2018-12-06T14:18:31Z
tests/sources/git.py: Test unlisted submodules warning appearing after track
-
a0297625
by Tristan Van Berkom
at 2018-12-06T14:18:31Z
tests/sources/git.py: Test invalid submodules warning appearing after track
-
60e96781
by Tristan Van Berkom
at 2018-12-06T15:14:37Z
Merge branch 'tristan/submodule-warnings' into 'master'
Implement submodule warnings
See merge request BuildStream/buildstream!996
-
7fe55143
by Jonathan Maw
at 2018-12-06T15:48:12Z
Fix bst source-checkout not working with open workspaces
-
2a6879a5
by Jonathan Maw
at 2018-12-06T17:46:33Z
Merge branch 'jonathan/source-checkout-workspace' into 'master'
Fix bst source-checkout not working with open workspaces
Closes #807
See merge request BuildStream/buildstream!997
-
e61e6e50
by Tristan Van Berkom
at 2018-12-07T07:57:13Z
_yaml.py: Added `allow_none` parameter to _yaml.node_get()
This allows specifying whether or not the code believes a None
value is acceptable if explicitly expressed to be None in the YAML.
A previous patch 3ba544b80f9f268be8ffe62fc8589b30212ec4a2 changed
the behavior to accept None for all values across the board, along
with allowing explicitly setting the `default_value` to `None` which
caused the code to be much more readable (this was the main motivation
of the patch, but it had the side effect of allowing None for everything).
In the majority of cases we load YAML however, it either has a value or
it is not specified in the YAML, and None is hardly ever acceptable
to be explicitly specified (it may be in the case you want to override
something with nothing in the YAML, but it is the edge case).
This had the side effect of causing unexpected crashes where the
YAML specifies an empty dictionary for instance.
Instead of forcing the caller to handle a possible None value, give
the choice to the caller if they really want to accept a None value,
and raise the LoadError otherwise.
This fixes issue #803, which is a crash due to receival of an
unexpected None value through _yaml.node_get()
-
504ecb3e
by Tristan Van Berkom
at 2018-12-07T07:57:56Z
plugin.py: Added new `allow_none` parameter to Plugin.node_get_member(), defaulting to False.
Allow plugins to conveniently leverage the new `allow_none` parameter,
and have Plugin.node_get_member() raise a LoadError in the usual
case where None is not an acceptable value to be explicitly set in
the user provided YAML.
-
642ae4e8
by Tristan Van Berkom
at 2018-12-07T07:57:56Z
tests/format/project.py: Added new regression test for empty dependency dictionaries
This adds a regression test for issue #803
-
2a0676c3
by Tristan Van Berkom
at 2018-12-07T08:33:14Z
Merge branch 'tristan/yaml-optionally-allow-none' into 'master'
Only optionally allow None values in user provided YAML
Closes #803
See merge request BuildStream/buildstream!999
-
56c2c2cc
by Javier Jardón
at 2018-12-09T16:59:26Z
buildstream/data/projectconfig.yaml: Remove default strip-binaries
They are too specific to be included by default
Recommendation is if you are building in Linux is to use the
ones begin used in the freedesktop-sdk project, for example
See #645
-
dcb6b5e8
by Javier Jardón
at 2018-12-09T16:59:26Z
NEWS: Update for removal of default strip-commands
-
ae0afce5
by Javier Jardón
at 2018-12-09T17:08:16Z
buildstream/buildelement.py: Document where to find strip commands for Linux
-
873b618c
by Javier Jardón
at 2018-12-10T09:49:52Z
Merge branch 'jjardon/strip-binaries-removal' into 'master'
Remove default strip-commands
Closes #645
See merge request BuildStream/buildstream!987
-
f2fcc2f6
by Valentin David
at 2018-12-11T12:12:08Z
Force updating tags when fetching git repository
When using aliases there are multiple remotes used in the cache
repository. When fetching, tags are not updated if the were previously
fetched from a different remote. Commits that not in a branch and only
tagged do not get fetched if the tag is not fetched.
Fixes #812
-
717c10d1
by Valentin David
at 2018-12-11T12:44:56Z
Merge branch 'valentindavid/git_force_fetch_tags' into 'master'
Force updating tags when fetching git repository
Closes #812
See merge request BuildStream/buildstream!1000
-
7892287a
by Jonathan Maw
at 2018-12-11T12:56:32Z
utils.py: Add a helper for searching upwards for files
i.e. with a given directory and filename, check parent directories until
either a directory with the filename is found, or you reach the root of
the filesystem.
This is a part of #222
-
67c7a58d
by Jonathan Maw
at 2018-12-11T12:56:32Z
Create and store data inside projects when opening workspaces
Changes to _context.py:
* Context has been extended to contain a WorkspaceProjectCache, as there
are times when we want to use it before a Workspaces can be initialised
(looking up a WorkspaceProject to find the directory that the project is
in)
Changes to _stream.py:
* Removed staging the elements from workspace_open() and workspace_reset()
Changes in _workspaces.py:
* A new WorkspaceProject contains all the information needed to refer back
to a project from its workspace (currently this is the project path and
the element used to create this workspace)
* This is stored within a new WorkspaceProjectCache object, which keeps
WorkspaceProjects around so they don't need to be loaded from disk
repeatedly.
* Workspaces has been extended to contain the WorkspaceProjectCache, and
will use it when opening and closing workspaces.
* Workspaces.create_workspace has been extended to handle the staging of
the element into the workspace, in addition to creating the equivalent
WorkspaceProject file.
This is a part of #222
-
64836b18
by Jonathan Maw
at 2018-12-11T12:56:32Z
_project.py: Rename _ensure_project_dir to _find_project_dir
This is a part of #222
-
7deeb2c3
by Jonathan Maw
at 2018-12-11T12:56:32Z
cli.py: Use utils' search upwards helper when searching for project.conf
This is a part of #222
-
496f0ab7
by Jonathan Maw
at 2018-12-11T12:56:32Z
_project.py: Find project from workspace if outside of a project
This is a part of #222
-
921f2bcb
by Jonathan Maw
at 2018-12-11T12:56:32Z
_workspaces.py: Do not include .bstproject.yaml in the cache key
This is a part of #222
-
f145a3e4
by Jonathan Maw
at 2018-12-11T12:56:32Z
cli: Interactively warn if the user is trying to close the workspace they're using to load the project
This involves changes in:
* _stream.py:
* Add the helper Stream.workspace_is_required()
* userconfig.yaml:
* Add a default value for prompt.really-workspace-close-project-inaccessible
* _context.py:
* Load the prompt 'really-workspace-close-project-inaccessible' from
user config.
* cli.py:
* If buildstream is invoked interactively, prompt the user to confirm
that they want to close the workspace they're using to load this
project.
This is a part of #222
-
494cb7c6
by Jonathan Maw
at 2018-12-11T14:12:54Z
tests: Test bst commands from an external workspace
This is a part of #222
-
fd1c5c5a
by Jonathan Maw
at 2018-12-11T14:12:54Z
NEWS: Add an entry for being able to run commands from a workspace
This is a part of #222
-
4219a6b4
by Jonathan Maw
at 2018-12-11T16:00:55Z
Merge branch 'jonathan/workspace-fragment-create' into 'master'
Support invoking buildstream from a workspace outside a project
See merge request BuildStream/buildstream!924
-
e564251b
by Jonathan Maw
at 2018-12-11T16:15:19Z
_stream.py: Fix ugly error when opening a workspace using a relative path
-
7cf83343
by Jonathan Maw
at 2018-12-11T16:15:19Z
_context.py: Store the directory buildstream was invoked on
-
a1dee91e
by Jonathan Maw
at 2018-12-11T16:15:19Z
Make specifying elements optional in bst commands
Known issues:
* `bst shell` works, but `bst shell COMMANDS...` doesn't, because click
has no way of separating optional args from variable-length args.
* `bst checkout` and `bst source-checkout`'s usage strings mark LOCATION
as an optional argument. Because click gets confused if there's an
optional argument before a mandatory argument, I had to mark LOCATION
as optional internally.
* `bst workspace open` makes no sense with element being optional, so
I skipped it.
* `bst workspace close` will probably need to be revisited when multiple
projects can own one workspace.
* `bst workspace reset` will happily delete the directory you're
currently in, requiring you to `cd $PWD` to see the contents of your
directory.
I could exclude the top-level directory of the workspace being
deleted, but it is entirely valid to run workspace commands from deeper
in the workspace.
This is a part of #222
-
9b192015
by Jonathan Maw
at 2018-12-11T16:15:19Z
tests: Add tests for guessing element names
This is a part of #222
-
e340371f
by Jonathan Maw
at 2018-12-11T16:15:19Z
Add NEWS for guessing the element in commands
This is a part of #222
-
bbf81c3d
by Jonathan Maw
at 2018-12-11T16:15:19Z
doc: Point out that element is optional when commands are run from the workspace
This is a part of #222
-
a5a53ddd
by Jonathan Maw
at 2018-12-12T13:44:09Z
Merge branch 'jonathan/workspace-fragment-guess-element' into 'master'
Guess the element when running commands in a workspace
See merge request BuildStream/buildstream!954
-
d55b9e39
by Phil Dawson
at 2018-12-12T13:55:19Z
Add --tar option to source-checkout command
This commit is part of the work towards #672
-
733aab53
by Phil Dawson
at 2018-12-12T13:55:19Z
Add --force / -f option to source-checkout command
-
c2efeba0
by Phil Dawson
at 2018-12-12T14:43:40Z
Add option to source-checkout command to generate build scripts
-
85c61894
by Phil Dawson
at 2018-12-12T14:43:40Z
Remove source bundle command
This is part of the work towards #672
-
a322d5c0
by Phil Dawson
at 2018-12-12T14:43:40Z
NEWS: Add entry about the removal of source-bundle command
-
ec909605
by Phil Dawson
at 2018-12-12T15:45:42Z
Merge branch 'phil/source-checkout-options' into 'master'
Retire bst source bundle command
Closes #672
See merge request BuildStream/buildstream!959
-
3697a611
by Richard Maw
at 2018-12-12T16:31:38Z
cli: Add support for auto-completing artifact ref names
-
b3dceb16
by Richard Maw
at 2018-12-12T16:32:41Z
cli: Add artifact command group
-
ba08a0cd
by Richard Maw
at 2018-12-12T16:32:41Z
cli: Add artifact log command
-
70fb9554
by Richard Maw
at 2018-12-12T16:32:41Z
tests: Add tests for artifact group commands
-
f773e746
by Richard Maw
at 2018-12-12T16:33:02Z
NEWS: Mention `bst artifact log`
-
b6528441
by richardmaw-codethink
at 2018-12-12T18:00:59Z
Merge branch 'richardmaw/artifact-log' into 'master'
Add artifact log command
See merge request BuildStream/buildstream!920
-
d03bf316
by Benjamin Schubert
at 2018-12-13T10:24:11Z
Mock storage space checks for tests.
Fix #530
- Extract free space computation in a function for easier
mocking
- Mock space computation during cache quota tests
- Mock cache size during cachque quota tests
- Add two more tests when the configuration would require
to much storage space
-
a116f576
by Tristan Van Berkom
at 2018-12-13T10:58:46Z
Merge branch 'BenjaminSchubert/fix-quota-tests' into 'master'
Mock storage space checks for tests.
Closes #530
See merge request BuildStream/buildstream!702
-
180fa774
by Benjamin Schubert
at 2018-12-13T12:05:15Z
element.py: fix unbounded variable in nested python exception
As per https://docs.python.org/3.7/reference/compound_stmts.html#except
variables used in the "except" clause are erased at the end of the
block.
We had a conflict with a nested exception using the same variable as
before.
This renames the nested variable to avoid the clash
-
224aa4c2
by Benjamin Schubert
at 2018-12-13T12:34:41Z
Merge branch 'bschubert/fix-unbound-variable-exception' into 'master'
element.py: fix unbounded variable in nested python exception
See merge request BuildStream/buildstream!1006
-
053beb66
by Tristan Van Berkom
at 2018-12-13T14:23:19Z
manual plugin: Support virtual directories
-
29ab271c
by Tristan Van Berkom
at 2018-12-13T14:23:19Z
makemaker plugin: Support virtual directories
-
ba955cf0
by Tristan Van Berkom
at 2018-12-13T14:23:19Z
pip element plugin: Support virtual directories
-
6010b5a4
by Tristan Van Berkom
at 2018-12-13T14:23:19Z
modulebuild plugin: Support virtual directories
-
3a6d27a4
by Tristan Van Berkom
at 2018-12-13T14:23:19Z
distutils plugin: Support virtual directories
-
4c0e602c
by Tristan Van Berkom
at 2018-12-13T14:23:19Z
BuildElement: Don't enable batching of prepare and assemble by default
Some external plugins depend derive from BuildElement and are broken
by BuildElement enabling this batching by default.
Instead, enable it in all of the individual build element plugin
implementations.
This fixes issue #800
-
60ddeeb9
by Tristan Van Berkom
at 2018-12-13T14:58:28Z
Merge branch 'tristan/dont-batch-prepare-assemble-by-default' into 'master'
Dont batch prepare assemble by default
Closes #800
See merge request BuildStream/buildstream!1009
-
bf72cc42
by Angelos Evripiotis
at 2018-12-13T17:31:17Z
contributing: mandate end-to-end tests, allow others
Update the section on adding tests, to fulfill these broad points:
- Mandate end-to-end testing, with rationale.
- Mention internal APIs as endpoints for testing.
- Warn against pitfalls of unit-testing.
It's more text than I would have liked, perhaps in later work we'll be
able to say it with less.
This change integrates feedback from the mailing list
'Guidance on Unit Tests' thread:
https://mail.gnome.org/archives/buildstream-list/2018-November/msg00045.html
-
13eb7ed2
by Angelos Evripiotis
at 2018-12-13T18:01:56Z
Merge branch 'aevri/contributing_e2e_tests' into 'master'
contributing: more clarity on testing
See merge request BuildStream/buildstream!973
-
629a6e52
by Chandan Singh
at 2018-12-14T19:34:20Z
Introduce new "source" command group
Following the message thread
https://mail.gnome.org/archives/buildstream-list/2018-November/msg00106.html,
implement a new command group called `source`. Move existing `track`,
`fetch`, and the recently added `source-checkout` commands under this
group.
For `track` and `fetch`, this is a BREAKING change, as the old commands
have been marked as obsolete. Using them will result in an error message
that refers people to use the new versions, like `bst source fetch`
instead of old `bst fetch`. `source-checkout` will now become
`source checkout` (the dash has turned into a space), and is not a
breaking change as it was added in the current development cycle.
Note that the functionality to hide commands from help output was added
only recently in Click, so the minimum version of Click that we now
require is 7.0.
Summary of changes:
* _frontend/cli.py: Add `source` command group, mark previous versions
as obsolete and hide them from the help output.
* _frontend/complete.py: Fix completion for hidden commands.
* setup.py: Bump Click minimum version to 7.0.
* tests: Update to cope with the new command names.
Fixes #814.
-
f894c0a8
by Chandan Singh
at 2018-12-14T19:34:20Z
NEWS: Add entry for new source command group
Also, change the existing entry about `source-checkout` to be
`source checkout` instead. And, while we are there, move it near the
other announcements about the source command group.
-
b23bec55
by Chandan Singh
at 2018-12-14T20:07:13Z
Merge branch 'chandan/source-subgroup' into 'master'
Introduce new "source" command group
Closes #814
See merge request BuildStream/buildstream!1003
-
d2105909
by Raoul Hidalgo Charman
at 2018-12-18T11:13:56Z
Add remote execution instance option
This is used when sending execution requests, to specify which instance of the
execution server to use.
Partial fix for #627.
-
a3bbec23
by Jim MacArthur
at 2018-12-18T11:13:56Z
sandboxremote: Add server/storage config with defaults
Adds instance name support for the remote execution storage service.
-
89219f61
by Jim MacArthur
at 2018-12-18T11:13:56Z
_cascache.py: Add instance names to GRPC calls
Adds the 'instance_name' parameter, which may be None, to most GRPC
calls in the CASCache object. ByteStream requests already have
instance_name supplied in the resource name, so do not need the
parameter.
Closes #627.
-
3dc20963
by Jim MacArthur
at 2018-12-18T11:13:56Z
Documentation: Include instance-name in remote execution documentation
-
644d8b28
by Jim MacArthur
at 2018-12-18T11:45:30Z
Merge branch 'raoul/627-RE-instance-config' into 'master'
Remote-execution instance configuration support
Closes #627
See merge request BuildStream/buildstream!952
-
e29aea36
by William Salmon
at 2018-12-19T13:23:19Z
Basic options for shell --build to use buildtrees
Fixes issue #740
-
898a23a5
by Will Salmon
at 2018-12-19T14:35:41Z
Merge branch 'willsalmon/shellBuildTrees' into 'master'
Shell --build has optional buildtrees
Closes #740
See merge request BuildStream/buildstream!986
-
a2f1d879
by Javier Jardón
at 2018-12-19T15:36:11Z
README.rst: Add license badge
-
aae5e4b3
by Javier Jardón
at 2018-12-19T16:09:48Z
Merge branch 'jjardon/license_badge' into 'master'
README.rst: Add license badge
See merge request BuildStream/buildstream!1014
-
2b767fe8
by Jürg Billeter
at 2018-12-20T10:06:11Z
Move fetch logic from FetchQueue to Element
The queue shouldn't need to know about individual sources. This is in
line with _track() and _get_consistency().
-
7a102144
by Jürg Billeter
at 2018-12-20T10:06:11Z
element.py: Do not call fetch() for cached sources
-
b325989e
by Jürg Billeter
at 2018-12-20T10:07:20Z
tests/sources: Test that fetch() is not called for cached sources
