[Notes] [Git][BuildStream/buildstream][tpollard/774] 3 commits: man/: up

Jürg Billeter pushed to branch tpollard/774 at BuildStream / buildstream

Commits:

1f31aefc

by Angelos Evripiotis at 2018-11-30T09:34:38Z

man/: update with changes since Apr 2018

It seems that the regenerated man pages haven't been committed for some
time. Add them all in one lump now.

Had to manually strip the version number from the generated files.

Had to manually fix the NAME field of bst.1 to not be 'bst - None'.

Oddly had to invoke click-man directly in order to get the changes to
be applied, the usual setup.py route mentioned in contributing.rst
didn't work.

Eyeballed the result of invoking man on each of the commands, seems
good.

033a5ad9

by Jürg Billeter at 2018-11-30T10:09:11Z

Merge branch 'aevri/update_man' into 'master'

man/: update with changes since Apr 2018

See merge request BuildStream/buildstream!981

1123b9a1

by Tom Pollard at 2018-11-30T13:30:35Z

_stream.py: Ability to pull missing buildtrees outside of pull/build

Adds helper function _buildtree_pull_required() to determine if a
pullqueue should be constructed, for commands outside of bst pull
and build where it is determined that an element's buildtree
artifact is to be required given the respective semantics and
config. Utilised in push() to attempt to mitigate skipping the push
of partial elements without the user having to have preceded it with
an explicit pull.

cli.py: Add new behaviour to push command description

element.py: Move _cached_buildtree() to be non local private method,
use _KeyStrength types to reduce duplication.

tests/integration/pullbuildtrees.py also updated to cover this
use-case.

23 changed files:

buildstream/_frontend/cli.py
buildstream/_stream.py
buildstream/element.py
+ man/bst-artifact-server.1
man/bst-build.1
man/bst-checkout.1
man/bst-fetch.1
+ man/bst-help.1
man/bst-init.1
man/bst-pull.1
man/bst-push.1
man/bst-shell.1
man/bst-show.1
man/bst-source-bundle.1
+ man/bst-source-checkout.1
man/bst-track.1
man/bst-workspace-close.1
man/bst-workspace-list.1
man/bst-workspace-open.1
man/bst-workspace-reset.1
man/bst-workspace.1
man/bst.1
tests/integration/pullbuildtrees.py

Changes:

buildstream/_frontend/cli.py

@@ -469,6 +469,10 @@ def push(app, elements, deps, remote):
      The default destination is the highest priority configured cache. You can
      override this by passing a different cache URL with the `--remote` flag.
 +    If bst has been configured to include build trees on artifact pulls,
 +    an attempt will be made to pull any required build trees to avoid the
 +    skipping of partial artifacts being pushed.
++
      Specify `--deps` to control which artifacts to push:
      \b

buildstream/_stream.py

@@ -327,6 +327,10 @@ class Stream():
      # If `remote` specified as None, then regular configuration will be used
      # to determine where to push artifacts to.
+     #
 +    # If any of the given targets are missing their expected buildtree artifact,
 +    # a pull queue will be created if user context and available remotes allow for
 +    # attempting to fetch them.
 +    #
      def push(self, targets, *,
               selection=PipelineSelection.NONE,
               remote=None):
@@ -345,8 +349,17 @@ class Stream():
              raise StreamError("No artifact caches available for pushing artifacts")
          self._pipeline.assert_consistent(elements)
 -        self._add_queue(PushQueue(self._scheduler))
 -        self._enqueue_plan(elements)
++
 +        # Check if we require a pull queue, with given artifact state and context
 +        require_buildtrees = self._buildtree_pull_required(elements)
 +        if require_buildtrees:
 +            self._message(MessageType.INFO, "Attempting to fetch missing artifact buildtrees")
 +            self._add_queue(PullQueue(self._scheduler))
 +            self._enqueue_plan(require_buildtrees)
++
 +        push_queue = PushQueue(self._scheduler)
 +        self._add_queue(push_queue)
 +        self._enqueue_plan(elements, queue=push_queue)
          self._run()
      # checkout()
@@ -1237,3 +1250,28 @@ class Stream():
              parts.append(element.normal_name)
          return os.path.join(directory, *reversed(parts))
++
 +    # _buildtree_pull_required()
 +    #
 +    # Check if current task, given config, requires element buildtree artifact
 +    #
 +    # Args:
 +    #    elements (list): elements to check if buildtrees are required
 +    #
 +    # Returns:
 +    #    (list): elements requiring buildtrees
 +    #
 +    def _buildtree_pull_required(self, elements):
 +        required_list = []
++
 +        # If context is set to not pull buildtrees, or no fetch remotes, return empty list
 +        if not (self._context.pull_buildtrees or self._artifacts.has_fetch_remotes()):
 +            return required_list
++
 +        for element in elements:
 +            # Check if element is partially cached without its buildtree, as the element
 +            # artifact may not be cached at all
 +            if element._cached() and not element._cached_buildtree():
 +                required_list.append(element)
++
 +        return required_list

buildstream/element.py

@@ -1422,7 +1422,7 @@ class Element(Plugin):
                                                   .format(workspace.get_absolute_path())):
                              workspace.stage(temp_staging_directory)
                  # Check if we have a cached buildtree to use
 -                elif self.__cached_buildtree():
 +                elif self._cached_buildtree():
                      artifact_base, _ = self.__extract()
                      import_dir = os.path.join(artifact_base, 'buildtree')
                  else:
@@ -1808,7 +1808,7 @@ class Element(Plugin):
          # Do not push elements that aren't cached, or that are cached with a dangling buildtree
          # artifact unless element type is expected to have an an empty buildtree directory
 -        if not self.__cached_buildtree():
 +        if not self._cached_buildtree():
              return True
          # Do not push tainted artifact
@@ -1998,6 +1998,29 @@ class Element(Plugin):
      def _get_source_element(self):
          return self
 +    # _cached_buildtree()
 +    #
 +    # Check if element artifact contains expected buildtree. An
 +    # element's buildtree artifact will not be present if the rest
 +    # of the partial artifact is not cached.
 +    #
 +    # Returns:
 +    #     (bool): True if artifact cached with buildtree, False if
 +    #             element not cached or missing expected buildtree.
 +    #
 +    def _cached_buildtree(self):
 +        context = self._get_context()
++
 +        if not self._cached():
 +            return False
++
 +        key_strength = _KeyStrength.STRONG if context.get_strict() else _KeyStrength.WEAK
 +        if not self.__artifacts.contains_subdir_artifact(self, self._get_cache_key(strength=key_strength),
 +                                                         'buildtree'):
 +            return False
++
 +        return True
++
      #############################################################
      #                   Private Local Methods                   #
      #############################################################
@@ -2764,27 +2787,6 @@ class Element(Plugin):
          return True
 -    # __cached_buildtree():
 -    #
 -    # Check if cached element artifact contains expected buildtree
 -    #
 -    # Returns:
 -    #     (bool): True if artifact cached with buildtree, False if
 -    #             element not cached or missing expected buildtree
 -    #
 -    def __cached_buildtree(self):
 -        context = self._get_context()
+-
 -        if not self._cached():
 -            return False
 -        elif context.get_strict():
 -            if not self.__artifacts.contains_subdir_artifact(self, self.__strict_cache_key, 'buildtree'):
 -                return False
 -        elif not self.__artifacts.contains_subdir_artifact(self, self.__weak_cache_key, 'buildtree'):
 -            return False
+-
 -        return True
+-
      # __pull_directories():
+     #
      # Which directories to include or exclude given the current

man/bst-artifact-server.1

 +.TH "BST-ARTIFACT-SERVER" "1" "29-Nov-2018" "" "bst-artifact-server Manual"
 +.SH NAME
 +bst-artifact-server \- CAS Artifact Server
 +.SH SYNOPSIS
 +.B bst-artifact-server
 +[OPTIONS] REPO
 +.SH OPTIONS
 +.TP
 +\fB\-p,\fP \-\-port INTEGER
 +Port number  [required]
 +.TP
 +\fB\-\-server\-key\fP TEXT
 +Private server key for TLS (PEM-encoded)
 +.TP
 +\fB\-\-server\-cert\fP TEXT
 +Public server certificate for TLS (PEM-encoded)
 +.TP
 +\fB\-\-client\-certs\fP TEXT
 +Public client certificates for TLS (PEM-encoded)
 +.TP
 +\fB\-\-enable\-push\fP
 +Allow clients to upload blobs and update artifact cache
 \ No newline at end of file

man/bst-build.1

 -.TH "BST BUILD" "1" "26-Apr-2018" "" "bst build Manual"
 +.TH "BST BUILD" "1" "29-Nov-2018" "" "bst build Manual"
  .SH NAME
  bst\-build \- Build elements in a pipeline
  .SH SYNOPSIS

man/bst-checkout.1

 -.TH "BST CHECKOUT" "1" "26-Apr-2018" "" "bst checkout Manual"
 +.TH "BST CHECKOUT" "1" "29-Nov-2018" "" "bst checkout Manual"
  .SH NAME
  bst\-checkout \- Checkout a built artifact
  .SH SYNOPSIS
@@ -12,7 +12,7 @@ Checkout a built artifact to the specified location
  \fB\-f,\fP \-\-force
  Allow files to be overwritten
  .TP
 -\fB\-f,\fP \-\-deps
 +\fB\-d,\fP \-\-deps [run|none]
  The dependencies to checkout (default: run)
  .TP
  \fB\-\-integrate\fP / \-\-no\-integrate
@@ -22,5 +22,4 @@ Whether to run integration commands
  Checkout hardlinks instead of copies (handle with care)
  .TP
  \fB\-\-tar\fP
 -Create a tarball from the artifact contents instead of a file tree. If
 -LOCATION is '-', the tarball will be dumped to the standard output.
 +Create a tarball from the artifact contents instead of a file tree. If LOCATION is '-', the tarball will be dumped to the standard output.
 \ No newline at end of file

man/bst-fetch.1

 -.TH "BST FETCH" "1" "26-Apr-2018" "" "bst fetch Manual"
 +.TH "BST FETCH" "1" "29-Nov-2018" "" "bst fetch Manual"
  .SH NAME
  bst\-fetch \- Fetch sources in a pipeline
  .SH SYNOPSIS
@@ -6,14 +6,14 @@ bst\-fetch \- Fetch sources in a pipeline
  [OPTIONS] [ELEMENTS]...
  .SH DESCRIPTION
  Fetch sources required to build the pipeline
+-
 +.PP
  By default this will only try to fetch sources which are
  required for the build plan of the specified target element,
  omitting sources for any elements which are already built
  and available in the artifact cache.
+-
 +.PP
  Specify `--deps` to control which sources to fetch:
+-
 +.PP
      none:  No dependencies, just the element itself
      plan:  Only dependencies required for the build plan

man/bst-help.1

 +.TH "BST HELP" "1" "29-Nov-2018" "" "bst help Manual"
 +.SH NAME
 +bst\-help \- Print usage information
 +.SH SYNOPSIS
 +.B bst help
 +[OPTIONS] COMMAND
 +.SH DESCRIPTION
 +Print usage information about a given command
++
 \ No newline at end of file

man/bst-init.1

 -.TH "BST INIT" "1" "26-Apr-2018" "" "bst init Manual"
 +.TH "BST INIT" "1" "29-Nov-2018" "" "bst init Manual"
  .SH NAME
  bst\-init \- Initialize a new BuildStream project
  .SH SYNOPSIS
@@ -6,10 +6,10 @@ bst\-init \- Initialize a new BuildStream project
  [OPTIONS]
  .SH DESCRIPTION
  Initialize a new BuildStream project
+-
 +.PP
  Creates a new BuildStream project.conf in the project
  directory.
+-
 +.PP
  Unless `--project-name` is specified, this will be an
  interactive session.
  .SH OPTIONS
@@ -18,7 +18,7 @@ interactive session.
  The project name to use
  .TP
  \fB\-\-format\-version\fP INTEGER
 -The required format version (default: 8)
 +The required format version (default: 18)
  .TP
  \fB\-\-element\-path\fP PATH
  The subdirectory to store elements in (default: elements)

man/bst-pull.1

 -.TH "BST PULL" "1" "26-Apr-2018" "" "bst pull Manual"
 +.TH "BST PULL" "1" "29-Nov-2018" "" "bst pull Manual"
  .SH NAME
  bst\-pull \- Pull a built artifact
  .SH SYNOPSIS
@@ -6,13 +6,13 @@ bst\-pull \- Pull a built artifact
  [OPTIONS] [ELEMENTS]...
  .SH DESCRIPTION
  Pull a built artifact from the configured remote artifact cache.
+-
 +.PP
  By default the artifact will be pulled one of the configured caches
  if possible, following the usual priority order. If the `--remote` flag
  is given, only the specified cache will be queried.
+-
 +.PP
  Specify `--deps` to control which artifacts to pull:
+-
 +.PP
      none:  No dependencies, just the element itself
      all:   All dependencies

man/bst-push.1

 -.TH "BST PUSH" "1" "26-Apr-2018" "" "bst push Manual"
 +.TH "BST PUSH" "1" "29-Nov-2018" "" "bst push Manual"
  .SH NAME
  bst\-push \- Push a built artifact
  .SH SYNOPSIS
@@ -6,12 +6,12 @@ bst\-push \- Push a built artifact
  [OPTIONS] [ELEMENTS]...
  .SH DESCRIPTION
  Push a built artifact to a remote artifact cache.
+-
 +.PP
  The default destination is the highest priority configured cache. You can
  override this by passing a different cache URL with the `--remote` flag.
+-
 +.PP
  Specify `--deps` to control which artifacts to push:
+-
 +.PP
      none:  No dependencies, just the element itself
      all:   All dependencies

man/bst-shell.1

 -.TH "BST SHELL" "1" "26-Apr-2018" "" "bst shell Manual"
 +.TH "BST SHELL" "1" "29-Nov-2018" "" "bst shell Manual"
  .SH NAME
  bst\-shell \- Shell into an element's sandbox environment
  .SH SYNOPSIS
@@ -6,18 +6,18 @@ bst\-shell \- Shell into an element's sandbox environment
  [OPTIONS] ELEMENT [COMMAND]...
  .SH DESCRIPTION
  Run a command in the target element's sandbox environment
+-
 +.PP
  This will stage a temporary sysroot for running the target
  element, assuming it has already been built and all required
  artifacts are in the local cache.
+-
 +.PP
  Use the --build option to create a temporary sysroot for
  building the element instead.
+-
 +.PP
  Use the --sysroot option with an existing failed build
  directory or with a checkout of the given target, in order
  to use a specific sysroot.
+-
 +.PP
  If no COMMAND is specified, the default is to attempt
  to run an interactive shell.
  .SH OPTIONS

man/bst-show.1

 -.TH "BST SHOW" "1" "26-Apr-2018" "" "bst show Manual"
 +.TH "BST SHOW" "1" "29-Nov-2018" "" "bst show Manual"
  .SH NAME
  bst\-show \- Show elements in the pipeline
  .SH SYNOPSIS
@@ -6,25 +6,25 @@ bst\-show \- Show elements in the pipeline
  [OPTIONS] [ELEMENTS]...
  .SH DESCRIPTION
  Show elements in the pipeline
+-
 +.PP
  By default this will show all of the dependencies of the
  specified target element.
+-
 +.PP
  Specify `--deps` to control which elements to show:
+-
 +.PP
      none:  No dependencies, just the element itself
      plan:  Dependencies required for a build plan
      run:   Runtime dependencies, including the element itself
      build: Build time dependencies, excluding the element itself
      all:   All dependencies
+-
 +.PP
  FORMAT
  ~~~~~~
  The --format option controls what should be printed for each element,
  the following symbols can be used in the format string:
+-
 +.PP
      %{name}           The element name
      %{key}            The abbreviated cache key (if all sources are consistent)
@@ -36,17 +36,17 @@ the following symbols can be used in the format string:
      %{public}         Public domain data
      %{workspaced}     If the element is workspaced
      %{workspace-dirs} A list of workspace directories
+-
 +.PP
  The value of the %{symbol} without the leading '%' character is understood
  as a pythonic formatting string, so python formatting features apply,
  examle:
+-
 +.PP
      bst show target.bst --format \
          'Name: %{name: ^20} Key: %{key: ^8} State: %{state}'
+-
 +.PP
  If you want to use a newline in a format string in bash, use the '$' modifier:
+-
 +.PP
      bst show target.bst --format \
          $'---------- %{name} ----------\n%{vars}'
@@ -62,7 +62,4 @@ The dependencies to show (default: all)
  Staging or alphabetic ordering of dependencies
  .TP
  \fB\-f,\fP \-\-format FORMAT
 -Format string for each element
 -.TP
 -\fB\-\-downloadable\fP
 -Refresh downloadable state
 \ No newline at end of file
 +Format string for each element
 \ No newline at end of file

man/bst-source-bundle.1

 -.TH "BST SOURCE-BUNDLE" "1" "26-Apr-2018" "" "bst source-bundle Manual"
 +.TH "BST SOURCE-BUNDLE" "1" "29-Nov-2018" "" "bst source-bundle Manual"
  .SH NAME
  bst\-source-bundle \- Produce a build bundle to be manually executed
  .SH SYNOPSIS
  .B bst source-bundle
 -[OPTIONS] TARGET
 +[OPTIONS] ELEMENT
  .SH DESCRIPTION
  Produce a source bundle to be manually executed
@@ -16,10 +16,10 @@ Elements to except from the tarball
  Compress the tar file using the given algorithm.
  .TP
  \fB\-\-track\fP
 -Track new source references before building
 +Track new source references before bundling
  .TP
  \fB\-f,\fP \-\-force
 -Overwrite files existing in checkout directory
 +Overwrite an existing tarball
  .TP
  \fB\-\-directory\fP TEXT
  The directory to write the tarball to
 \ No newline at end of file

man/bst-source-checkout.1

 +.TH "BST SOURCE-CHECKOUT" "1" "29-Nov-2018" "" "bst source-checkout Manual"
 +.SH NAME
 +bst\-source-checkout \- Checkout sources for an element
 +.SH SYNOPSIS
 +.B bst source-checkout
 +[OPTIONS] ELEMENT LOCATION
 +.SH DESCRIPTION
 +Checkout sources of an element to the specified location
++
 +.SH OPTIONS
 +.TP
 +\fB\-\-except\fP PATH
 +Except certain dependencies
 +.TP
 +\fB\-d,\fP \-\-deps [build|none|run|all]
 +The dependencies whose sources to checkout (default: none)
 +.TP
 +\fB\-\-fetch\fP
 +Fetch elements if they are not fetched
 \ No newline at end of file

man/bst-track.1

 -.TH "BST TRACK" "1" "26-Apr-2018" "" "bst track Manual"
 +.TH "BST TRACK" "1" "29-Nov-2018" "" "bst track Manual"
  .SH NAME
  bst\-track \- Track new source references
  .SH SYNOPSIS
@@ -7,12 +7,12 @@ bst\-track \- Track new source references
  .SH DESCRIPTION
  Consults the specified tracking branches for new versions available
  to build and updates the project with any newly available references.
+-
 +.PP
  By default this will track just the specified element, but you can also
  update a whole tree of dependencies in one go.
+-
 +.PP
  Specify `--deps` to control which sources to track:
+-
 +.PP
      none:  No dependencies, just the specified elements
      all:   All dependencies of all specified elements

man/bst-workspace-close.1

 -.TH "BST WORKSPACE CLOSE" "1" "26-Apr-2018" "" "bst workspace close Manual"
 +.TH "BST WORKSPACE CLOSE" "1" "29-Nov-2018" "" "bst workspace close Manual"
  .SH NAME
  bst\-workspace\-close \- Close workspaces
  .SH SYNOPSIS

man/bst-workspace-list.1

 -.TH "BST WORKSPACE LIST" "1" "26-Apr-2018" "" "bst workspace list Manual"
 +.TH "BST WORKSPACE LIST" "1" "29-Nov-2018" "" "bst workspace list Manual"
  .SH NAME
  bst\-workspace\-list \- List open workspaces
  .SH SYNOPSIS
  .B bst workspace list
  [OPTIONS]
  .SH DESCRIPTION
 -List open workspaces
 \ No newline at end of file
 +List open workspaces

man/bst-workspace-open.1

 -.TH "BST WORKSPACE OPEN" "1" "26-Apr-2018" "" "bst workspace open Manual"
 +.TH "BST WORKSPACE OPEN" "1" "29-Nov-2018" "" "bst workspace open Manual"
  .SH NAME
  bst\-workspace\-open \- Open a new workspace
  .SH SYNOPSIS
  .B bst workspace open
 -[OPTIONS] ELEMENT DIRECTORY
 +[OPTIONS] ELEMENTS...
  .SH DESCRIPTION
  Open a workspace for manual source modification
  .SH OPTIONS
@@ -12,7 +12,10 @@ Open a workspace for manual source modification
  Do not checkout the source, only link to the given directory
  .TP
  \fB\-f,\fP \-\-force
 -Overwrite files existing in checkout directory
 +The workspace will be created even if the directory in which it will be created is not empty or if a workspace for that element already exists
  .TP
  \fB\-\-track\fP
 -Track and fetch new source references before checking out the workspace
 \ No newline at end of file
 +Track and fetch new source references before checking out the workspace
 +.TP
 +\fB\-\-directory\fP DIRECTORY
 +Only for use when a single Element is given: Set the directory to use to create the workspace
 \ No newline at end of file

man/bst-workspace-reset.1

 -.TH "BST WORKSPACE RESET" "1" "26-Apr-2018" "" "bst workspace reset Manual"
 +.TH "BST WORKSPACE RESET" "1" "29-Nov-2018" "" "bst workspace reset Manual"
  .SH NAME
  bst\-workspace\-reset \- Reset a workspace to its original state
  .SH SYNOPSIS
@@ -8,6 +8,9 @@ bst\-workspace\-reset \- Reset a workspace to its original state
  Reset a workspace to its original state
  .SH OPTIONS
  .TP
 +\fB\-\-soft\fP
 +Reset workspace state without affecting its contents
 +.TP
  \fB\-\-track\fP
  Track and fetch the latest source before resetting
  .TP

man/bst-workspace.1

 -.TH "BST WORKSPACE" "1" "26-Apr-2018" "" "bst workspace Manual"
 +.TH "BST WORKSPACE" "1" "29-Nov-2018" "" "bst workspace Manual"
  .SH NAME
  bst\-workspace \- Manipulate developer workspaces
  .SH SYNOPSIS
@@ -11,18 +11,15 @@ Manipulate developer workspaces
  \fBopen\fP
    Open a new workspace
    See \fBbst workspace-open(1)\fP for full documentation on the \fBopen\fP command.
+-
 -.PP
 -\fBlist\fP
 -  List open workspaces
 -  See \fBbst workspace-list(1)\fP for full documentation on the \fBlist\fP command.
+-
  .PP
  \fBclose\fP
    Close workspaces
    See \fBbst workspace-close(1)\fP for full documentation on the \fBclose\fP command.
+-
  .PP
  \fBreset\fP
    Reset a workspace to its original state
    See \fBbst workspace-reset(1)\fP for full documentation on the \fBreset\fP command.
 +.PP
 +\fBlist\fP
 +  List open workspaces
 +  See \fBbst workspace-list(1)\fP for full documentation on the \fBlist\fP command.
 \ No newline at end of file

man/bst.1

 -.TH "BST" "1" "26-Apr-2018" "" "bst Manual"
 +.TH "BST" "1" "29-Nov-2018" "" "bst Manual"
  .SH NAME
  bst \- Build and manipulate BuildStream projects...
  .SH SYNOPSIS
@@ -6,15 +6,15 @@ bst \- Build and manipulate BuildStream projects...
  [OPTIONS] COMMAND [ARGS]...
  .SH DESCRIPTION
  Build and manipulate BuildStream projects
+-
 +.PP
  Most of the main options override options in the
  user preferences configuration file.
  .SH OPTIONS
  .TP
  \fB\-\-version\fP
+-
 +.PP
  .TP
 -\fB\-c,\fP \-\-config PATH
 +\fB\-c,\fP \-\-config FILE
  Configuration file to use
  .TP
  \fB\-C,\fP \-\-directory DIRECTORY
@@ -61,58 +61,62 @@ Elements must be rebuilt when their dependencies have changed
  .TP
  \fB\-o,\fP \-\-option OPTION VALUE
  Specify a project option
 +.TP
 +\fB\-\-default\-mirror\fP TEXT
 +The mirror to fetch from first, before attempting other mirrors
 +.TP
 +\fB\-\-pull\-buildtrees\fP
 +Include an element's build tree when pulling remote element artifacts
  .SH COMMANDS
  .PP
 -\fBcheckout\fP
 -  Checkout a built artifact
 -  See \fBbst-checkout(1)\fP for full documentation on the \fBcheckout\fP command.
+-
 -.PP
 -\fBfetch\fP
 -  Fetch sources in a pipeline
 -  See \fBbst-fetch(1)\fP for full documentation on the \fBfetch\fP command.
+-
 -.PP
 -\fBsource-bundle\fP
 -  Produce a build bundle to be manually executed
 -  See \fBbst-source-bundle(1)\fP for full documentation on the \fBsource-bundle\fP command.
+-
 -.PP
 -\fBshow\fP
 -  Show elements in the pipeline
 -  See \fBbst-show(1)\fP for full documentation on the \fBshow\fP command.
+-
 -.PP
 -\fBworkspace\fP
 -  Manipulate developer workspaces
 -  See \fBbst-workspace(1)\fP for full documentation on the \fBworkspace\fP command.
+-
 -.PP
 -\fBtrack\fP
 -  Track new source references
 -  See \fBbst-track(1)\fP for full documentation on the \fBtrack\fP command.
+-
 +\fBhelp\fP
 +  Print usage information
 +  See \fBbst-help(1)\fP for full documentation on the \fBhelp\fP command.
  .PP
  \fBinit\fP
    Initialize a new BuildStream project
    See \fBbst-init(1)\fP for full documentation on the \fBinit\fP command.
+-
 -.PP
 -\fBshell\fP
 -  Shell into an element's sandbox environment
 -  See \fBbst-shell(1)\fP for full documentation on the \fBshell\fP command.
+-
  .PP
  \fBbuild\fP
    Build elements in a pipeline
    See \fBbst-build(1)\fP for full documentation on the \fBbuild\fP command.
+-
 +.PP
 +\fBfetch\fP
 +  Fetch sources in a pipeline
 +  See \fBbst-fetch(1)\fP for full documentation on the \fBfetch\fP command.
 +.PP
 +\fBtrack\fP
 +  Track new source references
 +  See \fBbst-track(1)\fP for full documentation on the \fBtrack\fP command.
  .PP
  \fBpull\fP
    Pull a built artifact
    See \fBbst-pull(1)\fP for full documentation on the \fBpull\fP command.
+-
  .PP
  \fBpush\fP
    Push a built artifact
    See \fBbst-push(1)\fP for full documentation on the \fBpush\fP command.
 +.PP
 +\fBshow\fP
 +  Show elements in the pipeline
 +  See \fBbst-show(1)\fP for full documentation on the \fBshow\fP command.
 +.PP
 +\fBshell\fP
 +  Shell into an element's sandbox environment
 +  See \fBbst-shell(1)\fP for full documentation on the \fBshell\fP command.
 +.PP
 +\fBcheckout\fP
 +  Checkout a built artifact
 +  See \fBbst-checkout(1)\fP for full documentation on the \fBcheckout\fP command.
 +.PP
 +\fBsource-checkout\fP
 +  Checkout sources for an element
 +  See \fBbst-source-checkout(1)\fP for full documentation on the \fBsource-checkout\fP command.
 +.PP
 +\fBworkspace\fP
 +  Manipulate developer workspaces
 +  See \fBbst-workspace(1)\fP for full documentation on the \fBworkspace\fP command.
 +.PP
 +\fBsource-bundle\fP
 +  Produce a build bundle to be manually executed
 +  See \fBbst-source-bundle(1)\fP for full documentation on the \fBsource-bundle\fP command.
 \ No newline at end of file

tests/integration/pullbuildtrees.py

@@ -38,7 +38,8 @@ def test_pullbuildtrees(cli, tmpdir, datafiles, integration_cache):
      # Create artifact shares for pull & push testing
      with create_artifact_share(os.path.join(str(tmpdir), 'share1')) as share1,\
 -        create_artifact_share(os.path.join(str(tmpdir), 'share2')) as share2:
 +        create_artifact_share(os.path.join(str(tmpdir), 'share2')) as share2,\
 +        create_artifact_share(os.path.join(str(tmpdir), 'share3')) as share3:
          cli.configure({
              'artifacts': {'url': share1.repo, 'push': True},
              'artifactdir': os.path.join(str(tmpdir), 'artifacts')
@@ -123,6 +124,32 @@ def test_pullbuildtrees(cli, tmpdir, datafiles, integration_cache):
          assert share2.has_artifact('test', element_name, cli.get_element_key(project, element_name))
          default_state(cli, tmpdir, share1)
 +        # Assert that bst push will automatically attempt to pull a missing buildtree
 +        # if pull-buildtrees is set, however as share3 is the only defined remote and is empty,
 +        # assert that no element artifact buildtrees are pulled (no available remote buildtree) and thus the
 +        # artifact cannot be pushed.
 +        result = cli.run(project=project, args=['pull', element_name])
 +        assert element_name in result.get_pulled_elements()
 +        cli.configure({'artifacts': {'url': share3.repo, 'push': True}})
 +        result = cli.run(project=project, args=['--pull-buildtrees', 'push', element_name])
 +        assert "Attempting to fetch missing artifact buildtrees" in result.stderr
 +        assert element_name not in result.get_pulled_elements()
 +        assert not os.path.isdir(buildtreedir)
 +        assert element_name not in result.get_pushed_elements()
 +        assert not share3.has_artifact('test', element_name, cli.get_element_key(project, element_name))
++
 +        # Assert that if we add an extra remote that has the buildtree artfact cached, bst push will
 +        # automatically attempt to pull it and will be successful, leading to the full artifact being pushed
 +        # to the empty share3. This gives the ability to attempt push currently partial artifacts to a remote,
 +        # without exlipictly requiring a bst pull.
 +        cli.configure({'artifacts': [{'url': share1.repo, 'push': False}, {'url': share3.repo, 'push': True}]})
 +        result = cli.run(project=project, args=['--pull-buildtrees', 'push', element_name])
 +        assert "Attempting to fetch missing artifact buildtrees" in result.stderr
 +        assert element_name in result.get_pulled_elements()
 +        assert os.path.isdir(buildtreedir)
 +        assert element_name in result.get_pushed_elements()
 +        assert share3.has_artifact('test', element_name, cli.get_element_key(project, element_name))
++
  # Ensure that only valid pull-buildtrees boolean options make it through the loading
  # process.

[Notes] [Git][BuildStream/buildstream][tpollard/774] 3 commits: man/: update with changes since Apr 2018

Jürg Billeter pushed to branch tpollard/774 at BuildStream / buildstream

Commits:

23 changed files:

Changes: