[Notes] [Git][BuildStream/bst-external][1-coverage-report-not-combined]

Jonathan Maw pushed to branch 1-coverage-report-not-combined at BuildStream / bst-external

Commits:

af59f10f

by Sam Thursfield at 2017-11-06T12:05:17Z

Remove pre- and post- commands

Part of https://gitlab.com/BuildStream/buildstream/issues/121

b2ccbf2d

by Tristan Van Berkom at 2017-11-06T12:32:15Z

Merge branch 'sam/remove-pre-post' into 'master'

Remove pre- and post- commands

See merge request BuildStream/bst-external!2

1f7eb29a

by Sam Thursfield at 2017-11-06T12:50:56Z

integration-tests: Fix `run-tests.sh --sources DIR` when DIR doesn't exist

We need to canonicalize the path using the `realpath -m` or
`realpath --canonicalize-missing`, otherwise if the directory doesn't
exist we get an error:

    realpath: /tmp/foo/bar/: No such file or directory

The tests still work if this happens but the sources go into the
default location because the BST_SOURCE_CACHE variable goes unset.

cf24924c

by Sam Thursfield at 2017-11-09T16:11:41Z

x86image plugin: Allow setting kernel args with a variable

It's very likely that users will want to customize the kernel
commandline so let's make that simple.

58b09485

by Sam Thursfield at 2017-11-09T16:21:27Z

x86image plugin: Fix totally broken code

This was my fault, although I was definitely assisted by the fact
that `pip3 install --editable --user .` somehow fails to work for
BuildStream plugin repos.

bb27c66f

by Sam Thursfield at 2017-11-09T16:44:35Z

Merge branch 'sam/fix-x86image' into 'master'

x86image plugin: Fix totally broken code

See merge request BuildStream/bst-external!4

90edaa55

by Sam Thursfield at 2017-11-10T11:29:29Z

Merge branch 'sam/x86image-kernel-args' into 'master'

x86image plugin: Allow setting kernel args with a variable

See merge request BuildStream/bst-external!5

8f159c80

by Sam Thursfield at 2017-11-14T11:14:05Z

Move element plugins under a bst_external package namespace

If every BuildStream plugins repo tries to install a top-level package
named 'elements' they will all conflict with each other. There is also
already a package on PyPI with that name:
https://pypi.python.org/pypi/Elements

Better to give the top-level package a (hopefully) unique name and have
an 'elements' subpackage.

0da38365

by Tristan Van Berkom at 2017-11-15T11:44:48Z

Merge branch 'sam/namespace' into 'master'

Move element plugins under a bst_external package namespace

See merge request BuildStream/bst-external!7

4ae5d018
by Sam Thursfield at 2017-11-24T18:13:20Z
```
dpkg-build-test: Fix name in project.conf
```

8f60b513

by Sam Thursfield at 2017-11-27T11:13:04Z

dpkg_deploy element: Fix for API break in BuildStream core

Fixes the following traceback with BuildStream versions from
commit aa79d317ed373bc563deb789d8d62d49a0766f50:

    [00:00:23][f68506e0][build:dpkg-deploy-test.bst          ] BUG     dpkg-build-test/dpkg-deploy-test/f68506e0-build.8340.log

        An unhandled exception occured:

        Traceback (most recent call last):
          File "/usr/lib/python3.6/site-packages/buildstream/_scheduler/job.py", line 265, in child_action
            result = self.action(element)
          File "/usr/lib/python3.6/site-packages/buildstream/_scheduler/buildqueue.py", line 34, in process
            element._assemble()
          File "/usr/lib/python3.6/site-packages/buildstream/element.py", line 1092, in _assemble
            self.stage(sandbox)
          File "/usr/lib/python3.6/site-packages/bst_external/elements/dpkg_deploy.py", line 191, in stage
            utils.link_files(src, dst, package_splits)
        TypeError: link_files() takes 2 positional arguments but 3 were given

6c694e73

by Jürg Billeter at 2017-11-27T11:42:22Z

Merge branch 'sam/fix' into 'master'

dpkg_deploy element: Fix for API break in BuildStream core

See merge request BuildStream/bst-external!10

9f9dd2e3

by Sam Thursfield at 2017-12-11T15:56:01Z

Fix documentation generation

This was working by coincidence rather than as intended. The Makefile
step to generate .rst files for each plugin was not working. However,
because in this module the plugins are in the toplevel package, the
call to sphinx-apidoc was generating .rst files for them instead.

The problem with that approach is that the plugin docs got a bunch
of useless function references at the bottom, where we should only
be including the docstring at the top.

To fix this, the sphinx-apidoc call is removed (this module doesn't
have any public API other than the plugins themselves) and the
Makefile step to generate .rst files is copied more closely from the
one in buildstream.git.

d1a3c489

by Jonathan Maw at 2017-12-11T16:19:24Z

Merge branch 'sam/doc-fixes' into 'master'

Fix documentation generation

See merge request BuildStream/bst-external!12

bc6db43e

by Sam Thursfield at 2017-12-11T17:59:26Z

.gitlab-ci.yml: Use fixed version of Docker image

For some reason `make` has vanished from newer images.

1811ebdd

by Sam Thursfield at 2018-01-05T12:42:56Z

doc: Explain what this repo provides and how to use the plugins

ef7b3d04

by Tristan Van Berkom at 2018-01-05T12:59:42Z

Merge branch 'sam/better-docs' into 'master'

doc: Explain what this repo provides and how to use the plugins

See merge request BuildStream/bst-external!14

ba2ef303

by Jonathan Maw at 2018-01-31T16:09:51Z

Fix incorrect formatting of public data documentation

4cf6b6b3

by Jonathan Maw at 2018-01-31T16:25:01Z

dpkg_deploy: Fix obsolete calls to Plugin.error()

7b651aee

by Jonathan Maw at 2018-01-31T16:25:11Z

dpkg_build: Fix obsolete calls to Plugin.error()

ccac75e8

by Jonathan Maw at 2018-01-31T16:25:23Z

dpkg_deploy: Fix builds failing if split-rules weren't all file paths

i.e. split-rules can be globs, but we were assuming they were just file
paths, as that was what dpkg_build exports.

db543278

by Tristan Van Berkom at 2018-02-01T11:35:12Z

Merge branch 'jonathan/dpkg-fixes' into 'master'

Fix things that caused problems while creating a deb from a generic artifact

See merge request BuildStream/bst-external!15

5dfe3073
by Tristan Maat at 2018-02-08T12:50:12Z
```
Add LICENSE
```
629753ba
by Tristan Maat at 2018-02-09T11:55:57Z
```
x86image.yaml: Don't require GNU tools
```

69d6759d

by Sam Thursfield at 2018-02-09T16:38:39Z

Merge branch 'busybox-friendly' into 'master'

x86image.yaml: Don't require GNU tools

See merge request BuildStream/bst-external!13

e9e1b585

by Sam Thursfield at 2018-02-12T09:29:55Z

Add Docker source plugin

This is for pulling binaries from Docker registries such as
<https://hub.docker.com>.

It adds an external dependency on the 'requests' library.

The source is working fine against Docker Hub itself,
and comes with an integration test.

A Docker Registry is basically a content-addressed blob store,
and we take advantage of this to verify cached data. If the
manifest file is saved but its content does not match its digest,
we raise an error like this:

    Manifest /home/sam/cache/buildstream/sources/docker/sha256:4b8ffaaa896d40622ac10dc6662204f429f1c8c5714be62a6493a7895f664098.manifest.json
    is corrupt; Got content hash of sha256:22e88c7d6da9b73fbb515ed6a8f6d133c680527a799e3069ca7ce346d90649b2

Similar checks are done for the blobs (image layers).

Only the "Image Manifest V 2, Schema 2" format[1] is supported.
This seems to be unsupported by <https://quay.io>, so the plugin
doesn't work with that registry at present.

I haven't tested with any other Docker Registry instances.

1. https://docs.docker.com/registry/spec/manifest-v2-2/

10426336
by Tristan Maat at 2018-02-12T09:30:11Z
```
.gitlab-ci.yml: Add requests dependency
```

a3507fc2

by Sam Thursfield at 2018-02-12T09:34:50Z

.gitlab-ci.yml: Update to latest BuildStream

The utils.save_file_atomic() helper function that we need is now
available in 'master'.

This requires updating the tests to use the new method of plugin
discovery. This was an API-incompatible change that happened in
BuildStream prior to the stable 1.0 release.

c4cc1046

by Jürg Billeter at 2018-02-12T14:13:32Z

Merge branch 'sam/docker-source' into 'master'

Add Docker source plugin

See merge request BuildStream/bst-external!9

c7def7cf

by Sam Thursfield at 2018-02-14T15:50:55Z

Add flatpak-image plugin

Taken from the Freedesktop SDK project:

  https://gitlab.com/freedesktop-sdk/freedesktop-sdk

598dccda

by Sam Thursfield at 2018-02-14T15:50:55Z

Rename flatpak-image to flatpak_image

I prefer the hyphenated form, but it's better to be consistent with the
existing plugins in this repo like dpkg_build and dpkg_deploy.

b28f0715

by Tristan Van Berkom at 2018-02-20T13:32:49Z

Merge branch 'sam/flatpak-image-plugin' into 'master'

Add flatpak_image plugin

See merge request BuildStream/bst-external!18

bb3720b5
by Phillip Smyth at 2018-02-20T14:01:11Z
```
Removed Quilt from DPKG source
```

d35fb5a5

by Phillip Smyth at 2018-02-22T11:36:34Z

Added Quilt source plugin and its test

Wrote the source plugin for Quilt
Created a test element and project for the integration test
added a shell script to test Quilt

aa8304e3

by Jonathan Maw at 2018-02-22T11:46:56Z

Merge branch 'quilt' into 'master'

Quilt Source Plugin

See merge request BuildStream/bst-external!19

e6bc4936

by Javier Jardón at 2018-02-27T13:11:08Z

README.rst: current recommended way to use this repo is as a submodule

1a1f602e

by Tristan Van Berkom at 2018-02-27T13:36:58Z

Merge branch 'jjardon/recommend_submodules' into 'master'

README.rst: current recommended way to use this repo is as a submodule

See merge request BuildStream/bst-external!20

446c1d63

by Sam Thursfield at 2018-02-27T16:16:08Z

x86image plugin: Avoid moving the whole filesystem around needlessly

On the default Linux platform, BuildStream creates the staging area using
hardlinks that point back into the cache. If we `mv` the entire contents of
the staging area into /buildstream then we end up duplicating all of that
data rather than just moving the hardlinks, so it is very slow.

This patch changes the plugin to build an ext4 filesystem directly from the
%{build-root} directory which ensures we copy the data directly into the
disk image.

This reduces a build of a 10GB VM image from 50 minutes to 25 minutes for me.

675f8eff

by Jonathan Maw at 2018-02-28T15:39:43Z

Merge branch 'sam/x86image-faster' into 'master'

x86image plugin: Avoid moving the whole filesystem around needlessly

See merge request BuildStream/bst-external!16

375fcbfa

by James Ennis at 2018-03-15T15:30:00Z

set up bst-external's setup.py to run pytest

ec3393b7
by James Ennis at 2018-03-15T15:30:00Z
```
Created a conftest.py file
```

293fb221

by James Ennis at 2018-03-15T15:30:00Z

Added tests/ directory which mimics the one in the main BuildStream repo

a8cbe129
by James Ennis at 2018-03-15T15:30:00Z
```
project.conf: Remove dead code
```
d3ceda06
by James Ennis at 2018-03-16T12:20:37Z
```
Create setup.cfg file
```

50131ef4