GNOME.org
 

[Notes] [Git][BuildStream/buildstream][aevri/doc_arch_deps] 11 commits: .gitlab-ci.yml: Enable parallel testing with 2 CPUs

Title: GitLab

Angelos Evripiotis pushed to branch aevri/doc_arch_deps at BuildStream / buildstream

Commits:

5 changed files:

Changes:

  • .gitlab-ci.yml
    ... ... @@ -12,7 +12,7 @@ stages:
    12 12 
     variables:
    13 13 
       PYTEST_ADDOPTS: "--color=yes"
    14 14 
       INTEGRATION_CACHE: "${CI_PROJECT_DIR}/cache/integration-cache"
    15 
    -  TEST_COMMAND: "tox -- --color=yes --integration"
    15 
    +  TEST_COMMAND: "tox -- --color=yes --integration -n 2"
    16 16 
       COVERAGE_PREFIX: "${CI_JOB_NAME}."
    17 17
    18 18

  • doc/source/arch_cachekeys.rst
    ... ... @@ -10,14 +10,14 @@ for the purpose of reusing artifacts in a well-defined, predictable way.
    10 10
    11 11 
     Structure
    12 12 
     ---------
    13 
    -Cache keys are SHA256 hash values generated from a pickled Python dict that
    13 
    +Cache keys are SHA256 hash values generated from a UTF-8 JSON document that
    14 14 
     includes:
    15 15
    16 
    -* Environment (e.g., project configuration and variables)
    17 
    -* Element configuration (details depend on element kind, ``Element.get_unique_key()``)
    18 
    -* Sources (``Source.get_unique_key()``)
    19 
    -* Dependencies (depending on cache key type, see below)
    20 
    -* Public data
    16 
    +* Environment (e.g., project configuration and variables).
    17 
    +* Element configuration (details depend on element kind, ``Element.get_unique_key()``).
    18 
    +* Sources (``Source.get_unique_key()``).
    19 
    +* Dependencies (depending on cache key type, see below).
    20 
    +* Public data.
    21 21
    22 22 
     Cache key types
    23 23 
     ---------------
    ... ... @@ -42,6 +42,9 @@ or the environment changes but it will not change when a dependency is updated.
    42 42 
     For elements without build dependencies the ``strong`` cache key is identical
    43 43 
     to the ``weak`` cache key.
    44 44
    45 
    +Note that dependencies which are not required at build time do not affect
    46 
    +either kind of key.
    47 
    +
    45 48 
     Strict build plan
    46 49 
     -----------------
    47 50 
     This is the default build plan that exclusively uses ``strong`` cache keys

  • doc/source/arch_dependency_model.rst
    ... ... @@ -42,7 +42,8 @@ Scope
    42 42 
     * **Scope.RUN**
    43 43
    44 44 
       In the :func:`Scope.RUN <buildstream.types.Scope.RUN>` scope, only elements
    45 
    -  which are required to run are considered, including the element itself.
    45 
    +  which are required to run are considered, including the element itself. Note
    46 
    +  that these are transitive - the service also requires the base runtime.
    46 47
    47 48 
       This is used when for example, launching a ``bst shell`` environment
    48 49 
       for the purpose of running, or in any case we need to consider which
    ... ... @@ -60,7 +61,7 @@ Scope
    60 61 
       .. image:: images/arch-dependency-model-build.svg
    61 62 
          :align: center
    62 63
    63 
    -  Note that build type dependencies are not transient, which is why the
    64 
    +  Note that build type dependencies are not transitive, which is why the
    64 65 
       *Bootstrap* element is not selected when pulling in the *Compiler* to
    65 66 
       build the *Application*.
    66 67

  • doc/source/arch_sandboxing.rst
    ... ... @@ -51,7 +51,7 @@ well.
    51 51 
     Filesystem access
    52 52 
     ~~~~~~~~~~~~~~~~~
    53 53
    54 
    -The filesystem inside sandboxes should be read only during element assembly,
    54 
    +The filesystem inside sandboxes should be read-only during element assembly,
    55 55 
     except for certain directories which element plugins can mark as being
    56 56 
     read/write. Most elements plugins derive from :mod:`BuildElement
    57 57 
     <buildstream.buildelement>`, which marks ``%{build-root}`` and
    ... ... @@ -158,17 +158,17 @@ and will refuse to push any artifacts built on such a system to a remote cache.
    158 158 
     For more information, see `issue #92
    159 159 
     <https://gitlab.com/BuildStream/buildstream/issues/92>`_.
    160 160
    161 
    -The Linux platform can operate as a standard user provided user namespace
    161 
    +The Linux platform can operate as a standard user, if user namespace
    162 162 
     support is available. If user namespace support is not available you have the
    163 163 
     option of installing bubblewrap as a setuid binary to avoid needing to run the
    164 164 
     entire ``bst`` process as the ``root`` user.
    165 165
    166 
    -The artifact cache on Linux systems is implemented using `OSTree
    167 
    -<https://github.com/ostreedev/ostree>`_, which can allow us to stage artifacts
    168 
    -using hardlinks instead of copying them. To avoid cache corruption it is
    169 
    -vital that hardlinked files cannot be overwritten. In cases where the root
    170 
    -filesystem inside the sandbox needs to be writable, a custom FUSE filesystem
    171 
    -named SafeHardlinks is used which provides a copy-on-write layer.
    166 
    +The artifact cache on Linux systems is implemented using a content-addressable
    167 
    +hardlink farm, which can allow us to stage artifacts using hardlinks instead of
    168 
    +copying them. To avoid cache corruption it is vital that hardlinked files
    169 
    +cannot be overwritten. In cases where the root filesystem inside the sandbox
    170 
    +needs to be writable, a custom FUSE filesystem named SafeHardlinks is used
    171 
    +which provides a copy-on-write layer.
    172 172
    173 173 
     Some of the operations on filesystem metadata listed above are not prohibited
    174 174 
     by the sandbox, but will instead be silently dropped when an artifact is

  • doc/source/arch_scheduler.rst
    ... ... @@ -19,9 +19,9 @@ The Job class has the following responsibilities:
    19 19 
     * Offering an abstract method for subclasses to handle the outcome of
    20 20 
       a job when it completes.
    21 21
    22 
    -* Forcefully terminating it's subprocess.
    22 
    +* Forcefully terminating its subprocess.
    23 23
    24 
    -* Suspending and resuming it's subprocess.
    24 
    +* Suspending and resuming its subprocess.
    25 25
    26 26 
     * Declaring the types of resources it will require, and which resources
    27 27 
       it will require exclusively.
    ... ... @@ -73,7 +73,7 @@ The Queue implementations are:
    73 73
    74 74 
       The pull queue tries to obtain a built artifact from a remote artifact server,
    75 75 
       it should be placed in advance of the fetch queue if used, since we can safely
    76 
    -  avoid fetching if fetching is not imerative, and we already have a cached
    76 
    +  avoid fetching if fetching is not imperative, and we already have a cached
    77 77 
       artifact.
    78 78
    79 79 
     * **Fetch**
    ... ... @@ -84,7 +84,7 @@ The Queue implementations are:
    84 84
    85 85 
     * **Build**
    86 86
    87 
    -  The build queue attempts to build the element if it's artifact is not locally
    87 
    +  The build queue attempts to build the element if its artifact is not locally
    88 88 
       present.
    89 89
    90 90 
     * **Push**

    • —
    View it on GitLab.
    You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.

    [Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]