[Notes] [Git][BuildStream/buildstream][aevri/doc_arch_deps] 11 commits: .gitlab-ci.yml: Enable parallel testing with 2 CPUs



Title: GitLab

Angelos Evripiotis pushed to branch aevri/doc_arch_deps at BuildStream / buildstream

Commits:

5 changed files:

Changes:

  • .gitlab-ci.yml
    ... ... @@ -12,7 +12,7 @@ stages:
    12 12
     variables:
    
    13 13
       PYTEST_ADDOPTS: "--color=yes"
    
    14 14
       INTEGRATION_CACHE: "${CI_PROJECT_DIR}/cache/integration-cache"
    
    15
    -  TEST_COMMAND: "tox -- --color=yes --integration"
    
    15
    +  TEST_COMMAND: "tox -- --color=yes --integration -n 2"
    
    16 16
       COVERAGE_PREFIX: "${CI_JOB_NAME}."
    
    17 17
     
    
    18 18
     
    

  • doc/source/arch_cachekeys.rst
    ... ... @@ -10,14 +10,14 @@ for the purpose of reusing artifacts in a well-defined, predictable way.
    10 10
     
    
    11 11
     Structure
    
    12 12
     ---------
    
    13
    -Cache keys are SHA256 hash values generated from a pickled Python dict that
    
    13
    +Cache keys are SHA256 hash values generated from a UTF-8 JSON document that
    
    14 14
     includes:
    
    15 15
     
    
    16
    -* Environment (e.g., project configuration and variables)
    
    17
    -* Element configuration (details depend on element kind, ``Element.get_unique_key()``)
    
    18
    -* Sources (``Source.get_unique_key()``)
    
    19
    -* Dependencies (depending on cache key type, see below)
    
    20
    -* Public data
    
    16
    +* Environment (e.g., project configuration and variables).
    
    17
    +* Element configuration (details depend on element kind, ``Element.get_unique_key()``).
    
    18
    +* Sources (``Source.get_unique_key()``).
    
    19
    +* Dependencies (depending on cache key type, see below).
    
    20
    +* Public data.
    
    21 21
     
    
    22 22
     Cache key types
    
    23 23
     ---------------
    
    ... ... @@ -42,6 +42,9 @@ or the environment changes but it will not change when a dependency is updated.
    42 42
     For elements without build dependencies the ``strong`` cache key is identical
    
    43 43
     to the ``weak`` cache key.
    
    44 44
     
    
    45
    +Note that dependencies which are not required at build time do not affect
    
    46
    +either kind of key.
    
    47
    +
    
    45 48
     Strict build plan
    
    46 49
     -----------------
    
    47 50
     This is the default build plan that exclusively uses ``strong`` cache keys
    

  • doc/source/arch_dependency_model.rst
    ... ... @@ -42,7 +42,8 @@ Scope
    42 42
     * **Scope.RUN**
    
    43 43
     
    
    44 44
       In the :func:`Scope.RUN <buildstream.types.Scope.RUN>` scope, only elements
    
    45
    -  which are required to run are considered, including the element itself.
    
    45
    +  which are required to run are considered, including the element itself. Note
    
    46
    +  that these are transitive - the service also requires the base runtime.
    
    46 47
     
    
    47 48
       This is used when for example, launching a ``bst shell`` environment
    
    48 49
       for the purpose of running, or in any case we need to consider which
    
    ... ... @@ -60,7 +61,7 @@ Scope
    60 61
       .. image:: images/arch-dependency-model-build.svg
    
    61 62
          :align: center
    
    62 63
     
    
    63
    -  Note that build type dependencies are not transient, which is why the
    
    64
    +  Note that build type dependencies are not transitive, which is why the
    
    64 65
       *Bootstrap* element is not selected when pulling in the *Compiler* to
    
    65 66
       build the *Application*.
    
    66 67
     
    

  • doc/source/arch_sandboxing.rst
    ... ... @@ -51,7 +51,7 @@ well.
    51 51
     Filesystem access
    
    52 52
     ~~~~~~~~~~~~~~~~~
    
    53 53
     
    
    54
    -The filesystem inside sandboxes should be read only during element assembly,
    
    54
    +The filesystem inside sandboxes should be read-only during element assembly,
    
    55 55
     except for certain directories which element plugins can mark as being
    
    56 56
     read/write. Most elements plugins derive from :mod:`BuildElement
    
    57 57
     <buildstream.buildelement>`, which marks ``%{build-root}`` and
    
    ... ... @@ -158,17 +158,17 @@ and will refuse to push any artifacts built on such a system to a remote cache.
    158 158
     For more information, see `issue #92
    
    159 159
     <https://gitlab.com/BuildStream/buildstream/issues/92>`_.
    
    160 160
     
    
    161
    -The Linux platform can operate as a standard user provided user namespace
    
    161
    +The Linux platform can operate as a standard user, if user namespace
    
    162 162
     support is available. If user namespace support is not available you have the
    
    163 163
     option of installing bubblewrap as a setuid binary to avoid needing to run the
    
    164 164
     entire ``bst`` process as the ``root`` user.
    
    165 165
     
    
    166
    -The artifact cache on Linux systems is implemented using `OSTree
    
    167
    -<https://github.com/ostreedev/ostree>`_, which can allow us to stage artifacts
    
    168
    -using hardlinks instead of copying them. To avoid cache corruption it is
    
    169
    -vital that hardlinked files cannot be overwritten. In cases where the root
    
    170
    -filesystem inside the sandbox needs to be writable, a custom FUSE filesystem
    
    171
    -named SafeHardlinks is used which provides a copy-on-write layer.
    
    166
    +The artifact cache on Linux systems is implemented using a content-addressable
    
    167
    +hardlink farm, which can allow us to stage artifacts using hardlinks instead of
    
    168
    +copying them. To avoid cache corruption it is vital that hardlinked files
    
    169
    +cannot be overwritten. In cases where the root filesystem inside the sandbox
    
    170
    +needs to be writable, a custom FUSE filesystem named SafeHardlinks is used
    
    171
    +which provides a copy-on-write layer.
    
    172 172
     
    
    173 173
     Some of the operations on filesystem metadata listed above are not prohibited
    
    174 174
     by the sandbox, but will instead be silently dropped when an artifact is
    

  • doc/source/arch_scheduler.rst
    ... ... @@ -19,9 +19,9 @@ The Job class has the following responsibilities:
    19 19
     * Offering an abstract method for subclasses to handle the outcome of
    
    20 20
       a job when it completes.
    
    21 21
     
    
    22
    -* Forcefully terminating it's subprocess.
    
    22
    +* Forcefully terminating its subprocess.
    
    23 23
     
    
    24
    -* Suspending and resuming it's subprocess.
    
    24
    +* Suspending and resuming its subprocess.
    
    25 25
     
    
    26 26
     * Declaring the types of resources it will require, and which resources
    
    27 27
       it will require exclusively.
    
    ... ... @@ -73,7 +73,7 @@ The Queue implementations are:
    73 73
     
    
    74 74
       The pull queue tries to obtain a built artifact from a remote artifact server,
    
    75 75
       it should be placed in advance of the fetch queue if used, since we can safely
    
    76
    -  avoid fetching if fetching is not imerative, and we already have a cached
    
    76
    +  avoid fetching if fetching is not imperative, and we already have a cached
    
    77 77
       artifact.
    
    78 78
     
    
    79 79
     * **Fetch**
    
    ... ... @@ -84,7 +84,7 @@ The Queue implementations are:
    84 84
     
    
    85 85
     * **Build**
    
    86 86
     
    
    87
    -  The build queue attempts to build the element if it's artifact is not locally
    
    87
    +  The build queue attempts to build the element if its artifact is not locally
    
    88 88
       present.
    
    89 89
     
    
    90 90
     * **Push**
    



  • [Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]