GNOME.org
 

[Notes] [Git][BuildGrid/buildgrid][mablanch/51-user-facing-docs] 9 commits: Removing redundant tests

Title: GitLab

Martin Blanchard pushed to branch mablanch/51-user-facing-docs at BuildGrid / buildgrid

Commits:

20 changed files:

Changes:

  • .pylintrc
    ... ... @@ -7,7 +7,7 @@ extension-pkg-whitelist=
    7 7
    8 8 
     # Add files or directories to the blacklist. They should be base names, not
    9 9 
     # paths.
    10 
    -#ignore=CVS,tests,doc
    10 
    +#ignore=tests,docs
    11 11
    12 12 
     # Add files or directories matching the regex patterns to the blacklist. The
    13 13 
     # regex matches against base names, not paths.

  • BuildGrid.doap
    1 
    +<?xml version="1.0" encoding="UTF-8"?>
    2 
    +<Project xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    3 
    +         xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
    4 
    +         xmlns:foaf="http://xmlns.com/foaf/0.1/"
    5 
    +         xmlns="http://usefulinc.com/ns/doap#">
    6 
    +
    7 
    +  <name xml:lang="en">BuildGrid</name>
    8 
    +  <shortdesc xml:lang="en">Remote execution service implementing Google's REAPI &amp; RWAPI</shortdesc>
    9 
    +  <homepage rdf:resource="https://buildgrid.build" />
    10 
    +  <mailing-list rdf:resource="https://lists.buildgrid.build/cgi-bin/mailman/listinfo/buildgrid" />
    11 
    +  <download-page rdf:resource="https://gitlab.com/BuildGrid/buildgrid" />
    12 
    +  <bug-database rdf:resource="https://gitlab.com/BuildGrid/buildgrid/issues" />
    13 
    +
    14 
    +  <programming-language>python3</programming-language>
    15 
    +
    16 
    +  <maintainer>
    17 
    +    <foaf:Person>
    18 
    +      <foaf:name>Finn Ball</foaf:name>
    19 
    +      <foaf:mbox rdf:resource="mailto:finn ball codethink co uk" />
    20 
    +    </foaf:Person>
    21 
    +  </maintainer>
    22 
    +</Project>
    \ No newline at end of file

  • CONTRIBUTING.rst
    1 
    -Contributing to BuildGrid
    2 
    -=========================
    3 
    -Some guidelines for people wanting to contribute. Also please always feel free to speak to us, we're very friendly :-)
    1 
    +Contributing
    2 
    +============
    4 3
    5 
    -Feature Additions
    4 
    +Some guidelines for people wanting to contribute. Also please always feel free
    5 
    +to speak to us, we're very friendly :-)
    6 
    +
    7 
    +
    8 
    +.. _feature-additions:
    9 
    +
    10 
    +Feature additions
    6 11 
     -----------------
    7 12
    8 
    -We welcome contributions in the form of bug fixes or feature additions / enhancements. Please discuss with us before submitting anything, as we may well have some important context which will could help guide your efforts.
    13 
    +We welcome contributions in the form of bug fixes or feature additions. Please
    14 
    +discuss with us before submitting anything, as we may well have some important
    15 
    +context which will could help guide your efforts.
    9 16
    10 
    -Any major feature additions should be raised as a proposal on the `Mailing List <https://lists.buildgrid.build/cgi-bin/mailman/listinfo/buildgrid/>`_ to be discussed, and then eventually followed up with an issue here on gitlab. We recommend that you propose the feature in advance of commencing work. We are also on irc - you can find us on #buildgrid on freenode.
    17 
    +Any major feature additions should be raised first as a proposal on the
    18 
    +`BuildGrid mailing list`_ to be discussed, and then eventually followed up with
    19 
    +an issue on GitLab. We recommend that you propose the feature in advance of
    20 
    +commencing work.
    11 21
    12 
    -The author of any patch is expected to take ownership of that code and is to support it for a reasonable time-frame. This means addressing any unforeseen side effects and quirks the feature may have introduced. More on this below in 'Granting Committer Access'.
    22 
    +The author of any patch is expected to take ownership of that code and is to
    23 
    +support it for a reasonable time-frame. This means addressing any unforeseen
    24 
    +side effects and quirks the feature may have introduced. More on this below in
    25 
    +:ref:`Committer access <committer-access>`.
    13 26
    14 
    -Patch Submissions
    15 
    ------------------
    27 
    +.. _BuildGrid mailing list: https://lists.buildgrid.build/cgi-bin/mailman/listinfo/buildgrid
    16 28
    17 
    -We will be running `trunk based development <https://trunkbaseddevelopment.com>`_. The idea behind this is that merge requests to the trunk will be small and made often, thus making the review and merge process as fast as possible. We do not want to end up with a huge backlog of outstanding merge requests. If possible,
    18 
    -it is preferred that merge requests address specific points and clearly outline what problem they are solving.
    19 29
    20 
    -Branches must be submitted as merge requests on gitlab and should be associated with an issue report on gitlab, whenever possible. If it's a small change, we'll accept an MR without it being associated to a gitlab issue, but generally we prefer an issue to be raised in advance. This is so that we can track the work that is currently in progress on the project - please see our Gitlab policy below.
    30 
    +.. _patch-submissions:
    31 
    +
    32 
    +Patch submissions
    33 
    +-----------------
    34 
    +
    35 
    +We are running `trunk based development`_. The idea behind this is that merge
    36 
    +requests to the trunk will be small and made often, thus making the review and
    37 
    +merge process as fast as possible. We do not want to end up with a huge backlog
    38 
    +of outstanding merge requests. If possible, it is preferred that merge requests
    39 
    +address specific points and clearly outline what problem they are solving.
    21 40
    22 
    -Each commit should address a specific gitlab issue number in the commit message. This is really important for provenance reasons.
    41 
    +Branches must be submitted as merge requests (MR) on GitLab and should be
    42 
    +associated with an issue, whenever possible. If it's a small change, we'll
    43 
    +accept an MR without it being associated to an issue, but generally we prefer an
    44 
    +issue to be raised in advance. This is so that we can track the work that is
    45 
    +currently in progress on the project.
    23 46
    24 
    -Merge requests that are not yet ready for review must be prefixed with the `WIP:` identifier, but if we stick to trunk based development then the 'WIP:' identifier will not stay around for very long on a merge request.
    47 
    +Below is a list of good patch submission good practice:
    25 48
    26 
    -When a merge request is ready for review, please find someone willing to do the review (ideally a maintainer) and assign them the MR on gitlab, leaving a comment asking for their review.
    49 
    +- Each commit should address a specific issue number in the commit message. This
    50 
    +  is really important for provenance reasons.
    51 
    +- Merge requests that are not yet ready for review must be prefixed with the
    52 
    +  ``WIP:`` identifier, but if we stick to trunk based development then the
    53 
    +  ``WIP:`` identifier will not stay around for very long on a merge request.
    54 
    +- When a merge request is ready for review, please find someone willing to do
    55 
    +  the review (ideally a maintainer) and assign them the MR, leaving a comment
    56 
    +  asking for their review.
    57 
    +- Submitted branches should not contain a history of work done.
    58 
    +- Unit tests should be a separate commit.
    27 59
    28 
    -Submitted branches should not contain a history of work done.
    60 
    +.. _trunk based development: https://trunkbaseddevelopment.com
    29 61
    30 
    -Unit tests should be a separate commit.
    31 62
    32 63 
     Commit messages
    33 64 
     ~~~~~~~~~~~~~~~
    34 
    -Commit messages must be formatted with a brief summary line, optionally followed by an empty line and then a
    35 
    -free form detailed description of the change.
    36 65
    37 
    -The summary line must start with what changed, followed by a colon and a very brief description of the
    38 
    -change.
    39 
    -
    40 
    -If there is an associated issue, it **must** be mentioned somewhere in the commit message.
    66 
    +Commit messages must be formatted with a brief summary line, optionally followed
    67 
    +by an empty line and then a free form detailed description of the change. The
    68 
    +summary line must start with what changed, followed by a colon and a very brief
    69 
    +description of the change. If there is an associated issue, it **must** be
    70 
    +mentioned somewhere in the commit message.
    41 71
    42 72 
     **Example**::
    43 73
    44 
    -  worker.py: Fixed to be more human than human
    74 
    +   worker.py: Fixed to be more human than human
    75 
    +
    76 
    +   Gifted the worker with a past so we can create
    77 
    +   a cushion or a pillow for their emotions and
    78 
    +   consequently, we can control them better.
    79 
    +
    80 
    +   This fixes issue #8.
    81 
    +
    82 
    +For more tips, please read `The seven rules of a great Git commit message`_.
    45 83
    46 
    -  Gifted the worker with a past so we can create
    47 
    -  a cushion or a pillow for their emotions and
    48 
    -  consequently, we can control them better.
    49 
    -
    50 
    -  This fixes issue #8
    84 
    +.. _The seven rules of a great Git commit message: https://chris.beams.io/posts/git-commit/#seven-rules
    51 85
    52 
    -
    53 
    -For more tips, please see `this <https://chris.beams.io/posts/git-commit/#seven-rules/>`_ article.
    86 
    +
    87 
    +.. _coding-style:
    54 88
    55 89 
     Coding style
    56 90 
     ------------
    57 
    -Coding style details for BuildGrid.
    58 91
    92 
    +Python coding style for BuildGrid is `PEP 8`_. We do have a couple of minor
    93 
    +exceptions to this standard, we dont want to compromise code readability by
    94 
    +being overly restrictive on line length for instance.
    95 
    +
    96 
    +BuildGrid's test suite includes a PEP8 style compliance check phase (using
    97 
    +`pep8`_) and a code linting phase (using `pylint`_). That test suite is
    98 
    +automatically run for every change submitted to the GitLab server and the merge
    99 
    +request sytem requires the test suite execution to succed before changes can
    100 
    +be pulled upstream. This means you have to respect the BuildGrid coding style.
    101 
    +
    102 
    +Configuration and exceptions for ``pep8`` and ``pylint`` can be found in:
    59 103
    60 
    -Style guide
    61 
    -~~~~~~~~~~~
    62 
    -Python coding style for BuildGrid is pep8, which is documented here: https://www.python.org/dev/peps/pep-0008/
    104 
    +- The `setup.cfg`_ file for ``pep8``.
    105 
    +- The `.pylintrc`_ file for ``pylint``.
    63 106
    64 
    -We have a couple of minor exceptions to this standard, we dont want to compromise
    65 
    -code readability by being overly restrictive on line length for instance.
    107 
    +.. _PEP 8: https://www.python.org/dev/peps/pep-0008
    108 
    +.. _pep8: https://pep8.readthedocs.io
    109 
    +.. _pylint: https://pylint.readthedocs.io
    110 
    +.. _setup.cfg: https://gitlab.com/BuildGrid/buildgrid/blob/master/setup.cfg
    111 
    +.. _.pylintrc: https://gitlab.com/BuildGrid/buildgrid/blob/master/.pylintrc
    66 112
    67 113
    68 114 
     Imports
    69 115 
     ~~~~~~~
    116 
    +
    70 117 
     Module imports inside BuildGrid are done with relative ``.`` notation
    71 118
    72 119 
     Good::
    ... ... @@ -77,99 +124,85 @@ Bad::
    77 124
    78 125 
       from buildgrid.worker import Worker
    79 126
    80 
    -Ordering
    81 
    -''''''''
    82 
    -For better readability and consistency, we try to keep private symbols below
    83 
    -public symbols. In the case of public modules where we may have a mix of
    84 
    -*API private* and *local private* symbols, *API private* symbols should come
    85 
    -before *local private* symbols.
    86 
    -
    87 127
    88 128 
     Symbol naming
    89 129 
     '''''''''''''
    90 
    -Any private symbol must start with a single leading underscore for two reasons:
    91 
    -
    92 
    -* So that it does not bleed into documentation and *public API*.
    93 
    -
    94 
    -* So that it is clear to developers which symbols are not used outside of the declaring *scope*
    95 
    -
    96 
    -Remember that with python, the modules (python files) are also symbols
    97 
    -within their containing *package*, as such; modules which are entirely
    98 
    -private to BuildGrid are named as such, e.g. ``_roy.py``.
    99 
    -
    100 
    -Granting Committer Access
    101 
    --------------------------
    102 130
    103 
    -We'll hand out commit access to anyone who has successfully landed a single patch to the code base. Please request this via irc or the mailing list.
    104 
    -
    105 
    -This of course relies on contributors being responsive and show willingness to address problems after landing branches there should not be any problems here.
    106 
    -
    107 
    -What we are expecting of committers here in general is basically to
    108 
    -escalate the review in cases of uncertainty:
    109 
    -
    110 
    -* If the patch/branch is very trivial (obvious few line changes or typos etc), and you are confident of the change, there is no need for review.
    111 
    -
    112 
    -* If the patch/branch is non trivial, please obtain a review from another committer who is familiar with the area which the branch effects. An approval from someone who is not the patch author will be needed before any merge.
    113 
    -
    114 
    -We don't have any detailed policy for "bad actors", but will of course handle things on a case by case basis - commit access should not result in commit wars or be used as a tool to subvert the project when disagreements arise, such incidents (if any) would surely lead to temporary suspension of commit rights.
    131 
    +Any private symbol must start with a single leading underscore for two reasons:
    115 132
    116 
    -BuildGrid policy for use of Gitlab features
    117 
    --------------------------------------------
    133 
    +- So that it does not bleed into documentation and *public API*.
    134 
    +- So that it is clear to developers which symbols are not used outside of the
    135 
    +  declaring *scope*.
    118 136
    119 
    -We intend to make use of some of gitlab's features in order to structure the activity of the BuildGrid project. In doing so we are trying to achieve the following goals:
    137 
    +Remember that with python, the modules (python files) are also symbols within
    138 
    +their containing *package*, as such; modules which are entirely private to
    139 
    +BuildGrid are named as such, e.g. ``_roy.py``.
    120 140
    121 
    -* Full transparency of the current WIP items
    122 
    -* Provide a view of all current and planned activity which is relatively easy for the viewer to digest
    123 
    -* Ensure that we keep it simple and easy to contribute to the project
    124 141
    125 
    -We propose to make use of the following Gitlab features:
    142 
    +.. _committer-access:
    126 143
    127 
    -* Milestones
    128 
    -* Labels
    129 
    -* Boards
    130 
    -* Templates
    144 
    +Committer access
    145 
    +----------------
    131 146
    132 
    -Milestones
    133 
    -~~~~~~~~~~
    134 
    -`Milestones <https://docs.gitlab.com/ee/user/project/milestones/>`_ are based on periods of time and what we want to achieve within those periods of time.
    147 
    +We'll hand out commit access to anyone who has successfully landed a single
    148 
    +patch to the code base. Please request this via irc or the mailing list.
    135 149
    136 
    -We have seen them used in the same way as `Epics <https://docs.gitlab.com/ee/user/group/epics/index.html#doc-nav/>`_ in other projects (since the Epic feature is only available with GitLab Ultimate) and this does not work. Milestones must be time-line based.
    150 
    +This of course relies on contributors being responsive and show willingness to
    151 
    +address problems after landing branches there should not be any problems here.
    137 152
    138 
    -Milestones can overlap, and we can be working towards multiple milestones at any one time. They allow us to group together all sub tasks into an overall aim.
    153 
    +What we are expecting of committers here in general is basically to escalate the
    154 
    +review in cases of uncertainty:
    139 155
    140 
    -Labels
    141 
    -~~~~~~
    142 
    -`Labels <https://docs.gitlab.com/ee/user/project/labels.html/>`_ allow us to filter tickets on gitlab in useful ways. They do complexity and effort as they grow in number and usage, though, so the general approach is to have the minimum possible.
    156 
    +- If the change is very trivial (obvious few line changes, typos…), and you are
    157 
    +  confident of the change, there is no need for review.
    158 
    +- If the change is non trivial, please obtain a review from another committer
    159 
    +  who is familiar with the area which the branch effects. An approval from
    160 
    +  someone who is not the patch author will be needed before any merge.
    143 161
    144 
    -Type Labels
    145 
    -'''''''''''
    146 
    -We have:
    162 
    +.. note::
    147 163
    148 
    -* Bug
    149 
    -* Documentation
    150 
    -* Enhancement
    151 
    -* Tests
    164 
    +   We don't have any detailed policy for "bad actors", but will of course handle
    165 
    +   things on a case by case basis - commit access should not result in commit
    166 
    +   wars or be used as a tool to subvert the project when disagreements arise.
    167 
    +   Such incidents (if any) would surely lead to temporary suspension of commit
    168 
    +   rights.
    152 169
    153 
    -This is useful for filtering different types of issues. We may expand this at some point.
    154 170
    155 
    -Priority Labels
    156 
    -'''''''''''''''
    157 
    -For now, we only have 'High Priority', which indicates an urgent task. We may add more granularity if we get more contributors.
    171 
    +.. _gitlab-features:
    158 172
    159 
    -Status
    160 
    -'''''
    161 
    -We have:
    173 
    +GitLab features
    174 
    +---------------
    162 175
    163 
    -* ToDo
    164 
    -* Doing
    176 
    +We intend to make use of some of GitLab's features in order to structure the
    177 
    +activity of the BuildGrid project. In doing so we are trying to achieve the
    178 
    +following goals:
    165 179
    166 
    -These labels are used when structuring tickets on a Board. GitLab issues start life in the 'Backlog' column by default, and we move them into 'ToDo' when they are coming up in the next few weeks. 'Doing' is only for when an item is currently being worked on. These labels don't have to be manually applied, they are applied by GitLab when moving the issue from column to column when using a Board - see below.
    180 
    +- Full transparency of the current work in progress items.
    181 
    +- Provide a view of all current and planned activity which is relatively easy
    182 
    +  for the viewer to digest.
    183 
    +- Ensure that we keep it simple and easy to contribute to the project.
    167 184
    168 
    -Issue Boards
    169 
    -~~~~~~~~~~~~
    170 
    -`Boards <https://docs.gitlab.com/ee/user/project/issue_board.html#doc-nav/>`_ allow you to visualise and manage issues in a simple way, and we can create different types of board by filtering labels. For now, we are just utilising Boards in order to be able to see all of the currently in flight items at a glance.
    185 
    +We are currenlty using the following GitLab features:
    171 186
    172 
    -Templates
    173 
    -~~~~~~~~~
    174 
    -`Issue templates <https://docs.gitlab.com/ee/user/project/description_templates.html#doc-nav/>`_ help us to receive good quality information in issues.
    187 
    +- `Milestones`_: we have seen them used in the same way as `Epics`_ in other
    188 
    +  projects. BuildGrid milestones must be time-line based, can overlap and we can
    189 
    +  be working towards multiple milestones at any one time. They allow us to group
    190 
    +  together all sub tasks into an overall aim. See our `BuildGrid milestones`_.
    191 
    +- `Labels`_: allow us to filter tickets in useful ways. They do complexity and
    192 
    +  effort as they grow in number and usage, though, so the general approach is
    193 
    +  to have the minimum possible. See our `BuildGrid labels`_.
    194 
    +- `Boards`_: allow us to visualise and manage issues and labels in a simple way.
    195 
    +  For now, we are only utilising one boards. Issues start life in the
    196 
    +  ``Backlog`` column by default, and we move them into ``ToDo`` when they are
    197 
    +  coming up in the next few weeks. ``Doing`` is only for when an item is
    198 
    +  currently being worked on. Moving an issue from column to column automatically
    199 
    +  adjust the tagged labels. See our `BuildGrid boards`_.
    175 200
    201 
    +.. _Milestones: https://docs.gitlab.com/ee/user/project/milestones
    202 
    +.. _Epics: https://docs.gitlab.com/ee/user/group/epics
    203 
    +.. _BuildGrid milestones: https://gitlab.com/BuildGrid/buildgrid/milestones
    204 
    +.. _Labels: https://docs.gitlab.com/ee/user/project/labels.html
    205 
    +.. _BuildGrid labels: https://gitlab.com/BuildGrid/buildgrid/labels
    206 
    +.. _Boards: https://docs.gitlab.com/ee/user/project/issue_board.html
    207 
    +.. _BuildGrid boards: https://gitlab.com/BuildGrid/buildgrid/boards
    208 
    +.. _Templates: https://docs.gitlab.com/ee/user/project/description_templates.html

  • README.rst
    1 
    -BuildGrid
    2 
    -=========
    1 
    +About
    2 
    +=====
    3 3
    4 4 
     .. image:: https://gitlab.com/Buildgrid/buildgrid/badges/master/pipeline.svg
    5 5 
        :target: https://gitlab.com/BuildStream/buildstream/commits/master
    ... ... @@ -7,79 +7,38 @@ BuildGrid
    7 7 
     .. image:: https://gitlab.com/BuildGrid/buildgrid/badges/master/coverage.svg?job=coverage
    8 8 
        :target: https://gitlab.com/BuildGrid/buildgrid/commits/master
    9 9
    10 
    +BuildGrid is a Python remote execution service which implements Google's
    11 
    +`Remote Execution API`_ and the `Remote Workers API`_. The project's goal is to
    12 
    +be able to execute build jobs remotely on a grid of computers in order to
    13 
    +massively speed up build times. Workers on the grid should be able to run with
    14 
    +different environments. It is designed to work with but not exclusively
    15 
    +`BuildStream`_.
    10 16
    11 
    -BuildGrid is a python remote execution service which implements the `Remote Execution API <https://github.com/bazelbuild/remote-apis//>`_ and the `Remote Workers API <https://docs.google.com/document/d/1s_AzRRD2mdyktKUj2HWBn99rMg_3tcPvdjx3MPbFidU/edit#heading=h.1u2taqr2h940/>`_.
    17 
    +.. _Remote Execution API: https://github.com/bazelbuild/remote-apis
    18 
    +.. _Remote Workers API: https://docs.google.com/document/d/1s_AzRRD2mdyktKUj2HWBn99rMg_3tcPvdjx3MPbFidU/edit#heading=h.1u2taqr2h940
    19 
    +.. _BuildStream: https://wiki.gnome.org/Projects/BuildStream
    12 20
    13 
    -The goal is to be able to execute build jobs remotely on a grid of computers to massively speed up build times. Workers on the system will also be able to run with different environments. It is designed to work with but not exclusively with `BuildStream <https://wiki.gnome.org/Projects/BuildStream/>`_.
    14 21
    15 
    -Install
    16 
    --------
    22 
    +.. _getting-started:
    17 23
    18 
    -To install::
    24 
    +Getting started
    25 
    +---------------
    19 26
    20 
    -   git clone https://gitlab.com/BuildGrid/buildgrid.git
    21 
    -   cd buildgrid
    22 
    -   pip3 install --user -e .
    27 
    +Please refer to the `documentation`_ for `installation`_ and `usage`_
    28 
    +instructions.
    23 29
    24 
    -This will install BuildGrid's python dependencies into your user’s homedir in ~/.local
    25 
    -and will run BuildGrid directly from the git checkout. It is recommended you adjust
    26 
    -your path with::
    30 
    +.. _documentation: https://buildgrid.gitlab.io/buildgrid
    31 
    +.. _installation: https://buildgrid.gitlab.io/buildgrid/installation.html
    32 
    +.. _usage: https://buildgrid.gitlab.io/buildgrid/using.html
    27 33
    28 
    -  export PATH="${PATH}:${HOME}/.local/bin"
    29 34
    30 
    -Which you can add to the end of your `~/.bashrc`.
    35 
    +.. _about-resources:
    31 36
    32 
    -Instructions for a Dummy Work
    33 
    -----------------------
    37 
    +Resources
    38 
    +---------
    34 39
    35 
    -In one terminal, start a server::
    36 
    -
    37 
    -  bgd server start
    38 
    -
    39 
    -In another terminal, send a request for work::
    40 
    -
    41 
    -  bgd execute request-dummy
    42 
    -
    43 
    -The stage should show as `QUEUED` as it awaits a bot to pick up the work::
    44 
    -
    45 
    -  bgd execute list
    46 
    -
    47 
    -Create a bot session::
    48 
    -
    49 
    -  bgd bot dummy
    50 
    -
    51 
    -Show the work as completed::
    52 
    -
    53 
    -  bgd execute list
    54 
    -
    55 
    -Instructions for a Simple Build
    56 
    --------------------------------
    57 
    -
    58 
    -This example covers a simple build. The user will upload a directory containing a C file and a command to the CAS. The bot will then fetch the uploaded directory and command which will then be run inside a temporary directory. The result will then be uploaded to the CAS and downloaded by the user. This is an early demo and still lacks a few features such as symlink support and checking to see if files exist in the CAS before executing a command.
    59 
    -
    60 
    -Create a new directory called `test-buildgrid/` and place the following C file in it called `hello.c`::
    61 
    -
    62 
    -  #include <stdio.h>
    63 
    -  int main()
    64 
    -  {
    65 
    -   printf("Hello, World!\n");
    66 
    -   return 0;
    67 
    -  }
    68 
    -
    69 
    -Now start a BuildGrid server, passing it a directory it can write a CAS to::
    70 
    -
    71 
    -  bgd server start --cas disk --cas-cache disk --cas-disk-directory /path/to/empty/directory
    72 
    -
    73 
    -Start the following bot session::
    74 
    -
    75 
    -  bgd bot temp-directory
    76 
    -
    77 
    -Upload the directory containing the C file::
    78 
    -
    79 
    -  bgd cas upload-dir /path/to/test-buildgrid
    80 
    -
    81 
    -Now we send an execution request to the bot with the name of the epxected `output-file`, a boolean describing if it is executeable, the path to the directory we uploaded in order to calculate the digest and finally the command to run on the bot::
    82 
    -
    83 
    -  bgd execute command --output-file hello True /path/to/test-buildgrid -- gcc -Wall hello.c -o hello
    84 
    -
    85 
    -The resulting executeable should have returned to a new directory called `testing/`
    40 
    +- Homepage: https://buildgrid.build
    41 
    +- GitLab repository: https://gitlab.com/BuildGrid/buildgrid
    42 
    +- Bug tracking: https://gitlab.com/BuildGrid/buildgrid/issues
    43 
    +- Mailing list: https://lists.buildgrid.build/cgi-bin/mailman/listinfo/buildgrid
    44 
    +- IRC channel: irc://chat.freenode.net/#buildgrid

  • app/cli.py
    ... ... @@ -60,9 +60,7 @@ class BuildGridCLI(click.MultiCommand):
    60 60 
             try:
    61 61 
                 mod = __import__(name='app.commands.cmd_{}'.format(name),
    62 62 
                                  fromlist=['cli'])
    63 
    -        except ModuleNotFoundError as e:
    64 
    -            raise Exception(e)
    65 
    -        except ImportError as e:
    63 
    +        except ImportError:
    66 64 
                 return None
    67 65 
             return mod.cli
    68 66

  • docs/source/about.rst
    1 
    +.. include:: ../../README.rst
    \ No newline at end of file

  • docs/source/conf.py
    ... ... @@ -24,8 +24,8 @@ from _version import __version__
    24 24 
     # -- Project information -----------------------------------------------------
    25 25
    26 26 
     project = 'BuildGrid'
    27 
    -copyright = '2018, The BuildGrid Contributors'
    28 
    -author = 'The BuildGrid Contributors'
    27 
    +copyright = '2018, Bloomberg LP'
    28 
    +author = None
    29 29
    30 30 
     # The short X.Y version
    31 31 
     version = __version__

  • docs/source/contributing.rst
    1 
    +.. include:: ../../CONTRIBUTING.rst
    \ No newline at end of file

  • docs/source/index.rst
    ... ... @@ -3,19 +3,27 @@
    3 3 
        You can adapt this file completely to your liking, but it should at least
    4 4 
        contain the root `toctree` directive.
    5 5
    6 
    +
    6 7 
     BuildGrid's documentation
    7 8 
     =========================
    8 9
    10 
    +Remote execution service implementing Google's REAPI and RWAPI.
    11 
    +
    9 12 
     .. toctree::
    10 13 
        :maxdepth: 1
    11 14 
        :caption: Contents:
    12 15
    16 
    +   about
    17 
    +   installation
    18 
    +   using
    19 
    +   contributing
    20 
    +
    13 21
    14 22 
     Resources
    15 23 
     ---------
    16 24
    17 
    -* Homepage: https://buildgrid.build
    18 
    -* GitLab repository: https://gitlab.com/BuildGrid/buildgrid
    19 
    -* Bug Tracking: https://gitlab.com/BuildGrid/buildgrid/issues
    20 
    -* Mailing list: https://lists.buildgrid.build/cgi-bin/mailman/listinfo/buildgrid
    21 
    -* IRC Channel: irc://chat.freenode.net/#buildgrid
    25 
    +- Homepage: https://buildgrid.build
    26 
    +- GitLab repository: https://gitlab.com/BuildGrid/buildgrid
    27 
    +- Bug tracking: https://gitlab.com/BuildGrid/buildgrid/issues
    28 
    +- Mailing list: https://lists.buildgrid.build/cgi-bin/mailman/listinfo/buildgrid
    29 
    +- IRC channel: irc://chat.freenode.net/#buildgrid

  • docs/source/installation.rst
    1 
    +Installation
    2 
    +============
    3 
    +
    4 
    +How to install BuildGrid onto your machine.
    5 
    +
    6 
    +.. note::
    7 
    +
    8 
    +   BuildGrid server currently only support *Linux*, *macOS* and *Windows*
    9 
    +   platforms are **not** supported.
    10 
    +
    11 
    +
    12 
    +.. _install-prerequisites:
    13 
    +
    14 
    +Prerequisites
    15 
    +-------------
    16 
    +
    17 
    +BuildGrid only supports ``python3 >= 3.5`` but has no system requirements. Main
    18 
    +Python dependencies, automatically handle during installation, includes:
    19 
    +
    20 
    +- `boto3`_: the Amazon Web Services (AWS) SDK for Python.
    21 
    +- `click`_: a Python composable command line library.
    22 
    +- `grpcio`_: Google's `gRPC`_ Python interface.
    23 
    +- `protobuf`_: Google's `protocol-buffers`_ Python interface.
    24 
    +
    25 
    +.. _boto3: https://pypi.org/project/boto3
    26 
    +.. _click: https://pypi.org/project/click
    27 
    +.. _grpcio: https://pypi.org/project/grpcio
    28 
    +.. _gRPC: https://grpc.io
    29 
    +.. _protobuf: https://pypi.org/project/protobuf
    30 
    +.. _protocol-buffers: https://developers.google.com/protocol-buffers
    31 
    +
    32 
    +
    33 
    +.. _source-install:
    34 
    +
    35 
    +Install from sources
    36 
    +--------------------
    37 
    +
    38 
    +BuildGrid has ``setuptools`` support. In order to install it to your home
    39 
    +directory, typically under ``~/.local``, simply run:
    40 
    +
    41 
    +.. code-block:: sh
    42 
    +
    43 
    +   git clone https://gitlab.com/BuildGrid/buildgrid.git && cd buildgrid
    44 
    +   pip3 install --user --editable .
    45 
    +
    46 
    +Additionally, and if your distribution does not already includes it, you may
    47 
    +have to adjust your ``PATH``, in ``~/.bashrc``, with:
    48 
    +
    49 
    +.. code-block:: sh
    50 
    +
    51 
    +   export PATH="${PATH}:${HOME}/.local/bin"
    52 
    +
    53 
    +.. note::
    54 
    +
    55 
    +   The ``setup.py`` script defines two extra targets, ``docs`` and ``tests``,
    56 
    +   that declare required dependency for, respectively, generating documentation
    57 
    +   and running unit-tests. They can be use as helpers for setting up a
    58 
    +   development environment. To use them simply run:
    59 
    +
    60 
    +   .. code-block:: sh
    61 
    +
    62 
    +      pip3 install --user --editable ".[docs,tests]"

  • docs/source/using.rst
    1 
    +Using
    2 
    +=====
    3 
    +
    4 
    +This section covers how to run an use the BuildGrid build service.
    5 
    +
    6 
    +
    7 
    +Instructions for a Dummy Work
    8 
    +-----------------------------
    9 
    +
    10 
    +In one terminal, start a server:
    11 
    +
    12 
    +.. code-block:: sh
    13 
    +
    14 
    +   bgd server start
    15 
    +
    16 
    +In another terminal, send a request for work:
    17 
    +
    18 
    +.. code-block:: sh
    19 
    +
    20 
    +   bgd execute request-dummy
    21 
    +
    22 
    +The stage should show as ``QUEUED`` as it awaits a bot to pick up the work:
    23 
    +
    24 
    +.. code-block:: sh
    25 
    +
    26 
    +   bgd execute list
    27 
    +
    28 
    +Create a bot session:
    29 
    +
    30 
    +.. code-block:: sh
    31 
    +
    32 
    +   bgd bot dummy
    33 
    +
    34 
    +Show the work as completed:
    35 
    +
    36 
    +.. code-block:: sh
    37 
    +
    38 
    +   bgd execute list
    39 
    +
    40 
    +
    41 
    +Instructions for a Simple Build
    42 
    +-------------------------------
    43 
    +
    44 
    +This example covers a simple build. The user will upload a directory containing
    45 
    +a C file and a command to the CAS. The bot will then fetch the uploaded
    46 
    +directory and command which will then be run inside a temporary directory. The
    47 
    +result will then be uploaded to the CAS and downloaded by the user. This is an
    48 
    +early demo and still lacks a few features such as symlink support and checking
    49 
    +to see if files exist in the CAS before executing a command.
    50 
    +
    51 
    +Create a new directory called `test-buildgrid/` and place the following C file
    52 
    +in it called `hello.c`:
    53 
    +
    54 
    +.. code-block:: C
    55 
    +
    56 
    +   #include <stdio.h>
    57 
    +   int main()
    58 
    +   {
    59 
    +     printf("Hello, World!\n");
    60 
    +     return 0;
    61 
    +   }
    62 
    +
    63 
    +Now start a BuildGrid server, passing it a directory it can write a CAS to:
    64 
    +
    65 
    +.. code-block:: sh
    66 
    +
    67 
    +   bgd server start --cas disk --cas-cache disk --cas-disk-directory /path/to/empty/directory
    68 
    +
    69 
    +Start the following bot session:
    70 
    +
    71 
    +.. code-block:: sh
    72 
    +
    73 
    +   bgd bot temp-directory
    74 
    +
    75 
    +Upload the directory containing the C file:
    76 
    +
    77 
    +.. code-block:: sh
    78 
    +
    79 
    +   bgd cas upload-dir /path/to/test-buildgrid
    80 
    +
    81 
    +Now we send an execution request to the bot with the name of the epxected
    82 
    +``output-file``, a boolean describing if it is executeable, the path to the
    83 
    +directory we uploaded in order to calculate the digest and finally the command
    84 
    +to run on the bot:
    85 
    +
    86 
    +.. code-block:: sh
    87 
    +
    88 
    +   bgd execute command --output-file hello True /path/to/test-buildgrid -- gcc -Wall hello.c -o hello
    89 
    +
    90 
    +The resulting executeable should have returned to a new directory called ``testing``.

  • setup.py
    ... ... @@ -89,7 +89,7 @@ tests_require = [
    89 89 
         'coverage == 4.4.0',
    90 90 
         'moto',
    91 91 
         'pep8',
    92 
    -    'pytest >= 3.1.0',
    92 
    +    'pytest == 3.6.4',
    93 93 
         'pytest-cov >= 2.5.0',
    94 94 
         'pytest-pep8',
    95 95 
         'pytest-pylint',

  • tests/action_cache.py
    ... ... @@ -15,9 +15,11 @@
    15 15 
     # Authors:
    16 16 
     #        Carter Sande <csande bloomberg net>
    17 17
    18 
    -from buildgrid._protos.build.bazel.remote.execution.v2 import remote_execution_pb2
    18 
    +# pylint: disable=redefined-outer-name
    19 
    +
    19 20 
     import pytest
    20 21
    22 
    +from buildgrid._protos.build.bazel.remote.execution.v2 import remote_execution_pb2
    21 23 
     from buildgrid.server import action_cache
    22 24 
     from buildgrid.server.cas.storage import lru_memory_cache
    23 25

  • tests/cas/test_services.py
    ... ... @@ -15,12 +15,14 @@
    15 15 
     # Authors:
    16 16 
     #        Carter Sande <csande bloomberg net>
    17 17
    18 
    +# pylint: disable=redefined-outer-name
    19 
    +
    18 20 
     import io
    19 21
    20 
    -from buildgrid._protos.google.bytestream import bytestream_pb2
    21 
    -from buildgrid._protos.build.bazel.remote.execution.v2 import remote_execution_pb2 as re_pb2
    22 22 
     import pytest
    23 23
    24 
    +from buildgrid._protos.google.bytestream import bytestream_pb2
    25 
    +from buildgrid._protos.build.bazel.remote.execution.v2 import remote_execution_pb2 as re_pb2
    24 26 
     from buildgrid.server.cas.storage.storage_abc import StorageABC
    25 27 
     from buildgrid.server.cas.bytestream_service import ByteStreamService
    26 28 
     from buildgrid.server.cas.content_addressable_storage_service import ContentAddressableStorageService
    ... ... @@ -33,10 +35,11 @@ class SimpleStorage(StorageABC):
    33 35 
         Does not attempt to delete old entries, so this is only useful for testing.
    34 36 
         """
    35 37
    36 
    -    def __init__(self, existing_data=[]):
    38 
    +    def __init__(self, existing_data=None):
    37 39 
             self.data = {}
    38 
    -        for datum in existing_data:
    39 
    -            self.data[(HASH(datum).hexdigest(), len(datum))] = datum
    40 
    +        if existing_data:
    41 
    +            for datum in existing_data:
    42 
    +                self.data[(HASH(datum).hexdigest(), len(datum))] = datum
    40 43
    41 44 
         def has_blob(self, digest):
    42 45 
             return (digest.hash, digest.size_bytes) in self.data
    ... ... @@ -59,7 +62,8 @@ class SimpleStorage(StorageABC):
    59 62
    60 63
    61 64 
     class MockObject:
    62 
    -    pass
    65 
    +    def __init__(self):
    66 
    +        self.abort = None
    63 67
    64 68
    65 69 
     class MockException(Exception):
    ... ... @@ -148,7 +152,8 @@ def test_bytestream_write_rejects_wrong_hash():
    148 152 
         context.abort = raise_mock_exception
    149 153 
         with pytest.raises(MockException):
    150 154 
             servicer.Write(requests, context)
    151 
    -    assert len(storage.data) == 0
    155 
    +
    156 
    +    assert len(storage.data) is 0
    152 157
    153 158
    154 159 
     @pytest.mark.parametrize("instance", instances)

  • tests/cas/test_storage.py
    ... ... @@ -15,14 +15,15 @@
    15 15 
     # Authors:
    16 16 
     #        Carter Sande <csande bloomberg net>
    17 17
    18 
    +# pylint: disable=redefined-outer-name
    19 
    +
    18 20 
     import tempfile
    19 21
    20 22 
     import boto3
    21 
    -from buildgrid._protos.build.bazel.remote.execution.v2.remote_execution_pb2 import Digest
    22 23 
     from moto import mock_s3
    23 24 
     import pytest
    24 25
    25 
    -from buildgrid.server.cas.storage.storage_abc import StorageABC
    26 
    +from buildgrid._protos.build.bazel.remote.execution.v2.remote_execution_pb2 import Digest
    26 27 
     from buildgrid.server.cas.storage.lru_memory_cache import LRUMemoryCache
    27 28 
     from buildgrid.server.cas.storage.disk import DiskStorage
    28 29 
     from buildgrid.server.cas.storage.s3 import S3Storage

  • tests/integration/action_cache_service.py
    ... ... @@ -15,14 +15,15 @@
    15 15 
     # Authors:
    16 16 
     #        Finn Ball <finn ball codethink co uk>
    17 17
    18 
    -import grpc
    19 
    -import pytest
    18 
    +# pylint: disable=redefined-outer-name
    20 19
    21 20 
     from unittest import mock
    22 21
    22 
    +import grpc
    23 23 
     from grpc._server import _Context
    24 
    -from buildgrid._protos.build.bazel.remote.execution.v2 import remote_execution_pb2
    24 
    +import pytest
    25 25
    26 
    +from buildgrid._protos.build.bazel.remote.execution.v2 import remote_execution_pb2
    26 27 
     from buildgrid.server import action_cache
    27 28 
     from buildgrid.server.cas.storage import lru_memory_cache
    28 29 
     from buildgrid.server.execution import action_cache_service

  • tests/integration/bot_session.py
    ... ... @@ -12,13 +12,13 @@
    12 12 
     # See the License for the specific language governing permissions and
    13 13 
     # limitations under the License.
    14 14
    15 
    -import grpc
    16 
    -import pytest
    15 
    +# pylint: disable=redefined-outer-name
    16 
    +
    17 17 
     import uuid
    18 18
    19 
    -from unittest import mock
    19 
    +import pytest
    20 20
    21 
    -from buildgrid.bot import bot_session, bot_interface
    21 
    +from buildgrid.bot import bot_session
    22 22
    23 23
    24 24 
     @pytest.mark.parametrize("docker_value", ["True", "False"])
    ... ... @@ -35,14 +35,14 @@ def test_create_device_key_fail():
    35 35 
         properties = {'voight': 'kampff'}
    36 36
    37 37 
         with pytest.raises(KeyError):
    38 
    -        device = bot_session.Device(properties)
    38 
    +        bot_session.Device(properties)
    39 39
    40 40
    41 41 
     def test_create_device_value_fail():
    42 42 
         properties = {'docker': True}
    43 43
    44 44 
         with pytest.raises(ValueError):
    45 
    -        device = bot_session.Device(properties)
    45 
    +        bot_session.Device(properties)
    46 46
    47 47
    48 48 
     def test_create_worker():

  • tests/integration/bots_service.py
    ... ... @@ -15,20 +15,20 @@
    15 15 
     # Authors:
    16 16 
     #        Finn Ball <finn ball codethink co uk>
    17 17
    18 
    +# pylint: disable=redefined-outer-name
    19 
    +
    18 20 
     import copy
    19 
    -import grpc
    20 
    -import pytest
    21 21 
     import uuid
    22 
    -
    23 22 
     from unittest import mock
    24 23
    24 
    +import grpc
    25 25 
     from grpc._server import _Context
    26 
    -from buildgrid._protos.build.bazel.remote.execution.v2 import remote_execution_pb2
    27 
    -from buildgrid._protos.google.devtools.remoteworkers.v1test2 import bots_pb2, worker_pb2
    28 
    -from google.protobuf import any_pb2
    26 
    +import pytest
    29 27
    28 
    +from buildgrid._protos.build.bazel.remote.execution.v2 import remote_execution_pb2
    29 
    +from buildgrid._protos.google.devtools.remoteworkers.v1test2 import bots_pb2
    30 30 
     from buildgrid.server import scheduler, job
    31 
    -from buildgrid.server.job import ExecuteStage, LeaseState
    31 
    +from buildgrid.server.job import LeaseState
    32 32 
     from buildgrid.server.worker import bots_interface, bots_service
    33 33
    34 34
    ... ... @@ -115,7 +115,7 @@ def test_update_bot_session_zombie(bot_session, context, instance):
    115 115 
         request = bots_pb2.UpdateBotSessionRequest(name=bot.name,
    116 116 
                                                    bot_session=bot)
    117 117
    118 
    -    response = instance.UpdateBotSession(request, context)
    118 
    +    instance.UpdateBotSession(request, context)
    119 119
    120 120 
         context.set_code.assert_called_once_with(grpc.StatusCode.INVALID_ARGUMENT)
    121 121
    ... ... @@ -134,7 +134,7 @@ def test_number_of_leases(number_of_jobs, bot_session, context, instance):
    134 134 
         request = bots_pb2.CreateBotSessionRequest(parent='',
    135 135 
                                                    bot_session=bot_session)
    136 136 
         # Inject work
    137 
    -    for n in range(0, number_of_jobs):
    137 
    +    for _ in range(0, number_of_jobs):
    138 138 
             action_digest = remote_execution_pb2.Digest()
    139 139 
             instance._instance._scheduler.append_job(job.Job(action_digest))
    140 140
    ... ... @@ -192,7 +192,7 @@ def test_update_leases_work_complete(bot_session, context, instance):
    192 192 
                                                    bot_session=response)
    193 193 
         response = copy.deepcopy(instance.UpdateBotSession(request, context))
    194 194
    195 
    -    assert len(response.leases) == 0
    195 
    +    assert len(response.leases) is 0
    196 196
    197 197
    198 198 
     def test_work_rejected_by_bot(bot_session, context, instance):
    ... ... @@ -216,51 +216,6 @@ def test_work_rejected_by_bot(bot_session, context, instance):
    216 216 
         context.set_code.assert_called_once_with(grpc.StatusCode.UNIMPLEMENTED)
    217 217
    218 218
    219 
    -@pytest.mark.parametrize("state", [LeaseState.LEASE_STATE_UNSPECIFIED, LeaseState.PENDING])
    220 
    -def test_work_out_of_sync_from_pending(state, bot_session, context, instance):
    221 
    -    request = bots_pb2.CreateBotSessionRequest(parent='',
    222 
    -                                               bot_session=bot_session)
    223 
    -    # Inject work
    224 
    -    action_digest = remote_execution_pb2.Digest(hash='gaff')
    225 
    -    instance._instance._scheduler.append_job(job.Job(action_digest))
    226 
    -    # Simulated the severed binding between client and server
    227 
    -    bot = copy.deepcopy(instance.CreateBotSession(request, context))
    228 
    -
    229 
    -    request = bots_pb2.UpdateBotSessionRequest(name=bot.name,
    230 
    -                                               bot_session=bot)
    231 
    -
    232 
    -    response = copy.deepcopy(instance.UpdateBotSession(request, context))
    233 
    -
    234 
    -    response.leases[0].state = state.value
    235 
    -
    236 
    -    request = bots_pb2.UpdateBotSessionRequest(name=response.name,
    237 
    -                                               bot_session=response)
    238 
    -
    239 
    -    response = instance.UpdateBotSession(request, context)
    240 
    -
    241 
    -    context.set_code.assert_called_once_with(grpc.StatusCode.DATA_LOSS)
    242 
    -
    243 
    -
    244 
    -@pytest.mark.parametrize("state", [LeaseState.LEASE_STATE_UNSPECIFIED, LeaseState.PENDING])
    245 
    -def test_work_out_of_sync_from_active(state, bot_session, context, instance):
    246 
    -    request = bots_pb2.CreateBotSessionRequest(parent='',
    247 
    -                                               bot_session=bot_session)
    248 
    -    # Inject work
    249 
    -    action_digest = remote_execution_pb2.Digest(hash='gaff')
    250 
    -    instance._instance._scheduler.append_job(job.Job(action_digest))
    251 
    -    # Simulated the severed binding between client and server
    252 
    -    bot = copy.deepcopy(instance.CreateBotSession(request, context))
    253 
    -
    254 
    -    request = bots_pb2.UpdateBotSessionRequest(name=bot.name,
    255 
    -                                               bot_session=bot)
    256 
    -
    257 
    -    response = copy.deepcopy(instance.UpdateBotSession(request, context))
    258 
    -
    259 
    -    response = instance.UpdateBotSession(request, context)
    260 
    -
    261 
    -    context.set_code.assert_called_once_with(grpc.StatusCode.UNIMPLEMENTED)
    262 
    -
    263 
    -
    264 219 
     @pytest.mark.parametrize("state", [LeaseState.LEASE_STATE_UNSPECIFIED, LeaseState.PENDING])
    265 220 
     def test_work_out_of_sync_from_pending(state, bot_session, context, instance):
    266 221 
         request = bots_pb2.CreateBotSessionRequest(parent='',

  • tests/integration/execution_service.py
    ... ... @@ -15,17 +15,16 @@
    15 15 
     # Authors:
    16 16 
     #        Finn Ball <finn ball codethink co uk>
    17 17
    18 
    -import grpc
    19 
    -import pytest
    20 
    -import uuid
    18 
    +# pylint: disable=redefined-outer-name
    21 19
    20 
    +import uuid
    22 21 
     from unittest import mock
    23 22
    24 23 
     from grpc._server import _Context
    24 
    +import pytest
    25 
    +
    25 26 
     from buildgrid._protos.build.bazel.remote.execution.v2 import remote_execution_pb2
    26 27 
     from buildgrid._protos.google.longrunning import operations_pb2
    27 
    -from google.protobuf import any_pb2
    28 
    -
    29 28 
     from buildgrid.server import action_cache, scheduler, job
    30 29 
     from buildgrid.server.cas.storage import lru_memory_cache
    31 30 
     from buildgrid.server.execution import execution_instance, execution_service
    ... ... @@ -71,25 +70,24 @@ def test_execute(skip_cache_lookup, instance, context):
    71 70 
         assert uuid.UUID(result.name, version=4)
    72 71 
         assert result.done is False
    73 72
    74 
    -"""
    75 
    -def test_wait_execution(instance, context):
    73 
    +
    74 
    +# def test_wait_execution(instance, context):
    76 75 
         # TODO: Figure out why next(response) hangs on the .get()
    77 76 
         # method when running in pytest.
    78 
    -    action_digest = remote_execution_pb2.Digest()
    79 
    -    action_digest.hash = 'zhora'
    77 
    +#     action_digest = remote_execution_pb2.Digest()
    78 
    +#     action_digest.hash = 'zhora'
    80 79
    81 
    -    j = job.Job(action_digest, None)
    82 
    -    j._operation.done = True
    80 
    +#     j = job.Job(action_digest, None)
    81 
    +#     j._operation.done = True
    83 82
    84 
    -    request = remote_execution_pb2.WaitExecutionRequest(name=j.name)
    83 
    +#     request = remote_execution_pb2.WaitExecutionRequest(name=j.name)
    85 84
    86 
    -    instance._instance._scheduler.jobs[j.name] = j
    85 
    +#     instance._instance._scheduler.jobs[j.name] = j
    87 86
    88 
    -    action_result_any = any_pb2.Any()
    89 
    -    action_result = remote_execution_pb2.ActionResult()
    90 
    -    action_result_any.Pack(action_result)
    87 
    +#     action_result_any = any_pb2.Any()
    88 
    +#     action_result = remote_execution_pb2.ActionResult()
    89 
    +#     action_result_any.Pack(action_result)
    91 90
    92 
    -    instance._instance._scheduler._update_execute_stage(j, job.ExecuteStage.COMPLETED)
    91 
    +#     instance._instance._scheduler._update_execute_stage(j, job.ExecuteStage.COMPLETED)
    93 92
    94 
    -    response = instance.WaitExecution(request, context)
    95 
    -"""
    93 
    +#     response = instance.WaitExecution(request, context)

  • tests/integration/operations_service.py
    ... ... @@ -15,19 +15,21 @@
    15 15 
     # Authors:
    16 16 
     #        Finn Ball <finn ball codethink co uk>
    17 17
    18 
    -import grpc
    19 
    -import pytest
    18 
    +# pylint: disable=redefined-outer-name
    20 19
    21 20 
     from unittest import mock
    22 21
    22 
    +import grpc
    23 23 
     from grpc._server import _Context
    24 
    +import pytest
    25 
    +
    26 
    +from google.protobuf import any_pb2
    27 
    +
    24 28 
     from buildgrid._protos.build.bazel.remote.execution.v2 import remote_execution_pb2
    25 29 
     from buildgrid._protos.google.longrunning import operations_pb2
    26 
    -
    27 
    -from buildgrid.server import scheduler, job
    30 
    +from buildgrid.server import scheduler
    28 31 
     from buildgrid.server.execution._exceptions import InvalidArgumentError
    29 32 
     from buildgrid.server.execution import execution_instance, operations_service
    30 
    -from google.protobuf import any_pb2
    31 33
    32 34
    33 35 
     # Can mock this
    ... ... @@ -78,7 +80,7 @@ def test_get_operation(instance, execute_request, context):
    78 80
    79 81 
     def test_get_operation_fail(instance, context):
    80 82 
         request = operations_pb2.GetOperationRequest()
    81 
    -    response = instance.GetOperation(request, context)
    83 
    +    instance.GetOperation(request, context)
    82 84
    83 85 
         context.set_code.assert_called_once_with(grpc.StatusCode.INVALID_ARGUMENT)
    84 86
    ... ... @@ -127,7 +129,7 @@ def test_delete_operation(instance, execute_request, context):
    127 129 
                                                       execute_request.skip_cache_lookup)
    128 130 
         request = operations_pb2.DeleteOperationRequest()
    129 131 
         request.name = response_execute.name
    130 
    -    response = instance.DeleteOperation(request, context)
    132 
    +    instance.DeleteOperation(request, context)
    131 133
    132 134 
         request = operations_pb2.GetOperationRequest()
    133 135 
         request.name = response_execute.name
    ... ... @@ -150,6 +152,6 @@ def test_cancel_operation(instance, context):
    150 152
    151 153
    152 154 
     def _pack_any(pack):
    153 
    -    any = any_pb2.Any()
    154 
    -    any.Pack(pack)
    155 
    -    return any
    155 
    +    some_any = any_pb2.Any()
    156 
    +    some_any.Pack(pack)
    157 
    +    return some_any

    • —
    View it on GitLab.
    You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.

    [Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]