GNOME.org
 

[Notes] [Git][BuildGrid/buildgrid][mablanch/51-clients-usage] docs: Add Bazel and BuildStream configuration instructions

Title: GitLab

Martin Blanchard pushed to branch mablanch/51-clients-usage at BuildGrid / buildgrid

Commits:

5 changed files:

Changes:

  • docs/source/data/bazel-example-server.conf
    1 
    +server:
    2 
    +  port: 50051
    3 
    +  insecure-mode: true
    4 
    +
    5 
    +instances:
    6 
    +  - name: main
    7 
    +
    8 
    +    storages:
    9 
    +      - !lru-storage &main-storage
    10 
    +        size: 512MB
    11 
    +
    12 
    +    services:
    13 
    +      - !action-cache &main-action
    14 
    +        storage: *main-storage
    15 
    +        max_cached_refs: 256
    16 
    +        allow_updates: true
    17 
    +      - !execution
    18 
    +        storage: *main-storage
    19 
    +        action_cache: *main-action
    20 
    +      - !cas
    21 
    +        storage: *main-storage
    22 
    +      - !bytestream
    23 
    +        storage: *main-storage
    \ No newline at end of file

  • docs/source/using.rst
    ... ... @@ -9,5 +9,5 @@ This section covers how to run an use the BuildGrid build service.
    9 9 
     .. toctree::
    10 10 
        :maxdepth: 2
    11 11
    12 
    -   using_dummy_build.rst
    13 
    -   using_simple_build.rst
    12 
    +   using_internal.rst
    13 
    +   using_bazel.rst

  • docs/source/using_bazel.rst
    1 
    +
    2 
    +.. _bazel-builds:
    3 
    +
    4 
    +Bazel builds
    5 
    +============
    6 
    +
    7 
    +`Bazel`_ is a *“fast, scalable, multi-language and extensible build system”*
    8 
    +that supports remote build execution using the remote execution API (REAPI) v2
    9 
    +since its `0.17 release`_.
    10 
    +
    11 
    +.. _Bazel: https://bazel.build
    12 
    +.. _0.17 release: https://blog.bazel.build/2018/09/14/bazel-0.17.html
    13 
    +
    14 
    +
    15 
    +.. _bazel-configuration:
    16 
    +
    17 
    +Configuration
    18 
    +-------------
    19 
    +
    20 
    +Bazel accepts many options that can either be specified as command line
    21 
    +arguments when involking the ``bazel`` tool or stored in a `.bazelrc`_
    22 
    +configuration file. In order to activate remote build execution, the
    23 
    +``bazel build`` subcommand needs to be given specific `build options`_,
    24 
    +including:
    25 
    +
    26 
    +- ``--remote_executor``: remote execution endpoint's location, ``{host}:{port}``.
    27 
    +- ``--remote_instance_name``: remote execution instance's name.
    28 
    +- ``--spawn_strategy``: action execution method.
    29 
    +- ``--genrule_strategy``: `genrules`_ execution method.
    30 
    +
    31 
    +Spawn and genrule strategies need to be set to ``remote`` for remote execution.
    32 
    +
    33 
    +As an example, in order to activate remote execution on the ``main`` instance of
    34 
    +the remote execution server available at ``controller.grid.build`` on port
    35 
    +``50051`` you should amend your ``.bazelrc`` with:
    36 
    +
    37 
    +.. code-block:: sh
    38 
    +
    39 
    +   build --spawn_strategy=remote --genrule_strategy=remote --remote_executor=controller.grid.build:50051 --remote_instance_name=main
    40 
    +
    41 
    +.. _.bazelrc: https://docs.bazel.build/versions/master/user-manual.html#bazelrc
    42 
    +.. _build options: https://docs.bazel.build/versions/master/command-line-reference.html#build-options
    43 
    +.. _genrules: https://docs.bazel.build/versions/master/be/general.html#genrule
    44 
    +
    45 
    +
    46 
    +.. _bazel-example:
    47 
    +
    48 
    +Example build
    49 
    +-------------
    50 
    +
    51 
    +The `bazel-examples`_ repository contains example Bazel workspaces that can be
    52 
    +used for testing purpose. We'll focus here on instructions on how to build the
    53 
    +`stage3 CPP example`_ running Bazel and BuildGrid on your local machine,
    54 
    +compiling the C++ source code using host-tools.
    55 
    +
    56 
    +First, you need to checkout the bazel-examples repository sources:
    57 
    +
    58 
    +.. code-block:: sh
    59 
    +
    60 
    +   git clone https://github.com/bazelbuild/examples.git bazel-examples
    61 
    +
    62 
    +Next, change the current directory to the Bazel workspace root:
    63 
    +
    64 
    +.. code-block:: sh
    65 
    +
    66 
    +   cd bazel-examples/cpp-tutorial/stage3
    67 
    +
    68 
    +.. note::
    69 
    +
    70 
    +   All the commands in the instructions below are expected to be executed from
    71 
    +   that root directory (the stage3 example workspace's root directory).
    72 
    +
    73 
    +Before running Bazel and building the example workspace, you'll have to setup
    74 
    +and run a BuildGrid server and bot. A minimal server's configuration is given
    75 
    +below, paste it in a ``server.conf`` file in the root directory:
    76 
    +
    77 
    +.. literalinclude:: ./data/bazel-example-server.conf
    78 
    +   :language: yaml
    79 
    +
    80 
    +This defines a single ``main`` server instance implementing a
    81 
    +``ContentAddressableStorage`` (CAS) + ``ByteStream`` service together with an
    82 
    +``Execution`` + ``ActionCache`` service, both using the same in-memory storage.
    83 
    +You can then start the BuildGrid server deamon using that configuration by
    84 
    +running:
    85 
    +
    86 
    +.. code-block:: sh
    87 
    +
    88 
    +   bgd server start server.conf
    89 
    +
    90 
    +In order to perform the actual build work, you need to attach a worker bot to
    91 
    +that server for that ``main`` instance. A simple host-tools based bot should be
    92 
    +enough to build the stage3 CPP example. Once you've make sure that your machine
    93 
    +has ``gcc`` installed, run:
    94 
    +
    95 
    +.. code-block:: sh
    96 
    +
    97 
    +   bgd bot --remote=http://localhost:50051 --parent=main temp-directory
    98 
    +
    99 
    +The ``--remote`` option is used to specify the server location (running on the
    100 
    +same machine here, and listenning to port 50051). The ``--parent`` option is
    101 
    +used to specifiy the server instance you except the bot to be attached to.
    102 
    +
    103 
    +The BuildGrid server is now ready to accept jobs and execute them. Bazel needs
    104 
    +some :ref:`configuration <bazel-configuration>` in order to run remote builds.
    105 
    +Below are the minimal build options you should use, paste them in a ``.bazelrc``
    106 
    +file in the root directory:
    107 
    +
    108 
    +.. code-block:: sh
    109 
    +
    110 
    +   build --spawn_strategy=remote --genrule_strategy=remote --remote_executor=localhost:50051 --remote_instance_name=main
    111 
    +
    112 
    +This ativates Bazel's remote execution mode and points to the ``main`` remote
    113 
    +execution server instance at ``localhost:50051``.
    114 
    +
    115 
    +You can finally have Bazel to build the example workspace by running:
    116 
    +
    117 
    +.. code-block:: sh
    118 
    +
    119 
    +   bazel build //main:hello-world
    120 
    +
    121 
    +You can verify that the example has been successfully build by runnig the
    122 
    +generated executable. Simply invoke:
    123 
    +
    124 
    +.. code-block:: sh
    125 
    +
    126 
    +   ./bazel-bin/main/hello-world
    127 
    +
    128 
    +.. _bazel-examples: https://github.com/bazelbuild/examples
    129 
    +.. _stage3 CPP example: https://github.com/bazelbuild/examples/tree/master/cpp-tutorial/stage3

  • docs/source/using_dummy_build.rst deleted
    1 
    -.. _dummy-build:
    2 
    -
    3 
    -Dummy build
    4 
    -===========
    5 
    -
    6 
    -In one terminal, start a server:
    7 
    -
    8 
    -.. code-block:: sh
    9 
    -
    10 
    -   bgd server start buildgrid/_app/settings/default.yml
    11 
    -
    12 
    -In another terminal, send a request for work:
    13 
    -
    14 
    -.. code-block:: sh
    15 
    -
    16 
    -   bgd execute request-dummy
    17 
    -
    18 
    -The stage should show as ``QUEUED`` as it awaits a bot to pick up the work:
    19 
    -
    20 
    -.. code-block:: sh
    21 
    -
    22 
    -   bgd execute list
    23 
    -
    24 
    -Create a bot session:
    25 
    -
    26 
    -.. code-block:: sh
    27 
    -
    28 
    -   bgd bot dummy
    29 
    -
    30 
    -Show the work as completed:
    31 
    -
    32 
    -.. code-block:: sh
    33 
    -
    34 
    -   bgd execute list

  • docs/source/using_simple_build.rst → docs/source/using_internal.rst
    1 
    +
    2 
    +.. _internal-builds:
    3 
    +
    4 
    +Internal builds
    5 
    +===============
    6 
    +
    7 
    +
    8 
    +.. _dummy-test:
    9 
    +
    10 
    +Dummy test
    11 
    +----------
    12 
    +
    13 
    +In one terminal, start a server:
    14 
    +
    15 
    +.. code-block:: sh
    16 
    +
    17 
    +   bgd server start buildgrid/_app/settings/default.yml
    18 
    +
    19 
    +In another terminal, send a request for work:
    20 
    +
    21 
    +.. code-block:: sh
    22 
    +
    23 
    +   bgd execute request-dummy
    24 
    +
    25 
    +The stage should show as ``QUEUED`` as it awaits a bot to pick up the work:
    26 
    +
    27 
    +.. code-block:: sh
    28 
    +
    29 
    +   bgd execute list
    30 
    +
    31 
    +Create a bot session:
    32 
    +
    33 
    +.. code-block:: sh
    34 
    +
    35 
    +   bgd bot dummy
    36 
    +
    37 
    +Show the work as completed:
    38 
    +
    39 
    +.. code-block:: sh
    40 
    +
    41 
    +   bgd execute list
    42 
    +
    43 
    +
    1 44 
     .. _simple-build:
    2 45
    3 46 
     Simple build
    4 
    -============
    47 
    +------------
    5 48
    6 49 
     This example covers a simple build. The user will upload a directory containing
    7 50 
     a C file and a command to the CAS. The bot will then fetch the uploaded

    • —
    View it on GitLab.
    You're receiving this email because of your account on gitlab.com. If you'd like to receive fewer emails, you can adjust your notification settings.

    [Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]