[Notes] [Git][BuildStream/buildstream][chandan/pip-install-instructions] 7 commits: element.py: chmod 777 directories if delete fails.



Title: GitLab

Chandan Singh pushed to branch chandan/pip-install-instructions at BuildStream / buildstream

Commits:

6 changed files:

Changes:

  • HACKING.rst
    ... ... @@ -465,12 +465,12 @@ If you want to run a specific test or a group of tests, you
    465 465
     can specify a prefix to match. E.g. if you want to run all of
    
    466 466
     the frontend tests you can do::
    
    467 467
     
    
    468
    -  ./setup.py test --addopts '-k tests/frontend/'
    
    468
    +  ./setup.py test --addopts 'tests/frontend/'
    
    469 469
     
    
    470 470
     Specific tests can be chosen by using the :: delimeter after the test module.
    
    471 471
     If you wanted to run the test_build_track test within frontend/buildtrack.py you could do::
    
    472 472
     
    
    473
    -  ./setup.py test --adopts '-k tests/frontend/buildtrack.py::test_build_track'
    
    473
    +  ./setup.py test --addopts 'tests/frontend/buildtrack.py::test_build_track'
    
    474 474
     
    
    475 475
     We also have a set of slow integration tests that are disabled by
    
    476 476
     default - you will notice most of them marked with SKIP in the pytest
    

  • buildstream/element.py
    ... ... @@ -1361,8 +1361,12 @@ class Element(Plugin):
    1361 1361
                 if not vdirectory.is_empty():
    
    1362 1362
                     raise ElementError("Staging directory '{}' is not empty".format(vdirectory))
    
    1363 1363
     
    
    1364
    -            with tempfile.TemporaryDirectory() as temp_staging_directory:
    
    1364
    +            # While mkdtemp is advertised as using the TMP environment variable, it
    
    1365
    +            # doesn't, so this explicit extraction is necesasry.
    
    1366
    +            tmp_prefix = os.environ.get("TMP", None)
    
    1367
    +            temp_staging_directory = tempfile.mkdtemp(prefix=tmp_prefix)
    
    1365 1368
     
    
    1369
    +            try:
    
    1366 1370
                     workspace = self._get_workspace()
    
    1367 1371
                     if workspace:
    
    1368 1372
                         # If mount_workspaces is set and we're doing incremental builds,
    
    ... ... @@ -1377,6 +1381,19 @@ class Element(Plugin):
    1377 1381
                             source._stage(temp_staging_directory)
    
    1378 1382
     
    
    1379 1383
                     vdirectory.import_files(temp_staging_directory)
    
    1384
    +
    
    1385
    +            finally:
    
    1386
    +                # Staging may produce directories with less than 'rwx' permissions
    
    1387
    +                # for the owner, which will break tempfile, so we need to use chmod
    
    1388
    +                # occasionally.
    
    1389
    +                def make_dir_writable(fn, path, excinfo):
    
    1390
    +                    os.chmod(os.path.dirname(path), 0o777)
    
    1391
    +                    if os.path.isdir(path):
    
    1392
    +                        os.rmdir(path)
    
    1393
    +                    else:
    
    1394
    +                        os.remove(path)
    
    1395
    +                shutil.rmtree(temp_staging_directory, onerror=make_dir_writable)
    
    1396
    +
    
    1380 1397
             # Ensure deterministic mtime of sources at build time
    
    1381 1398
             vdirectory.set_deterministic_mtime()
    
    1382 1399
             # Ensure deterministic owners of sources at build time
    

  • doc/source/install_linux_distro.rst
    ... ... @@ -152,8 +152,29 @@ for advice on this.
    152 152
     
    
    153 153
     Installing
    
    154 154
     ~~~~~~~~~~
    
    155
    -Once you have the base system dependencies, you can clone the BuildStream
    
    156
    -git repository and install it as a regular user::
    
    155
    +Once you have the base system dependencies, you can install the BuildStream
    
    156
    +python package as a regular user.
    
    157
    +
    
    158
    +Via PyPI (recommended)
    
    159
    +++++++++++++++++++++++
    
    160
    +::
    
    161
    +
    
    162
    +  pip3 install --user BuildStream
    
    163
    +
    
    164
    +This will install latest stable version of BuildStream and its pure python
    
    165
    +dependencies into your user's homedir in ``~/.local``.
    
    166
    +
    
    167
    +Keep following the instructions below to ensure that the ``bst``
    
    168
    +command is in your ``PATH`` and to enable bash completions for it.
    
    169
    +
    
    170
    +.. note::
    
    171
    +
    
    172
    +  If you want a specific version of BuildStream, you can install it using
    
    173
    +  ``pip install --user BuildStream==<version-number>``
    
    174
    +
    
    175
    +Via Git checkout
    
    176
    +++++++++++++++++
    
    177
    +::
    
    157 178
     
    
    158 179
       git clone https://gitlab.com/BuildStream/buildstream.git
    
    159 180
       cd buildstream
    
    ... ... @@ -206,9 +227,19 @@ to your ``~/.bash_completion``:
    206 227
     
    
    207 228
     Upgrading BuildStream
    
    208 229
     ~~~~~~~~~~~~~~~~~~~~~
    
    209
    -Assuming you have followed the default instructions above, all
    
    210
    -you need to do to upgrade BuildStream is to update your local git
    
    211
    -checkout::
    
    230
    +
    
    231
    +Via PyPI
    
    232
    +++++++++
    
    233
    +
    
    234
    +If you installed BuildStream from PyPI, you can update it like so::
    
    235
    +
    
    236
    +  pip install --user --upgrade BuildStream
    
    237
    +
    
    238
    +Via Git checkout
    
    239
    +++++++++++++++++
    
    240
    +
    
    241
    +If you installed BuildStream from a local git checkout using ``-e`` option, all
    
    242
    +you need to do to upgrade BuildStream is to update your local git checkout::
    
    212 243
     
    
    213 244
       cd /path/to/buildstream
    
    214 245
       git pull --rebase
    

  • tests/sources/tar.py
    ... ... @@ -3,6 +3,7 @@ import pytest
    3 3
     import tarfile
    
    4 4
     import tempfile
    
    5 5
     import subprocess
    
    6
    +from shutil import copyfile, rmtree
    
    6 7
     
    
    7 8
     from buildstream._exceptions import ErrorDomain
    
    8 9
     from buildstream import _yaml
    
    ... ... @@ -257,3 +258,47 @@ def test_stage_default_basedir_lzip(cli, tmpdir, datafiles, srcdir):
    257 258
         original_contents = list_dir_contents(original_dir)
    
    258 259
         checkout_contents = list_dir_contents(checkoutdir)
    
    259 260
         assert(checkout_contents == original_contents)
    
    261
    +
    
    262
    +
    
    263
    +# Test that a tarball that contains a read only dir works
    
    264
    +@pytest.mark.datafiles(os.path.join(DATA_DIR, 'read-only'))
    
    265
    +def test_read_only_dir(cli, tmpdir, datafiles):
    
    266
    +    try:
    
    267
    +        project = os.path.join(datafiles.dirname, datafiles.basename)
    
    268
    +        generate_project(project, tmpdir)
    
    269
    +
    
    270
    +        # Get the tarball in tests/sources/tar/read-only/content
    
    271
    +        #
    
    272
    +        # NOTE that we need to do this because tarfile.open and tar.add()
    
    273
    +        # are packing the tar up with writeable files and dirs
    
    274
    +        tarball = os.path.join(str(datafiles), 'content', 'a.tar.gz')
    
    275
    +        if not os.path.exists(tarball):
    
    276
    +            raise FileNotFoundError('{} does not exist'.format(tarball))
    
    277
    +        copyfile(tarball, os.path.join(str(tmpdir), 'a.tar.gz'))
    
    278
    +
    
    279
    +        # Because this test can potentially leave directories behind
    
    280
    +        # which are difficult to remove, ask buildstream to use
    
    281
    +        # our temp directory, so we can clean up.
    
    282
    +        tmpdir_str = str(tmpdir)
    
    283
    +        if not tmpdir_str.endswith(os.path.sep):
    
    284
    +            tmpdir_str += os.path.sep
    
    285
    +        env = {"TMP": tmpdir_str}
    
    286
    +
    
    287
    +        # Track, fetch, build, checkout
    
    288
    +        result = cli.run(project=project, args=['track', 'target.bst'], env=env)
    
    289
    +        result.assert_success()
    
    290
    +        result = cli.run(project=project, args=['fetch', 'target.bst'], env=env)
    
    291
    +        result.assert_success()
    
    292
    +        result = cli.run(project=project, args=['build', 'target.bst'], env=env)
    
    293
    +        result.assert_success()
    
    294
    +
    
    295
    +    finally:
    
    296
    +
    
    297
    +        # Make tmpdir deletable no matter what happens
    
    298
    +        def make_dir_writable(fn, path, excinfo):
    
    299
    +            os.chmod(os.path.dirname(path), 0o777)
    
    300
    +            if os.path.isdir(path):
    
    301
    +                os.rmdir(path)
    
    302
    +            else:
    
    303
    +                os.remove(path)
    
    304
    +        rmtree(str(tmpdir), onerror=make_dir_writable)

  • tests/sources/tar/read-only/content/a.tar.gz
    No preview for this file type
  • tests/sources/tar/read-only/target.bst
    1
    +kind: import
    
    2
    +description: The kind of this element is irrelevant.
    
    3
    +sources:
    
    4
    +- kind: tar
    
    5
    +  url: tmpdir:/a.tar.gz
    
    6
    +  ref: foo



  • [Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]