[BuildStream] Proposal: bst artifact subcommand group

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

Hi all,

There are a few functionalities which have been requested for some
time, such as viewing the build log of an artifact[0], or deleting an
existing artifact from the local cache.

It was recently raised that we cannot determine the build date of an
existing artifact either[1], as it's not a part of artifact metadata,
this discussion evolved to proposing that we have a subgroup of CLI
commands for dealing with specific artifacts.

As such I'd like to propose the following, probably uncontroversial
enhancements:

  o Add the build date to the artifact metadata, as a timezone neutral
    unix timestamp (seconds since epoch in UTC).

  o Standardize on the name of an artifact, this is not really exposed
    in the UX except the user may see "artifact names" at times.

    An artifact name is essentially:

       ${project-name}/${element-name}/${cache-key}

    Exposing this to the user essentially means that we have a handy
    syntax for the user to express an artifact on the command line.

  o Add the `bst artifact` subgroup with the following commands:

    o bst artifact list <artifact name glob pattern>

      By default listing only the artifacts which match the glob
      pattern.

      An additional `--long/-l` argument can get us more human
      readable information, similar to what was outlined here[2]

      List the artifacts in order of build date, displaying the build
      date in local time along with the size of the files portion of
      the artifact, and the active workspace if it was created locally
      with a workspace.

    o bst artifact log <artifact name or pattern>

      Display the artifact build log in the $PAGER, similar to how we
      implement the "log" choice on the prompt of a failing build.

      If a pattern is specified then the log of each will be sent to
      the system pager in series (matching to the behavior of
      specifying wildcards to programs like "less").

    o bst artifact delete <artifact name or glob pattern>

      Delete artifacts from the local cache which match the pattern

    o bst artifact list-content <artifact name or glob pattern>

      This can be useful for scriptability purposes, where one wants to
      generate a manifest of what an artifact contains, or simply for
      a curious user to see what files an artifact contains.

      Similarly, this should have `--long/-l` options to show detailed
      information about the files in the artifact, such as
      user/group/everyone permission bits, ownership bits, file size,
      etc (I think `tar -t` offers this with a `-vv` switch or such).

    o bst artifact diff <artifact name> <artifact name>

      Show differing added/removed and differing files in two artifacts

Beyond this:

  o It will be interesting to allow artifact operations to specify
    elements instead of artifact names, for the cases where you just
    want to use the artifact who's cache key corresponds to the project
    state.

  o It would also be interesting to move `checkout`, `push` and `pull`
    commands under the new `artifact` group (deprecating the existing
    commands).

    This will open up the door to performing checkouts etc at the
    artifact level instead of only supporting the artifacts who's
    cache keys correspond to the project state.

If nobody opposes the proposal, I will go ahead and roadmap this by
creating a flagship issue on gitlab with a task list, each task
pointing to individual separate issues for the individually proposed
commands.

Best Regards,
    -Tristan

[0]: https://gitlab.com/BuildStream/buildstream/issues/234
[1]: https://gitlab.com/BuildStream/buildstream/issues/416
[2]: https://gitlab.com/BuildStream/buildstream/issues/416#note_83263225