[BuildStream] Proposal: Configuration format for CAS/artifact server split

[Tristan Daniël Maat <tristan maat codethink co uk>]

Hi,

I've started looking at [supporting separate endpoints for
CAS/artifact services][1]. While the issue seems pretty
straightforward, we'll need to change the configuration format
slightly, and I wanted to check if I'm going in the right direction
since I've not really played with CAS all that much.

The configuration format I'd like to use looks like this:

```yaml
artifacts:
  # Un-split caches can still be defined using the old spec
  - url: https://foo.com:11001
    server-cert: foo.crt

  # Split caches are made up of two separate specs like this
  - metadata:   # Can anyone think of more obvious names for users?
      url: https://foo2.com:11001
      server-cert: foo2.crt
    blobs:      # Can anyone think of more obvious names for users?
      url: https://foo3.com:11001
      server-cert: foo3.crt

# We'll also want to support this for sources, if only for the sake
# of keeping the code paths similar
sources:
  - metadata:
      url: https://foo2.com::11001
      server-cert: foo2.crt
    blobs:
      url: https://foo3.com::11001
      server-cert: foo3.crt
```

Specifying both metadata and blobs will be mandatory, since it makes
no sense to store artifact metadata without its blob, nor does it make
sense to store a blob we can't later access anymore.

The effect in BuildStream's internals will be that it instantiates two
separate remotes, and uses one to store artifact/source protos, and
the other to store the blobs those consist of. Doing that might take
some juggling of classes, which will almost certainly interfere with
Jürg's work to use [buildbox-casd][2], but we'll cross that bridge
when we get there :)

This should mean that we can continue to use buildgrid to store
artifacts in CAS while using a separate remote to store the metadata
we need for BuildStream.

With that I believe we hit all points of the [issue][1], are there any
other use cases for this not considered here?

Regards,

Tristan Maat

[1]: https://gitlab.com/BuildStream/buildstream/issues/1041
[2]: https://gitlab.com/BuildStream/buildstream/merge_requests/1499