Re: [BuildStream] bst-artifact-server plans

[Chandan Singh <chandan chandansingh net>]

Hi,

Juerg, thanks for writing this. The plan makes sense to me. A couple
of comments inline.

<snip>

> Why PostgreSQL in particular as the first effort? I have no strong
> feelings on that but I think it would be best for the first backend to
> have a ready user/use-case.

o/ I am ready for this backend :) I absolutely agree that the first
backend  has to have a ready use-case, and we do have at least one. In
our case,  using a proper database as a backend will allow us to make
the artifact servers themselves stateless, and hence more easily
scalable.

> > # Query support
> >
> > To manage a server with a large number of artifacts, supporting queries
> > and deletions will be important. The details are not defined yet,
> > however, I expect us to create a CLI tool in the same repository for
> > this.

You probably already had this in mind, but I'd like to call out that
it will be nice to be able to delete the objects from the actual CAS,
and not just the database. In fact, keeping the metadata in the
database might be preferred in some cases. It should be much smaller
comparatively and might be useful for audit trails. The deletions can
always be done externally, but will be nice if the CLI/API supported
this already.

> The details hopefully will not depend on the backend however, that is
> probably unavoidable. I think it makes sense to either have an api for
> low-level interactions with the backend or support plugins to minimise
> maintenance.

Yeah, we will probably need some sort of abstraction around what
constitutes a backend. I expect it won't be much more complicated than
supporting CRUD operations. It will certainly be nice to keep the
backend-specific limited to the backend implementations.

Thanks,
Chandan