Re: [BuildStream] Proposal: Decouple source tracking and building



Hi,

On Oct 18, 2019, at 10:56 PM, Chandan Singh <chandan chandansingh net> wrote:

Hi all,

tl;dr: Remove `--track-*` options from `bst build` and simplify codebase.

---

I wanted to propose a clearer separation between tracking sources and building
them in BuildStream. Currently `bst build` offers a few `--track-*` options,
all related to whether or not BuildStream should attempt to track sources
before starting the build.

In my opinion, these are two different kinds of operations, and don't really
need to be part of the same pipeline.

I disagree with this.

The reason why we can track and build in the same session is an optimization, we should not have to wait 
until all the elements finish tracking before being able to build the first elements which reach a consistent 
state. I understand the code would be easier to maintain if we didn’t have this feature but I think it really 
provides good value.

I do however agree that the UX for this is undesirable and could be improved, one suggestion somebody made on 
IRC some months ago was to make things a bit scriptable for the more complex invocations, and for running 
multiple commands in a single session (so we could also have build + checkout for instance, which has been 
requested as a convenience by some users).

This might mean removing some of the more complex CLI options (or not) and adding a new option to read 
commands from a file (or a quoted string on the command line, or stdin or such).

Not sure how popular this idea might be, just throwing it out there.

Cheers,
    -Tristan


My concerns about this can be broken down
into two broad areas:

1. Simplifying scheduling/update_state() logic
2. UI/UX

First, let's talk about scheduler. At present, when we start a build, we expect
the elements being tracked to be resolved only after they've been tracked. This
means that we can't be sure that all elements in the pipeline are fully resolved
and ready to be built.

Decoupling tracking and build operations should simplify the `update_state()`
logic as it would eliminate one of the needs for doing so.

Moving on to UI/UX issues, `bst build --track-*` essentially duplicates
functionality from `bst source track`. Hence we end up with two ways of doing
the same thing and have to support both.

Since tracking sources and building are rather different kinds of operations,
I am proposing to have them clearly separated. More concretely, I am proposing
to remove the following options from `bst build`:

 * `--track`
 * `--track-all`
 * `--track-except`
 * `--track-cross-junctions`

 * There's also `--track-save` which is deprecated and ignored

`bst source track` already supports all functionality offered by the above
options, so that won't require any additional features.

I can understand the desire to update all source references in a CI build, or
for testing purposes. But, I think the same can be achieved by running
`bst source track` followed by `bst build`. Can people think of any other use
cases that would be negatively affected by this change, other than having to
type two commands instead of one?

Let me know what you think.

Thanks!
Chandan
_______________________________________________
buildstream-list mailing list
buildstream-list gnome org
https://mail.gnome.org/mailman/listinfo/buildstream-list




[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]