Re: [BuildStream] Questiont: How to handle monorepos in buildstream

[Jonathan Maw <jonathan maw codethink co uk>]

On 2019-01-21 19:11, Yates, Matthew (StoreOnce) wrote:
> Hi Jonathan,
>
> Thanks for getting back to me. Filter elements look like they would do
> the job,
> but I still have some concerns:
> * The spilt-rules for python and ruby libs are quite simple, but we
> have some
>   legacy projects that have gotten quite complicated over years, and
> tend to
>   scatter their files across the filesystem with little or no regards
> to the
>   FHS. Writing generic rules for these elements would quickly devolve
> down to
>   file manifests, that I'd rather not have to maintain. Having a
> dedicated
>   build element for each subproject make this easier as the manifest
> of a build
>   is whatever is in %{install-root}.
>
> * When using workspaces, the usual cycle of edit-build slows down as
> the build
>   step now includes subprojects unrelated to what the developer is
> currently
>   editing (of course incremental builds would mitigate this issue
> quite a bit,
>   but that's not guaranteed.). Also I can't take advantage of a
> non-strict build
>   plan to speed up my builds on the bench.
>
> * I can't use the builtin build elements ('distutils', 'autotools',
> 'cmake'
>   etc.) and would probably have to roll my own which is an aggregation
> the
>   subprojects' build systems. If you look in the protobuf repo you'll
> see there
>   isn't an overarching build system (well there is Bazel, but that's
> difficult
>   to setup in buildstream at the moment so I'm ignoring it for now);
> there's the
>   autotools files at the root for building the c shared lib,
> python/setup.py for
>   the python package, ruby/Rakefile etc. And our repo's share the same
> setup,
>   the internal buildsystems are usually left to the project owners.
>
> None of these are deal breakers, but they do make the system a bit
> cumbersome to
> use.
>
> Our first implementation will likely use split-rules, but I think I'm
> going to
> keep working at the ElementSource plugin.
>
> Thanks again,
> Matt Yates

Hi Matt,

Thanks for explaining your use-case further. Under those constraints, it 
does sound like Abderrahim Kitouni's include-based solution 
(https://mail.gnome.org/archives/buildstream-list/2019-January/msg00045.html) 
is a better fit for your project.

Thanks again for your input, the mono-repo use-case is one that's had a 
lot of discussion in the past, and the more real-world use-cases we see, 
the better.

Best regards,

Jonathan

--
Jonathan Maw, Software Engineer, Codethink Ltd.
Codethink privacy policy: https://www.codethink.co.uk/privacy.html