|
> Hi Matt, thanks for your interest.
> > This is a problem that I've considered before, and led to the creation of filter elements (https://docs.buildstream.build/elements/filter.html#module-elements.filter). > > > With a filter element, you'd have a structure similar to number 3. > > protobuf-src.bst builds all the output that you need in total, and defines split-rules that separate out the files. > > e.g. it might look like: > ``` > kind: autotools > sources: > - kind: git > url: https://github.com/protocolbuffers/protobuf.git > track: v3.6.1.2 > ref: 9c793fc9e80ce6287bfe6f6cdf349055d1477406 > public: > bst: > split-rules: > python: > - %{includedir}/protobuf-python.h > ruby: > - %{includedir}/protobuf-ruby.h > ``` > > You would then have filter elements that select only the split-domains you want ("python" and "ruby" respectively). > > e.g. protobuf-ruby.bst would look something like: > ``` > kind: filter > config: > include: > - ruby > build-depends: > - protobuf-src.bst > ``` > > And then service-b would depend on protobuf-ruby as normal. > > protobuf-src.bst is a build-dependency on protobuf-ruby.bst, so if anything runtime-depends on protobuf-ruby, it won't also depend on protobuf-src. > > > I hope this helps, let me know if you have any further questions. > > Best regards, > > Jonathan > -- > Jonathan Maw, Software Engineer, Codethink Ltd. > Codethink privacy policy: https://www.codethink.co.uk/privacy.html > --- Hi Jonathan, Thanks for getting back to me. Filter elements look like they would do the job, but I still have some concerns: * The spilt-rules for python and ruby libs are quite simple, but we have some legacy projects that have gotten quite complicated over years, and tend to scatter their files across the filesystem with little or no regards to the FHS. Writing generic rules for these elements would quickly devolve down to file manifests, that I'd rather not have to maintain. Having a dedicated build element for each subproject make this easier as the manifest of a build is whatever is in %{install-root}. * When using workspaces, the usual cycle of edit-build slows down as the build step now includes subprojects unrelated to what the developer is currently editing (of course incremental builds would mitigate this issue quite a bit, but that's not guaranteed.). Also I can't take advantage of a non-strict build plan to speed up my builds on the bench. * I can't use the builtin build elements ('distutils', 'autotools', 'cmake' etc.) and would probably have to roll my own which is an aggregation the subprojects' build systems. If you look in the protobuf repo you'll see there isn't an overarching build system (well there is Bazel, but that's difficult to setup in buildstream at the moment so I'm ignoring it for now); there's the autotools files at the root for building the c shared lib, python/setup.py for the python package, ruby/Rakefile etc. And our repo's share the same setup, the internal buildsystems are usually left to the project owners. None of these are deal breakers, but they do make the system a bit cumbersome to use. Our first implementation will likely use split-rules, but I think I'm going to keep working at the ElementSource plugin. Thanks again, Matt Yates |