Re: [BuildStream] Proposal: Allowing download-only sources to work on local files

[Tom Mewett <tom mewett codethink co uk>]

Hi Ben, thanks for your good comments. I have thought over them for a
while.

On Wed, 11 Dec 2019 09:45:32 +0000
Benjamin Schubert <contact benschubert me> wrote:
> I disagree with changing the terminology here. Nothing in 'URL' means it should be only to
> target a resource on the web. A local file path _is_ a valid url.

As Jurg pointed out, that last part isn't always true. Plus,
technically URIs must have a scheme, which relative paths don't. But my
concern was actually that a less-knowledgable user might assume a 'url'
key could only take remote resources.

> While this would work in theory, I don't think this should be done in practice.
> The reason for this is that we would then have a set of sources (say Zip,Deb,Tar) that
> would work with both a local path or a remote one. However other plugins like 'git'
> would then only be supporting remote? Why such a discrepancy?

I agree with Jurg here that this discrepancy is unimportant.
Additionally, I think it's unnatural to consider a Git repository as a
file one downloads. It is more like database that is accessed remotely
in a Git-specific way. To this end, I don't think it would be
surprising that the Git source doesn't support this feature.

(However this point does make a slight case for renaming the 'url'
option for "file-based sources," to hint at this difference.)

> This would lead to two radically different handling of the _same_ source type, making it
> harder ot understand what is happening and how to write a source element correctly.
> I think this is too complex and will lead to confusion for newcomers.

It is certainly added complexity, but I think it's worth it. Consider
that for the majority of use-cases, nothing is changing. Also, this
functionality will be present over multiple, very similar plugins so
will only need to be learned once. I'm hoping users won't have too many
issues with it. I could ask some coworkers of mine what they think?

And I don't see how it will make writing a source plugin more
difficult. It will be largely similar to deriving from
DownloadableFileSource currently.

> On Monday, 02 December 2019 10:19, Tom Mewett via buildstream-list <buildstream-list gnome org> wrote:
> > Another advantage would be that local files imported into the project
> > can be given refs, meaning that they would not need to be present to
> > compute cache keys of depending elements. (This is not possible with
> > the current 'local' source.)
>
> That would mean that depending on whether a ref is set or not, a change will
> be picked up or not. I think this will lead to too many confusion with editing
> said files.

Reading again, I'm actually not too sure how my original idea there
would have worked :) but I'm also not sure what you mean. So I think we
should discuss that when there's something more concrete.

> Here is my alternate proposition, which builds upon Tristan's one a lot:
> ...

I share the same concerns as Jurg with this idea.

--

Thanks again for your comments, they made me think more deeply about
the proposal.

-- 
Tom Mewett <tom mewett codethink co uk>