Re: GrlNetWc cache?

[Bastien Nocera <hadess hadess net>]

Hey!

On Fri, 2014-02-21 at 19:30 +0100, Juan A. Suarez Romero wrote:
> On Fri, 2014-02-21 at 17:57 +0100, Bastien Nocera wrote:
> > Hey,
> >
> > Is the cache from GrlNetWc supposed to be persistent? I can see it
> > being destroyed unconditionally, and with always changing paths.
>
> Current implementation is not persistent. But fyi, I already thought
> about adding a persistent cache option. And it is in my TODO. I'll file
> a bug about it.

Great, thank you. I'd be happy to implement it if you can tell me what
you want as an API.

> > I could see myself requiring a persistent cache for certain requests
> > (checking the "latest" items, but only if it's more than a certain age).
>
> Yep. But take in account that GrlNet will not cache per sé, but use
> libsoup cache feature. So at the end, it will be libsoup the one doing
> the cache.

Right. I was wondering how libsoup does its caching, and whether we can
avoid hitting the web server when we know the data won't have changed
(because the web server doesn't tell us in the headers, but somewhere in
the data for example).

> > If we implement this, I'd also like for those caches to be named after
> > each plugin, to avoid conflicts.
>
> Already contemplated. The idea is that sources will need to provide a
> name or identifier to the cache in order to use it (for instance, the
> source name).

Sounds good.

> > This might also mean that we need to be careful about using a single
> > GrlNetWc in each plugin or at least source to share the cache data.
>
> Unless I forget something, I think each source uses its own grlnet
> object. For the case of persistance, I'll add in the documentation that
> is not recommended at all use the same grlnet object at the same time.

I was more concerned about whether multiple GrlNetWc with the same cache
directory wouldn't overwrite each other's entries. The soup_cache_dump()
and soup_cache_load() call make me think that it's a bad idea to use
separate caches (and thus GrlNetWc) for a single cache directory.

So, let me know what sort of API you want, and I can try and implement
it or we can discuss further in the bug.

Cheers