Re: [Tracker] HitsAdded, HitsRemoved and HitsModified for Xesam

[Jamie McCracken <jamiemcc blueyonder co uk>]

On Wed, 2008-04-30 at 17:14 +0200, Philip Van Hoof wrote:
> On Wed, 2008-04-30 at 11:05 -0400, Jamie McCracken wrote:
>
> > > The best mechanism is simply reusing the same mechanism that was used
> > > initially. And that is the query that we converted from the Xesam XML
> > > stuff into the SQL query used to get the GetHits/GetHitsData and fed to
> > > us during the NewSearch.
> >
> > yes of course - the live query result will be stored in a tmp table
> > along with its sql, service types and mime types affected
>
> Aha, here's the disconnect then. Can you elaborate on how this should be
> implemented (the temporary table) and why it's necessary?

we need the original results of the query (that is the URI hits) in
order to compare with the new ones. Only then can we emit the correct
signals

> > Determining if a query is affected will need to check service and mime 
> > types to see if they affect query
>
> Because we can also equally easily determine whether a live-search is
> affected by joining a table that contains "affected ServiceIDs", which
> I'd call "Events", together with Services: Only the ones in "Events"
> will be evaluated against the live-query.

yes we can do that anyhow - but service IDs are no good for newly
created items and so have to be iterated over. They only work for
existing ones

> No need to create any temporary tables, that way.
>
> We only store (and per frequency dispose) "events" and we don't copy all
> currently matching items of a live-query to a temporary table.
>
> This way, a live-query doesn't consume memory in that temporary table
> when it's just sitting there, doing nothing (and no indexing is
> happening, in which case the indexer causes the changed items between
> the last periodic handle and 'now()' to have their ServiceIDs in the
> Events table).
>
>
>
> > > Else we'll be throwing all HitsAdded, HitsRemoved and HitsModified to
> > > all live queries (because there's no way to determine whether or not the
> > > live query we're currently evaluating was affected by a specific event).
> > >
> > > So the "iterate over that list and determine if live query needs
> > > refreshing" is a hard problem to solve ;-), it's not simply.
> >
> > I know - its easy to do hard to get it optimal :)
>
> Voila :)