On Wed, 2008-01-23 at 16:39 -0500, Colin Walters wrote: > Hi, > > Over the last few days I've been trying to design and code a new polling > system for Mugshot. The main goal is to reduce the database traffic > we're currently doing now checking for updates. This in turn should > improve the Online Desktop server by helping it stay up longer. > > It's called Firehose. The current design page is here: > http://developer.mugshot.org/wiki/Firehose > > I'll commit some code soon to the Mugshot SVN. Some questions: - How are "tasksets" sent from the master to the slave? - How does a slave get details about information used to poll a task? (like the URL to poll or whatever) The wiki page only describes a task as a family/id pair. - How is data needed for polling like private keys distributed to the slave? - Is there any affinity for tasks? Do we always execute the same task on the same slave or each run of each task or is each run assigned to a task independently? - Do we have any way of sending If-Modified headers when applicable? If we are running this on EC2, we'll be paying per gigabyte of downloaded data. - Is the list of tasks persistently saved on the master, or does the server send tasks again on restart? - If we wanted to "sync up" the set of tasks that the master is executing with the set that we should be running (I could imagine them getting out of sync for various reasons if we keep the tasks around persistently), how do we do it? - Does the master implement "poll tasks faster after changes"? Thanks, - Owen
Attachment:
signature.asc
Description: This is a digitally signed message part