Re: Improving GNOME's ldap resiliency

[Olav Vitters <olav vitters nl>]

On Fri, Sep 24, 2010 at 07:17:10AM -0700, Jeff Schroeder wrote:
> As we move to evolve the GNOME infrastructure, it has came to my attention that:
>     1.) Our ldap services sometimes go haywire and services we provide
> go with it.
>     2.) Our ldap master, label.gnome.org, does not have an ldap client
> configured due to the chicken/egg problem.
>     3.) The ldap slave in our backup (Canonical) datacenter is flaky
> causing issues with services hosted there such as damned lies[1].

#3 is different. One version of OpenLDAP used by that machine didn't
like the fact that it didn't get a full mirror (userPassword is hidden).
This made the openldap version fail to start.

I think I mentioned it before, but we should also deploy a real mirror,
but then on the Red Hat infrastructure, not outside of it.

> The sssd[2] is this nifty project written by mostly redhatters which
[..]
> Setting up sssd on our servers fixes several existing issues:
>     1.) When label goes down, users can no longer commit to gnome git.
> This would have been a much bigger issue in the svn days. Yay for
> dvcs!
>     2.) Other services on the ldap master won't have problems if their
> init script runs before ldap comes up. Example:
>             Starting httpd: httpd: bad group name bugzilla [FAILED]

Yeah, I was wondering if we should mirror those entries in /etc/passwd
via Puppet, but that just seems messy. Real cache would be much better.

[..]
> In the future, sssd will support caching ssh keys (from ldap) locally
> in it's own ldb cache. Do we want to explore this avenue or do we want
> to continue using the the create-auth scripts? If we want to entertain
> this, we should work together with upstream to integrate with our
> custom ssh key ldap schema. The developers expressed they will work by
> default with the openssh-lpk schema which we sadly do not use.

Interesting option. We need to ensure 100% that sysadmins can always
login to any machine, even if most stuff is broken. Due to create-auth,
we currently only need a running sshd. With openssh-lpk we'd also rely
on 1) sssd and 2) having the sysadmin info cached. I guess that would be
ok?

Having changes in ssh keys and new users setup be available immediately
would be pretty awesome IMO. Always a bit annoying that you have to
inform them that their account only works after a while.

Different schema would mean migrating, updating Mango, and probably
deploy the new schema to all the LDAP master and slave machines. Not
impossible, though would prefer someone finishing the Django port of
Mango for me.

-- 
Regards,
Olav