Re: The documentation web

[Shaun McCance <shaunm gnome org>]

On Wed, 2007-02-28 at 15:35 -0500, Eric S. Raymond wrote:
> Here is a look at what I'm thinking about in regard to documentation
> infrastructure.  Read this with my previous post "Duck season! Rabbit season! 
> Mallard season!" in mind.
> 
> The novel idea at the center of Mallard is that documentation needs to
> be organized and styled in a particular, different, topic-centered way
> than it now is in order to to be more effective. I agree with this.  

That's the starting point, at least.  The novel ideas come
in when we start talking about how users are expected to
navigate a quagmire of topics.  There is actually extensive
research in pedagogical psychology on how users deal with
navigation and information context in hypertext learning
systems.

One of the largest problems with non-linear documents is that
readers tend to get lost.  When readers do find what they need
(frequently through search), they're isolated on one topic,
unable to use that as a starting point for further reading.
This seriously hinders exploratory learning.

In the free software world, DQA teams are hard to come by.
To ensure consistent navigation (e.g. this links there so
that links here), we need to automate as much as possible,
reducing writer redundancy wherever we can.  This is the
essence of "cross-linked up the wazoo".

> On-line docs that just read back the interface at me, and that tell me
> nothing about troubleshooting when the app behaves unexpectedly, piss 
> me off too.

This is more an issue of how we write than anything else.
You can restate the interface in DocBook, HTML, Mallard,
wiki, man, info, or any other format.  I do believe that
the format can encourage (or discourage) organization of
thoughts, but it ultimately comes down to the writer.

> But when thinking about the state of Linux documentation, I start from
> a slightly different place than Shaun McCance does.
> 
> It bothers me that serious users have to learn finger habits for as
> many as five different help browsers: a web browser, man(1), info(1),
> the Emacs text editor, a dedicated GUI help browser such as GNOME Yelp
> or the KDE Help Center, and the venerable more(1).

In terms of finger habits, man == less, info == emacs, and
Yelp == Epiphany (alternately, KDE Help Center == Konqueror).
Sure there are some differences, but they're a result of
domain-specific functionality.  Yelp's Alt+Up and Alt+Down,
for instance, don't make sense when you're reading Planet
GNOME in Epiphany.

> These are driven by
> as many as five different presentation formats (HTML, man, texinfo,
> XML, and plain text) that don't share a uniform namespace; thus,
> hotlinks that should exist don't and can't.

We *can* link from our help pages to man and info pages,
but I can't see how man and info pages are going to link
to help pages without being horribly broken when you view
them with man(1) and info(1).  Gnome doesn't have control
over what's written in most of the man and info pages on
your computer, so it's doubtful there's much we can do.

> Additionally, I think it's perverse that Linux users have better
> ability to search the external Web via Google than they do to search
> the documentation on their own systems.

This is, again, a writer problem.  We can pat ourselves
on the back all day for having built clever documentation
tools, but if writers don't write the content, we're not
worth a damn thing.  Writers are king.

> The disease of which these are symptoms is that the presentation of
> Linux documentation is sadly fragmented.  Multiple composition formats
> are not a bad thing, but having multiple *presentation* formats and
> viewers to go with each one of them them just sucks.  And the
> viewers...if it weren't for man(1) being a Hallowed Unix Tradition,
> we'd laugh at it in 2007.  And rightly so.

Yelp turns everything into HTML for the reader, so we
do have a single presentation format.  Not that readers
care what's going on behind the scenes, of course.  But
if there are problems with Yelp's transformations (which
there are), then they just need to be fixed.  Baby and
the bath water and all that jazz.

> My radical idea is that it's time to shoot man(1), info(1) *and yelp*
> through the head and present everything through the user's choice of
> web browser, with HTML as a uniform presentation format.  Actually,
> I'm not quite that radical.  We can keep yelp and the other
> specialized browsers around as backward-compatibility hacks if we're
> clever about it.

The Epiphany-as-your-help-viewer idea has come up numerous
times.  Heck, before Yelp came along, Nautilus was the help
viewer.  I've defended Yelp from this dangerous idea in the
past, and I'll do it again.  (Perhaps "dangerous idea" are
strong words, but certainly no stronger than "shoot Yelp
through the head".)

Let's look at this:

* We're not about to make Epiphany able to transform
  DocBook, Mallard, man, and info into HTML on the fly.
* Thus, we have to install the presentation format.
* Installing the presentation format is 87% of what's
  wrong with info.
* Installing the presentation format seriously hinders
  our ability to adjust the formatting for accessibility
  and localization needs.

>From another perspective:

* Epiphany provides a lot of UI that isn't necessary
  for browsing help pages.
* Epiphany doesn't provide a lot of UI that is useful
  for browsing help pages.

Yelp is not a compatibility hack.  It is an application
whose features and interface have been tuned explicitly
for the task of finding information in documentation.

> I think this plan actually dovetails pretty well with the topic-centered 
> documentation idea.  It doesn't even exclude writing a Shaun's Magic
> New Language -- as long as that's rendered to HTML for user viewing.
> 
> HTML and its tools have the advantage of being distro-independent and
> desktop-independent, too.  I'm not just after helping GNOME; I want a
> solution that any distro and any desktop can use, and a browser gets
> us a long way towards that.

Taking the least-common-denominator route doesn't sound
like pushing things forward to me.  It sounds more like
giving up.

> In my view of the infrastructure problem, the big grotty issue is not
> "how do we write new documentation?". If SNML just turns into HTML for
> presentation, almost all of the Mallard architecture just works.  The
> big question is, "how do we pull all the legacy stuff into the Web
> world?".
> 
> I've solved the biggest part of that problem.  My program doclifter 
> lifts manual-page sources to well-structured XML-DocBook.  Texinfo
> can be translated to XML-DocBook with makeinfo.  Shaun's objections
> to XML-DocBook as a composition format have some weight, but it
> makes a dandy interchange format and is easy to generate HTML from.

I'd call man pages a very small part of the problem.
Honestly, the sort of content that's generally found
in man pages is not stuff most people care about.  We
provide access to them in Yelp because, for the people
who do care, it's nice.  It's a cool feature.

But if I have an hour in the evening to work on Yelp,
and I have the choice between developing tools to help
us create a better user guide and making man pages work
a bit better, guess which one I'll choose.

> The problem I want to focus on is how to build the software that will
> check existing documentation such as manual pages into a system database,
> automatically lifting it if it's manual pages and rendering to HTML to
> be integrated into the local web.

So you want a system daemon that finds all installed
documentation in any format, converts it all to HTML,
and stores that HTML so it can be viewed in a web
browser?  That sounds like a lot of wasted space,
especially since we've been successfully doing the
transformations on the fly for years.

You are, of course, free to work on whatever you like.
Maybe it will turn out to be the next big thing, and
Gnome will jump behind it.  But at the moment, I'm not
really interested in going in that direction.  Maybe
others on the list are.

--
Shaun