Yelp patches: man page speedup
- From: Brent Smith <gnome nextreality net>
- To: gnome-doc-devel-list gnome org
- Cc: federico ximian com
- Subject: Yelp patches: man page speedup
- Date: Tue, 08 Nov 2005 23:19:24 -0700
Well since everyone else is doing it, I decided to jump on the
bandwagon.
Let me apologize in advance about the length of this email:
I've been working on this for a couple weeks, so I kind of
have a lot to say.
I've have been looking at fixing the startup time for yelp when
man page support is enabled (as it is by default in Ubuntu).
During my research I found one major bottleneck in when yelp
starts up and there are a large number of man pages to index.
(In my case 40,503 - needless to say I haven't reinstalled
in quite awhile).
The bottleneck occurs in the yelp_doc_info_new function, in a
call to gnome_vfs_make_uri_from_input_with_dirs(). Everytime
this function is called, a system call is made to access the
current working directory and then an attempt to the access
the file. If you do an "strace -e file" on yelp HEAD, you
will system calls like the following:
getcwd("/extra/cvs/gnome2/yelp-head5", 4096) = 29
access("/extra/cvs/gnome2/yelp-head5/man:/usr/share/man/man1/msgcomm.1.gz",
F_OK) = -1 ENOENT (No such file or directory)
More disk access occurs in the convert_man_uri function, due
to a g_file_test() call:
stat64("/usr/share/man/man1/msgcomm.1.gz", {st_mode=S_IFREG|0644,
st_size=1648, ...}) = 0
Since we are reading all the man files from the disk anyway,
there's no reason to check for relative paths, and to see if
the file exists; we know it does - therefore, I made a change to the API
for the following functions:
-static gchar * convert_man_uri (gchar *uri);
+static gchar * convert_man_uri (gchar *uri, gboolean trust_uri);
-yelp_doc_info_new (const gchar *uri)
+yelp_doc_info_new (const gchar *uri, gboolean trust_uri)
-yelp_doc_info_get (const gchar *uri)
+yelp_doc_info_get (const gchar *uri, gboolean trust_uri)
Basically, if we know the uri we are passing exists and is an
absolute uri, we set trust_uri = TRUE; this in turn bypasses
the call to gnome_vfs_make_uri_from_input_with_dirs() with a
call to gnome_vfs_make_uri_from_input(), which doesn't
suffer the performance issues of getting the cwd and checking
for the existence of a relative path. The g_file_test()
in convert_man_uri is also bypassed.
In addition to this simple change, I've been working on a
cache patch, which on first run creates an index of man pages
in ~/.gnome2/yelp.d/manindex.xml. Any subsequent running
of yelp will use this file instead of scanning the disk. In
conjunction with the API changes above, on reboot yelp comes
up pretty quickly (knocked about 8 seconds off from my tests)
The disadvantage with the cache patch is that it doesn't
check the mtime of the directories to see if any new manpages
have been added, and recreate the index. However, I designed
the patch with this in mind, so it shouldn't be too hard to
add this functionality.
I've attached three patches to this email.
[1] yelp-head-timing.patch
[2] yelp-trust-timing.patch
[3] yelp-mancache-timing.patch
[1] simply adds a function to print timing information on
how long process_mandir_pending takes.
[2] adds the API changes discussed above and the timing print stuff
[3] is the full blown cache, API changes, and timing print stuff
I've included the results from my tests below. I've performed a
reboot before each test, and than ran yelp twice and recorded the
results. The cache + API + timing patch is definitely the best
since on reboot it doesn't have to touch the disk to create the
TOC for the man pages.
[1] yelp-head-timing.patch
[2] yelp-trust-timing.patch
[3] yelp-mancache-timing.patch (no cache at ~/.gnome2/yelp.d/manindex.xml)
[4] yelp-mancache-timing.patch (cache present at
~/.gnome2/yelp.d/manindex.xml)
Actual timings after a reboot:
[1] 1st run: elapsed: sec=9, usec=117386
2nd run: elapsed: sec=0, usec=805525
[2] 1st run: elapsed: sec=1, usec=278251
2nd run: elapsed: sec=0, usec=652525
[3] 1st run: elapsed: sec=1, usec=497169 (cache file gets created
during this run)
2nd run: elapsed: sec=0, usec=664619 (cache file exists for this run)
[4] 1st run: elapsed: sec=0, usec=766423
2nd run: elapsed: sec=0, usec=644318
All patches are against yelp HEAD.
Again sorry for the length. Any comments/testing are greatly
appreciated.
--
Brent Smith <gnome nextreality net>
IRC: smitten
Index: src/yelp-toc-pager.c
===================================================================
RCS file: /cvs/gnome/yelp/src/yelp-toc-pager.c,v
retrieving revision 1.52
diff -u -r1.52 yelp-toc-pager.c
--- src/yelp-toc-pager.c 2 Nov 2005 21:49:48 -0000 1.52
+++ src/yelp-toc-pager.c 9 Nov 2005 05:21:54 -0000
@@ -151,6 +151,39 @@
static YelpTocPager *toc_pager;
+static gboolean
+do_timing (void)
+{
+ static gboolean first_call = TRUE;
+ static struct timeval before;
+ static struct timeval after;
+
+
+ if (first_call) {
+ gettimeofday (&before, NULL);
+ first_call = FALSE;
+ } else {
+ gint sec=0, usec=0;
+
+ gettimeofday (&after, NULL);
+
+ sec = after.tv_sec - before.tv_sec;
+ if (after.tv_sec > before.tv_sec) {
+ if (after.tv_usec < before.tv_usec) {
+ usec = (1000000 - before.tv_usec) + after.tv_usec;
+ sec--;
+ } else usec = after.tv_usec - before.tv_usec;
+ } else usec = after.tv_usec - before.tv_usec;
+
+ g_print ("before : sec=%d, usec=%d\n", before.tv_sec, before.tv_usec);
+ g_print ("after : sec=%d, usec=%d\n", after.tv_sec, after.tv_usec);
+ g_print ("elapsed: sec=%d, usec=%d\n", sec, usec);
+ }
+
+ /* needed to indicate we are done processing */
+ return FALSE;
+}
+
GType
yelp_toc_pager_get_type (void)
{
@@ -356,7 +389,9 @@
process_omf_pending,
process_xslt,
#ifdef ENABLE_MAN
+ do_timing,
process_mandir_pending,
+ do_timing,
process_xslt,
#endif
#ifdef ENABLE_INFO
? test.txt
Index: src/yelp-toc-pager.c
===================================================================
RCS file: /cvs/gnome/yelp/src/yelp-toc-pager.c,v
retrieving revision 1.51
diff -u -r1.51 yelp-toc-pager.c
--- src/yelp-toc-pager.c 21 Oct 2005 01:47:56 -0000 1.51
+++ src/yelp-toc-pager.c 9 Nov 2005 05:06:52 -0000
@@ -68,11 +68,13 @@
GSList *omf_pending;
#ifdef ENABLE_MAN
- gint mandirs_i;
- gint langs_i;
- gint manpaths_i;
- gchar **manpaths;
-
+ gint manpage_count;
+ xmlNodePtr root;
+ xmlNodePtr ins;
+ xmlDocPtr manindex_xml;
+ GSList *mandir_list;
+ GSList *mandir_ptr; /* ptr to current entry in mandir_list */
+ GSList *mandir_langpath; /* ptr to current entry in mandir_ptr */
GHashTable *man_secthash;
GHashTable *man_manhash;
#endif
@@ -151,6 +153,39 @@
static YelpTocPager *toc_pager;
+static gboolean
+do_timing (void)
+{
+ static gboolean first_call = TRUE;
+ static struct timeval before;
+ static struct timeval after;
+
+
+ if (first_call) {
+ gettimeofday (&before, NULL);
+ first_call = FALSE;
+ } else {
+ gint sec=0, usec=0;
+
+ gettimeofday (&after, NULL);
+
+ sec = after.tv_sec - before.tv_sec;
+ if (after.tv_sec > before.tv_sec) {
+ if (after.tv_usec < before.tv_usec) {
+ usec = (1000000 - before.tv_usec) + after.tv_usec;
+ sec--;
+ } else usec = after.tv_usec - before.tv_usec;
+ } else usec = after.tv_usec - before.tv_usec;
+
+ g_print ("before : sec=%d, usec=%d\n", before.tv_sec, before.tv_usec);
+ g_print ("after : sec=%d, usec=%d\n", after.tv_sec, after.tv_usec);
+ g_print ("elapsed: sec=%d, usec=%d\n", sec, usec);
+ }
+
+ /* needed to indicate we are done processing */
+ return FALSE;
+}
+
GType
yelp_toc_pager_get_type (void)
{
@@ -232,7 +267,7 @@
{
YelpDocInfo *doc_info;
- doc_info = yelp_doc_info_get ("x-yelp-toc:");
+ doc_info = yelp_doc_info_get ("x-yelp-toc:", FALSE);
toc_pager = (YelpTocPager *) g_object_new (YELP_TYPE_TOC_PAGER,
"document-info", doc_info,
@@ -356,7 +391,9 @@
process_omf_pending,
process_xslt,
#ifdef ENABLE_MAN
+ do_timing,
process_mandir_pending,
+ do_timing,
process_xslt,
#endif
#ifdef ENABLE_INFO
@@ -509,7 +546,7 @@
omf_seriesid =
xmlXPathEvalExpression ("string(/omf/resource/relation/@seriesid)", omf_xpath);
- doc_info = yelp_doc_info_get (omf_url->stringval);
+ doc_info = yelp_doc_info_get (omf_url->stringval, FALSE);
if (!doc_info)
goto done;
yelp_doc_info_set_title (doc_info, omf_title->stringval);
@@ -561,23 +598,165 @@
}
#ifdef ENABLE_MAN
-static gboolean
-process_mandir_pending (YelpTocPager *pager)
+static void
+add_man_page_to_toc (YelpTocPager *pager, gchar *dirname, gchar *filename)
{
- xmlNodePtr tmp;
- gchar *manpath;
+ xmlNodePtr tmp = NULL;
+ gchar *manname = NULL;
+ gchar *mansect = NULL;
+ gchar *manman = NULL;
+ gchar *c1 = NULL;
+ gchar *c2 = NULL;
+
+ YelpTocPagerPriv *priv = YELP_TOC_PAGER (pager)->priv;
+
+ c1 = g_strrstr (filename, ".bz2");
- gchar *dirname, *filename;
- GDir *dir;
+ if (c1 && strlen (c1) != 4)
+ c1 = NULL;
- const gchar * const * langs = g_get_language_names ();
+ if (!c1) {
+ c1 = g_strrstr (filename, ".gz");
+ if (c1 && strlen (c1) != 3)
+ c1 = NULL;
+ }
+
+ if (c1)
+ c2 = g_strrstr_len (filename, c1 - filename, ".");
+ else
+ c2 = g_strrstr (filename, ".");
+
+ if (c2) {
+ manname = g_strndup (filename, c2 - filename);
+ if (c1)
+ mansect = g_strndup (c2 + 1, c1 - c2 - 1);
+ else
+ mansect = g_strdup (c2 + 1);
+ }
+
+ manman = g_strconcat (manname, ".", mansect, NULL);
+
+ if (g_hash_table_lookup (priv->man_manhash, manman) == NULL) {
+ tmp = g_hash_table_lookup (priv->man_secthash, mansect);
+
+ if (tmp == NULL && strlen (mansect) > 1) {
+ gchar *mansect0 = g_strndup (mansect, 1);
+ tmp = g_hash_table_lookup (priv->man_secthash, mansect0);
+ g_free (mansect0);
+ }
+
+ if (tmp) {
+ gchar *tooltip, *url_full, *url_short;
+ YelpDocInfo *info;
+
+ url_full = g_strconcat ("man:", dirname, "/", filename, NULL);
+ url_short = g_strconcat ("man:", manname, ".", mansect, NULL);
+ info = yelp_doc_info_get (url_full, TRUE);
+
+ if (info) {
+ yelp_doc_info_add_uri (info, url_short, YELP_URI_TYPE_MAN);
+ tmp = xmlNewChild (tmp, NULL, "doc", NULL);
+ xmlNewNsProp (tmp, NULL, "href", url_full);
+ xmlNewChild (tmp, NULL, "title", manname);
+ tooltip = g_strdup_printf (_("Read man page for %s"), manname);
+ xmlNewChild (tmp, NULL, "tooltip", tooltip);
+
+ g_free (tooltip);
+ }
+
+ g_free (url_full);
+ g_free (url_short);
+ } else {
+ d (g_warning ("Could not locate section %s for %s\n", mansect, manman));
+ }
+
+ g_hash_table_insert (priv->man_manhash, g_strdup (manman), priv);
+ }
+
+ g_free (manname);
+ g_free (mansect);
+ g_free (manman);
+}
+
+static void
+create_toc_from_index (YelpTocPager *pager, gchar *index_file)
+{
+ xmlXPathContextPtr xpath = NULL;
+ xmlXPathObjectPtr objsect = NULL;
+ xmlDocPtr manindex_xml = NULL;
+ gint i, j, k;
+
+ YelpTocPagerPriv *priv = YELP_TOC_PAGER (pager)->priv;
+
+ manindex_xml = xmlReadFile (index_file, NULL, XML_PARSE_NOBLANKS |
+ XML_PARSE_NOCDATA | XML_PARSE_NOERROR |
+ XML_PARSE_NONET);
+
+ xpath = xmlXPathNewContext (manindex_xml);
+ objsect = xmlXPathEvalExpression ("/manindex/mansect", xpath);
+
+ for (i=0; i < objsect->nodesetval->nodeNr; i++) {
+ xmlXPathObjectPtr objdirs;
+
+ if (!priv->man_manhash)
+ priv->man_manhash = g_hash_table_new_full (g_str_hash, g_str_equal,
+ g_free, NULL);
+
+ xpath->node = objsect->nodesetval->nodeTab[i];
+ objdirs = xmlXPathEvalExpression ("dir", xpath);
+
+ for (j=0; j < objdirs->nodesetval->nodeNr; j++) {
+ xmlXPathObjectPtr objdirname;
+ xmlXPathObjectPtr objmanpages;
+ xmlNodePtr dirnode = objdirs->nodesetval->nodeTab[j];
+ xmlNodePtr namenode = NULL;
+ xmlChar *dirname = NULL;
+
+ xpath->node = dirnode;
+ objdirname = xmlXPathEvalExpression ("name[1]", xpath);
+
+ namenode = objdirname->nodesetval->nodeTab[0];
+ dirname = xmlNodeListGetString (manindex_xml,
+ namenode->xmlChildrenNode, 1);
+
+ objmanpages = xmlXPathEvalExpression ("page", xpath);
+
+ for (k=0; k < objmanpages->nodesetval->nodeNr; k++) {
+ xmlNodePtr node = objmanpages->nodesetval->nodeTab[k];
+ xmlChar *manpage = NULL;
+
+ manpage = xmlNodeListGetString (manindex_xml,
+ node->xmlChildrenNode, 1);
+
+ add_man_page_to_toc (pager, dirname, manpage);
+ priv->manpage_count++;
+ }
+ }
+
+ if (priv->man_manhash) {
+ g_hash_table_destroy (priv->man_manhash);
+ priv->man_manhash = NULL;
+ }
+ }
+
+ return;
+}
+
+static gboolean
+process_mandir_pending (YelpTocPager *pager)
+{
+ static gchar *index_file = NULL;
+ gchar *filename = NULL;
+ gchar *dirname = NULL;
+ GDir *dir = NULL;
+ gint i, j, k;
YelpTocPagerPriv *priv = YELP_TOC_PAGER (pager)->priv;
if (!priv->toc_doc) {
xmlXPathContextPtr xpath;
xmlXPathObjectPtr obj;
- gint i;
+
priv->toc_doc = xmlCtxtReadFile (priv->parser, DATADIR "/yelp/man.xml", NULL,
XML_PARSE_NOBLANKS | XML_PARSE_NOCDATA |
XML_PARSE_NOENT | XML_PARSE_NOERROR |
@@ -586,6 +765,7 @@
xpath = xmlXPathNewContext (priv->toc_doc);
obj = xmlXPathEvalExpression ("//toc", xpath);
+
for (i = 0; i < obj->nodesetval->nodeNr; i++) {
xmlNodePtr node = obj->nodesetval->nodeTab[i];
xmlChar *sect = xmlGetProp (node, "sect");
@@ -595,143 +775,141 @@
xml_trim_titles (node);
}
}
+
+ if (!index_file)
+ index_file = g_build_filename (yelp_dot_dir(), "manindex.xml", NULL);
- if (priv->langs_i == 0 && priv->manpaths_i == 0) {
- if (priv->man_manhash)
- g_hash_table_destroy (priv->man_manhash);
- priv->man_manhash = g_hash_table_new_full (g_str_hash, g_str_equal,
- g_free, NULL);
- }
-
- if (!priv->manpaths) {
- if (!g_spawn_command_line_sync ("manpath", &manpath, NULL, NULL, NULL))
- manpath = g_strdup (g_getenv ("MANPATH"));
-
- if (manpath) {
- g_strstrip (manpath);
- priv->manpaths = g_strsplit (manpath, G_SEARCHPATH_SEPARATOR_S, -1);
- g_free (manpath);
+ /* On first call to this function, create a list of directories that we
+ * need to process: we actually make a linked list (mandir_list) whose
+ * members who are linked lists containing directories for a particular
+ * man directory such as "man1", "man2", etc.. We do this because we
+ * need a separate hash for every mandir */
+ if (!priv->mandir_list) {
+ const gchar * const * langs = g_get_language_names ();
+ gchar *manpath = NULL;
+ gchar **manpaths = NULL;
+
+ /* check for the existence of the xml cache file in ~/.gnome2/yelp.d/
+ * if it exists, use it as the source for man pages instead of
+ * searching the hard disk for them - should make startup much faster */
+ if (g_file_test (index_file, G_FILE_TEST_EXISTS)) {
+
+ create_toc_from_index (pager, index_file);
+
+ /* we are done.. */
+ g_print ("number of manpages added to toc: %d\n", priv->manpage_count);
+ return FALSE;
} else {
- goto done;
- }
- }
-
- if (g_str_equal (langs[priv->langs_i], "C"))
- dirname = g_build_filename (priv->manpaths[priv->manpaths_i],
- mandirs[priv->mandirs_i],
- NULL);
- else
- dirname = g_build_filename (priv->manpaths[priv->manpaths_i],
- langs[priv->langs_i],
- mandirs[priv->mandirs_i],
- NULL);
- dir = g_dir_open (dirname, 0, NULL);
- if (dir) {
- while ((filename = (gchar *) g_dir_read_name (dir))) {
- gchar *c1 = NULL, *c2 = NULL, *manname, *mansect, *manman;
-
- c1 = g_strrstr (filename, ".bz2");
- if (c1 && strlen (c1) != 4)
- c1 = NULL;
-
- if (!c1) {
- c1 = g_strrstr (filename, ".gz");
- if (c1 && strlen (c1) != 3)
- c1 = NULL;
- }
-
- if (c1)
- c2 = g_strrstr_len (filename, c1 - filename, ".");
- else
- c2 = g_strrstr (filename, ".");
-
- if (c2) {
- manname = g_strndup (filename, c2 - filename);
- if (c1)
- mansect = g_strndup (c2 + 1, c1 - c2 - 1);
- else
- mansect = g_strdup (c2 + 1);
- } else {
- mansect = g_strdup (mandirs[priv->mandirs_i] + 3);
- if (c1)
- manname = g_strndup (filename, c1 - filename);
- else
- manname = g_strdup (filename);
+ priv->manindex_xml = xmlNewDoc (BAD_CAST "1.0");
+ priv->root = xmlNewNode (NULL, BAD_CAST "manindex");
+ priv->ins = priv->root;
+
+ xmlDocSetRootElement (priv->manindex_xml, priv->root);
+
+ xmlAddChild (priv->root, xmlNewText ("\n "));
+
+ if (!g_spawn_command_line_sync ("manpath", &manpath, NULL, NULL, NULL))
+ manpath = g_strdup (g_getenv ("MANPATH"));
+
+ if (!manpath) {
+ manpath = g_strdup ("/usr/share/man");
}
+
+ g_strstrip (manpath);
+ manpaths = g_strsplit (manpath, G_SEARCHPATH_SEPARATOR_S, -1);
+ g_free (manpath);
+
+ for (i=0; mandirs[i] != NULL; i++) {
+ GSList *tmplist = NULL;
+
+ for (j=0; langs[j] != NULL; j++) {
+ for (k=0; manpaths[k] != NULL; k++) {
+ if (g_str_equal (langs[j], "C"))
+ dirname = g_build_filename (manpaths[k], mandirs[i], NULL);
+ else
+ dirname = g_build_filename (manpaths[k], langs[j],
+ mandirs[i],
+ NULL);
- manman = g_strconcat (manname, ".", mansect, NULL);
-
- if (g_hash_table_lookup (priv->man_manhash, manman) == NULL) {
- tmp = g_hash_table_lookup (priv->man_secthash, mansect);
- if (tmp == NULL && strlen (mansect) > 1) {
- gchar *mansect0 = g_strndup (mansect, 1);
- tmp = g_hash_table_lookup (priv->man_secthash, mansect0);
- }
-
- if (tmp) {
- gchar *tooltip, *url_full, *url_short;
- YelpDocInfo *info;
-
- url_full = g_strconcat ("man:", dirname, "/", filename, NULL);
- url_short = g_strconcat ("man:", manname, ".", mansect, NULL);
- info = yelp_doc_info_get (url_full);
-
- if (info) {
- yelp_doc_info_add_uri (info, url_short, YELP_URI_TYPE_MAN);
- tmp = xmlNewChild (tmp, NULL, "doc", NULL);
- xmlNewNsProp (tmp, NULL, "href", url_full);
-
- xmlNewChild (tmp, NULL, "title", manname);
- tooltip = g_strdup_printf (_("Read man page for %s"), manname);
- xmlNewChild (tmp, NULL, "tooltip", tooltip);
- g_free (tooltip);
+ tmplist = g_slist_prepend (tmplist, dirname);
}
-
- g_free (url_full);
- g_free (url_short);
- } else {
- g_warning ("Could not locate section %s for %s\n",
- mansect, manman);
}
-
- g_hash_table_insert (priv->man_manhash, g_strdup (manman), priv);
+
+ priv->mandir_list = g_slist_prepend (priv->mandir_list, tmplist);
}
- g_free (manman);
- g_free (manname);
- g_free (mansect);
- }
- g_dir_close (dir);
- }
-
- done:
- g_free (dirname);
-
- if (priv->manpaths) {
- priv->manpaths_i++;
- if (priv->manpaths[priv->manpaths_i] == NULL) {
- priv->manpaths_i = 0;
- priv->langs_i++;
- }
- }
- if (langs[priv->langs_i] == NULL) {
- priv->langs_i = 0;
- priv->mandirs_i++;
- if (priv->man_manhash) {
- g_hash_table_destroy (priv->man_manhash);
- priv->man_manhash = NULL;
+ priv->mandir_ptr = priv->mandir_list;
+ if (priv->mandir_list && priv->mandir_list->data) {
+ priv->mandir_langpath = priv->mandir_list->data;
+ }
}
}
- if (mandirs[priv->mandirs_i] == NULL) {
- if (priv->manpaths) {
- g_strfreev (priv->manpaths);
- priv->manpaths = NULL;
+ /* iterate through our previously created linked lists and create the
+ * table of contents from them */
+ else {
+ if (!priv->man_manhash) {
+ priv->man_manhash = g_hash_table_new_full (g_str_hash, g_str_equal,
+ g_free, NULL);
+
+ g_print ("adding section\n");
+ priv->ins = xmlNewChild (priv->root, NULL, "mansect", NULL);
+ xmlAddChild (priv->ins, xmlNewText ("\n "));
}
- if (priv->man_secthash) {
- g_hash_table_destroy (priv->man_secthash);
- priv->man_secthash = NULL;
+
+ if (priv->mandir_langpath && priv->mandir_langpath->data) {
+ dirname = priv->mandir_langpath->data;
+
+ if ((dir = g_dir_open (dirname, 0, NULL))) {
+ g_print ("processing dir = %s\n", dirname);
+
+ priv->ins = xmlNewChild (priv->ins, NULL, "dir", NULL);
+ xmlAddChild (priv->ins, xmlNewText ("\n "));
+ xmlNewChild (priv->ins, NULL, "name", dirname);
+ xmlAddChild (priv->ins, xmlNewText ("\n "));
+ xmlAddChild (priv->ins->parent, xmlNewText ("\n "));
+
+ while ((filename = (gchar *) g_dir_read_name (dir))) {
+
+ xmlNewChild (priv->ins, NULL, "page", filename);
+ xmlAddChild (priv->ins, xmlNewText ("\n "));
+
+ add_man_page_to_toc (pager, dirname, filename);
+ priv->manpage_count++;
+ }
+
+ priv->ins = priv->ins->parent;
+
+ g_dir_close (dir);
+ }
+
+ priv->mandir_langpath = g_slist_next (priv->mandir_langpath);
+
+ } else {
+ priv->mandir_ptr = g_slist_next (priv->mandir_ptr);
+
+ if (priv->mandir_ptr && priv->mandir_ptr->data) {
+ priv->mandir_langpath = priv->mandir_ptr->data;
+
+ if (priv->man_manhash) {
+ g_hash_table_destroy (priv->man_manhash);
+ priv->man_manhash = NULL;
+ }
+ } else {
+ /* FIXME: free mandir_list and mandir_langpath */
+ FILE *newindex = NULL;
+
+ if (!(newindex = g_fopen (index_file, "w")))
+ g_warning ("unable to create '%s'\n", index_file);
+ else {
+ xmlDocDump (newindex, priv->manindex_xml);
+ fclose (newindex);
+ }
+
+ /* done processing */
+ g_print ("number of manpages added to toc: %d\n", priv->manpage_count);
+ return FALSE;
+ }
}
- return FALSE;
}
return TRUE;
Index: src/yelp-utils.c
===================================================================
RCS file: /cvs/gnome/yelp/src/yelp-utils.c,v
retrieving revision 1.25
diff -u -r1.25 yelp-utils.c
--- src/yelp-utils.c 22 Feb 2005 23:44:28 -0000 1.25
+++ src/yelp-utils.c 9 Nov 2005 05:06:52 -0000
@@ -17,7 +17,8 @@
* Free Software Foundation, Inc., 59 Temple Place - Suite 330,
* Boston, MA 02111-1307, USA.
*
- * Author: Shaun McCance <shaunm gnome org>
+ * Authors: Shaun McCance <shaunm gnome org>
+ * Brent Smith <gnome nextreality net>
*/
#ifdef HAVE_CONFIG_H
@@ -29,6 +30,7 @@
#include <libgnomevfs/gnome-vfs.h>
#include <libgnomevfs/gnome-vfs-mime-utils.h>
#include <libgnome/gnome-program.h>
+#include <libgnome/gnome-init.h>
#include "yelp-utils.h"
@@ -69,11 +71,34 @@
static YelpDocType get_doc_type (gchar *uri);
static gchar * convert_ghelp_uri (gchar *uri);
-static gchar * convert_man_uri (gchar *uri);
+static gchar * convert_man_uri (gchar *uri, gboolean trust_uri);
static gchar * convert_info_uri (gchar *uri);
+static gchar *dot_dir = NULL;
+
+const char *
+yelp_dot_dir (void)
+{
+ if (dot_dir == NULL) {
+ dot_dir = g_build_filename (g_get_home_dir(), GNOME_DOT_GNOME,
+ "yelp.d", NULL);
+
+ if (!g_file_test (dot_dir, G_FILE_TEST_EXISTS | G_FILE_TEST_IS_DIR))
+ mkdir (dot_dir, 0750);
+ }
+
+ return dot_dir;
+}
+
+/* @uri: the uri to interpret for the new YelpDocInfo struct
+ * @trust_uri: if the uri is absolute and is known to exist,
+ * then this should be set. Only makes sense for local
+ * files. This is here for performance reasons, adding
+ * 40,000 man pages and accessing the disk for each one
+ * can get pretty expensive.
+ */
YelpDocInfo *
-yelp_doc_info_new (const gchar *uri)
+yelp_doc_info_new (const gchar *uri, gboolean trust_uri)
{
YelpDocInfo *doc;
gchar *doc_uri = NULL;
@@ -87,10 +112,13 @@
d (g_print ("yelp_doc_info_new\n"));
d (g_print (" uri = \"%s\"\n", uri));
- full_uri =
+ if (trust_uri)
+ full_uri = gnome_vfs_make_uri_from_input (uri);
+ else
+ full_uri =
gnome_vfs_make_uri_from_input_with_dirs (uri,
GNOME_VFS_MAKE_URI_DIR_CURRENT);
-
+
if (g_str_has_prefix (full_uri, "file:")) {
if ((cur = strchr (full_uri, '#')))
doc_uri = g_strndup (full_uri, cur - full_uri);
@@ -107,7 +135,7 @@
uri_type = YELP_URI_TYPE_GHELP;
}
else if (g_str_has_prefix (full_uri, "man:")) {
- doc_uri = convert_man_uri (full_uri);
+ doc_uri = convert_man_uri (full_uri, trust_uri);
doc_type = YELP_DOC_TYPE_MAN;
uri_type = YELP_URI_TYPE_MAN;
}
@@ -152,7 +180,7 @@
}
YelpDocInfo *
-yelp_doc_info_get (const gchar *uri)
+yelp_doc_info_get (const gchar *uri, gboolean trust_uri)
{
YelpDocInfo *doc;
gint i;
@@ -182,7 +210,7 @@
doc = (YelpDocInfo *) g_hash_table_lookup (doc_info_table, doc_uri);
if (!doc) {
- doc = yelp_doc_info_new (doc_uri);
+ doc = yelp_doc_info_new (doc_uri, trust_uri);
if (doc && doc->type != YELP_DOC_TYPE_EXTERNAL) {
YelpDocInfo *old_doc;
@@ -682,7 +710,7 @@
}
static gchar *
-convert_man_uri (gchar *uri)
+convert_man_uri (gchar *uri, gboolean trust_uri)
{
gchar *path, *cur;
gchar *doc_uri = NULL;
@@ -708,7 +736,9 @@
/* An absolute file path after man: */
if (path[0] == '/') {
- if (g_file_test (path, G_FILE_TEST_IS_REGULAR))
+ if (trust_uri)
+ doc_uri = g_strconcat ("file://", path, NULL);
+ else if (g_file_test (path, G_FILE_TEST_IS_REGULAR))
doc_uri = g_strconcat ("file://", path, NULL);
goto done;
}
Index: src/yelp-utils.h
===================================================================
RCS file: /cvs/gnome/yelp/src/yelp-utils.h,v
retrieving revision 1.11
diff -u -r1.11 yelp-utils.h
--- src/yelp-utils.h 22 Feb 2005 19:17:01 -0000 1.11
+++ src/yelp-utils.h 9 Nov 2005 05:06:52 -0000
@@ -94,8 +94,11 @@
gchar *toc_id;
};
-YelpDocInfo * yelp_doc_info_new (const gchar *uri);
-YelpDocInfo * yelp_doc_info_get (const gchar *uri);
+const char * yelp_dot_dir (void);
+YelpDocInfo * yelp_doc_info_new (const gchar *uri,
+ gboolean trust_uri);
+YelpDocInfo * yelp_doc_info_get (const gchar *uri,
+ gboolean trust_uri);
void yelp_doc_info_add_uri (YelpDocInfo *doc_info,
const gchar *uri,
YelpURIType type);
Index: src/yelp-window.c
===================================================================
RCS file: /cvs/gnome/yelp/src/yelp-window.c,v
retrieving revision 1.177
diff -u -r1.177 yelp-window.c
--- src/yelp-window.c 15 Oct 2005 15:31:10 -0000 1.177
+++ src/yelp-window.c 9 Nov 2005 05:06:54 -0000
@@ -728,7 +728,7 @@
priv = window->priv;
- doc_info = yelp_doc_info_get (uri);
+ doc_info = yelp_doc_info_get (uri, FALSE);
if (!doc_info) {
GError *error = NULL;
g_set_error (&error, YELP_ERROR, YELP_ERROR_NO_DOC,
Index: src/yelp-toc-pager.c
===================================================================
RCS file: /cvs/gnome/yelp/src/yelp-toc-pager.c,v
retrieving revision 1.52
diff -u -r1.52 yelp-toc-pager.c
--- src/yelp-toc-pager.c 2 Nov 2005 21:49:48 -0000 1.52
+++ src/yelp-toc-pager.c 9 Nov 2005 05:20:07 -0000
@@ -151,6 +151,39 @@
static YelpTocPager *toc_pager;
+static gboolean
+do_timing (void)
+{
+ static gboolean first_call = TRUE;
+ static struct timeval before;
+ static struct timeval after;
+
+
+ if (first_call) {
+ gettimeofday (&before, NULL);
+ first_call = FALSE;
+ } else {
+ gint sec=0, usec=0;
+
+ gettimeofday (&after, NULL);
+
+ sec = after.tv_sec - before.tv_sec;
+ if (after.tv_sec > before.tv_sec) {
+ if (after.tv_usec < before.tv_usec) {
+ usec = (1000000 - before.tv_usec) + after.tv_usec;
+ sec--;
+ } else usec = after.tv_usec - before.tv_usec;
+ } else usec = after.tv_usec - before.tv_usec;
+
+ g_print ("before : sec=%d, usec=%d\n", before.tv_sec, before.tv_usec);
+ g_print ("after : sec=%d, usec=%d\n", after.tv_sec, after.tv_usec);
+ g_print ("elapsed: sec=%d, usec=%d\n", sec, usec);
+ }
+
+ /* needed to indicate we are done processing */
+ return FALSE;
+}
+
GType
yelp_toc_pager_get_type (void)
{
@@ -232,7 +265,7 @@
{
YelpDocInfo *doc_info;
- doc_info = yelp_doc_info_get ("x-yelp-toc:");
+ doc_info = yelp_doc_info_get ("x-yelp-toc:", FALSE);
toc_pager = (YelpTocPager *) g_object_new (YELP_TYPE_TOC_PAGER,
"document-info", doc_info,
@@ -356,7 +389,9 @@
process_omf_pending,
process_xslt,
#ifdef ENABLE_MAN
+ do_timing,
process_mandir_pending,
+ do_timing,
process_xslt,
#endif
#ifdef ENABLE_INFO
@@ -509,7 +544,7 @@
omf_seriesid =
xmlXPathEvalExpression (BAD_CAST "string(/omf/resource/relation/@seriesid)", omf_xpath);
- doc_info = yelp_doc_info_get ((const gchar *) omf_url->stringval);
+ doc_info = yelp_doc_info_get ((const gchar *) omf_url->stringval, FALSE);
if (!doc_info)
goto done;
yelp_doc_info_set_title (doc_info, (gchar *) omf_title->stringval);
@@ -675,7 +710,7 @@
url_full = g_strconcat ("man:", dirname, "/", filename, NULL);
url_short = g_strconcat ("man:", manname, ".", mansect, NULL);
- info = yelp_doc_info_get (url_full);
+ info = yelp_doc_info_get (url_full, TRUE);
if (info) {
yelp_doc_info_add_uri (info, url_short, YELP_URI_TYPE_MAN);
Index: src/yelp-utils.c
===================================================================
RCS file: /cvs/gnome/yelp/src/yelp-utils.c,v
retrieving revision 1.27
diff -u -r1.27 yelp-utils.c
--- src/yelp-utils.c 2 Nov 2005 21:49:48 -0000 1.27
+++ src/yelp-utils.c 9 Nov 2005 05:20:07 -0000
@@ -17,7 +17,8 @@
* Free Software Foundation, Inc., 59 Temple Place - Suite 330,
* Boston, MA 02111-1307, USA.
*
- * Author: Shaun McCance <shaunm gnome org>
+ * Authors: Shaun McCance <shaunm gnome org>
+ * Brent Smith <gnome nextreality net>
*/
#ifdef HAVE_CONFIG_H
@@ -29,6 +30,7 @@
#include <libgnomevfs/gnome-vfs.h>
#include <libgnomevfs/gnome-vfs-mime-utils.h>
#include <libgnome/gnome-program.h>
+#include <libgnome/gnome-init.h>
#include "yelp-utils.h"
@@ -69,11 +71,34 @@
static YelpDocType get_doc_type (gchar *uri);
static gchar * convert_ghelp_uri (gchar *uri);
-static gchar * convert_man_uri (gchar *uri);
+static gchar * convert_man_uri (gchar *uri, gboolean trust_uri);
static gchar * convert_info_uri (gchar *uri);
+static gchar *dot_dir = NULL;
+
+const char *
+yelp_dot_dir (void)
+{
+ if (dot_dir == NULL) {
+ dot_dir = g_build_filename (g_get_home_dir(), GNOME_DOT_GNOME,
+ "yelp.d", NULL);
+
+ if (!g_file_test (dot_dir, G_FILE_TEST_EXISTS | G_FILE_TEST_IS_DIR))
+ mkdir (dot_dir, 0750);
+ }
+
+ return dot_dir;
+}
+
+/* @uri: the uri to interpret for the new YelpDocInfo struct
+ * @trust_uri: if the uri is absolute and is known to exist,
+ * then this should be set. Only makes sense for local
+ * files. This is here for performance reasons, adding
+ * 40,000 man pages and accessing the disk for each one
+ * can get pretty expensive.
+ */
YelpDocInfo *
-yelp_doc_info_new (const gchar *uri)
+yelp_doc_info_new (const gchar *uri, gboolean trust_uri)
{
YelpDocInfo *doc;
gchar *doc_uri = NULL;
@@ -87,10 +112,13 @@
d (g_print ("yelp_doc_info_new\n"));
d (g_print (" uri = \"%s\"\n", uri));
- full_uri =
+ if (trust_uri)
+ full_uri = gnome_vfs_make_uri_from_input (uri);
+ else
+ full_uri =
gnome_vfs_make_uri_from_input_with_dirs (uri,
GNOME_VFS_MAKE_URI_DIR_CURRENT);
-
+
if (g_str_has_prefix (full_uri, "file:")) {
if ((cur = strchr (full_uri, '#')))
doc_uri = g_strndup (full_uri, cur - full_uri);
@@ -107,7 +135,7 @@
uri_type = YELP_URI_TYPE_GHELP;
}
else if (g_str_has_prefix (full_uri, "man:")) {
- doc_uri = convert_man_uri (full_uri);
+ doc_uri = convert_man_uri (full_uri, trust_uri);
doc_type = YELP_DOC_TYPE_MAN;
uri_type = YELP_URI_TYPE_MAN;
}
@@ -157,7 +185,7 @@
}
YelpDocInfo *
-yelp_doc_info_get (const gchar *uri)
+yelp_doc_info_get (const gchar *uri, gboolean trust_uri)
{
YelpDocInfo *doc;
gint i;
@@ -187,7 +215,7 @@
doc = (YelpDocInfo *) g_hash_table_lookup (doc_info_table, doc_uri);
if (!doc) {
- doc = yelp_doc_info_new (doc_uri);
+ doc = yelp_doc_info_new (doc_uri, trust_uri);
if (doc && doc->type != YELP_DOC_TYPE_EXTERNAL) {
YelpDocInfo *old_doc = NULL;
@@ -687,7 +715,7 @@
}
static gchar *
-convert_man_uri (gchar *uri)
+convert_man_uri (gchar *uri, gboolean trust_uri)
{
gchar *path, *cur;
gchar *doc_uri = NULL;
@@ -713,7 +741,9 @@
/* An absolute file path after man: */
if (path[0] == '/') {
- if (g_file_test (path, G_FILE_TEST_IS_REGULAR))
+ if (trust_uri)
+ doc_uri = g_strconcat ("file://", path, NULL);
+ else if (g_file_test (path, G_FILE_TEST_IS_REGULAR))
doc_uri = g_strconcat ("file://", path, NULL);
goto done;
}
Index: src/yelp-utils.h
===================================================================
RCS file: /cvs/gnome/yelp/src/yelp-utils.h,v
retrieving revision 1.12
diff -u -r1.12 yelp-utils.h
--- src/yelp-utils.h 28 Oct 2005 20:37:37 -0000 1.12
+++ src/yelp-utils.h 9 Nov 2005 05:20:07 -0000
@@ -93,8 +93,11 @@
gchar *toc_id;
};
-YelpDocInfo * yelp_doc_info_new (const gchar *uri);
-YelpDocInfo * yelp_doc_info_get (const gchar *uri);
+const char * yelp_dot_dir (void);
+YelpDocInfo * yelp_doc_info_new (const gchar *uri,
+ gboolean trust_uri);
+YelpDocInfo * yelp_doc_info_get (const gchar *uri,
+ gboolean trust_uri);
void yelp_doc_info_add_uri (YelpDocInfo *doc_info,
const gchar *uri,
YelpURIType type);
Index: src/yelp-window.c
===================================================================
RCS file: /cvs/gnome/yelp/src/yelp-window.c,v
retrieving revision 1.180
diff -u -r1.180 yelp-window.c
--- src/yelp-window.c 2 Nov 2005 21:49:49 -0000 1.180
+++ src/yelp-window.c 9 Nov 2005 05:20:09 -0000
@@ -732,7 +732,7 @@
priv = window->priv;
- doc_info = yelp_doc_info_get (uri);
+ doc_info = yelp_doc_info_get (uri, FALSE);
if (!doc_info) {
GError *error = NULL;
g_set_error (&error, YELP_ERROR, YELP_ERROR_NO_DOC,
[
Date Prev][
Date Next] [
Thread Prev][
Thread Next]
[
Thread Index]
[
Date Index]
[
Author Index]
