From muellerw@pc7143.unige.ch Wed Mar 14 17:29:16 2001 Return-Path: Delivered-To: gnome-kde-list@gnome.org Received: from pc7143.unige.ch (pc7143.unige.ch [129.194.71.43]) by mail.gnome.org (Postfix) with ESMTP id B2BEF2BE13 for ; Wed, 14 Mar 2001 17:29:15 -0500 (EST) Received: from muellerw by pc7143.unige.ch with local (Exim 3.12 #1 (Debian)) id 14dJuz-0007Rd-00; Wed, 14 Mar 2001 23:38:57 +0100 From: Wolfgang Mueller MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15023.62209.390726.580552@pc7143.unige.ch> Date: Wed, 14 Mar 2001 23:38:57 +0100 (CET) To: gnome-kde-list@gnome.org Subject: FdL a Project for Intelligence for the Free Desktop X-Mailer: VM 6.72 under 21.1 (patch 10) "Capitol Reef" XEmacs Lucid Sender: gnome-kde-list-admin@gnome.org Errors-To: gnome-kde-list-admin@gnome.org X-BeenThere: gnome-kde-list@gnome.org X-Loop: gnome-kde-list@gnome.org X-Mailman-Version: 2.0beta5 Precedence: bulk List-Id: GNOME/KDE interoperability Hi, Fer-de-Lance is a project for building a truly content-aware desktop environment, i.e. a finder that really helps you find any kind of files/images etc. . This is the first step towards an intelligent desktop. The link http://muellerw.free.fr/ points to the foundation document of the Fer-de-Lance project, and describes how to combine and extend GNOME/KDE and the GNU Image-Finding Tool to build something new and exciting. I would like FdL to work on GNOME and KDE, and possibly other desktops, so I guess this is the place to think about how to achieve this. Cheers, Wolfgang -- Wolfgang Müller, assistant-doctorant == PhD student (2001), teaching assistant Personal page: http://cui.unige.ch/~vision/members/WolfgangMueller.html Maintainer, GNU Image Finding Tool (http://www.gnu.org/software/gift) From dank@alumni.caltech.edu Sun Mar 18 02:46:53 2001 Return-Path: Delivered-To: gnome-kde-list@gnome.org Received: from lsmls02.we.mediaone.net (lsmls02.we.mediaone.net [24.130.1.15]) by mail.gnome.org (Postfix) with ESMTP id 1E3A52DC73 for ; Sun, 18 Mar 2001 02:46:53 -0500 (EST) Received: from alumni.caltech.edu (we-24-130-77-137.we.mediaone.net [24.130.77.137]) by lsmls02.we.mediaone.net (8.11.1/8.11.1) with ESMTP id f2I7kfS19237; Sat, 17 Mar 2001 23:46:41 -0800 (PST) Message-ID: <3AB468D1.6AA56541@alumni.caltech.edu> Date: Sat, 17 Mar 2001 23:50:41 -0800 From: Dan Kegel Reply-To: dank@alumni.caltech.edu X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.14-5.0 i686) X-Accept-Language: en MIME-Version: 1.0 To: Peter Godman , gnome-kde-list@gnome.org Subject: re: RealPlayer 8 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: gnome-kde-list-admin@gnome.org Errors-To: gnome-kde-list-admin@gnome.org X-BeenThere: gnome-kde-list@gnome.org X-Loop: gnome-kde-list@gnome.org X-Mailman-Version: 2.0beta5 Precedence: bulk List-Id: GNOME/KDE interoperability Hi Peter, I saw your post (http://lists.kde.org/?l=kfm-devel&m=98355172614068&w=2) and wanted to encourage you to join the gnome-kde-list mailing list, which is a good place to bring up integration issues like the ones you mention (at least if you want to interoperate not just with kde but also with gnome). See http://mail.gnome.org/mailman/listinfo/gnome-kde-list for more info. - Dan From muellerw@pc7143.unige.ch Mon Mar 19 10:01:01 2001 Return-Path: Delivered-To: gnome-kde-list@gnome.org Received: from pc7143.unige.ch (pc7143.unige.ch [129.194.71.43]) by mail.gnome.org (Postfix) with ESMTP id A5BB72BAF1 for ; Mon, 19 Mar 2001 10:01:00 -0500 (EST) Received: from muellerw by pc7143.unige.ch with local (Exim 3.12 #1 (Debian)) id 14f1Ja-0000Am-00 for ; Mon, 19 Mar 2001 16:11:22 +0100 From: Wolfgang Mueller MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15030.8600.500587.508056@pc7143.unige.ch> Date: Mon, 19 Mar 2001 16:11:20 +0100 (CET) To: gnome-kde-list@gnome.org Subject: FdL, some interesting questions from Rebecca X-Mailer: VM 6.72 under 21.1 (patch 10) "Capitol Reef" XEmacs Lucid Sender: gnome-kde-list-admin@gnome.org Errors-To: gnome-kde-list-admin@gnome.org X-BeenThere: gnome-kde-list@gnome.org X-Loop: gnome-kde-list@gnome.org X-Mailman-Version: 2.0beta5 Precedence: bulk List-Id: GNOME/KDE interoperability Hi, >>>>> "RS" == Rebecca Schulman writes: RS> It would be cool if you could explain what technologies you RS> already have developed, and some others I haven't heard of. RS> If it's not clear, I guess knowing more about your expectations RS> of how technologies you've been talking about would fit RS> into/create a the larger interface you've described. OK pretty big question. RS> Specifically, what is the GIFT? What does it do, how does it work, why does it RS> work? The GIFT is the GNU Image Finding Tool. It extracts automatically visual features from images, and translates each image into a chain of integers (which are IDs of the visual features) with associated floats (which are weights). The resulting documents are structurally equivalent to text (I call them pseudo-words making a pseudo-text). This permits us to index them like text in an inverted file using statistical weighting methods (those of you who do not know what's an inverted file, just retain that this is THE way of indexing text). Now we can take any image, do the same feature extraction on it, and use this as an inverted file query by example. Note: you give an example image, you retrieve visually similar images, which are likely to be semantically similar to the query. We can also give feedback by marking some of the result images as positive or negative. This works for some stuff pretty well, for some stuff not so well, in any case, this is more flexible than fixed hierarchies, and in many (most?) cases better than looking sifting the collection by hand. [Just as an aside: credits for the features-to-inverted-file idea go to David Squire, my ex boss] RS> Also, what is MRML? As content-based image retrieval (this is the term for what the GIFT is doing) is an area of active research, there is the challenge to find a client-server communication protocol that is apt to be extended in unforeseen directions. MRML is an XML-based communication protocol designed with this purpose in mind. MRML is an XML-wrapper for about all kinds of query languages you might imagine. In the future, MRML is likely to be used by quite a number of CBIRS researchers. We are just starting to get partners, so I think things are well under way. The advantage for FdL in using MRML would be, that if we make a really good and flexible query engine, this engine could be compared to other systems easily, and in the same way, it would be easy to incorporate "foreign" query engines. In the short term: the GIFT already uses MRML, so why not keep that? RS> You are right to see that I haven't been very interested RS> in statistical searching methods. I have done some work with RS> them, and my undergraduate thesis was actually on using statistical RS> language information from corpora to find relevant documents. RS> I'm not philosophically opposed to them, but I want to create RS> technology that is easy to use, and one of the criteria of this, RS> in my opinion, is that its behavior be something that you can RS> become more familiar with over time. I haven't really RS> seen statistical methods that meet this criterion. I'd be RS> glad to hear your feelings on the matter. I think rule-based stuff is good for constrained areas. E.g. if you constrain yourself to the english language, and maybe a certain subset of it. (Or, to give an image processing example: if you limit yourself to images of soccer games etc.) . In these cases trying to understand completely the given image/text will get you very far. In other cases, statistical information processing will do cheating which is more successful than "serious" trying to understand. In any case, true image understanding is still unheard of in this world. So in this area we are limited to statistical methods. I do not think it is feasible to give perfection, at the current state. However, we can do things which are *far better* than doing nothing at all, and we can do things which are better than any desktop environment has seen so far. -- To come back to the smaller question that is interesting for you and me: Medusa/GIFT/Fdl?? The GIFT is done explicitly for having multiple query processors evaluate the same query and then merge the resutlts. This should work with about anything. So a paradigm mix might create some work but would be rather a proof of concept of the architecture. -- The big question or: how do I imagine things: Konqueror/Nautilus (in alphabetical order) are able to present documents by their thumbnails etc. . This is information we could use. What I would like to in the beginning would be a: "Find similar documents" popup menu item in the file manager. If this is chosen, the URL of the image will be sent to the GIFT via MRML, the GIFT will retrieve similar images (or texts given a text retrieval engine), the file manager will show them as thumbnails, (possibly with scores), and it will give the possibility to enhance the query using relevance feedback (marking images positive and negative), leading to other thumbnails with other scores. Imagine the same stuff in the GNOME/KDE file-open dialogue, and in the "recently used files" menus of applications and of the desktop. Of course, it would be good for the search engine to be notified of saved files, if we do not want to do a find every now and then. This would create some work for the desktop people, but a large chunk of the work would be to make the GIFT flexible enough to do queries on any kind of multimedia data, and not only on images. A first step might be to let the GIFT and a text retrieval engine live "side by side" and merge the results, but I guess in the long term it would be best to have a search engine which tries to combine the maximum amount of cues in one search process. Cheers, Wolfgang -- Wolfgang Müller, assistant-doctorant == PhD student (2001), teaching assistant Personal page: http://cui.unige.ch/~vision/members/WolfgangMueller.html Maintainer, GNU Image Finding Tool (http://www.gnu.org/software/gift) From pgodman@real.com Thu Mar 22 12:21:40 2001 Return-Path: Delivered-To: gnome-kde-list@gnome.org Received: from two90.dev.prognet.com (tog-wakko2.prognet.com [207.188.29.249]) by mail.gnome.org (Postfix) with ESMTP id 506632DC71 for ; Thu, 22 Mar 2001 12:21:40 -0500 (EST) Received: from localhost (pgodman@localhost) by two90.dev.prognet.com (8.9.3/8.9.3) with ESMTP id KAA14191; Thu, 22 Mar 2001 10:21:50 -0800 X-Authentication-Warning: two90.dev.prognet.com: pgodman owned process doing -bs Date: Thu, 22 Mar 2001 10:21:50 -0800 (PST) From: Peter Godman X-Sender: pgodman@two90 To: Dan Kegel Cc: gnome-kde-list@gnome.org Subject: re: RealPlayer 8 In-Reply-To: <3AB468D1.6AA56541@alumni.caltech.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: gnome-kde-list-admin@gnome.org Errors-To: gnome-kde-list-admin@gnome.org X-BeenThere: gnome-kde-list@gnome.org X-Loop: gnome-kde-list@gnome.org X-Mailman-Version: 2.0beta5 Precedence: bulk List-Id: GNOME/KDE interoperability Hello Dan: Thanks for the invitation! I will do that: there will be many gnome questions like the kde ones coming up :) Thanks, Pete On Sat, 17 Mar 2001, Dan Kegel wrote: > Hi Peter, > I saw your post (http://lists.kde.org/?l=kfm-devel&m=98355172614068&w=2) > and wanted to encourage you to join the gnome-kde-list mailing list, > which is a good place to bring up integration issues like the ones > you mention (at least if you want to interoperate not just with kde > but also with gnome). > See http://mail.gnome.org/mailman/listinfo/gnome-kde-list > for more info. > - Dan > From muellerw@pc7143.unige.ch Wed Mar 14 17:29:16 2001 Return-Path: Delivered-To: gnome-kde-list@gnome.org Received: from pc7143.unige.ch (pc7143.unige.ch [129.194.71.43]) by mail.gnome.org (Postfix) with ESMTP id B2BEF2BE13 for ; Wed, 14 Mar 2001 17:29:15 -0500 (EST) Received: from muellerw by pc7143.unige.ch with local (Exim 3.12 #1 (Debian)) id 14dJuz-0007Rd-00; Wed, 14 Mar 2001 23:38:57 +0100 From: Wolfgang Mueller MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15023.62209.390726.580552@pc7143.unige.ch> Date: Wed, 14 Mar 2001 23:38:57 +0100 (CET) To: gnome-kde-list@gnome.org Subject: FdL a Project for Intelligence for the Free Desktop X-Mailer: VM 6.72 under 21.1 (patch 10) "Capitol Reef" XEmacs Lucid Sender: gnome-kde-list-admin@gnome.org Errors-To: gnome-kde-list-admin@gnome.org X-BeenThere: gnome-kde-list@gnome.org X-Loop: gnome-kde-list@gnome.org X-Mailman-Version: 2.0beta5 Precedence: bulk List-Id: GNOME/KDE interoperability Hi, Fer-de-Lance is a project for building a truly content-aware desktop environment, i.e. a finder that really helps you find any kind of files/images etc. . This is the first step towards an intelligent desktop. The link http://muellerw.free.fr/ points to the foundation document of the Fer-de-Lance project, and describes how to combine and extend GNOME/KDE and the GNU Image-Finding Tool to build something new and exciting. I would like FdL to work on GNOME and KDE, and possibly other desktops, so I guess this is the place to think about how to achieve this. Cheers, Wolfgang -- Wolfgang Müller, assistant-doctorant == PhD student (2001), teaching assistant Personal page: http://cui.unige.ch/~vision/members/WolfgangMueller.html Maintainer, GNU Image Finding Tool (http://www.gnu.org/software/gift) From dank@alumni.caltech.edu Sun Mar 18 02:46:53 2001 Return-Path: Delivered-To: gnome-kde-list@gnome.org Received: from lsmls02.we.mediaone.net (lsmls02.we.mediaone.net [24.130.1.15]) by mail.gnome.org (Postfix) with ESMTP id 1E3A52DC73 for ; Sun, 18 Mar 2001 02:46:53 -0500 (EST) Received: from alumni.caltech.edu (we-24-130-77-137.we.mediaone.net [24.130.77.137]) by lsmls02.we.mediaone.net (8.11.1/8.11.1) with ESMTP id f2I7kfS19237; Sat, 17 Mar 2001 23:46:41 -0800 (PST) Message-ID: <3AB468D1.6AA56541@alumni.caltech.edu> Date: Sat, 17 Mar 2001 23:50:41 -0800 From: Dan Kegel Reply-To: dank@alumni.caltech.edu X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.14-5.0 i686) X-Accept-Language: en MIME-Version: 1.0 To: Peter Godman , gnome-kde-list@gnome.org Subject: re: RealPlayer 8 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: gnome-kde-list-admin@gnome.org Errors-To: gnome-kde-list-admin@gnome.org X-BeenThere: gnome-kde-list@gnome.org X-Loop: gnome-kde-list@gnome.org X-Mailman-Version: 2.0beta5 Precedence: bulk List-Id: GNOME/KDE interoperability Hi Peter, I saw your post (http://lists.kde.org/?l=kfm-devel&m=98355172614068&w=2) and wanted to encourage you to join the gnome-kde-list mailing list, which is a good place to bring up integration issues like the ones you mention (at least if you want to interoperate not just with kde but also with gnome). See http://mail.gnome.org/mailman/listinfo/gnome-kde-list for more info. - Dan From muellerw@pc7143.unige.ch Mon Mar 19 10:01:01 2001 Return-Path: Delivered-To: gnome-kde-list@gnome.org Received: from pc7143.unige.ch (pc7143.unige.ch [129.194.71.43]) by mail.gnome.org (Postfix) with ESMTP id A5BB72BAF1 for ; Mon, 19 Mar 2001 10:01:00 -0500 (EST) Received: from muellerw by pc7143.unige.ch with local (Exim 3.12 #1 (Debian)) id 14f1Ja-0000Am-00 for ; Mon, 19 Mar 2001 16:11:22 +0100 From: Wolfgang Mueller MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15030.8600.500587.508056@pc7143.unige.ch> Date: Mon, 19 Mar 2001 16:11:20 +0100 (CET) To: gnome-kde-list@gnome.org Subject: FdL, some interesting questions from Rebecca X-Mailer: VM 6.72 under 21.1 (patch 10) "Capitol Reef" XEmacs Lucid Sender: gnome-kde-list-admin@gnome.org Errors-To: gnome-kde-list-admin@gnome.org X-BeenThere: gnome-kde-list@gnome.org X-Loop: gnome-kde-list@gnome.org X-Mailman-Version: 2.0beta5 Precedence: bulk List-Id: GNOME/KDE interoperability Hi, >>>>> "RS" == Rebecca Schulman writes: RS> It would be cool if you could explain what technologies you RS> already have developed, and some others I haven't heard of. RS> If it's not clear, I guess knowing more about your expectations RS> of how technologies you've been talking about would fit RS> into/create a the larger interface you've described. OK pretty big question. RS> Specifically, what is the GIFT? What does it do, how does it work, why does it RS> work? The GIFT is the GNU Image Finding Tool. It extracts automatically visual features from images, and translates each image into a chain of integers (which are IDs of the visual features) with associated floats (which are weights). The resulting documents are structurally equivalent to text (I call them pseudo-words making a pseudo-text). This permits us to index them like text in an inverted file using statistical weighting methods (those of you who do not know what's an inverted file, just retain that this is THE way of indexing text). Now we can take any image, do the same feature extraction on it, and use this as an inverted file query by example. Note: you give an example image, you retrieve visually similar images, which are likely to be semantically similar to the query. We can also give feedback by marking some of the result images as positive or negative. This works for some stuff pretty well, for some stuff not so well, in any case, this is more flexible than fixed hierarchies, and in many (most?) cases better than looking sifting the collection by hand. [Just as an aside: credits for the features-to-inverted-file idea go to David Squire, my ex boss] RS> Also, what is MRML? As content-based image retrieval (this is the term for what the GIFT is doing) is an area of active research, there is the challenge to find a client-server communication protocol that is apt to be extended in unforeseen directions. MRML is an XML-based communication protocol designed with this purpose in mind. MRML is an XML-wrapper for about all kinds of query languages you might imagine. In the future, MRML is likely to be used by quite a number of CBIRS researchers. We are just starting to get partners, so I think things are well under way. The advantage for FdL in using MRML would be, that if we make a really good and flexible query engine, this engine could be compared to other systems easily, and in the same way, it would be easy to incorporate "foreign" query engines. In the short term: the GIFT already uses MRML, so why not keep that? RS> You are right to see that I haven't been very interested RS> in statistical searching methods. I have done some work with RS> them, and my undergraduate thesis was actually on using statistical RS> language information from corpora to find relevant documents. RS> I'm not philosophically opposed to them, but I want to create RS> technology that is easy to use, and one of the criteria of this, RS> in my opinion, is that its behavior be something that you can RS> become more familiar with over time. I haven't really RS> seen statistical methods that meet this criterion. I'd be RS> glad to hear your feelings on the matter. I think rule-based stuff is good for constrained areas. E.g. if you constrain yourself to the english language, and maybe a certain subset of it. (Or, to give an image processing example: if you limit yourself to images of soccer games etc.) . In these cases trying to understand completely the given image/text will get you very far. In other cases, statistical information processing will do cheating which is more successful than "serious" trying to understand. In any case, true image understanding is still unheard of in this world. So in this area we are limited to statistical methods. I do not think it is feasible to give perfection, at the current state. However, we can do things which are *far better* than doing nothing at all, and we can do things which are better than any desktop environment has seen so far. -- To come back to the smaller question that is interesting for you and me: Medusa/GIFT/Fdl?? The GIFT is done explicitly for having multiple query processors evaluate the same query and then merge the resutlts. This should work with about anything. So a paradigm mix might create some work but would be rather a proof of concept of the architecture. -- The big question or: how do I imagine things: Konqueror/Nautilus (in alphabetical order) are able to present documents by their thumbnails etc. . This is information we could use. What I would like to in the beginning would be a: "Find similar documents" popup menu item in the file manager. If this is chosen, the URL of the image will be sent to the GIFT via MRML, the GIFT will retrieve similar images (or texts given a text retrieval engine), the file manager will show them as thumbnails, (possibly with scores), and it will give the possibility to enhance the query using relevance feedback (marking images positive and negative), leading to other thumbnails with other scores. Imagine the same stuff in the GNOME/KDE file-open dialogue, and in the "recently used files" menus of applications and of the desktop. Of course, it would be good for the search engine to be notified of saved files, if we do not want to do a find every now and then. This would create some work for the desktop people, but a large chunk of the work would be to make the GIFT flexible enough to do queries on any kind of multimedia data, and not only on images. A first step might be to let the GIFT and a text retrieval engine live "side by side" and merge the results, but I guess in the long term it would be best to have a search engine which tries to combine the maximum amount of cues in one search process. Cheers, Wolfgang -- Wolfgang Müller, assistant-doctorant == PhD student (2001), teaching assistant Personal page: http://cui.unige.ch/~vision/members/WolfgangMueller.html Maintainer, GNU Image Finding Tool (http://www.gnu.org/software/gift) From pgodman@real.com Thu Mar 22 12:21:40 2001 Return-Path: Delivered-To: gnome-kde-list@gnome.org Received: from two90.dev.prognet.com (tog-wakko2.prognet.com [207.188.29.249]) by mail.gnome.org (Postfix) with ESMTP id 506632DC71 for ; Thu, 22 Mar 2001 12:21:40 -0500 (EST) Received: from localhost (pgodman@localhost) by two90.dev.prognet.com (8.9.3/8.9.3) with ESMTP id KAA14191; Thu, 22 Mar 2001 10:21:50 -0800 X-Authentication-Warning: two90.dev.prognet.com: pgodman owned process doing -bs Date: Thu, 22 Mar 2001 10:21:50 -0800 (PST) From: Peter Godman X-Sender: pgodman@two90 To: Dan Kegel Cc: gnome-kde-list@gnome.org Subject: re: RealPlayer 8 In-Reply-To: <3AB468D1.6AA56541@alumni.caltech.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: gnome-kde-list-admin@gnome.org Errors-To: gnome-kde-list-admin@gnome.org X-BeenThere: gnome-kde-list@gnome.org X-Loop: gnome-kde-list@gnome.org X-Mailman-Version: 2.0beta5 Precedence: bulk List-Id: GNOME/KDE interoperability Hello Dan: Thanks for the invitation! I will do that: there will be many gnome questions like the kde ones coming up :) Thanks, Pete On Sat, 17 Mar 2001, Dan Kegel wrote: > Hi Peter, > I saw your post (http://lists.kde.org/?l=kfm-devel&m=98355172614068&w=2) > and wanted to encourage you to join the gnome-kde-list mailing list, > which is a good place to bring up integration issues like the ones > you mention (at least if you want to interoperate not just with kde > but also with gnome). > See http://mail.gnome.org/mailman/listinfo/gnome-kde-list > for more info. > - Dan > From muellerw@pc7143.unige.ch Wed Mar 14 17:29:16 2001 Return-Path: Delivered-To: gnome-kde-list@gnome.org Received: from pc7143.unige.ch (pc7143.unige.ch [129.194.71.43]) by mail.gnome.org (Postfix) with ESMTP id B2BEF2BE13 for ; Wed, 14 Mar 2001 17:29:15 -0500 (EST) Received: from muellerw by pc7143.unige.ch with local (Exim 3.12 #1 (Debian)) id 14dJuz-0007Rd-00; Wed, 14 Mar 2001 23:38:57 +0100 From: Wolfgang Mueller MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15023.62209.390726.580552@pc7143.unige.ch> Date: Wed, 14 Mar 2001 23:38:57 +0100 (CET) To: gnome-kde-list@gnome.org Subject: FdL a Project for Intelligence for the Free Desktop X-Mailer: VM 6.72 under 21.1 (patch 10) "Capitol Reef" XEmacs Lucid Sender: gnome-kde-list-admin@gnome.org Errors-To: gnome-kde-list-admin@gnome.org X-BeenThere: gnome-kde-list@gnome.org X-Loop: gnome-kde-list@gnome.org X-Mailman-Version: 2.0beta5 Precedence: bulk List-Id: GNOME/KDE interoperability Hi, Fer-de-Lance is a project for building a truly content-aware desktop environment, i.e. a finder that really helps you find any kind of files/images etc. . This is the first step towards an intelligent desktop. The link http://muellerw.free.fr/ points to the foundation document of the Fer-de-Lance project, and describes how to combine and extend GNOME/KDE and the GNU Image-Finding Tool to build something new and exciting. I would like FdL to work on GNOME and KDE, and possibly other desktops, so I guess this is the place to think about how to achieve this. Cheers, Wolfgang -- Wolfgang Müller, assistant-doctorant == PhD student (2001), teaching assistant Personal page: http://cui.unige.ch/~vision/members/WolfgangMueller.html Maintainer, GNU Image Finding Tool (http://www.gnu.org/software/gift) From dank@alumni.caltech.edu Sun Mar 18 02:46:53 2001 Return-Path: Delivered-To: gnome-kde-list@gnome.org Received: from lsmls02.we.mediaone.net (lsmls02.we.mediaone.net [24.130.1.15]) by mail.gnome.org (Postfix) with ESMTP id 1E3A52DC73 for ; Sun, 18 Mar 2001 02:46:53 -0500 (EST) Received: from alumni.caltech.edu (we-24-130-77-137.we.mediaone.net [24.130.77.137]) by lsmls02.we.mediaone.net (8.11.1/8.11.1) with ESMTP id f2I7kfS19237; Sat, 17 Mar 2001 23:46:41 -0800 (PST) Message-ID: <3AB468D1.6AA56541@alumni.caltech.edu> Date: Sat, 17 Mar 2001 23:50:41 -0800 From: Dan Kegel Reply-To: dank@alumni.caltech.edu X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.14-5.0 i686) X-Accept-Language: en MIME-Version: 1.0 To: Peter Godman , gnome-kde-list@gnome.org Subject: re: RealPlayer 8 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: gnome-kde-list-admin@gnome.org Errors-To: gnome-kde-list-admin@gnome.org X-BeenThere: gnome-kde-list@gnome.org X-Loop: gnome-kde-list@gnome.org X-Mailman-Version: 2.0beta5 Precedence: bulk List-Id: GNOME/KDE interoperability Hi Peter, I saw your post (http://lists.kde.org/?l=kfm-devel&m=98355172614068&w=2) and wanted to encourage you to join the gnome-kde-list mailing list, which is a good place to bring up integration issues like the ones you mention (at least if you want to interoperate not just with kde but also with gnome). See http://mail.gnome.org/mailman/listinfo/gnome-kde-list for more info. - Dan From muellerw@pc7143.unige.ch Mon Mar 19 10:01:01 2001 Return-Path: Delivered-To: gnome-kde-list@gnome.org Received: from pc7143.unige.ch (pc7143.unige.ch [129.194.71.43]) by mail.gnome.org (Postfix) with ESMTP id A5BB72BAF1 for ; Mon, 19 Mar 2001 10:01:00 -0500 (EST) Received: from muellerw by pc7143.unige.ch with local (Exim 3.12 #1 (Debian)) id 14f1Ja-0000Am-00 for ; Mon, 19 Mar 2001 16:11:22 +0100 From: Wolfgang Mueller MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15030.8600.500587.508056@pc7143.unige.ch> Date: Mon, 19 Mar 2001 16:11:20 +0100 (CET) To: gnome-kde-list@gnome.org Subject: FdL, some interesting questions from Rebecca X-Mailer: VM 6.72 under 21.1 (patch 10) "Capitol Reef" XEmacs Lucid Sender: gnome-kde-list-admin@gnome.org Errors-To: gnome-kde-list-admin@gnome.org X-BeenThere: gnome-kde-list@gnome.org X-Loop: gnome-kde-list@gnome.org X-Mailman-Version: 2.0beta5 Precedence: bulk List-Id: GNOME/KDE interoperability Hi, >>>>> "RS" == Rebecca Schulman writes: RS> It would be cool if you could explain what technologies you RS> already have developed, and some others I haven't heard of. RS> If it's not clear, I guess knowing more about your expectations RS> of how technologies you've been talking about would fit RS> into/create a the larger interface you've described. OK pretty big question. RS> Specifically, what is the GIFT? What does it do, how does it work, why does it RS> work? The GIFT is the GNU Image Finding Tool. It extracts automatically visual features from images, and translates each image into a chain of integers (which are IDs of the visual features) with associated floats (which are weights). The resulting documents are structurally equivalent to text (I call them pseudo-words making a pseudo-text). This permits us to index them like text in an inverted file using statistical weighting methods (those of you who do not know what's an inverted file, just retain that this is THE way of indexing text). Now we can take any image, do the same feature extraction on it, and use this as an inverted file query by example. Note: you give an example image, you retrieve visually similar images, which are likely to be semantically similar to the query. We can also give feedback by marking some of the result images as positive or negative. This works for some stuff pretty well, for some stuff not so well, in any case, this is more flexible than fixed hierarchies, and in many (most?) cases better than looking sifting the collection by hand. [Just as an aside: credits for the features-to-inverted-file idea go to David Squire, my ex boss] RS> Also, what is MRML? As content-based image retrieval (this is the term for what the GIFT is doing) is an area of active research, there is the challenge to find a client-server communication protocol that is apt to be extended in unforeseen directions. MRML is an XML-based communication protocol designed with this purpose in mind. MRML is an XML-wrapper for about all kinds of query languages you might imagine. In the future, MRML is likely to be used by quite a number of CBIRS researchers. We are just starting to get partners, so I think things are well under way. The advantage for FdL in using MRML would be, that if we make a really good and flexible query engine, this engine could be compared to other systems easily, and in the same way, it would be easy to incorporate "foreign" query engines. In the short term: the GIFT already uses MRML, so why not keep that? RS> You are right to see that I haven't been very interested RS> in statistical searching methods. I have done some work with RS> them, and my undergraduate thesis was actually on using statistical RS> language information from corpora to find relevant documents. RS> I'm not philosophically opposed to them, but I want to create RS> technology that is easy to use, and one of the criteria of this, RS> in my opinion, is that its behavior be something that you can RS> become more familiar with over time. I haven't really RS> seen statistical methods that meet this criterion. I'd be RS> glad to hear your feelings on the matter. I think rule-based stuff is good for constrained areas. E.g. if you constrain yourself to the english language, and maybe a certain subset of it. (Or, to give an image processing example: if you limit yourself to images of soccer games etc.) . In these cases trying to understand completely the given image/text will get you very far. In other cases, statistical information processing will do cheating which is more successful than "serious" trying to understand. In any case, true image understanding is still unheard of in this world. So in this area we are limited to statistical methods. I do not think it is feasible to give perfection, at the current state. However, we can do things which are *far better* than doing nothing at all, and we can do things which are better than any desktop environment has seen so far. -- To come back to the smaller question that is interesting for you and me: Medusa/GIFT/Fdl?? The GIFT is done explicitly for having multiple query processors evaluate the same query and then merge the resutlts. This should work with about anything. So a paradigm mix might create some work but would be rather a proof of concept of the architecture. -- The big question or: how do I imagine things: Konqueror/Nautilus (in alphabetical order) are able to present documents by their thumbnails etc. . This is information we could use. What I would like to in the beginning would be a: "Find similar documents" popup menu item in the file manager. If this is chosen, the URL of the image will be sent to the GIFT via MRML, the GIFT will retrieve similar images (or texts given a text retrieval engine), the file manager will show them as thumbnails, (possibly with scores), and it will give the possibility to enhance the query using relevance feedback (marking images positive and negative), leading to other thumbnails with other scores. Imagine the same stuff in the GNOME/KDE file-open dialogue, and in the "recently used files" menus of applications and of the desktop. Of course, it would be good for the search engine to be notified of saved files, if we do not want to do a find every now and then. This would create some work for the desktop people, but a large chunk of the work would be to make the GIFT flexible enough to do queries on any kind of multimedia data, and not only on images. A first step might be to let the GIFT and a text retrieval engine live "side by side" and merge the results, but I guess in the long term it would be best to have a search engine which tries to combine the maximum amount of cues in one search process. Cheers, Wolfgang -- Wolfgang Müller, assistant-doctorant == PhD student (2001), teaching assistant Personal page: http://cui.unige.ch/~vision/members/WolfgangMueller.html Maintainer, GNU Image Finding Tool (http://www.gnu.org/software/gift) From pgodman@real.com Thu Mar 22 12:21:40 2001 Return-Path: Delivered-To: gnome-kde-list@gnome.org Received: from two90.dev.prognet.com (tog-wakko2.prognet.com [207.188.29.249]) by mail.gnome.org (Postfix) with ESMTP id 506632DC71 for ; Thu, 22 Mar 2001 12:21:40 -0500 (EST) Received: from localhost (pgodman@localhost) by two90.dev.prognet.com (8.9.3/8.9.3) with ESMTP id KAA14191; Thu, 22 Mar 2001 10:21:50 -0800 X-Authentication-Warning: two90.dev.prognet.com: pgodman owned process doing -bs Date: Thu, 22 Mar 2001 10:21:50 -0800 (PST) From: Peter Godman X-Sender: pgodman@two90 To: Dan Kegel Cc: gnome-kde-list@gnome.org Subject: re: RealPlayer 8 In-Reply-To: <3AB468D1.6AA56541@alumni.caltech.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: gnome-kde-list-admin@gnome.org Errors-To: gnome-kde-list-admin@gnome.org X-BeenThere: gnome-kde-list@gnome.org X-Loop: gnome-kde-list@gnome.org X-Mailman-Version: 2.0beta5 Precedence: bulk List-Id: GNOME/KDE interoperability Hello Dan: Thanks for the invitation! I will do that: there will be many gnome questions like the kde ones coming up :) Thanks, Pete On Sat, 17 Mar 2001, Dan Kegel wrote: > Hi Peter, > I saw your post (http://lists.kde.org/?l=kfm-devel&m=98355172614068&w=2) > and wanted to encourage you to join the gnome-kde-list mailing list, > which is a good place to bring up integration issues like the ones > you mention (at least if you want to interoperate not just with kde > but also with gnome). > See http://mail.gnome.org/mailman/listinfo/gnome-kde-list > for more info. > - Dan > From muellerw@pc7143.unige.ch Wed Mar 14 17:29:16 2001 Return-Path: Delivered-To: gnome-kde-list@gnome.org Received: from pc7143.unige.ch (pc7143.unige.ch [129.194.71.43]) by mail.gnome.org (Postfix) with ESMTP id B2BEF2BE13 for ; Wed, 14 Mar 2001 17:29:15 -0500 (EST) Received: from muellerw by pc7143.unige.ch with local (Exim 3.12 #1 (Debian)) id 14dJuz-0007Rd-00; Wed, 14 Mar 2001 23:38:57 +0100 From: Wolfgang Mueller MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15023.62209.390726.580552@pc7143.unige.ch> Date: Wed, 14 Mar 2001 23:38:57 +0100 (CET) To: gnome-kde-list@gnome.org Subject: FdL a Project for Intelligence for the Free Desktop X-Mailer: VM 6.72 under 21.1 (patch 10) "Capitol Reef" XEmacs Lucid Sender: gnome-kde-list-admin@gnome.org Errors-To: gnome-kde-list-admin@gnome.org X-BeenThere: gnome-kde-list@gnome.org X-Loop: gnome-kde-list@gnome.org X-Mailman-Version: 2.0beta5 Precedence: bulk List-Id: GNOME/KDE interoperability Hi, Fer-de-Lance is a project for building a truly content-aware desktop environment, i.e. a finder that really helps you find any kind of files/images etc. . This is the first step towards an intelligent desktop. The link http://muellerw.free.fr/ points to the foundation document of the Fer-de-Lance project, and describes how to combine and extend GNOME/KDE and the GNU Image-Finding Tool to build something new and exciting. I would like FdL to work on GNOME and KDE, and possibly other desktops, so I guess this is the place to think about how to achieve this. Cheers, Wolfgang -- Wolfgang Müller, assistant-doctorant == PhD student (2001), teaching assistant Personal page: http://cui.unige.ch/~vision/members/WolfgangMueller.html Maintainer, GNU Image Finding Tool (http://www.gnu.org/software/gift) From dank@alumni.caltech.edu Sun Mar 18 02:46:53 2001 Return-Path: Delivered-To: gnome-kde-list@gnome.org Received: from lsmls02.we.mediaone.net (lsmls02.we.mediaone.net [24.130.1.15]) by mail.gnome.org (Postfix) with ESMTP id 1E3A52DC73 for ; Sun, 18 Mar 2001 02:46:53 -0500 (EST) Received: from alumni.caltech.edu (we-24-130-77-137.we.mediaone.net [24.130.77.137]) by lsmls02.we.mediaone.net (8.11.1/8.11.1) with ESMTP id f2I7kfS19237; Sat, 17 Mar 2001 23:46:41 -0800 (PST) Message-ID: <3AB468D1.6AA56541@alumni.caltech.edu> Date: Sat, 17 Mar 2001 23:50:41 -0800 From: Dan Kegel Reply-To: dank@alumni.caltech.edu X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.14-5.0 i686) X-Accept-Language: en MIME-Version: 1.0 To: Peter Godman , gnome-kde-list@gnome.org Subject: re: RealPlayer 8 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: gnome-kde-list-admin@gnome.org Errors-To: gnome-kde-list-admin@gnome.org X-BeenThere: gnome-kde-list@gnome.org X-Loop: gnome-kde-list@gnome.org X-Mailman-Version: 2.0beta5 Precedence: bulk List-Id: GNOME/KDE interoperability Hi Peter, I saw your post (http://lists.kde.org/?l=kfm-devel&m=98355172614068&w=2) and wanted to encourage you to join the gnome-kde-list mailing list, which is a good place to bring up integration issues like the ones you mention (at least if you want to interoperate not just with kde but also with gnome). See http://mail.gnome.org/mailman/listinfo/gnome-kde-list for more info. - Dan From muellerw@pc7143.unige.ch Mon Mar 19 10:01:01 2001 Return-Path: Delivered-To: gnome-kde-list@gnome.org Received: from pc7143.unige.ch (pc7143.unige.ch [129.194.71.43]) by mail.gnome.org (Postfix) with ESMTP id A5BB72BAF1 for ; Mon, 19 Mar 2001 10:01:00 -0500 (EST) Received: from muellerw by pc7143.unige.ch with local (Exim 3.12 #1 (Debian)) id 14f1Ja-0000Am-00 for ; Mon, 19 Mar 2001 16:11:22 +0100 From: Wolfgang Mueller MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15030.8600.500587.508056@pc7143.unige.ch> Date: Mon, 19 Mar 2001 16:11:20 +0100 (CET) To: gnome-kde-list@gnome.org Subject: FdL, some interesting questions from Rebecca X-Mailer: VM 6.72 under 21.1 (patch 10) "Capitol Reef" XEmacs Lucid Sender: gnome-kde-list-admin@gnome.org Errors-To: gnome-kde-list-admin@gnome.org X-BeenThere: gnome-kde-list@gnome.org X-Loop: gnome-kde-list@gnome.org X-Mailman-Version: 2.0beta5 Precedence: bulk List-Id: GNOME/KDE interoperability Hi, >>>>> "RS" == Rebecca Schulman writes: RS> It would be cool if you could explain what technologies you RS> already have developed, and some others I haven't heard of. RS> If it's not clear, I guess knowing more about your expectations RS> of how technologies you've been talking about would fit RS> into/create a the larger interface you've described. OK pretty big question. RS> Specifically, what is the GIFT? What does it do, how does it work, why does it RS> work? The GIFT is the GNU Image Finding Tool. It extracts automatically visual features from images, and translates each image into a chain of integers (which are IDs of the visual features) with associated floats (which are weights). The resulting documents are structurally equivalent to text (I call them pseudo-words making a pseudo-text). This permits us to index them like text in an inverted file using statistical weighting methods (those of you who do not know what's an inverted file, just retain that this is THE way of indexing text). Now we can take any image, do the same feature extraction on it, and use this as an inverted file query by example. Note: you give an example image, you retrieve visually similar images, which are likely to be semantically similar to the query. We can also give feedback by marking some of the result images as positive or negative. This works for some stuff pretty well, for some stuff not so well, in any case, this is more flexible than fixed hierarchies, and in many (most?) cases better than looking sifting the collection by hand. [Just as an aside: credits for the features-to-inverted-file idea go to David Squire, my ex boss] RS> Also, what is MRML? As content-based image retrieval (this is the term for what the GIFT is doing) is an area of active research, there is the challenge to find a client-server communication protocol that is apt to be extended in unforeseen directions. MRML is an XML-based communication protocol designed with this purpose in mind. MRML is an XML-wrapper for about all kinds of query languages you might imagine. In the future, MRML is likely to be used by quite a number of CBIRS researchers. We are just starting to get partners, so I think things are well under way. The advantage for FdL in using MRML would be, that if we make a really good and flexible query engine, this engine could be compared to other systems easily, and in the same way, it would be easy to incorporate "foreign" query engines. In the short term: the GIFT already uses MRML, so why not keep that? RS> You are right to see that I haven't been very interested RS> in statistical searching methods. I have done some work with RS> them, and my undergraduate thesis was actually on using statistical RS> language information from corpora to find relevant documents. RS> I'm not philosophically opposed to them, but I want to create RS> technology that is easy to use, and one of the criteria of this, RS> in my opinion, is that its behavior be something that you can RS> become more familiar with over time. I haven't really RS> seen statistical methods that meet this criterion. I'd be RS> glad to hear your feelings on the matter. I think rule-based stuff is good for constrained areas. E.g. if you constrain yourself to the english language, and maybe a certain subset of it. (Or, to give an image processing example: if you limit yourself to images of soccer games etc.) . In these cases trying to understand completely the given image/text will get you very far. In other cases, statistical information processing will do cheating which is more successful than "serious" trying to understand. In any case, true image understanding is still unheard of in this world. So in this area we are limited to statistical methods. I do not think it is feasible to give perfection, at the current state. However, we can do things which are *far better* than doing nothing at all, and we can do things which are better than any desktop environment has seen so far. -- To come back to the smaller question that is interesting for you and me: Medusa/GIFT/Fdl?? The GIFT is done explicitly for having multiple query processors evaluate the same query and then merge the resutlts. This should work with about anything. So a paradigm mix might create some work but would be rather a proof of concept of the architecture. -- The big question or: how do I imagine things: Konqueror/Nautilus (in alphabetical order) are able to present documents by their thumbnails etc. . This is information we could use. What I would like to in the beginning would be a: "Find similar documents" popup menu item in the file manager. If this is chosen, the URL of the image will be sent to the GIFT via MRML, the GIFT will retrieve similar images (or texts given a text retrieval engine), the file manager will show them as thumbnails, (possibly with scores), and it will give the possibility to enhance the query using relevance feedback (marking images positive and negative), leading to other thumbnails with other scores. Imagine the same stuff in the GNOME/KDE file-open dialogue, and in the "recently used files" menus of applications and of the desktop. Of course, it would be good for the search engine to be notified of saved files, if we do not want to do a find every now and then. This would create some work for the desktop people, but a large chunk of the work would be to make the GIFT flexible enough to do queries on any kind of multimedia data, and not only on images. A first step might be to let the GIFT and a text retrieval engine live "side by side" and merge the results, but I guess in the long term it would be best to have a search engine which tries to combine the maximum amount of cues in one search process. Cheers, Wolfgang -- Wolfgang Müller, assistant-doctorant == PhD student (2001), teaching assistant Personal page: http://cui.unige.ch/~vision/members/WolfgangMueller.html Maintainer, GNU Image Finding Tool (http://www.gnu.org/software/gift) From pgodman@real.com Thu Mar 22 12:21:40 2001 Return-Path: Delivered-To: gnome-kde-list@gnome.org Received: from two90.dev.prognet.com (tog-wakko2.prognet.com [207.188.29.249]) by mail.gnome.org (Postfix) with ESMTP id 506632DC71 for ; Thu, 22 Mar 2001 12:21:40 -0500 (EST) Received: from localhost (pgodman@localhost) by two90.dev.prognet.com (8.9.3/8.9.3) with ESMTP id KAA14191; Thu, 22 Mar 2001 10:21:50 -0800 X-Authentication-Warning: two90.dev.prognet.com: pgodman owned process doing -bs Date: Thu, 22 Mar 2001 10:21:50 -0800 (PST) From: Peter Godman X-Sender: pgodman@two90 To: Dan Kegel Cc: gnome-kde-list@gnome.org Subject: re: RealPlayer 8 In-Reply-To: <3AB468D1.6AA56541@alumni.caltech.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: gnome-kde-list-admin@gnome.org Errors-To: gnome-kde-list-admin@gnome.org X-BeenThere: gnome-kde-list@gnome.org X-Loop: gnome-kde-list@gnome.org X-Mailman-Version: 2.0beta5 Precedence: bulk List-Id: GNOME/KDE interoperability Hello Dan: Thanks for the invitation! I will do that: there will be many gnome questions like the kde ones coming up :) Thanks, Pete On Sat, 17 Mar 2001, Dan Kegel wrote: > Hi Peter, > I saw your post (http://lists.kde.org/?l=kfm-devel&m=98355172614068&w=2) > and wanted to encourage you to join the gnome-kde-list mailing list, > which is a good place to bring up integration issues like the ones > you mention (at least if you want to interoperate not just with kde > but also with gnome). > See http://mail.gnome.org/mailman/listinfo/gnome-kde-list > for more info. > - Dan >