Re: [Evolution-hackers] Camel plans and possibilities

[Jeffrey Stedfast <fejj ximian com>]

On Wed, 2003-06-25 at 23:54, Not Zed wrote:
> I put this (long) list together of a bunch of things that should be
> addressed in camel.
> 
> Its a bit like a TODO list for 1.6/making a separate camel, although not
> all of the stuff will likely be included because there's a ton of it.
> 
> (I used the .h files to mean the class contained, so a couple files
> might have been skipped.  I didn't go into much detail on the backend
> stores etc).
> 
> Cheers
>  Michael
> 
> 
> In all cases:
>  - should remove dependent headers from header files, force client code
>    to include appropriate header dependencies, a-la libc.
>  - do we remove non-threaded case code?
>  - can we control exported symbols from plugins using libtool?
>  - need to consider breaking up directory structure
>  - check all uses of strcasecmp, generally not correct since we must
> force us-ascii
>  - remove the last gal reference
>  - what to do about e-util - we only use a few non-gui related files
> from e-util,
>    could either copy, or move them to camel/e-util and rename it.
> 
> New classes and types
>  - CamelIterator (CamelI?) for iterators, for searches, camel-index,
> etc.
>  - CamelPlugin
>  - CamelProvider should be a real object
>  - CamelStreamPipe - for unnamed pipes
>  - Some sort of camel configuration, for global stuff, e.g. charsets,
> etc.
> 
> Priorities
>  - how to access the message in a raw form without having to re-encode,
>    or keep the whole raw form around.

if we didn't convert to utf-8 in camel-mime-part-utils, we could easily
keep the raw message content. the only problem we'd have then, is the
smarts for detecting windows charsets. perhaps we can add some smarts to
camel-mime-filter-charset to change the iconv_t on-the-fly internally if
it finds that the iso-8859-* charset is really a windows-cp125* charset?

would that even work? if so, it'd kick ass...

>  - plugins for sasl auth types, and providers
>  - namespace cleanup

I've namespaced the string-utils code so far...

> 
> Lower priorities
>  - iterating searches
>  - configuration stuff
> 
> Providers
> (very quick comments)
> 
> local
>  - its a bit of a mess, but works
> nntp
>  - unmaintained
> imap
>  - needs to be rewritten, if possible
>  - needs to be able to get messages and the like, incrementally
>  - be nice to have a fast-mode without support for mlist stuff
>    (i.e. based on ENVELOPE fetches)
> sendmail
>  - 'it works' - should use camel-process or something if it doesn't
> already
> smtp
>  - 'it works' too
> pop3
>  - already rewritten.  still relatively bit-rot free
> 
> Camel core
> 
> camel.h
>  - imho, remove this, or at least, change it not to inlude all of camel
>  - where to put camel-init?
>  - how to hook into camel-init?
>  - how to hook plugins?
> camel-object.h
>  - be nice to implement (inheritable?) interfaces
>  - object bag should support multiple outstanding reservations(?)
>  - need to reduce overhead for using hooks.  should probably use
>    g_recursive_mutex, or fix e_mutex to use native rec mutex's if
>    available.
>  - should be able to build without the magic tags
>  - remove CamelType, or rename CamelObjectClass, they're the same typdef
> anyway.
>  - uses e-util
>  - make camel_object_init non-recursive?
>  - make camel object class refcounted, and disposable?
> camel-arg.h
>  - not sure, implementation not ideal, or very efficient
> camel-exception.h
>  - add imapp longjmp exceptions?
>  - should fix api, setv is incorrectly named ...
>  - should define more, and more reasonable exceptions
>  - also, string mappings for exception types?
>  - extensible excption id's?
> camel-operation.h
>  - depends on e-util
> camel-types.h
>  - should just kill this
> 
> camel-private.h
>  - this is mainly used to share access to locks
>  - maybe locks should be moved to public api's, if they're public
> 
> camel-url.h
>  - s/camel_url_copy/camel_url_clone/ ?
>  - camel_url_new probably doesn't need an exception, it only sets one
>    type
>  - is the base stuff still used ... or was it moved to gtkhtml?
> 
> camel-address.h
> camel-news-address.h
> camel-internet-address.h
>  - ok
> 
> camel-index.h
>  - cursors to be replaced by/subclass camel iterators?
> camel-block-file.h
> camel-partition-table.h
> camel-text-index.h
>  - basically ok, but really need to find those looping and crashing
>    bugs :(
> 
> camel-charset-map.h
>  - remove the charset_to_windows, move it to the iconv charset handling
> stuff
>  - are we still using this?
> camel-charset-map-private.h
> camel-i18n.h
>  - doesn't glib cover this functionality?
> camel-iconv.h
>  - copy of gal/e-iconv, must decide what to use

just updated camel-iconv code to use the latest e-iconv stuff.

> camel-utf8.h
>  - g_string_append_u should be camel namespaced
>  - should probably just move to camel-string-utils
> 
> camel-certdb.h
>  - seems complicated for what it does
> camel-cipher-context.h
>  - needs api tweaks to cover all security context types as plugins
> camel-gpg-context.h
>  - uses gal/e-iconv
>  - object should be CamelGPGContext, not CamelGpgContext
> camel-pkcs7-context.h
>  - what is this for?
> camel-smime-context.h
>  - should derive from cipher-context, not cmscontext?
>  - the object should be CamelSMIMEContext, not CamelSMimeContext
> camel-smime.h
>  - should be moved into camel-smime-context or camel-smime-utils?
> camel-smime-utils.h
> camel-cms-context.h
>  - should derive from camel-cipher-context, or just not exist
> camel-pgp-mime.h
>  - should be moved into camel-gpg-context?

this just needs to be removed, it serves no purpose anymore.

> 
> camel-data-wrapper.h
>  - needs some work on the awful mime-type api parts
>  - rawtext should be removed (and implied)
> camel-medium.h
>  - needs a better header interface, currently cannot handle removal of
> duplicates
>    perhaps using a simple iterator?
>  - complex api for what it does
>  - privatise private parts
> camel-mime-part.h
>  - needs to honour content-languages
>  - should probably use camel_object_get* instead of bulky api (also
> allows easier override)
>  - should no longer implictly encode parts?
>  - uses gal/e-iconv
>  - privatise private parts
> camel-mime-part-utils.h
>  - should treat all content as opaque, and perform no encoding/decoding
>  - can this be more 'streamed'??
>  - uses gal/e-iconv
>  - privatise private parts
> camel-mime-message.h
>  - remove the mbox_from encoding to somewhere else (mime-utils?)
>  - use object_get* methods?
>  - uses gal/e-iconv (why??)
>  - privatise private parts
>  - it would be nice to have it more streamable
> camel-mime-parser.h
>  - object should be CamelMIMEParser
>  - namespace (HSCAN_* enums)
>  - remove unused virtual methods/events in class
> camel-mime-utils.h
>  - namespace namespace namespace
>  - could probably be split a little bit (e.g. transfer-encoding
> encoders/decoders)
>  - could export some more api (e.g. rfc822 type macro's)
>  - uses gal/e-iconv
> camel-multipart.h
>  - should be purely abstract, with a camel-mime-multipart subpart?
>  - it would be nice to have it more streamable
> camel-multipart-encrypted.h
>  - seems ok
> camel-multipart-signed.h
>  - should clean up the fugly 'parser' code internally
> 
> camel-file-utils.h
> camel-io.h
>  - remove, move methods into camel-file-utils

done.

> camel-data-cache.h
>  - need to finish all api, and api docs
> camel-uid-cache.h
>  - needs to be made incremental, and more robust.  this is a priority
> for pop
> 
> camel-lock.h
> camel-lock-client.h
>  - could probably use camel-io safe-write safe-read methods
> camel-lock-helper.h
>  - needs to pass back more detail in error codes (errno is all we have i
> think, so should do)
> 
> camel-session.h
>  - if its going to have a thread api, it needs to be more complete,
>    and have progress notification, cancellation, etc.  the implementer
>    should be able to override most thread stuff itself (e.g. use
>    gthreads if it wants).
>  - some of the api's seems a bit messy/inconsistent with each other
>    e.g. get_password should have service as second arg
>  - uses e-util
> camel-provider.h
>  - should be a plugin
>  - should be an active, actual object
>  - needs much better autodetect interface
>  - possibly change authtypes stuff
> camel-transport.h
>  - s/send_to/send/
> camel-service.h
>  - possible changes to authtypes
>  - move hostname lookup stuff somewhere else.  should probably also
>    use the glibc async dns api if possible and it is available.
> camel-store.h
>  - use camel-store-summary for all summary ops?
>  - remove camel_mkdir_heir, use camel-file-util

done.

>  - the store_noop thing is a bit weird (i.e. when to use it?)
>  - be nice to have a 'subscribed interface' ?
>  - needs acl interface
>  - should create, delete, rename, return unopened, blank folder
>    objects?  would this make store-summary redundant (or just
>    internal?)
>  - some locking should be finer grained/moved to subclasses
>  - server-side filtering??
>    should the filter-driver be set on the store, and
>    not requested from the session?  makes editing painful though.
> camel-store-summary.h
>  - api a little clumsy
>  - possibly should offer more virtual methods (get, etc)
>  - use iterators?
> camel-folder.h
>  - should this be able to be created, unopened?
>  - s/transfer_messages_to/transfer_messages/
>  - summary and search done via interfaces?
>  - should flag/tag get/set be done via messageinfo's?  (would require
> summary support).
>  - some locking should be finer grained/moved to subclasses
>  - some data queries should be object_get only
>  - what about a sorting interface?  incremental summary queries?  etc?
> camel-search-private.h
> camel-folder-summary.h
>  - api is troublesome for some backends, e.g. imap, which dont merely
> append items
>  - api is enourmous anyway, and summarising works strangely
>  - need to be able to append without reading/writing whole file?
>  - indexing needs to be done some other way (externally?)
>  - uses gal/e-iconv
>  - maybe it should just be an interface
>  - use iterators?
>  - CamelMessageInfo's should probably be self-contained (i.e. contain
>    a 'class' pointer) - will add slightly to memory use in
>    current model, but will allow e.g. disk-based summaries loaded on
>    demand using existing api's.
> camel-folder-search.h
>  - api is weird (set_summary, set_folder, etc, run search, they get
> unset implictly)
>  - execute expression is ugly code
>  - would be nice to make this return values incrementally, using some
> iterator type thing
>    cancellation could be cleaner too
>  - otoh this might not be worth it (e.g. trying to return values in
> summary order always)
>  - hide private bits
> camel-folder-thread.h
>  - really needs the incremental api to be done incrementally (then
>    again, its so fast anyway ...)
>  - should it just create a table of uid's instead, so that messageids
> can be loaded
>    on demand?
> camel-vee-store.h
>  - needs some pretty major architectural issues fixed
> camel-vee-folder.h
>  - needs some pretty major architectural issues fixed
>  - mainly how unmatched works and how vfolders are stored in memory
> camel-vtrash-folder.h
>  - should be child of a first-class vtrash provider/store?
> 
> camel-digest-store.h
>  - not sure this serves any purpose?
> camel-digest-folder.h
>  - more or less ok
> camel-digest-summary.h
>  - is a noop, should probably be removed (just use camel-folder-summary
> directly)
> 
> camel-disco-store.h
> camel-disco-folder.h
> camel-disco-diary.h
>  - please die and go to hell :)  disconnected operation should probably
> be moved
>    to camel-store, either as an interface, or a few extra api's to see
> if its
>    supported, etc.  The diary stuff should just be part of the backend.
> 
> camel-filter-driver.h
>  - possibly, should use virtual methods (and hence subclassing), for the
> various
>    callbacks
>  - actually, should probably subclass and add api to allow extension,
> and remove
>    all of the callbacks
>  - esexp would need to ignore unknown methods when running?
>  - 'shell' should be a builtin command, dont use gnome_exec*
>  - _filter_message api is bloated
>  - logfile format needs to be more parsable
>  - need to fix the run-once stuff
> camel-filter-search.h
>  - also needs to be extensible, should probably be an object which can
> be attached
>    to the filter-driver.
>  - uses gal/e-iconv
> 
> camel-html-parser.h
>  - probably ok, for what its for
>  - maybe should be moved into camel-mime-filter-html.c
> 
> camel-mime-filter.h
>  - this and all subclasses, s/CamelMime/CamelMIME/
>  - could probably have a check_room method
> camel-mime-filter-basic.h
>  - rename to CamelMIMEFilterEncoding, since thats what it does
> camel-mime-filter-bestenc.h
>  - basically ok
> camel-mime-filter-charset.h
>  - probably ok, although probably need some better fallback when iconv
> gives up
>  - uses gal/e-iconv
> camel-mime-filter-canon.h
> camel-mime-filter-crlf.h
>  - one of these is redundant and/or the functionality should be merged?
> camel-mime-filter-enriched.h
>  - shouldn't use alloca in a loop, but otherwise ok
>  - maybe should be renamed enriched_to_html?
> camel-mime-filter-from.h
>  - should remove as well as add >'s, e.g. >From  -> >>From  and >From ->
> >From in decode
>  - could probably be done as part of canonicalisation filter
> camel-mime-filter-html.h
>  - ok for indexing which is where its used
>  - perhaps should include camel-html-parser instead of separate file
>  - renamed html_to_plain ?
> camel-mime-filter-index.h
>  - seems ok, not sure what set_index is for though
> camel-mime-filter-linewrap.h
>  - ok
> camel-mime-filter-save.h
>  - ok
> camel-mime-filter-tohtml.h
>  - is citation colour used anymore?
>  - should it be using html tags for citations, not just colours?
> 
> camel-url-scanner.h
>  - uses own chartype tables (camel-mime-utils has some already)
>  - uses e-util/e-trie
> 
> camel-movemail.h
>  - probably needs to make runtime decision about solaris style mailbox
>  - should it just be part of fileutils?
> 
> camel-process.h
>  - should probably be an object, with programmable input/output streams
>  - need a stream-unamed-pipe or something too?
> 
> camel-sasl.h
>  - s/CamelSasl/CamelSASL/
>  - all sasl mechanisms need to be dynamic plugins
>  - service should be first arg to sasl_new
>  - authtypes perhaps should have more direct references to sasl ones?
> camel-sasl-anonymous.h
>  - not sure how this fits into being a plugin
>  - is this used anywhere?
> camel-sasl-cram-md5.h
> camel-sasl-digest-md5.h
>  - uses gal/e-iconv?
> camel-sasl-gssapi.h
> camel-sasl-kerberos4.h
> camel-sasl-login.h
> camel-sasl-ntlm.h
> camel-sasl-plain.h
> camel-sasl-popb4smtp.h
> 
> camel-stream.h
>  - use the glibc printf stuff to printf to a stream where available
>    (uses less copying)
> camel-seekable-stream.h
>  - should this just be based on a normal stream with a seekable flag?
> camel-seekable-substream.h
>  - not sure how useful, only used by camel-multipart.  although when we
> make camel
> camel-stream-buffer.h
>  - should be used more often (in output anyway)
> camel-stream-filter.h
>  - ok
> camel-stream-fs.h
>  - also need a non-seekable fd based stream
> camel-stream-mem.h
>  - maybe this should make the stream out of chunks, a bytearray is
> pretty inefficient
> camel-stream-null.h
> 
> camel-stream-ssl.h
>  - what is this?  if its openssl it should be stream-openssl
> 
> camel-tcp-stream.h
>  - s/CamelTcp/CamelTCP/
> camel-http-stream.h
>  - should probably get this more tested, and use it in evolution?
>  - ftp stream too?
> camel-tcp-stream-raw.h
> camel-tcp-stream-ssl.h
>  - is there any way to get rid of service argument?  e.g. w/ callback?
>    or just use session?
> 
> string-utils.h
>  - fix the filename - camel-string-utils.h

done.

>  - namespace any g_* functions out of g_*

done.

>  - maybe just move the utf8 stuff here
>  - also probably the character classes, and some of the camel-mime-utils
> stuff

Jeff

-- 
Jeffrey Stedfast
Evolution Hacker - Ximian, Inc.
fejj ximian com  - www.ximian.com