TNY_HEADER_FLAG_CACHED and TNY_HEADER_FLAG_PARTIAL

[Philip Van Hoof <spam pvanhoof be>]

Hey Dirk-Jan, and the mailing list,

Today there's something wrong with my Internet connection. My ISP is
somehow redirecting all detected HTTP traffic to a page that explains
that "their world wide web service is experiencing difficulties". Sigh.

All other ports work perfectly. I can of course setup a bunch of proxies
on one of my servers and stuff like that. However. It's not that urgent
and I can E-mail the diff, right?

This diff implements setting the two new flags:

TNY_HEADER_FLAG_CACHED and TNY_HEADER_FLAG_PARTIAL

The first means that the message is cached, the second one means that
the message is partially cached. The second one is only on of the first
one is on and the message is indeed partially cached.

A requirement was to make it possible to know whether (1) header-only is
received, (2) full e-mail or (3) emails without attachments.

This maps to:

(1): TNY_HEADER_FLAG_CACHED is off and TNY_HEADER_FLAG_PARTIAL is off
(2): TNY_HEADER_FLAG_CACHED is on and TNY_HEADER_FLAG_PARTIAL is off
(3): TNY_HEADER_FLAG_CACHED is on and TNY_HEADER_FLAG_PARTIAL is on

A next requirement is to make it possible to limit the amount of E-mails
and the individual message size of the E-mail.

Maybe we should do some chatting about this, Dirk-Jan? :-) Like how the
API for this should look?

My current idea is to add a size attribute to the CamelMessageInfoBase
structure and perhaps reflect that to a an API like "ssize_t
tny_header_get_msg_size (TnyHeader *self)" or something like that.

The possibility to choose whether or not to download it, can be
implemented in a TnyMsgReceiveStrategy: 

Yes it can, that type is not only implementable so that you can classify
between partial and full. You can also implement a "don't-receive-it"
strategy of course. For example one that returns a message that says
"this message is too big, it won't fit on this device. Consider turning
on partial retrieval.".


TnyCamelRefuseMsgReceiveStrategy inherits TnyCamelFullMsgReceiveStrategy
{
	TnyMsg * perform_get_msg (TnyHeader *header)
	{

		if (tny_header_get_msg_size (header) > 10000)
		{
			if (!refuse_msg)
				refuse_msg = create_refuse_msg ();
			return refuse_msg;
		} else
			return super->perform_get_msg (header);
	}

}

TnyFolder *folder = ...
TnyCamelRefuseMsgReceiveStrategy *s = new ...
TnyMsg *msg; TnyHeader *header = ...;
TnyMsgView *view = ...;

tny_folder_set_msg_receive_strategy (folder, s);
msg = tny_folder_get_msg (folder, header);

tny_msg_view_set_msg (view, msg);


Yadi yada ... in other words: let's talk about this? Monday?


-- 
Philip Van Hoof, software developer
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
http://www.pvanhoof.be/blog



Index: docs/devel/reference/tmpl/tny-enums.sgml
===================================================================
--- docs/devel/reference/tmpl/tny-enums.sgml	(revision 1404)
+++ docs/devel/reference/tmpl/tny-enums.sgml	(working copy)
@@ -121,6 +121,8 @@
 @TNY_HEADER_FLAG_ANSWERED_ALL: Unknown
 @TNY_HEADER_FLAG_JUNK: This message is detected as junk mail
 @TNY_HEADER_FLAG_SECURE: Unknown
+ TNY_HEADER_FLAG_CACHED: The message is cached
+ TNY_HEADER_FLAG_PARTIAL: The message is retrieved partially 
 @TNY_HEADER_FLAG_FOLDER_FLAGGED: Unknown
 @TNY_HEADER_FLAG_JUNK_LEARN: This message is flagged to be junk-learned
 @TNY_HEADER_FLAG_USER: Unknown
Index: libtinymail-camel/camel-lite/camel/camel-data-cache.h
===================================================================
--- libtinymail-camel/camel-lite/camel/camel-data-cache.h	(revision 1404)
+++ libtinymail-camel/camel-lite/camel/camel-data-cache.h	(working copy)
@@ -33,6 +33,7 @@
 
 #include <camel/camel-stream.h>
 #include <camel/camel-exception.h>
+#include <camel/camel-folder-summary.h>
 
 #define CAMEL_DATA_CACHE_TYPE     (camel_data_cache_get_type ())
 #define CAMEL_DATA_CACHE(obj)     (CAMEL_CHECK_CAST((obj), CAMEL_DATA_CACHE_TYPE, CamelFolder))
@@ -95,6 +96,8 @@
 void         camel_data_cache_set_partial (CamelDataCache *cache, const char *path,
 					      const char *uid, gboolean partial);
 
+void camel_data_cache_set_flags (CamelDataCache *cdc, const char *path, CamelMessageInfoBase *mi);
+
 /* Standard Camel function */
 CamelType camel_data_cache_get_type (void);
 
Index: libtinymail-camel/camel-lite/camel/providers/pop3/camel-pop3-folder.c
===================================================================
--- libtinymail-camel/camel-lite/camel/providers/pop3/camel-pop3-folder.c	(revision 1404)
+++ libtinymail-camel/camel-lite/camel/providers/pop3/camel-pop3-folder.c	(working copy)
@@ -121,6 +121,13 @@
 	}
 }
 
+static void
+camel_pop3_summary_set_extra_flags (CamelFolder *folder, CamelMessageInfoBase *mi)
+{
+	CamelPOP3Store *pop3_store = (CamelPOP3Store *)folder->parent_store;
+	camel_data_cache_set_flags (pop3_store->cache, "cache", mi);
+}
+
 CamelFolder *
 camel_pop3_folder_new (CamelStore *parent, CamelException *ex)
 {
@@ -132,9 +139,10 @@
 	
 	folder = CAMEL_FOLDER (camel_object_new (CAMEL_POP3_FOLDER_TYPE));
 	camel_folder_construct (folder, parent, "inbox", "inbox");
-	
+
 	summary_file = g_strdup_printf ("%s/summary.mmap", p3store->root);
 	folder->summary = camel_folder_summary_new (folder);
+	folder->summary->set_extra_flags_func = camel_pop3_summary_set_extra_flags;
 	camel_folder_summary_set_build_content (folder->summary, TRUE);
 	camel_folder_summary_set_filename (folder->summary, summary_file);
 
@@ -322,6 +330,8 @@
 			camel_message_info_free (mi);
 	}
 
+	camel_folder_summary_save (folder->summary);
+
 	if (pop3_store->engine->capa & CAMEL_POP3_CAP_UIDL) {
 		camel_pop3_engine_command_free(pop3_store->engine, pcu);
 	} else {
Index: libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-store.c
===================================================================
--- libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-store.c	(revision 1404)
+++ libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-store.c	(working copy)
@@ -1165,7 +1165,7 @@
 	cache = camel_imap_message_cache_new (folder_dir, summary, ex);
 	if (cache)
 		camel_imap_message_cache_clear (cache);
-	
+
 	camel_object_unref (cache);
 	camel_object_unref (summary);
 	
Index: libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-summary.c
===================================================================
--- libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-summary.c	(revision 1404)
+++ libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-summary.c	(working copy)
@@ -34,6 +34,7 @@
 #include "camel-imap-summary.h"
 #include "camel-imap-utils.h"
 #include "camel-file-utils.h"
+#include "camel-imap-folder.h"
 
 #define CAMEL_IMAP_SUMMARY_VERSION (3)
 
@@ -107,10 +108,18 @@
 }
 
 static void
+camel_imap_summary_set_extra_flags (CamelFolder *folder, CamelMessageInfoBase *mi)
+{
+	CamelImapFolder *imap_folder = CAMEL_IMAP_FOLDER (folder);
+	camel_imap_message_cache_set_flags (imap_folder->folder_dir, mi);
+}
+
+static void
 camel_imap_summary_init (CamelImapSummary *obj)
 {
 	CamelFolderSummary *s = (CamelFolderSummary *)obj;
 
+	s->set_extra_flags_func = camel_imap_summary_set_extra_flags;
 	/* subclasses need to set the right instance data sizes */
 	s->message_info_size = sizeof(CamelImapMessageInfo);
 	s->content_info_size = sizeof(CamelImapMessageContentInfo);
Index: libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-folder.c
===================================================================
--- libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-folder.c	(revision 1404)
+++ libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-folder.c	(working copy)
@@ -243,6 +243,10 @@
 	}
 
 	folder = CAMEL_FOLDER (camel_object_new (camel_imap_folder_get_type ()));
+	imap_folder = CAMEL_IMAP_FOLDER (folder);
+
+	imap_folder->folder_dir = g_strdup (folder_dir);
+
 	short_name = strrchr (folder_name, '/');
 	if (short_name)
 		short_name++;
@@ -267,13 +271,14 @@
 	g_free(state_file);
 	camel_object_state_read(folder);
 
-	imap_folder = CAMEL_IMAP_FOLDER (folder);
 	imap_folder->cache = camel_imap_message_cache_new (folder_dir, folder->summary, ex);
+
 	if (!imap_folder->cache) {
 		camel_object_unref (CAMEL_OBJECT (folder));
 		return NULL;
 	}
 
+
 	if (!g_ascii_strcasecmp (folder_name, "INBOX")) {
 		if ((imap_store->parameters & IMAP_PARAM_FILTER_INBOX))
 			folder->folder_flags |= CAMEL_FOLDER_FILTER_RECENT;
@@ -429,6 +434,9 @@
 	if (imap_folder->cache)
 		camel_object_unref (CAMEL_OBJECT (imap_folder->cache));
 
+	if (imap_folder->folder_dir)
+		g_free (imap_folder->folder_dir);
+
 #ifdef ENABLE_THREADS
 	g_static_mutex_free(&imap_folder->priv->search_lock);
 	g_static_rec_mutex_free(&imap_folder->priv->cache_lock);
Index: libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-folder.h
===================================================================
--- libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-folder.h	(revision 1404)
+++ libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-folder.h	(working copy)
@@ -53,6 +53,7 @@
 	unsigned int need_rescan:1;
 	unsigned int need_refresh:1;
 	unsigned int read_only:1;
+	gchar *folder_dir;
 };
 
 typedef struct {
Index: libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-message-cache.c
===================================================================
--- libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-message-cache.c	(revision 1404)
+++ libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-message-cache.c	(working copy)
@@ -328,8 +328,25 @@
 	return insert_finish (cache, uid, path, key, stream);
 }
 
+void
+camel_imap_message_cache_set_flags (const gchar *folder_dir, CamelMessageInfoBase *mi)
+{
+	char mystring [512];
+	snprintf (mystring, 512, "%s/%s.", folder_dir, mi->uid);
+	if (g_file_test (mystring, G_FILE_TEST_IS_REGULAR))
+	{
+		mi->flags |= CAMEL_MESSAGE_CACHED;
+		snprintf (mystring, 512, "%s/%s.partial", folder_dir, mi->uid);
+		if (g_file_test (mystring, G_FILE_TEST_IS_REGULAR))
+			mi->flags |= CAMEL_MESSAGE_PARTIAL;
+		else
+			mi->flags &= ~CAMEL_MESSAGE_PARTIAL;
+	} else {
+		mi->flags &= ~CAMEL_MESSAGE_CACHED;
+		mi->flags &= ~CAMEL_MESSAGE_PARTIAL;
+	}
+}
 
-
 gboolean
 camel_imap_message_cache_is_partial (CamelImapMessageCache *cache, const char *uid)
 {
Index: libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-message-cache.h
===================================================================
--- libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-message-cache.h	(revision 1404)
+++ libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-message-cache.h	(working copy)
@@ -107,6 +107,9 @@
 					      const char *dest_uid,
 					      CamelException *ex);
 
+void camel_imap_message_cache_set_flags (const gchar *folder_dir, CamelMessageInfoBase *mi);
+
+
 /* Standard Camel function */
 CamelType camel_imap_message_cache_get_type (void);
 
Index: libtinymail-camel/camel-lite/camel/camel-folder-summary.c
===================================================================
--- libtinymail-camel/camel-lite/camel/camel-folder-summary.c	(revision 1404)
+++ libtinymail-camel/camel-lite/camel/camel-folder-summary.c	(working copy)
@@ -131,6 +131,8 @@
 	g_free (key);
 }
 
+static void do_nothing (CamelFolder *folder, CamelMessageInfoBase *mi) { }
+ 
 static void
 camel_folder_summary_init (CamelFolderSummary *s)
 {
@@ -142,6 +144,7 @@
 	s->dump_lock = g_mutex_new ();
 	s->message_info_size = sizeof(CamelMessageInfoBase);
 	s->content_info_size = sizeof(CamelMessageContentInfo);
+	s->set_extra_flags_func = do_nothing;
 
 #if defined (DOESTRV) || defined (DOEPOOLV)
 	s->message_info_strings = CAMEL_MESSAGE_INFO_LAST;
@@ -1954,9 +1957,11 @@
 	ptrchr = camel_file_util_mmap_decode_uint32 (ptrchr, &mi->flags, FALSE);
 
 	mi->flags &= ~CAMEL_MESSAGE_INFO_NEEDS_FREE;
-	if (uidmf)
+	if (uidmf) /* TNY TODO: Is this correct? Please debug and check */
 		mi->flags &= ~CAMEL_MESSAGE_INFO_UID_NEEDS_FREE;
 
+	s->set_extra_flags_func (s->folder, mi);
+
 #ifdef NON_TINYMAIL_FEATURES
 	ptrchr = camel_file_util_mmap_decode_uint32 (ptrchr, &mi->size, FALSE);
 #else
Index: libtinymail-camel/camel-lite/camel/camel-data-cache.c
===================================================================
--- libtinymail-camel/camel-lite/camel/camel-data-cache.c	(revision 1404)
+++ libtinymail-camel/camel-lite/camel/camel-data-cache.c	(working copy)
@@ -226,6 +226,32 @@
 	g_dir_close(dir);
 }
 
+
+void 
+camel_data_cache_set_flags (CamelDataCache *cdc, const char *path, CamelMessageInfoBase *mi)
+{
+	char mystring [512];
+	gchar *mpath; char *dir;
+	guint32 hash;
+	hash = g_str_hash(mi->uid);
+	hash = (hash>>5)&CAMEL_DATA_CACHE_MASK;
+
+	snprintf (mystring, 512, "%s/%s/%02x/%s", cdc->path, path, hash, mi->uid);
+
+	if (g_file_test (mystring, G_FILE_TEST_IS_REGULAR))
+	{
+		mi->flags |= CAMEL_MESSAGE_CACHED;
+		snprintf (mystring, 512, "%s/%s/%02x/%s.ispartial", cdc->path, path, hash, mi->uid);
+		if (g_file_test (mystring, G_FILE_TEST_IS_REGULAR))
+			mi->flags |= CAMEL_MESSAGE_PARTIAL;
+		else
+			mi->flags &= ~CAMEL_MESSAGE_PARTIAL;
+	} else {
+		mi->flags &= ~CAMEL_MESSAGE_CACHED;
+		mi->flags &= ~CAMEL_MESSAGE_PARTIAL;
+	}
+}
+
 gboolean
 camel_data_cache_is_partial (CamelDataCache *cdc, const char *path,
 					      const char *uid)
Index: libtinymail-camel/camel-lite/camel/camel-folder-summary.h
===================================================================
--- libtinymail-camel/camel-lite/camel/camel-folder-summary.h	(revision 1404)
+++ libtinymail-camel/camel-lite/camel/camel-folder-summary.h	(working copy)
@@ -79,8 +79,10 @@
 	CAMEL_MESSAGE_JUNK = 1<<7,
 	CAMEL_MESSAGE_SECURE = 1<<8,
 	CAMEL_MESSAGE_FREED = 1<<9,
-	
 
+	CAMEL_MESSAGE_CACHED = 1<<10,
+	CAMEL_MESSAGE_PARTIAL = 1<<11,
+
 	/* following flags are for the folder, and are not really permanent flags */
 	CAMEL_MESSAGE_FOLDER_FLAGGED = 1<<16, /* for use by the folder implementation */
 
@@ -213,6 +215,7 @@
 	CAMEL_SUMMARY_DIRTY = 1<<0,
 } CamelFolderSummaryFlags;
 
+
 struct _CamelFolderSummary {
 	CamelObject parent;
 
@@ -245,6 +248,8 @@
 	unsigned char *filepos;
 	GMutex *dump_lock;
 	gboolean in_reload;
+
+	void (*set_extra_flags_func) (CamelFolder *folder, CamelMessageInfoBase *mi);
 };
 
 struct _CamelFolderSummaryClass {
@@ -294,6 +299,7 @@
 	gboolean (*info_set_user_flag)(CamelMessageInfo *mi, const char *id, gboolean state);
 	gboolean (*info_set_user_tag)(CamelMessageInfo *mi, const char *id, const char *val);
 	gboolean (*info_set_flags)(CamelMessageInfo *mi, guint32 mask, guint32 set);
+
 };
 
 /* Meta-summary info */
Index: ChangeLog
===================================================================
--- ChangeLog	(revision 1404)
+++ ChangeLog	(working copy)
@@ -1,3 +1,12 @@
+2007-01-14  Philip Van Hoof <pvanhoof gnome org>
+
+	* Implementation for TNY_HEADER_FLAG_CACHED and
+	TNY_HEADER_FLAG_PARTIAL for IMAP and POP3
+	* Introduction of TNY_HEADER_FLAG_CACHED and 
+	TNY_HEADER_FLAG_PARTIAL in the TnyHeader type
+
+	* This was a major API change
+
 2007-01-13  Philip Van Hoof <pvanhoof gnome org>
 
 	* Introduction of the TnyLockable type
Index: libtinymail/tny-enums.h
===================================================================
--- libtinymail/tny-enums.h	(revision 1404)
+++ libtinymail/tny-enums.h	(working copy)
@@ -79,6 +79,8 @@
 	TNY_HEADER_FLAG_ANSWERED_ALL = 1<<6,
 	TNY_HEADER_FLAG_JUNK = 1<<7,
 	TNY_HEADER_FLAG_SECURE = 1<<8,
+	TNY_HEADER_FLAG_CACHED = 1<<10,
+	TNY_HEADER_FLAG_PARTIAL = 1<<11,
 	TNY_HEADER_FLAG_FOLDER_FLAGGED = 1<<16,
 	TNY_HEADER_FLAG_JUNK_LEARN = 1<<30,
 	TNY_HEADER_FLAG_USER = 1<<31
Index: libtinymail/tny-header.h
===================================================================
--- libtinymail/tny-header.h	(revision 1404)
+++ libtinymail/tny-header.h	(working copy)
@@ -62,6 +62,8 @@
 	TNY_HEADER_FLAG_ANSWERED_ALL = 1<<6,
 	TNY_HEADER_FLAG_JUNK = 1<<7,
 	TNY_HEADER_FLAG_SECURE = 1<<8,
+	TNY_HEADER_FLAG_CACHED = 1<<10,
+	TNY_HEADER_FLAG_PARTIAL = 1<<11,
 	TNY_HEADER_FLAG_FOLDER_FLAGGED = 1<<16,
 	TNY_HEADER_FLAG_JUNK_LEARN = 1<<30,
 	TNY_HEADER_FLAG_USER = 1<<31