fish upload worked around, need better fix

[Pavel Roskin <proski gnu org>]

Hello!

I have finally found what's going on with upload over fish, which doesn't
work reliably with large files (100k and more).

The problem is that "dd" (GNU fileutils-4.1) doesn't reread a block if
read() returns less bytes than requested.  Unfortunately, this happens
when dd gets data from ssh (OpenSSH 3.4, linux 2.4.19-rc1, glibc 2.2.4).  
I believe that at ~100k dd just gets all the data that ssh has available,
and interprets the end of data as a "partial block".

The solution would be to use blocks 1 byte long - read() is guaranteed to
return at least one byte it it's successful.  But this approach would use
too many resources on the remote machine, since dd actually would call
read() and write() for every byte.  Using a bigger obs (output block size)  
helps, but not much, since we still have one read() per byte.

I don't know of any widespread and portable alternative to dd that would
read a certain number of bytes and then quit (unlike e.g. gzip, that will
search for the next file), that would reread partial blocks (unlike dd),
and that would not require non-trivial software to decode it's output (od
would need sed or awk capable of writing binary zeroes).

Maybe I'm missing a very simple solution.

The largest block that works for me is 256 bytes long.  That's the patch
I'm applying now, and it fixes fish upload for me, but better ideas are
welcome.  There is no guarantee that dd won't get partial reads with 256
bytes blocks on other operating systems or with outher versions of libc.

===================================
--- ChangeLog
+++ ChangeLog
@@ -2,2 +2,5 @@
 
+	* fish.c (file_store): Reduce input block size to 256 bytes,
+	otherwise dd gets partial blocks.
+
 	* ftpfs.c (login_server) [HSC_PROXY]: Fix crash.
--- fish.c
+++ fish.c
@@ -486,22 +486,22 @@ file_store(vfs *me, vfs_s_fh *fh, char *
     print_vfs_message(_("fish: store %s: sending command..."), name );
     /*
      * FIXME: Limit size to unsigned long for now.
-     * Files longer than 4096 * ULONG_MAX are not supported.
+     * Files longer than 256 * ULONG_MAX are not supported.
      */
     if (command (me, super, WAIT_REPLY,
 		 "#STOR %lu /%s\n"
 		 "> \"/%s\"\n"
 		 "echo '### 001'\n"
 		 "(\n"
-		   "dd bs=4096 count=%lu\n"
+		   "dd ibs=256 obs=4096 count=%lu\n"
 		   "dd bs=%lu count=1\n"
 		 ") 2>/dev/null | (\n"
 		   "cat > \"/%s\"\n"
 		   "cat > /dev/null\n"
 		 "); echo '### 200'\n",
 		 (unsigned long) s.st_size, name, name,
-		 (unsigned long) (s.st_size >> 12),
-		 ((unsigned long) s.st_size) & (4096 - 1), name)
+		 (unsigned long) (s.st_size >> 8),
+		 ((unsigned long) s.st_size) & (256 - 1), name)
 	!= PRELIM) 
         ERRNOR(E_REMOTE, -1);
 
===================================

-- 
Regards,
Pavel Roskin