Re: Your mc efforts is a great job !!! [Warning]: Long

[Nikolai Bezroukov <speditor optonline net>]

[Warning] Long, as I reiterated some points from the prev discussion.

Pavel Roskin wrote:

[snip]
> I'm not sure you are talking about mcfs. mcfs doesn't allow you to
> execute any remote commands other than whatever is already
> supported by mcserver. That's the standard file operations supported 
by FTP
> plus a few goodies like chmod.

I do not know the details of the actual implementation, but IMHO the 
idea itself is very elegant althouth slightly ahead of the codebase with 
its hardwired hotkeys. It's better to be preserved and extended it to 
the full functionality. We should  not throw out the child with a water. 

Your concerns about security might be slightly overstated, as IPsec, SSL 
or similar  secure or semi-secure channel can be used.  Moreover on 
switched networks that now prevail, the session is not that easy to 
sniff,  if switches are configured correctly and hardened against ARP 
poisoning. For example in the 3COM switches (SuperStack II) enabling 
"Port Security" on a port makes the switch remember the first MAC 
address it receives and locks that MAC address to that port until 
overridden by manual intervention. After that any hacker can hit the 
wall with his head as long as he wish. Also any attempt to use ARP 
poisoning in corp. environment usually trigger IDS.

Here I am discussing the general ("theoretical") value of client-server 
mode for this type of managers. And it is in this lights I stated in  
http://www.softpanorama.org/OFM/Ofm_04.shtml  and I am still convinced 
that mc client-server capabilities provides an important,  pioneering 
step in the right direction. If  keystrokes are parameterized into a 
"command language", not hardwired like in the current implementation, 
then this client server model is very natural with the server 
interpreting internal "command language" and client converting 
keystrokes to the command stream and updating panels.

The key advantage is the unique powerful and very attractive blend of 
FTP and telnet functionality in one interface. IMHO such a functionality 
might constitute a new protocol,  "a command line VNC", if you wish.

Probably equally important is the possibility of  avoiding those 
horrible "Alice in Wonderland" key binding problems that litter this 
list. The server is immune to the host key bindings and you can use just 
one client on, say, the most "mc-friendly"  Linux flavor (RH8 with the 
gnome-terminal 2.0.1-5 ;-) to administer all your servers.  That  
instantly makes mc much more appealing tool to the system administrators 
who need to work with a lot of servers and might help to increase the 
development resources by attracting new talented developers. 

[snip]

> I was confused by "VFS" meaning two different things on adjacent
> lines. I still don't know what NC5 XTree VFS is. I have never used NC5.

That's how I call "flat file VFS". As a PhD I have the responsibility to 
obscure things by inventing new terms ;-). See also 
http://www.softpanorama.org/OFM/Ofm_00.shtml.  The book discuses several 
implementations including NC5.

[snip]

> > The idea is to have access to top and bottom directory in the
> > current panel (which of course depends of your sorting mode) via 
hotkeys.
> > As I said, I'm not aware of such key in mc.

It's actually not defined either in OFM1999 standard or in OFM2004 
standard  (see http://www.softpanorama.org/OFM/Ofm_08.shtml and 
http://www.softpanorama.org/OFM/Ofm_09.shtml ). Here is what OFM1999 is 
saying  (sorry, it's slightly Windows-biased ;-) :

   The hotkey assignment for sorting is optional.  If the hotkey is
   defined its action should correspond to the F9-(LR)-S sequence for
   the current panel and display the same menu so that the proper order
   can be selected with the second key from the menu. Alternatively,
   Ctrl-F3, Ctrl-F4, Ctrl-F5, Ctrl-F6 and Ctrl-F7 can be used for
   sorting by name, extension, modification time, size and unsorted,
   correspondingly ("NETSU" order).

For Unix you need to add the ability to sort by sort by permissions, 
owner and group.  So the menu selection string became "NETSUPOG".

As for the hotkey, actually, how about Ctrl-A ?  A very convenient 
hotkey for a frequently used operation  (and resorting of columns is 
definitely very frequently used, at least by advanced users). Actually 
you are in a better position to suggest a suitable hotkey as I do not 
really understand all the tradeoffs of the Unix environment especially 
the potential conflicts with Emacs/XEmacs (I use Vim/Gvim).

[snip]

> > >>But it looks like the major architectural weakness of MC is the
> > >>absence of a build-in scripting language. There are rudimentary
> > >>attempts to invent some rudimentary scripting languages is some 
parts
> > >>of mc (for example in user menu), but they are just weak and
> > >>inconsistent attempts to solve a generic problem by inventing ad hoc
> > >>mini-languages for each case. Currently probably the best way to
> > >>script mc is to use a programmable keyboard (like EnduraPro/104
> > >>FOCUS FK-9200, or MCK-142) or use Expect.

> > >Again, it's something that needs to be implemented and maintained. 
There
> > >is not enough manpower to maintain even the existing featured in a 
good
> > >shape.

Very true, and that's my point too.  Here I would like to reiterate some 
points as we moved the discussion.

I consider your efforts to be really outstanding because you not only 
managed to revitalize the development, but also stop the feature creep. 
Without your efforts mc probably would scale down to version 4.35.  IMHO 
two of your recent decisions  (1) to clean the code of GUI-related staff 
and (2) Windows-port related staff were very timely.

Two things are now more or less clear: 

   -  mc is in the stabilization phase and it's level of maturity is
   reasonably high,

   -  the development resources are very scarce and just keeping up
   with the usual flow of bugs strains the resources.

That's why I think it make sense to bite the bullet and move this 
"cleaning" strategy on a new level by using the capabilities of S-lang. 
Adding S-lang and exposing relevant internal functions might took the 
same amount of efforts as a regular cleaning of code that is needed 
anyway,  but with much better return on the investment. Iit gives you an 
opportunity to concentrate on simplifying and refactoring the core 
codebase by removing several ad hoc mini-languages invented in usual 
hacker-style traditions. For example, the code implementing "user 
extension predicates" and "user menu visibility predicates" are two 
examples of mini-languages, implementations of which can instantly be 
replaced with S-lang with no or very little effort.

Also the pool of people who can develop/polish scriptable parts is 
bigger/deeper.  That's a natural separation of labor that the current 
development environment is lacking:  you need to be a jack of all trades 
and react to each and every hotkey problem (actually even in todays 
situation several things from the dscussions can probably be added to 
FAQ to cut the number of such issues).  Assuming the core and scripting 
part are separated you have a core team and the extension development 
community like in Emacs. that's a more powerful combination and I 
suspect that more people might be willing to help with the scripting,  
but few would dig into the current C codebase.

>> That might be not that simple and there might be some savings.
>> For example slang interpreter may be easily embedded
>> into mc to make it extensible. And that provides an  opportunity to 
cut "feature creep"
>> that takes a lot of efforts to
>> maintain by implementing some features via macros.

> I'd rather go with something more suitable for writing anything from 
syscalls to multi-dimension
> arrays. Also, it would be  helpful if the language was specifically 
designed to be embedded.
> I thing Perl or Guile  will be fine, but I didn't research
> this question. Anyway, it's  all in the far future.

True, but paradise is unachievable on the earth, you know :-). Something 
is always missing.   And that especially true about embeddable 
languages. I  mentioned S-lang because it might be the easiest to 
implement. The library is used by mc already, so it's almost in.  
Actually for system calls,  S-lang's "intrinsic function" mechanism can 
be used (see jed codebase). S-lang also has arrays and hashes, so it 
looks more or less  adequate for the purpose.  But the most important 
point is that it's already *almost in*.

Moreover S-lang usage in mc may provide stimulus for further improvement 
of both the S-lang  and its interpreter and thus lift some or all of the 
current limitations/reservations. There also may be some kind of 
cross-pollination / integration with jed, which is a much better editor 
then cooledit , so this is actually not a one time tradeoff when you can 
lose of win,  but a dynamic situation, a potentially mutually beneficial 
two-way process.

It is true that S-lang has its own warts and limitations, but  please 
understand that is such situation there is no best choice, only the 
worst choice and this might be inaction.  The truth is that even the 
worst choice (let's say adopting bash ;-) is better than none: scripting 
language is one of the few available avenues of addressing the feature 
creep problem .

The other solution might be exposing API and introducing a FAR-style 
plug-ins mechanism, although they have their own problems and having a 
two dozens or more of them installed lead to "overcrowding of the 
kitchen". But technically this is as simple as introducing plug-in menu 
in addition to the user menu or option to call a plug-in in the user menu.

I would agree with your point that  Perl might be a more "politically 
correct" candidate due to its very wide appeal to system administrators. 
It probably provides the best opportunity to enlarge the community of mc 
developers and users. As the success of vim-Perl, myperl and plperl had 
shown in VIM, MySQL and Postgress development communities, respectively, 
any product that adopts Perl as a scripting language instantly enjoys 
some additional share of developers and users.

Guile is essentially "TCL on steroids" or "TCLized Scheme" so, while 
like Perl it is capable to separate the core development and extensions 
development, the fact that it's Lisp-based means that it is does not has 
the appeal of Perl and probably will  be *less* appreciated by the 
current userbase (outside hard-core Emacs hackers crowd). In this sense 
it's equal to S-lang.

Both Perl and Guile are definitely more complex and labor-consuming 
undertakings that adopting S-lang, so pro and contra issues need to be 
weighted by  resources available for any such implementation.

Anyway, the earlier the decision is made and the transition plan 
outlined, the easier is to fight feature creep and clean the codebase. 
As Emacs demonstrated to the world long ago, a scripting language 
provides the natural separation of core functionality and user 
extensions. Moreover some non-essential, but man-power hungry features 
can be temporary suspended or even removed with the promise to be later 
reincarnated in the form of scripts.

If the recent crisis, that you resolved by stepping in,  is any 
indication about the future, a large pure-C open-source implementation 
like mc might not be sustainable due to the level of complexity and 
those of them, which cannot not attract enough resources need either to 
scale down of face the possibility of extinction. Just because the pool 
of really knowledgeable C developers is shrinking and they are in fact 
pretty close to becoming an endangered specie.  We all like free lunch 
but somebody needs to heat it even if the food is free ;-)

That's why making this decision and taking some steps now, when the mc 
development is clearly revitalized because of your efforts is, in my 
opinion,  so important.

Best Regards,

Dr. Nikolai Bezroukov