Terminal answers

[Anton Monroe <akm meer net>]

Several months ago I suggested that the MC documentation include some
information to help users solve terminal emulation problems.  Since then
I have managed to solve some of my particular problems and discovered a
few things.  Granted, my knowledge is still mostly gaps, but I'll offer
this for discussion, corrections, or additions.

comments in [] are gaps that need to be filled


This is a summary of what I have learned about terminal problems while
trying to get Midnight Commander (MC) to work when I telnet/ssh into a
Linux machine from OS/2.  It is very incomplete, but some of it might
be useful.

First, terminal problems are very common.  They aren't unique to any
application or operating system.  This document is aimed at Midnight
Commander users, but some of it would apply to terminal-emulation
problems in general.

If you are new to Unix-like operating systems, read this part:

   A PC operating system like DOS, Windows, OS/2, etc. deals directly    
   with the hardware.  Using it doesn't (normally) involve the concept
   of a "terminal" at all.

   Say you are using Norton Commander under DOS. You press the <Shift>
   key, then press the <F1> key.  Norton Commander not only knows you
   pressed <Shift>+<F1>, it can look to see which of the two <Shift>
   keys you pressed.  If you press the <End> key, it can tell whether it
   is the grey <End> key or the <End>/<1> key on the number keypad.

   Likewise, a DOS application can display text and colors on the screen
   by writing directly to the memory chip on your video card.

   Unix-based operating systems work differently.  Imagine you are
   sitting at a terminal that consists of just a keyboard and a screen. 
   It is connected by a cable to a computer that may be miles away, with
   Midnight Commander running on it.  Information about what keys you
   press is sent as a string of characters from the terminal to the
   computer.  If you press the <a> key, the terminal sends the letter
   "a".  If you press <Shift>+<a>, it sends the letter "A".  But
   keypresses like <F1> and <Shift>+<F1> are not letters.  The terminal
   sends some string of characters that represents that particular key
   or combination of keys. For instance, it might send "^[[[A" when you
   press the  <F1> key and ^[[25~ when you press <Shift>+<F1>.  (The
   "^[" in those examples represents what is called the escape
   character. It is often used with terminals and printers to indicate
   that the letters that follow have some special meaning and should not
   be interpreted as printable characters.)  Likewise, when MC puts
   information on the terminal's screen it often has to use escape
   sequences to tell the terminal where to put the text and what color
   to use.

   By now, you can guess what the problem is-- A Unix-style application
   like MC has to know what escape sequences your terminal uses. There
   are hundreds of different kinds of terminals, each with its own set
   of escape sequences.  Even worse, they don't all have the same
   capabilities-- some do not have all the keys you are used to on a PC
   keyboard, some have extra keys, they don't all have the same size of
   screen, some don't use color, etc. Each application cannot possibly
   know how to communicate with every kind of terminal. The Terminfo
   database is an attempt to address this problem.

   You say "but I'm not using a terminal, I'm using Linux on a PC!"  But
   you are using a terminal.  It's just that your terminal is in the
   same machine as the computer.  Consider the screen or window that you
   use for MC as a fake terminal provided by Linux.  And even Linux
   provides more than one kind of terminal; the terminal you are using
   may be "linux", "xterm", "eterm", "rxvt", etc.  They each have
   different capabilities.  This is important to understand-- any time
   you use Midnight Commander you must be using either a real terminal
   or some kind of software that pretends to be a terminal.  That is,
   software that provides terminal emulation.


About Terminfo:

   The Terminfo database attempts to solve the problem of multitudes of
   terminal types by providing a file for each terminal that describes
   what it can do and how it does it.  (Termcap is a similar scheme, but
   isn't used much now.  We will ignore Termcap.)  Somewhere on your
   computer is a directory called "terminfo" with a lot of directories
   and files under it.  Each of those files is a description of a
   terminal.  So when you run MC, it checks to see what your terminal
   type is set to.  (To find out, type "echo $TERM" at a command
   prompt.)  If your terminal type is "linux", MC reads the terminfo
   file called "linux" to find out how to talk to your terminal. 
   (Actually, MC does it indirectly, by using either the Slang or
   Ncurses libraries.  MC talks to Slang and Slang talks to the
   terminal.)
   
   Briefly, a terminfo entry defines a standard set of things that a
   terminal might be capable of, and defines how it does them.  For
   instance, it might tell the application that the terminal sends
   "^[[[A" when the <F1> key is pressed,  and that the application can
   send "^[[H^[[J" to clear the screen.
   

Getting Midnight Commander to work:

   In theory, Terminfo should solve the problem.  However, some of the
   terminfo entries are incomplete or wrong, and often a terminal
   emulator doesn't accurately emulate the terminal that it claims to
   be.  The result is that when you use MC, some of your keys don't work,
   or produce the wrong result, or the screen does not display what it
   should.

   Other applications may work correctly with your terminal.  If the
   only  problem you have is with MC, that does not necessarily mean MC
   is faulty.  It probably just means MC uses some capability of your
   terminal that the other application doesn't.  The fault is probably a
   mismatch between your terminal and the terminfo entry. You can try to
   fix your problems in several ways:

   1) Change your $TERM variable to match what your terminal emulator is
   emulating.  If your terminal emulates xterm, set TERM=xterm. People
   sometimes think that changing $TERM will change the capabilities of
   their terminal. It doesn't.  Setting TERM=xterm will not give you an
   xterm terminal, it just tells applications that you have an xterm
   terminal.  If you don't have xterm, nothing good will come of lying
   about it.

   2) MC has a limited ability to fix keyboard problems.  First, go to
   the 'Options/Learn Keys' item on the MC menu.  For instance, if MC
   doesn't recognize your arrow keys, you can teach it what escape
   sequences your terminal sends for those keys.  MC saves the sequences
   in your ~/.mc/ini file.  In fact, the mc.lib file already has many
   additions that fix common problems.  Look at /usr/share/mc/mc.lib or
   /usr/local/share/mc/mc.lib.  Note that the key assignments are listed
   under the terminal name.  If you use MC under several different terminal
   types, you need to fix the key assignments for each one separately.
   [This explanation probably needs to be expanded]

   3) You could try compiling MC with a different screen library. You
   have a choice of using the Slang library that comes with MC, or the
   version of Slang that is (probably) already on your computer, or the
   version of Ncurses that is (probably) already on your computer. Slang
   is reputedly better at dealing with broken terminfo entries than
   Ncurses, and MC uses Slang by default.  But I suppose Ncurses might
   be better in some circumstances.
   
   4) You might be able to reconfigure your terminal emulator.  

   5) It might be worthwhile to get an updated Terminfo database from
   ftp://invisible-island.net/ncurses/terminfo.src.gz.  Of course, there
   is always a chance that the newer version of your terminal's entry
   could be as bad or worse than the one you have.  The database is in a
   file called terminfo.src.  Use the 'tic' command (part of Ncurses) to
   compile and install it.  Some of the comments in terminfo.src might
   be worth reading.
   
   6) If all else fails, you can try to fix your terminfo entry to match
   what your emulator does. It could be that the entry is wrong, or your
   emulator is faulty. Try to figure out which is the case.  If your
   emulator provides a faulty emulation of a terminal, it would probably
   be a mistake to break a good terminfo entry to match it.  
   
   That was my situation. My SSH client for OS/2 claims to provide a
   "substantial subset" of the xterm-color terminal.  It doesn't say
   which version of xterm-color. I doubt if even the various Linux
   distributions all provide exactly the same terminfo files.  None of
   the various xterm-color files I could find were very close to what my
   terminal emulator actually does.  Rather than redefine Terminfo's
   idea of what xterm-color is, I decided to write a new terminfo entry
   that would provide an accurate description of my terminal.  I kept
   the parts of xterm-color that worked or that I didn't understand. 
   Then I tried to fix the other pieces one at a time, with the help of
   'tack' and 'infocmp'.
   

Some tips, in no particular order:
   
   The escape character is often represented by "^[", but Terminfo
   entries use "\E" and MC uses "\e".  
   
   If you want to see what character sequence your terminal sends for
   each key, use the command 'showkey -a'.  
   
   By all means, look at documentation.  Try 'man terminfo', as well as
   the manpages for the individual utilities.
   http://www.tldp.org/HOWTO/Text-Terminal-HOWTO.html might be useful.
   
   Each terminfo capability has a short name and a long name that is
   easier to understand.  Unfortunately, 'tack' and 'tput' only use the
   short forms.  Keep a list handy for reference.

   If you don't have Ncurses, go to 
   http://www.gnu.org/software/ncurses/ncurses.html.

   Ncurses comes with some very useful utilities, like infocmp, tack,
   and tput.

   'infocmp' will decode a terminfo entry into something that is   
   (hopefully) understandable by a human.  To see description of the   
   xterm entry, type "infocmp -L1f xterm | less".  You can redirect
   that into a file, edit the file, and then recompile it into a new
   terminfo entry with "tic xterm.txt".  Infocmp will also let you
   extract capabilities that two entries have in common or that are
   different.

   'tack' is a program to detect errors in a terminfo entry.  It will
   ask you to press keys to see if they are recognized and put things on
   the screen to see if they display correctly. The Alternate Character
   Set section can help figure out why line-drawing characters don't
   display correctly.
   
   'tput' will let you manually send commands to the terminal, using
   their terminfo names.  For instance, "tput clear" should clear the
   screen.  If it doesn't, the 'clear' sequence in your terminfo entry
   is probably wrong.
   
   If you press a cursor or function key and you get a spurious letter
   instead of what you expect, it means the terminal sent a sequence
   that the application doesn't understand.  For instance, you press
   <Home> and the MC editor displays 1~ instead of moving the cursor. 
   The terminal probably sent "^[[1~".  MC didn't recognize that and
   interpreted the last part of it to mean that you typed "1~".
   The easiest fix might be to add this line in your ~/.mc/ini file:
      home=\e[1~
      
   Don't expect perfection. Sometimes terminal emulation problems are
   not fixable, or aren't worth the frustration.  In the MC editor, I
   can use <Shift>+ Arrow keys to mark text, but <Shift>+<End> or
   <Shift>+<Home> insert spurious characters.  I don't understand why.
   I finally just edited ~/.mc/ini to tell MC to interpret them as
   <End> and <Home>, because I would rather have unwanted cursor
   movements than unwanted characters.  In particular, don't try to
   understand all about a terminfo entry; it will drive you mad.
   

Some questions I can't answer:

[How does MC read mc.lib?  If a key is defined in the [terminal:xterm]
section, does that apply only when $TERM = xterm, or to any terminal
that starts with  xterm, like xterm-color?]

[I don't quite understand the files that get installed in
/usr/share/mc/term. It seems like they ought to be mentioned here.
Or are they mostly obsolete?]

[Terminfo entries have a way to change a terminal keyboard from "normal mode"
to "application mode".  I don't really understand that, but I can see it
complicates things.  Does MC use application mode if the terminfo entry
provides for it?  What does a user lose if his terminal doesn't allow it?]

[Under xterm and related terminals, MC has some features that aren't
possible under other terminals.  What exactly are those features?   Is
this somehow related to how MC recognizes <Shift>, <Alt>, <Ctrl> key
combinations?]