termcap/terminfo: Where do not described escape sequences come from?

ogogon

ogogon

Colleagues, tell me, please, where does the escape sequences not described in termcap and terminfo come from in my program?

I use FreeBSD 11.4-RELEASE-p2 amd64
Env TERM=xterm

I wrote a small program that switches stdin to non-canonical mode and reads the input characters.
In particular, this is what the program reads when you press the cursor keys.

Cursor left - 0x1b, 0x5b, 0x44 ("\E[D")
Cursor right - 0x1b, 0x5b, 0x43 ("\E[C")
Cursor up - 0x1b, 0x5b, 0x41 ("\E[A")
Cursor down - 0x1b, 0x5b, 0x42 ("\E[B")

Although, according to both termcap and terminfo, these keys have completely different escape sequences:

Cursor left - "\EOD", Cursor right - "\EOC", Cursor up - "\EOA", Cursor down - "\EOB".

Here is the contents of the tgetent buffer containing the termcap for the xterm, with all the recursions.

Code: 
xterm|X11 terminal emulator:@7=\EOF:@8=\EOM:F1=\E[23~:F2=\E[24~:K2=\EOE:Km=\E[M:k1=\EOP:k2=\EOQ:k3=\EOR:k4=\EOS:k5=\E[15~:k6=\E[17~:k7=\E[18~:k8=\E[19~:k9=\E[20~:k;=\E[21~:kI=\E[2~:kN=\E[6~:kP=\E[5~:kd=\EOB:kh=\EOH:kl=\EOD:kr=\EOC:ku=\EOA:am:bs:km:mi:ms:ut:xn:AX:Co#8:co#80:kn#12:li#24:pa#64:AB=\E[4%dm:AF=\E[3%dm:AL=\E[%dL:DC=\E[%dP:DL=\E[%dM:DO=\E[%dB:LE=\E[%dD:RI=\E[%dC:UP=\E[%dA:ae=\E(B:al=\E[L:as=\E(0:bl=^G:cd=\E[J:ce=\E[K:cl=\E[H\E[2J:cm=\E[%i%d;%dH:cs=\E[%i%d;%dr:ct=\E[3g:dc=\E[P:dl=\E[M:ei=\E[4l:ho=\E[H:im=\E[4h:is=\E[!p\E[?3;4l\E[4l\E>:kD=\E[3~:kb=^H:ke=\E[?1l\E>:ks=\E[?1h\E=:kB=\E[Z:le=^H:md=\E[1m:me=\E[m:ml=\El:mr=\E[7m:mu=\Em:nd=\E[C:op=\E[39;49m:rc=\E8:rs=\E[!p\E[?3;4l\E[4l\E>:sc=\E7:se=\E[27m:sf=^J:so=\E[7m:sr=\EM:st=\EH:ue=\E[24m:up=\E[A:us=\E[4m:ve=\E[?12l\E[?25h:vi=\E[?25l:vs=\E[?12;25h:

Here is the contents of the terminfo.src file containing entries for the xterm, with all the recursions.

Code: 
xterm|xterm terminal emulator (X Window System),
        use=xterm-new,

xterm-new|modern xterm terminal emulator,
        npc,
        indn=\E[%p1%dS, kb2=\EOE, kcbt=\E[Z, kent=\EOM,
        rin=\E[%p1%dT, use=xterm+pcfkeys, use=xterm+tmux,
        use=xterm-basic,

xterm+pcfkeys|fragment for PC-style fkeys,
        use=xterm+app, use=xterm+pcf2, use=xterm+pcc2,
        use=xterm+pce2,

xterm+app|fragment with cursor keys in application mode,
        kcub1=\EOD, kcud1=\EOB, kcuf1=\EOC, kcuu1=\EOA, kend=\EOF,
        khome=\EOH,

xterm+pcf2|fragment with modifyFunctionKeys:2,
        kf1=\EOP, kf10=\E[21~, kf11=\E[23~, kf12=\E[24~,
        kf13=\E[1;2P, kf14=\E[1;2Q, kf15=\E[1;2R, kf16=\E[1;2S,
        kf17=\E[15;2~, kf18=\E[17;2~, kf19=\E[18;2~, kf2=\EOQ,
        kf20=\E[19;2~, kf21=\E[20;2~, kf22=\E[21;2~,
        kf23=\E[23;2~, kf24=\E[24;2~, kf25=\E[1;5P, kf26=\E[1;5Q,
        kf27=\E[1;5R, kf28=\E[1;5S, kf29=\E[15;5~, kf3=\EOR,
        kf30=\E[17;5~, kf31=\E[18;5~, kf32=\E[19;5~,
        kf33=\E[20;5~, kf34=\E[21;5~, kf35=\E[23;5~,
        kf36=\E[24;5~, kf37=\E[1;6P, kf38=\E[1;6Q, kf39=\E[1;6R,
        kf4=\EOS, kf40=\E[1;6S, kf41=\E[15;6~, kf42=\E[17;6~,
        kf43=\E[18;6~, kf44=\E[19;6~, kf45=\E[20;6~,
        kf46=\E[21;6~, kf47=\E[23;6~, kf48=\E[24;6~,
        kf49=\E[1;3P, kf5=\E[15~, kf50=\E[1;3Q, kf51=\E[1;3R,
        kf52=\E[1;3S, kf53=\E[15;3~, kf54=\E[17;3~,
        kf55=\E[18;3~, kf56=\E[19;3~, kf57=\E[20;3~,
        kf58=\E[21;3~, kf59=\E[23;3~, kf6=\E[17~, kf60=\E[24;3~,
        kf61=\E[1;4P, kf62=\E[1;4Q, kf63=\E[1;4R, kf7=\E[18~,
        kf8=\E[19~, kf9=\E[20~,

xterm+pcc2|fragment with modifyCursorKeys:2,
        kLFT=\E[1;2D, kRIT=\E[1;2C, kind=\E[1;2B, kri=\E[1;2A,
        kDN=\E[1;2B, kDN3=\E[1;3B, kDN4=\E[1;4B, kDN5=\E[1;5B,
        kDN6=\E[1;6B, kDN7=\E[1;7B, kLFT3=\E[1;3D, kLFT4=\E[1;4D,
        kLFT5=\E[1;5D, kLFT6=\E[1;6D, kLFT7=\E[1;7D,
        kRIT3=\E[1;3C, kRIT4=\E[1;4C, kRIT5=\E[1;5C,
        kRIT6=\E[1;6C, kRIT7=\E[1;7C, kUP=\E[1;2A, kUP3=\E[1;3A,
        kUP4=\E[1;4A, kUP5=\E[1;5A, kUP6=\E[1;6A, kUP7=\E[1;7A,

xterm+pce2|fragment with modifyCursorKeys:2,
        kDC=\E[3;2~, kEND=\E[1;2F, kHOM=\E[1;2H, kIC=\E[2;2~,
        kNXT=\E[6;2~, kPRV=\E[5;2~, kich1=\E[2~, knp=\E[6~,
        kpp=\E[5~, kDC3=\E[3;3~, kDC4=\E[3;4~, kDC5=\E[3;5~,
        kDC6=\E[3;6~, kDC7=\E[3;7~, kEND3=\E[1;3F, kEND4=\E[1;4F,
        kEND5=\E[1;5F, kEND6=\E[1;6F, kEND7=\E[1;7F,
        kHOM3=\E[1;3H, kHOM4=\E[1;4H, kHOM5=\E[1;5H,
        kHOM6=\E[1;6H, kHOM7=\E[1;7H, kIC3=\E[2;3~, kIC4=\E[2;4~,
        kIC5=\E[2;5~, kIC6=\E[2;6~, kIC7=\E[2;7~, kNXT3=\E[6;3~,
        kNXT4=\E[6;4~, kNXT5=\E[6;5~, kNXT6=\E[6;6~,
        kNXT7=\E[6;7~, kPRV3=\E[5;3~, kPRV4=\E[5;4~,
        kPRV5=\E[5;5~, kPRV6=\E[5;6~, kPRV7=\E[5;7~,
        use=xterm+edit,

xterm+edit|fragment for 6-key editing-keypad,
        kdch1=\E[3~, kich1=\E[2~, knp=\E[6~, kpp=\E[5~,
        use=xterm+pc+edit,

xterm+pc+edit|fragment for pc-style editing keypad,
        kend=\E[4~, khome=\E[1~,

xterm+tmux|advanced xterm features used in tmux,
        Cr=\E]112\007, Cs=\E]12;%p1%s\007,
        Ms=\E]52;%p1%s;%p2%s\007, Se=\E[2 q, Ss=\E[%p1%d q,

xterm-basic|modern xterm terminal emulator - common,
        OTbs, am, bce, km, mir, msgr, xenl, AX, XT,
        colors#8, cols#80, it#8, lines#24, pairs#64,
        acsc=``aaffggiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,
        bel=^G, blink=\E[5m, bold=\E[1m, cbt=\E[Z, civis=\E[?25l,
        clear=\E[H\E[2J, cnorm=\E[?12l\E[?25h, cr=^M,
        csr=\E[%i%p1%d;%p2%dr, cub=\E[%p1%dD, cub1=^H,
        cud=\E[%p1%dB, cud1=^J, cuf=\E[%p1%dC, cuf1=\E[C,
        cup=\E[%i%p1%d;%p2%dH, cuu=\E[%p1%dA, cuu1=\E[A,
        cvvis=\E[?12;25h, dch=\E[%p1%dP, dch1=\E[P, dl=\E[%p1%dM,
        dl1=\E[M, ech=\E[%p1%dX, ed=\E[J, el=\E[K, el1=\E[1K,
        flash=\E[?5h$<100/>\E[?5l, home=\E[H, hpa=\E[%i%p1%dG,
        ht=^I, hts=\EH, ich=\E[%p1%d@, il=\E[%p1%dL, il1=\E[L,
        ind=^J, invis=\E[8m, is2=\E[!p\E[?3;4l\E[4l\E>,
        kmous=\E[M, meml=\El, memu=\Em, op=\E[39;49m, rc=\E8,
        rev=\E[7m, ri=\EM, rmacs=\E(B, rmam=\E[?7l,
        rmcup=\E[?1049l, rmir=\E[4l, rmkx=\E[?1l\E>,
        rmm=\E[?1034l, rmso=\E[27m, rmul=\E[24m, rs1=\Ec,
        rs2=\E[!p\E[?3;4l\E[4l\E>, sc=\E7, setab=\E[4%p1%dm,
        setaf=\E[3%p1%dm,
        setb=\E[4%?%p1%{1}%=%t4%e%p1%{3}%=%t6%e%p1%{4}%=%t1%e%p1%{6}%=%t3%e%p1%d%;m,
        setf=\E[3%?%p1%{1}%=%t4%e%p1%{3}%=%t6%e%p1%{4}%=%t1%e%p1%{6}%=%t3%e%p1%d%;m,
        sgr=%?%p9%t\E(0%e\E(B%;\E[0%?%p6%t;1%;%?%p2%t;4%;%?%p1%p3%|%t;7%;%?%p4%t;5%;%?%p7%t;8%;m,
        sgr0=\E(B\E[m, smacs=\E(0, smam=\E[?7h, smcup=\E[?1049h,
        smir=\E[4h, smkx=\E[?1h\E=, smm=\E[?1034h, smso=\E[7m,
        smul=\E[4m, tbc=\E[3g, vpa=\E[%i%p1%dd, E3=\E[3;J,
        use=ansi+pp, use=xterm+kbs, use=vt100+enq,

ansi+pp|ansi printer port,
        mc5i,
        mc0=\E[i, mc4=\E[4i, mc5=\E[5i,

xterm+kbs|fragment for backspace key,
        kbs=^H,

vt100+enq|ncurses extension for vt100-style ENQ,
        u8=\E[?1;2c, use=ansi+enq,

ansi+enq|ncurses extension for ANSI ENQ,
        u6=\E[%i%d;%dR, u7=\E[6n, u8=\E[?%[;0123456789]c,
        u9=\E[c,

As we can see, nowhere are the cursor keys specified for the above escape sequences.

I ran my program from the machine's text console, from ssh clients on MacOS X and Linux Mint. The results are exactly the same everywhere.

Where do these escape sequences come from? What am I doing wrong?

Thanks in advance for your reply,
Ogogon.
 
If I understand your problem correctly, your keyboard is connected to a FreeBSD machine, which is running an xterm under some graphical user interface. You press the cursor up key, and the xterm sends the character sequence \E[A to the running application, but you expected \EOA. Correct so far?

OK, if you look at the definition of ANSI control sequences (which is what all the xterm stuff is based on), you'll find that ANSI terminals can send either of the two when a cursor key has been pressed. There is a control sequence called DECCKM, which is the two sequences \E[?3h and \E[?3l, which toggle the cursor keys. That's also known as "application cursor key mode".

Why does this even exist? Good question. Some engineer at Digital Equipment (probably in the mill) must have thought, about 45 or 50 years ago, that it was a good idea. I have no idea why, but it has been like this ever since. I have a slight hunch, which is the following: The sequence \E[A when sent to the terminal (not from the cursor key to the application) orders the terminal to move the cursor to the right, and it might have been simpler fo the code to be either able to echo the cursor key immediately back to the terminal, or deliberately distinguish between cursor key presses and echos. Remember, in those days we had machines with about 100K of RAM having to serve a dozen uses, to simplicity was king, and using intelligence in the terminal an important part of surviving.

Personally, I don't have an xterm running on FreeBSD, so I can't test it, but I just tried it on a Mac (using iTerm), and DECCKM works exactly as advertised.

And if you want a reference for all these codes: on Linux machines, "man console_codes" has a good overview, or you can find them by searching the web for "ansi control sequences". Much easier than going through termcap/terminfo.
 
ralphbsz said:
If I understand your problem correctly, your keyboard is connected to a FreeBSD machine, which is running an xterm under some graphical user interface. You press the cursor up key, and the xterm sends the character sequence \E[A to the running application, but you expected \EOA. Correct so far?
Click to expand...
No, somewhat differently.

I have a machine running FreeBSD 11.4 amd64. It doesn't have X11 installed. She has a regular text console.
In most cases, I access it over ssh, from machines running MacOS X or Linux. Not very often, I plug keyboard and monitor into it and login into it from its text console. In all cases, the environment variable TERM is "xterm". (I did not force set the value of the variable, I just read the one set by the system.)

In other words, I had a configuration where was used ssh connection from different clients. But then, to eliminate the potentially nuances associated with this, I ran the program from the non-graphical console of the same machine. Undoubtedly, in the case of the text console, incorrect or too arbitrary behavior of the ssh client is completely ruled out.

The program non-canonically reads its stdin and prints the readed bytes to the screen.
And in the case of a connection via ssh, and in the case of entering from the text console of the same machine, the program behavior is absolutely identical.
 
You're expecting "application mode" sequences like kcuu1/kcud1/kcub1/kcuf1 (ku/kd/kl/kr in termcap), but you didn't enable "application mode" using smkx (ks in termcap), so you're seeing cuu1/cud1/cub1/cuf1 (up/do/nd/le in termcap).

There's a fair bit of history dealing with escape sequences and control sequences on VTxx terminals that can cause a bit of a mess on terminals that recognize the independence of DECCKM being set/reset and DECKPAM/DECKPNM, but most VT100-compatible terminal descriptions (including xterm) keep things simple with the smkx/rmkx pair to turn application mode on/off (ks/ke in termcap).

By the way, FreeBSD still uses the old termcap format instead of terminfo, which is the source of the result of tgetent() in your original post. You can also cross-reference the termcap/terminfo names in terminfo(5) as necessary. For example, search for the termcap name @7 in terminfo(5) and look at the column to the left to find the terminfo name that you will see in the terminfo.src file.
 
You must log in or register to reply here.
Back
Top