User Input: Keyboard, Mouse

Keyboard

The user can input text using their keyboard, or on Android, using the on-screen keyboard which is open using a button beneath the screen. Supported are all printable characters, as well as many control keys, such as arrows, _Ctrl+letters_ and function keys. Sequences sent by function keys are based on VT102 and Xterm.

The codes sent by _Home_, _End_, _F1-F4_ and cursor keys are affected by various keyboard modes (_Application Cursor Keys_, _Application Numpad Mode_, _SS3 Fn Keys Mode_). Some can be set in the Terminal Settings, others via commands.

Here are some examples of control key codes:

KeyCodeKeyCode
Up `\e[A`,~`\eOA` F1 `\eOP`,~`\e[11\~`
Down `\e[B`,~`\eOB` F2 `\eOQ`,~`\e[12\~`
Right `\e[C`,~`\eOC` F3 `\eOR`,~`\e[13\~`
Left `\e[D`,~`\eOD` F4 `\eOS`,~`\e[14\~`
Home `\eOH`,~`\e[H`,~`\e[1\~` F5 `\e[15~`
End `\eOF`,~`\e[F`,~`\e[4\~` F6 `\e[17\~`
Insert `\e[2\~` F7 `\e[18\~`
Delete `\e[3\~` F8 `\e[19\~`
Page Up `\e[5\~` F9 `\e[20\~`
Page Down `\e[6\~` F10 `\e[21\~`
Enter `\r` (13) F11 `\e[23\~`
Ctrl+Enter `\n` (10) F12 `\e[24\~`
Tab `\t` (9) ESC `\e` (27)
Backspace `\b` (8) Ctrl+A..Z ASCII 1-26

Action buttons

The blue buttons under the screen send ASCII codes 1, 2, 3, 4, 5, which incidentally correspond to _Ctrl+A,B,C,D,E_. This choice was made to make button press parsing as simple as possible.

Mouse

ESPTerm implements standard mouse tracking modes based on Xterm. Mouse tracking can be used to implement powerful user interactions such as on-screen buttons, draggable sliders or dials, menus etc. ESPTerm's mouse tracking was tested using VTTest and should be compatible with all terminal applications that request mouse tracking.

Mouse can be tracked in different ways; some are easier to parse, others more powerful. The coordinates can also be encoded in different ways. All mouse tracking options are set using option commands: `CSI ? _n_ h` to enable, `CSI ? _n_ l` to disable option _n_.

Mouse Tracking Modes

All tracking modes produce three numbers which are then encoded and send to the application. First is the _event number_ N, then the _X and Y coordinates_, 1-based.

Mouse buttons are numbered: 1=left, 2=middle, 3=right. Wheel works as two buttons (4 and 5) which generate only press events (no release).

OptionNameDescription
`9` *X10~mode* This is the most basic tracking mode, in which only button presses are reported. N = button - 1: (0 left, 1 middle, 2 right, 3, 4 wheel).
`1000` *Normal~mode* In Normal mode, both button presses and releases are reported. The lower two bits of N indicate the button pressed: `00b` (0) left, `01b` (1) middle, `10b` (2) right, `11b` (3) button release. Wheel buttons are reported as 0 and 1 with added 64 (e.g. 64 and 65). Normal mode also supports tracking of modifier keys, which are added to N as bit masks: 4=_Shift_, 8=_Meta/Alt_, 16=_Control/Cmd_. Example: middle button with _Shift_ = 1 + 4 = `101b` (5).
`1002` *Button-Event tracking* This is similar to Normal mode (`1000`), but mouse motion with a button held is also reported. A motion event is generated when the mouse cursor moves between screen character cells. A motion event has the same N as a press event, but 32 is added. For example, drag-drop event with the middle button will produce N = 1 (press), 33 (dragging) and 3 (release).
`1003` *Any-Event tracking* This mode is almost identical to Button Event tracking (1002), but motion events are sent even when no mouse buttons are held. This could be used to draw on-screen mouse cursor, for example. Motion events with no buttons will use N = 32 + _11b_ (35).
`1004` *Focus~tracking* Focus tracking is a separate function from the other mouse tracking modes, therefore they can be enabled together. Focus tracking reports when the terminal window (in Xterm) gets or loses focus, or in ESPTerm's case, when any user is connected. This can be used to pause/resume a game or on-screen animations. Focus tracking mode sends `CSI I` when the terminal receives, and `CSI O` when it loses focus.

Mouse Report Encoding

The following encoding schemes can be used with any of the tracking modes (except Focus tracking, which is not affected).

OptionNameDescription
-- *Normal~encoding* This is the default encoding scheme used when no other option is selected. In this mode, a mouse report has the format `CSI M _n_ _x_ _y_`, where _n_, _x_ and _y_ are characters with ASCII value = 32 (space) + the respective number, e.g. 0 becomes 32 (space), 1 becomes 33 (!). The reason for adding 32 is to avoid producing control characters. Example: `\e[M !!` - left button press at coordinates 1,1 when using X10 mode.
`1005` *UTF-8~encoding* This scheme should encode each of the numbers as a UTF-8 code point, expanding the maximum possible value. Since ESPTerm's screen size is limited and this has no practical benefit, this serves simply as an alias to the normal scheme.
`1006` *SGR~encoding* In SGR encoding, the response looks like a SGR sequence with the three numbers as semicolon-separated ASCII values. In this case 32 is not added like in the Normal and UTF-8 schemes, because it would serve nor purpose here. Also, button release is not reported as 11b, but using the normal button code while changing the final SGR character: `M` for button press and `m` for button release. Example: `\e[2;80;24m` - the right button was released at row 80, column 24.
`1015` *URXVT~encoding* This is similar to SGR encoding, but the final character is always `M` and the numbers are like in the Normal scheme, with 32 added. This scheme has no real advantage over the previous schemes and was added solely for completeness.