I administrate linux servers from time to time, and a common need in those situation is monitoring what a different terminal is doing. Either because of a lost connection to an SSH terminal which runs a command, or wanting to know what an unrecognized terminal is doing. To that end I use a very useful tool called strace.
Strace is a system utility on linux systems that can trace all systems calls and signals that a process uses. To monitor a terminal we can make strace print all occurrences of the read and write system calls that the terminal process is using.
Note: This tutorial assumes you have root privileges otherwise you won't be able to use strace to connect and monitor a running process.
To see who’s connected to the machine we can list /dev/pts. The directory contains special character files for each connected pts.
To use strace for the task at hand, we first need to identify the terminal we want to connect to. We can use ps which lists all the running processes like so:
alpha@milkyway:~$ ps aux | grep pts alpha 943 0.0 0.0 44472 3344 pts/4 R+ 09:23 0:00 ps aux alpha 944 0.0 0.0 21536 1048 pts/4 S+ 09:23 0:00 grep --color=auto pts alpha 7978 0.0 0.0 29812 5172 pts/3 Ss+ 09:16 0:00 bash alpha 22659 0.0 0.0 29812 5192 pts/0 Ss 00:13 0:00 bash root 22877 0.0 0.0 72716 4304 pts/0 S 00:24 0:00 sudo su alchemist root 22878 0.0 0.0 72248 3884 pts/0 S 00:24 0:00 su alchemist alchemi+ 22891 0.0 0.0 29808 5100 pts/0 S+ 00:24 0:00 bash alpha 23022 0.0 0.0 29812 5104 pts/1 Ss 00:28 0:00 bash root 23034 0.0 0.0 72956 4424 pts/1 S 00:28 0:00 sudo su root 23035 0.0 0.0 72248 3780 pts/1 S 00:28 0:00 su root 23049 0.0 0.0 28712 3924 pts/1 S+ 00:28 0:00 bash alpha 24095 0.0 0.0 29812 5144 pts/2 Ss+ 01:15 0:00 bash
To see all the input and output of the pts we’re interested in we can call strace with the following arguments:
strace -f -s999 -e trace=write -p 7978
-f: This option tells strace to follow the system calls for the child processes as well (commands run in the terminal).
-s: by default strace truncates strings, we can set the truncate limit higher so we can see more of the output.
-e trace=write: the system calls we want to monitor. For our task we only want to see the output in the terminal. If you want to log what the user types you can add the read call as well. The format is trace=syscalls1,syscall2,..
-p: The pid of the process we’re monitoring.
The output is useful, but it would be more useful if the output was more human suitable. You’re in luck, I wrote a simple wrapper to make the output appear like a normal terminal. You can download it from here.