Device Drivers¶

Note: we will consider all commands from here on out as being run from the linux directory (source of your kernel tree).

1. Creating a shared folder¶

Let's make a shared folder between the host machine and the VM.

First we'll create a folder to be shared from our host machine:

mkdir ../shared_folder

Now we make a shared folder inside the virtual machine (we are going to add some more flags on our qemu command):

qemu-system-x86_64 \
    -drive file=../my_disk.raw,format=raw,index=0,media=disk \
    -m 2G -nographic \
    -kernel ./arch/x86_64/boot/bzImage \
    -append "root=/dev/sda rw console=ttyS0 loglevel=6" \
    -fsdev local,id=fs1,path=../shared_folder,security_model=none \
    -device virtio-9p-pci,fsdev=fs1,mount_tag=shared_folder \
    --enable-kvm

Now we need to determine the mountpoint of the shared folder, i.e., what directory inside the VM our shared folder will be mounted to. For this, inside the VM, edit the file /etc/fstab with your preferred editor (nano or vi), and add this line at the end of file:

# <device> <mountpoint> <type> <options> <dump> <pass>
shared_folder /root/host_folder 9p trans=virtio 0 0

Now reboot your VM and check if there is a /root/host_folder with the same files and folder of shared_folder in the host machine. For now, your shared_folder is still empty, so the existence of the /root/host_folder directory should be enough proof that it's working (you can create a random file inside it from the host just to check if it shows up in the VM as well).

2. Making a module¶

2.1 Compiling it out-of-tree¶

A module is a piece of kernel code that is loadable at runtime. On regular Linux systems, a sizeable chunk of the kernel's functionality is shipped as modules, instead of being embedded in the kernel image itself.

Let's look at the minimum boilerplate needed to make a kernel module. We'll start by creating a file hello.c inside the shared folder:

// SPDX-License-Identifier: GPL-2.0-only
/*  
 *  hello.c - The simplest kernel module.
 */
#include    <linux/init.h>      /* Needed for __init/__exit */
#include    <linux/module.h>    /* Needed for module_init/module_exit/MODULE_* */

static int __init hello_init(void)
{
    pr_info("Hello world\n");

    return 0; // A non 0 return value means init_module failed; module can't be loaded.
}

static void __exit hello_exit(void)
{
    pr_info("Goodbye world\n");
}

module_init(hello_init);
module_exit(hello_exit);

MODULE_AUTHOR("Your Name <your.email@domain.com>");
MODULE_DESCRIPTION("A simple hello / goodbye driver");
MODULE_LICENSE("GPL");

This module will print a 'Hello world' message to the kernel log when loaded, and a 'Goodbye' message when removed.

In the same folder, create a file called Makefile with the following content:

obj-m += hello.o

Compile out-of-tree module:

make -C . M=../shared_folder hello.ko

Now you have compiled your module, you can boot your VM with the shared folder:

qemu-system-x86_64 \
    -drive file=../my_disk.raw,format=raw,index=0,media=disk \
    -m 4G -nographic \
    -kernel ./arch/x86_64/boot/bzImage \
    -append "root=/dev/sda rw console=ttyS0 loglevel=6" \
    -fsdev local,id=fs1,path=../shared_folder,security_model=none \
    -device virtio-9p-pci,fsdev=fs1,mount_tag=shared_folder \
    --enable-kvm

And inside the VM, load the module:

cd host_folder
insmod hello.ko

Check the log:

dmesg

Unload your module and check the logs again:

rmmod hello
dmesg

You can now exit the VM.

3.2 Compiling it in-tree¶

Now we are going to make the module inside the kernel tree. So we are going to copy our work to the drivers folder in the linux kernel project:

mkdir drivers/lkcamp
cp ../shared_folder/hello.c drivers/lkcamp
cp ../shared_folder/Makefile drivers/lkcamp

Now, let's add a flag of configuration for the module to appear on make menuconfig. Let's create a file named Kconfig on drivers/lkcamp folder, with the following contents:

config HELLO
    default m
    tristate "Hello module"
    help
        This module prints the following message when loaded: 'hello',
        as well as the following when removed: 'goodbye'.
        This module was created for educational purposes, as part
        of the Linux Kernel Development Workshop organized by LKCAMP.
        The default configuration is 'm': the module will be compiled
        and it will be available to be loaded in execution time.

The Kconfig file will create a config option that can be toggled from menuconfig.

With the Kconfig file created, we need to make sure that this module configuration is actually being used when the driver is compiled. To do so, edit the drivers/lkcamp/Makefile file to replace the previous hard-coded m in obj-m with the value assigned to the CONFIG_HELLO variable:

obj-$(CONFIG_HELLO) += hello.o

We also need to include the reference to the new driver in the parent folder drivers/. To do this, add the following line to the drivers/Makefile file:

obj-y               += lkcamp/

And, in drivers/Kconfig, at the end of the file, before endmenu, add the line:

source "drivers/lkcamp/Kconfig"

Now you should be able to see your module on the menuconfig (from the root of the source tree):

make menuconfig

Search for your module: type /, followed by HELLO and then press Enter. You will notice that the entry for our newly added config option has a (1) next to it; that means you can press 1 and jump directly to it, instead of finding your way through the menus.

Or, if you prefer a more manual way, you can go into the Device Drivers option inside of menuconfig, then go down the list until you find your new module. If you added your driver source to the last line of Kconfig, the new module will be on the last line of the list.

You can see it is set by default with the m option. You can exit the menu and compile it.

Let's compile our kernel with the new module:

make -j$(nproc)

You should notice that at some point the following showed up while compiling the kernel:

  CC [M]  drivers/lkcamp/hello.o

Now we need to actually install the modules (e.g., copy them to the directory they live in, just like regular programs live in /bin, /usr/bin, etc). Normally, on Linux systems, they're installed to the /lib/modules/$(uname -r) folder (take a look inside that directory on your machine, for example).

The kernel already offers a convenient target for this: modules_install. However, if we run it on our host, the modules will get installed to our host's /lib/modules/, which is not what we want; we need them to live in our VM's filesystem. For this, we'll mount our VM disk again and use the INSTALL_MOD_PATH variable to define the path where the modules will be installed to:

sudo mount ../my_disk.raw ../mnt_partition
sudo make INSTALL_MOD_PATH=../mnt_partition modules_install
sudo umount ../mnt_partition

Now that we have added the modules to our VM rootfs, we can boot our machine:

qemu-system-x86_64 -drive file=../my_disk.raw,format=raw,index=0,media=disk -m 4G -nographic -kernel ./arch/x86_64/boot/bzImage -append "root=/dev/sda rw console=ttyS0 loglevel=6" -fsdev local,id=fs1,path=../shared_folder,security_model=none -device virtio-9p-pci,fsdev=fs1,mount_tag=shared_folder --enable-kvm

You can search for your module in /lib/modules to confirm that it was installed correctly:

find /lib/modules -name "hello*"

Now you can load/unload it and check the results:

modprobe hello
dmesg | tail
modprobe -r hello
dmesg | tail

dmesg | tail

3. Implementing more complex modules¶

3.1 Passing command line arguments to a module¶

Let's declare a variable to be passed as argument when initializing modules, add these lines on hello.c file:

int myint = -1;
module_param(myint, int, 0);

Then print the value when initialize the module, on init_module add this line:

    pr_info("Initialize with the value: %d\n", myint);

// SPDX-License-Identifier: GPL-2.0-only
/*
*  hello.c - The simplest kernel module.
*/
#include        <linux/init.h>          /* Needed for __init/__exit */
#include        <linux/module.h>        /* Needed for module_init/module_exit/MODULE_* */

int myint = -1;
module_param(myint, int, 0);

static int __init hello_init(void)
{
        pr_info("Hello world\n");
        pr_info("I received the following argument: myint=%d\n", myint);

        return 0; // A non 0 return means init_module failed; module can't be loaded.
}

static void __exit hello_exit(void)
{
        pr_info("Goodbye world\n");
}

module_init(hello_init);
module_exit(hello_exit);

MODULE_AUTHOR("Your Name <your.email@domain.com>");
MODULE_DESCRIPTION("A simple hello / goodbye driver");
MODULE_LICENSE("GPL");

Now you can compile and test it, same as before:

make -j$(nproc)
sudo mount ../my_disk.raw ../mnt_partition
sudo make INSTALL_MOD_PATH=../mnt_partition modules_install
sudo umount ../mnt_partition

qemu-system-x86_64 -drive file=../my_disk.raw,format=raw,index=0,media=disk -m 4G -nographic -kernel ./arch/x86_64/boot/bzImage -append "root=/dev/sda rw console=ttyS0 loglevel=6" -fsdev local,id=fs1,path=../shared_folder,security_model=none -device virtio-9p-pci,fsdev=fs1,mount_tag=shared_folder --enable-kvm

And inside your VM:

modprobe hello myint=10
dmesg | tail

You can add other arguments types, check https://tldp.org/LDP/lkmpg/2.6/html/x323.html for more.

3.2 Implementing a char driver module¶

For the next exercise we will consider a simple character driver. It only stores a status, which can be ON or OFF. You can query the current status by reading from it, and set it to ON or OFF by writing 1 or 0 to it, respectively.

The following code implements this behavior:

// SPDX-License-Identifier: GPL-2.0-only
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/uaccess.h>
#include <linux/fs.h>
#include <linux/cdev.h>
#include <linux/string.h>

static dev_t lkcamp_dev; // Holds the major and minor number for our driver
static struct cdev lkcamp_cdev; // Char device. Holds fops and device number

// Possible states for the driver
enum driver_state {
    STATUS_OFF = 0,
    STATUS_ON  = 1,
};

static enum driver_state status = STATUS_OFF;
static const char *status_strings[] = {"OFF\n", "ON\n"};

static ssize_t lkcamp_read(struct file *file, char __user *buf, size_t size,
               loff_t *ppos)
{
    // Return the string corresponding to the current driver state
    return simple_read_from_buffer(buf, size, ppos, status_strings[status],
                       strlen(status_strings[status]));
}

static ssize_t lkcamp_write(struct file *file, const char __user *buf,
                size_t size, loff_t *ppos)
{
    char value;

    // Copy the first character written to this device to 'value'
    if (copy_from_user(&value, buf, 1))
        return -EFAULT; // Something went very wrong

    if (value == '0')
        status = STATUS_OFF;
    else if (value == '1')
        status = STATUS_ON;
    else
        return -EINVAL;

    return 1; // We only read one character from the written string
}

// Define the functions that implement our file operations
static struct file_operations lkcamp_fops =
{
    .read = lkcamp_read,
    .write = lkcamp_write,
};

static int __init lkcamp_init(void)
{
    int ret;

    // Allocate a major and a minor number
    ret = alloc_chrdev_region(&lkcamp_dev, 0, 1, "lkcamp");
    if (ret)
        pr_err("Failed to allocate device number\n");

    // Initialize our character device structure
    cdev_init(&lkcamp_cdev, &lkcamp_fops);

    // Register our character device to our device number
    ret = cdev_add(&lkcamp_cdev, lkcamp_dev, 1);
    if (ret)
        pr_err("Char device registration failed\n");

    pr_info("LKCAMP driver initialized!\n");

    return 0;
}

static void __exit lkcamp_exit(void)
{
    // Clean up our mess
    cdev_del(&lkcamp_cdev);
    unregister_chrdev_region(lkcamp_dev, 1);

    pr_info("LKCAMP driver exiting!\n");
}

module_init(lkcamp_init); // Register our functions so they get called when our
module_exit(lkcamp_exit); // module is loaded and unloaded

MODULE_AUTHOR("LKCAMP");
MODULE_DESCRIPTION("LKCAMP's incredibly useful char driver");
MODULE_LICENSE("GPL");

You can add this code to a new file named lkcamp_char.c on your shared folder, let's compile it out-tree first. Just add this line to the Makefile:

obj-m       += lkcamp_char.o

You can compile it as before (out of tree):

make -C path/to/linux M=../shared_folder lkcamp_char.ko

sudo mount ../my_disk ../mnt_partition
sudo chroot ../mnt_partition

apt install strace man

sudo umount ../mnt_partition

You can boot into your VM and play with your module.

qemu-system-x86_64 -drive file=../my_disk.raw,format=raw,index=0,media=disk -m 4G -nographic -kernel ./arch/x86_64/boot/bzImage -append "root=/dev/sda rw console=ttyS0 loglevel=6" -fsdev local,id=fs1,path=../shared_folder,security_model=none -device virtio-9p-pci,fsdev=fs1,mount_tag=shared_folder --enable-kvm

Inside your VM, let's load and test it.

cd host_folder
insmod lkcamp_char.ko

As you load the module, you should see the "LKCAMP driver initialized!" message print on the kernel log.

Now to talk to the driver we have to create a character special file using mknod /dev/lkcamp c <driver major> 0. You should substitute <driver major> to its major number. But since it was dynamically allocated through alloc_chrdev_region(&lkcamp_dev, 0, 1, "lkcamp");, how can we know it?

The file you just created can now be used to read from and write to the driver. Query the drive's status with:

cat /dev/lkcamp

You can turn it off and on, respectively, with:

OFF:

echo -n '0' > /dev/lkcamp

ON:

echo -n '1' > /dev/lkcamp

You can try echo '0' > /dev/lkcamp, you will see an error message.

write(1, "0\n", 2) = 1
write(1, "\n", 1)  = -1 EINVAL (Invalid argument)

After you get bored of turning the driver on and off and checking its status, you can unload it with modprobe -r hello_char to see its exit message.

3.2.1 Add your char device in-tree¶

To add your char device in-tree follow similar steps for the previous device: you can add it on the same folder (in this case, just add new definitions on Makefile and Kconfig inside lkcamp folder), otherwise you can create a new folder and change configuration on the parent folder.

Makefile:

obj-$(CONFIG_LKCAMP_CHAR) += lkcamp_char.o

Kconfig:

config LKCAMP_CHAR
    default m
    tristate "LKCamp char module"
    help
        Provides a char module, to understand the 'read' and
        'write' functions of modules.
        This module was created for educational purposes, inside
        a lkcamp activity for linux kernel development workshop.
        The default configuration is 'm': the module will be compiled
        and it will be available to be loaded in execution time.