______ Laboratory: Adding a system call to Linux (CS 273 (OS), Fall 2020)
Home
>>    




Laboratory: Adding a system call to Linux

CS 273 (OS), Fall 2020

Instructions below are not yet revised for Fall 2019
ref for next time?

Note the following instructions are for a 64-bit x86 kernel, kernel version 5.0.21.

User-level invocation and implementation of system calls

Recall that each process carries out user-level code (written by the programmer) and kernel-level code. Here are the steps that take place when user-level code makes a system call.

  • User program calls a system-call library function, e.g., fork() or open().

  • Each system-call library function is implemented as an assembly language call to the operating system, in terms of a system call number __NR_call. For example, __NR_fork has the value 57; __NR_getuid has the value 102.

    These system call numbers must be defined for compiling both user-level code (e.g., when defining the C library functions such as fork() and getuid()) and for kernel source code (for implementing those system calls in the kernel).

    • For user-level code, the numbers are defined in the header file /usr/include/x86_64-linux-gnu/asm/unistd_64.h on your virtual machine's file system. This file is ordinarily accessed by including 

          #include <sys/syscall.h>
      when compiling user-level C code on your virtual machine.

    • For kernel code in your Linux source tree in /usr/src/linux-5.0.21/ directory on your virtual OS's file system, the numbers are defined for kernel computations in the kernel source file arch/x86/entry/syscalls/syscall_64.tbl. This file generates C source files such as arch/x86/include/generated/asm/syscalls_64.h (which only exists in the source tree after compilation).

    In spite of the different formats of these three files, note that they all associate the same system call numbers to specific system calls.

    Note: As illustrated above, we will use these fonts to distinguish between file path locations:

    • This font   indicates a file in the distributed Linux source code.

    • This font   indicates a "generated" file in the Linux source tree (only exists after using make to build the kernel).

    • This font   indicates a file in a user file system (and not in a kernel tree).

  • See lib.c for an example of user-level code that creates a new library function dub that invokes the system call dup2 (system call number 33). This uses the system call syscall to invoke the system call, within a new function dub(). This file lib.c is a system-call library module for user-level programmers to access your new system call.

    The file lib.h is a header file for using the functions defined in your library lib.c. Also, the file trylib.c shows how to use lib.h in a C program.

  • To compile a user-level program trylib.c that uses your system-call library function defined in lib.c, log into your virtual system, then carry out these steps.

    Note: Carry out these steps as an ordinary user on your virtual machine.

    • Use scp or another program to copy the files ~rab/os/lib.c, ~rab/os/lib.h, and ~rab/os/trylib.c from a link machine to a directory ~/testing under your (unprivileged) account's home directory on your virtual machine.

      Spring 2018 notes:

      • Your virtual OS may not recognize a link-computer name such as rns202-5.cs.stolaf.edu, so you may need to use a numerical IP address such as 162.210.91.22 instead. To determine the numerical IP address of a link machine, you can log in to a link machine (not on your virtual machine) and enter 

        % arp rns202-5.cs.stolaf.edu
rns202-5.cs.stolaf.edu (162.210.91.22) -- no entry
        Here % represents the shell prompt on a link machine, and the second line indicates the output from the arp program, which includes the numerical IP address for that computer rns202-5.

        • arp is the Address Resolution Protocol program, which attempts to show how domain-style names such as rns202-5.cs.stolaf.edu are translated into numerical IP addresses such as 162.210.91.22 .)

      • Now, on your virtual machine, log into (or switch users to) your unprivileged user account and enter 

        $ mkdir ~/testing
$ cd ~/testing
$ scp  username@ipaddress:~rab/os/lib.h  .
$ scp  username@ipaddress:~rab/os/lib.c  .
$ scp  username@ipaddress:~rab/os/trylib.c  .
        where username is your St. Olaf username (e.g., rab) and ipaddress is the numerical IP address of a link machine (e.g., 162.210.91.22).

    • Compile your library code. 

        $ gcc -c lib.c
      Here, the character $ represents whatever prompt your (non-root) user receives. (These steps could also be carried out by the root user, of course, but carrying them out with an ordinary user may become important for your system-call project later.)

    • Compile your test program. 

        $ gcc -c trylib.c

    • Link these modules to produce an executable. 

        $ gcc -o trylib trylib.o lib.o

    • Run your executable.

        $ ./trylib

    Notes:

    • The command ./trylib is shown for running your program, because the directory . may not appear in your ordinary user's path by default, due to security considerations.

    • Expected behavior: The code trylib.c calls the library function dub2() defined in lib.c which performs system call number 33 (otherwise known as dup2()). The call dub2(1, 5) thus should return the specified alternate file descriptor 5 for standard output. trylib.c then performs a write() call with that alternate file descriptor, which should print Hello, world! on standard output. The program should print 3 lines of output that reflect these steps.

Kernel-level implementation of system calls

  • In the kernel sources for our architecture (64-bit x86 processors, kernel version 5.0.21), system call numbers are defined in a file arch/x86/entry/syscalls/syscall_64.tbl. For example, this table specifies that system call number 1 is for write, and number 57 is for fork.

  • When a system call is performed in a running kernel, the kernel looks up the handler function for that system call using an array data structure called the system call table. (Don't confuse this runtime data structure in main memory with the source file ending in tbl on disk files!) The system call number is used as an index into this array to find that handler function.

Adding a system call to the kernel

    I. Decide names and specs

  1. Write a spec for your new system call. This forces you to make decisions about the system call name, arguments, etc., and can be used to describe your system call in your project report. The return value should be integer, with the value -1 indicating an error condition, as with other system calls. We will use the name rab_mycall for this example; you can use your own initials instead of rab. (Including your initials or username as part of the system call name will make it easier to avoid naming conflicts and easier to identify which calls are new.)

  2. Determine a system call number for your new call. Computationally, you can use any number that doesn't appear in arch/x86/entry/syscalls/syscall_64.tbl and is less than the generated value of the macro __NR_syscall_max. For this class you should choose the first unused system-call number in arch/x86/entry/syscalls/syscall_64.tbl, which is 335 for our setup in the case of your first new system call.

  3. Choose a name for your handler function, which will carry out the steps of your system call. In this example, we will choose the name __x64_sys_rab_mycall.

    • The prefix __x64_sys is required for system-call handlers in Linux version 5.0.21.

    • For this class, follow this   __x64_sys_uname_    pattern for your handler names, where uname is your username, to make it easier to identify your handlers.

    4. II. Update header files and the system call table

  4. Add a new line near the end of arch/x86/entry/syscalls/syscall_64.tbl to specify the information for your new system call. In our example, we add a line 

        335    common   rab_mycall    __x64_sys_rab_mycall

    The second column indicates machine-level calling conventions for your system-call handler, i.e., how parameters are passed. Technically, this is called the ABI, or Application Binary Interface choice. For our assumed 64-bit setup, common is the usual ABI; for a 32-bit setup, the only ABI choice available is i386.

    5. You only need to specify the first four columns in your new line; the fifth column (for 64-bit handler function names) is copied from the fourth column by default.

    Note: Use sudo to edit source files, e.g., 

        $ sudo emacs

    III. Add source code for your new system call

  5. Add the definition of your handler function (e.g., __x64_sys_rab_mycall) to the kernel sources.

    For this example, we will define rab_mycall to be a clone of Linux's getpid system call.

    • The system-call index page provides a list of all system calls in the (unmodified) Linux 5.0.21 source. This indicates that getpid is defined in the source file kernel/sys.c at line 891, in the following lines: 

          SYSCALL_DEFINE0(getpid)
    {
        return task_tgid_vnr(current);
    }
      Here, SYSCALL_DEFINE0(getpid) is a call of a preprocessor macro that produces the function header for the function sys_getpid. The source code uses that macro SYSCALL_DEFINE0 because the system call getpid has no arguments.

      Note: The source code uses macros SYSCALL_DEFINED1, SYSCALL_DEFINED2, etc., for system calls that require arguments. For example, using system-call index we see the 2-argument system call getpriority is defined on line 266 of that same file kernel/sys.c, using the preprocessor macro call 

          SYSCALL_DEFINE2(getpriority, int, which, int, who)
      (together with a much larger function body than getpid() above). Here,

      • getpriority is the name of the system call whose handler (sys_getpriority()) is being defined;

      • the second and third arguments of SYSCALL_DEFINE2 specify that sys_getpriority's first argument is named which and has type int; and

      • the fourth and fifth arguments of SYSCALL_DEFINE2 specify that sys_getpriority's second argument is named who and has type int.

    • Insert the following lines just before or just after the definition of getpid: 

          SYSCALL_DEFINE0(rab_mycall)
    {
        return task_tgid_vnr(current);
    }

    Note: We are adding this handler function definition to an existing source file. It is also possible to define your system calls in a separate new file of source-code, but making and testing minimal modifications is best when trying something new, because it's easier to isolate sources of error if something goes wrong (incremental development).

    6. IV. Build and test

  6. Recompile the kernel (in the top-level directory /usr/src/5.0.21), which creates a new kernel linux that also implements your new system call.

    • Note: If you don't need to make a configuration change, you can skip configuration and potentially reduce recompile time to a fraction of a complete recompile.

      However, modifying files that are #included in a lot of other files, such as syscalls.tbl and files generated by it, will also lengthen recompiles. (But modifying a source file involving a system call that has already been entered in syscall.tbl typically leads to a much shorter recompile.)

  7. Don't forget to install your new kernel if necessary. (This appears to happen automatically in Spring 2018.)

  8. Boot your new kernel, and login to a user account. The following three steps are performed within this login process.

  9. In your user level library source (e.g., lib.c) described above, make a new library function for the new system call (e.g., rab_mycall()), using the same system call number (e.g., __NR_rab_mycall) as you determined for the kernel. Also, add a declaration of your library function to the header file for that library, e.g., lib.h.

  10. Write a test program (named, e.g., trylib.c) that calls your new system call, so you can determine whether it is working properly.

    For the example, the new system call rab_mycall should behave exactly like the existing system call getpid, so a good test would be to print the results of both system calls.

  11. Compile your library and test program, and link them to create an executable, as described above. Run your test program in a terminal window on your virtual machine, and check for the desired behavior.

    For the example, verify whether the system calls getpid and rab_newcall return the same number (PID for the trylib process), as we expect. For an additional, more thorough test, you could add a call to sleep(15) (check the manual page...) to trylib and check that the ps program returns the same PID value, by running the following command in another terminal window within your running VM during the sleep() delay in trylib: 

    $  ps auxww | grep trylib

Note on debugging: If anything goes wrong, study the procedure above carefully to determine the stage when the error must have taken place. For instance, if something goes wrong with the example rab_newcall:

  • If you encounter a linking error when compiling trylib.c, such as the function rab_newcall() not being found, then look for an error involving your user-level library lib.c. You may have forgotten to link in your library lib.o (during step 10), or there may be an error when defining the library function rab_newcall() within lib.c (step 9), etc.

  • If you encounter a system message that your new system call doesn't exist, an examination of the steps above leads to several possible causes.

    • The system call number chosen in step 2 and used in steps 4 and 9 enables your user program to access your new system-call handler. Thus, there could be a problem involving the system call number __NR_rab_mycall - did you use the same number (e.g., 335) both within the kernel (in arch/x86/entry/syscalls/syscall_64.tbl) and in your user-level library lib.c? This error could also arise from failing to set up that system-call number in one of these locations.

    • The line you add to arch/x86/entry/syscalls/syscall_64.tbl within the kernel in step 4 is supposed to connect the system call number (e.g., 335) to your new handler function __x64_sys_rab_mycall, which is defined in step 5 using a macro SYSCALL_DEFINEn(). Check both of these steps to insure that the handler function was in fact defined and entered in syscall_64.tbl. Don't forget to look for a potential misspelling of "rab_mycall," etc.

    • It may well be that you did not boot with a new kernel that includes your system call. This could happen if you didn't recompile the kernel (step 6), or forgot to install your new kernel (step 7), or installed it but booted the wrong kernel (step 8). To determine which kernel you are using, try 

      $  uname -a
      which will typically display which version of kernel sources were used (5.0.21) and the time when that kernel's compile finished.

    • Also, insure that you are running your test program in a terminal window on your running virtual machine (step 11), not on a link computer or your laptop.

TO UPDATE: Adding files to the kernel

TO UPDATE: You can optionally define new system calls and other relevant code in new source files, instead of modifying source files from the Linux 5.0.21 distribution. See Patrick's wiki page for more information.




rab@stolaf.edu, October 25, 2020