WeensyOS Minilab 1

The WeensyOS problem sets are a series of little coding exercises that are also complete operating systems. You could boot a WeensyOS operating system on real x86-compatible hardware! The purpose of the WeensyOS exercises is first, to teach some of the concepts we use in class through example, and second, to demystify operating systems in general. I also hope they are sort of fun.

The first WeensyOS problem set concerns threads. The ThreadOS is a tiny operating system that supports the major thread primitives: creating a new thread, exiting a thread, and joining a thread that has exited. In this problem set, you will actually implement the code that creates a new thread. In the process, you'll see how system calls are implemented. You will also update the code that joins an exited thread so that user threads don't busy-wait.

In this simple problem set, you'll browse, partially understand, and change these tiny operating systems.

Handing in

You will electronically hand in code and a small writeup containing answers to the numbered exercises. The problem set code, weensyos1.tar.gz, unpacks into a directory called weensyos1. (We explain how to unpack it below.) You'll modify the code in this directory, and add a text file with your answers to the numbered exercises. When you're done, run the command gmake tarball. This should create a file named weensyos1-yourusername.tar.gz. You'll turn in this file to CourseWeb.

Answer the numbered exercises by editing the file named answers.txt. No Microsoft Word documents (or other binary format, except for PDF in special cases) will be accepted! For coding exercises, it's OK for answers.txt to just refer to your code (as long as you comment your code).

To review:

1. Download weensyos1.tar.gz and unpack it.
2. Do your work in the weensyos1 directory.
3. Put your answers to the numbered exercises in the answers.txt file in that directory.
4. When you're done, run gmake tarball from the weensyos1 directory. This will create a file named weensyos1-yourusername.tar.gz.
5. Submit that weensyos1-yourusername.tar.gz file to CourseWeb.

Setting up

You could take one of the disk image files this minilab builds, write it to your laptop's hard drive, and boot up your operating system directly if you wanted! However, it's much easier to work with a virtual machine or PC emulator.

An emulator mimics, or emulates, the behavior of a full hardware platform. A PC emulator acts like a Pentium-class PC: it emulates the execution of Intel x86 instructions, and the behavior of other PC hardware. For example, it can treat a normal file in your home directory as an emulated hard disk; when the program inside the emulator reads a sector from the disk, the emulator simply reads 512 bytes from the file. PC emulators are much slower than real hardware, since they do all of the regular CPU's job in software -- not to mention the disk controller's job, the console's job, and so forth. However, debugging with an emulator is a whole lot friendlier, and you can't screw up your machine!

We've used two PC emulators. The Bochs emulator has pretty nice debugging support. The QEMU package is fast and sleek, but it might be too fast for some of our purposes. If you work on your own machine, try QEMU. If you're interested in working from home, you can download the source for QEMU and/or Bochs and install your own copy using these instructions. Precompiled binaries for Windows and Mac OS X are available too.

You will also need a copy of GCC that compiles code for an x86 ELF target. Recent Linux PCs have the right compiler already set up. We've set up all the required tools on the machines in the Linux lab, and the Solaris machines on SEASnet. In the Linux lab, no special setup is required. On SEASnet, you need to set your environment to use our tools.

Read the minilab tools page and set up your environment appropriately.

Troubleshooting

Here are some common problems people have reported, and their solutions.

Q. When I type gmake, the compilation hangs at some point. Eventually it reports an "I/O error" (or some other error). What should I do?

A. Simply run gmake again; this seems to fix it.

Q. The minilab takes up a lot of disk space! How can I reduce the space?

A. Run gmake clean to get rid of the object files and binaries. We advise that you do this at the end of each minilab hacking session, to keep your account lean and speedy.

The Minilab

Now that you've got all the software set up (or you've just decided to use the Linux lab), it's time to download WeensyOS and take it out for a spin.

Unpack the source for weensyos1 using the following command.

% gzcat weensyos1.tar.gz | tar xf -

(On Linux, you can just say "gtar xzf weensyos1.tar.gz".) This should unpack the tarball into the weensyos1 directory.

% ls weensyos1
COPYRIGHT    console.c    mkbootdisk.c     threados-kern.c      threados.h
GNUmakefile  elf.h        threados-app.c   threados-kern.h      types.h
answers.txt  lib.c        threados-app.h   threados-symbols.ld  x86.h
bootstart.S  lib.h        threados-app2.c  threados-trap.S      x86struct.h
conf         mergedep.pl  threados-boot.c  threados-x86.c
%

Now that you've unpacked the source, it's time to give the OSes a whirl.

Change into the weensyos1 directory and run the gmake program.

The WeensyOS Makefile (well, GNUmakefile) builds a hard disk image called threados.img, which contains the ThreadOS "kernel" and a single application, threados-app.c.

Gmake's output should look something like this:

% gmake
+ hostcc mkbootdisk.c
+ as bootstart.S
+ cc threados-boot.c
+ ld threados-bootsector
+ cc threados-kern.c
+ cc threados-x86.c
+ as threados-trap.S
+ cc lib.c
+ ld threados-kern
+ cc threados-app.c
+ cc console.c
+ ld threados-app
+ mk threados.img
+ cc threados-app2.c
+ ld threados-app2
+ mk threados2.img
%

Now that you've built the OS disk image, it's time to run it! We've made it very easy to boot a given disk image; just run this command:

% gmake run-threados

This will start up Bochs, but not yet the emulated computer. (This is because Bochs is giving you a chance to set breakpoints on the emulated machine.) To start the emulated computer, type "c":

<bochs:1> c

After a moment you should see a window like this!

To quit Bochs, click the "Power" button in the upper-right corner. (Very funny, Bochs.)

QEMU Note. If you're running QEMU instead of Bochs, run the ThreadOS with qemu -hda threados.img. (The -hda option stands for Hard Disk A.) QEMU doesn't have a funky power button; just hit Control-C in the terminal to quit.

ThreadOS Application

You're now ready to start learning about the OS code!

Start first with the application, threados-app.c. This application simply starts a single child thread and waits for it to exit. It uses system calls that implement the thread functions we discussed in class: newthread starts a new thread that runs a named function; exit exits a thread; and join returns a thread's exit status.

Read and understand the code in threados-app.c.

You might wonder how those system calls are implemented! As discussed in class, to call a system call, the application program executes a trap instruction, which makes the processor save its state and transfer control to the kernel. The system call's arguments are often stored in machine registers, and that's how ThreadOS does it. Likewise, the system call's results are often returned in a machine register. On Intel 80386-compatible machines (colloquially called "x86s"), the trap instruction is called int, and registers have names like %eax, %ebx, and so forth. A special C language statement, called asm, can execute the trap instruction and connect register values with C-language variables. When the user executes a trap instruction, the processor:

1. Looks up the trap in an interrupt descriptor table. This tells the processor where to jump (the new instruction pointer), what mode to run (here, kernel mode), and where to save the old processor state (that is, the new stack).
2. Saves the old processor state onto the new stack.
3. Switches to the correct mode.
4. Sets the stack pointer and instruction pointer based on the interrupt descriptor table.
5. Continues execution, this time in kernel mode.

To sum up, when the user executes a trap instruction, the processor switches into kernel mode and starts running kernel code. This is called a kernel crossing.

Read and understand the comments in threados-app.h. This file defines ThreadOS's system calls. Also glance through the code, to see how system calls actually work!

The ThreadOS kernel handles these system calls.

This kernel is different from conventional operating system kernels in several ways, mostly to keep the kernel as small as possible. For one thing, the kernel shares an address space with user applications, so that user applications could write over the kernel if they wanted to. This isn't very robust, since the kernel is not isolated from user faults. In WeensyOS minilab 3, you will make a kernel whose address space is protected from user applications; but for now it is easier to keep everything in the same address space. Another difference is that ThreadOS implements cooperative multitasking, rather than preemptive multitasking. That is, threads give up control voluntarily, and if a thread went into an infinite loop, the machine would entirely stop. In preemptive multitasking, the kernel can preempt an uncooperative thread, which forces it to give up control. Preemptive multitasking is more robust than cooperative multitasking, meaning it's more resilient to errors, but it is slightly more complex. All modern PC-class operating systems use preemptive multitasking for user-level applications, but the kernel itself switches between internal tasks using cooperative multitasking. In WeensyOS minilab 2, you'll add preemptive multitasking to a minilab kernel.

ThreadOS's main kernel structures are as follows.

struct thread_t

This is the thread descriptor structure, which stores all the relevant information for each thread. It is defined in threados-kern.h.

thread_t threads[];

This is an array of thread descriptor structures, one for each possible thread. ThreadOS supports up to 15 concurrent threads, with thread IDs 1 to 15. The thread descriptor for thread I is stored in threads[I]. Initially, only one of these threads is active, namely threads[1]. The threads[0] entry is never used.

thread_t *current;

This points to the thread descriptor for the currently running thread.

The code in threados-kern.c sets up these structures. In particular, the start() function initializes all the thread descriptors.

Read and understand the code and comments in threados-kern.h. Then read and understand the memory map in thread-kern.c, the picture at the top that explains how ThreadOS's memory is laid out. Then look at start().

The code you'll be changing in ThreadOS is the function that responds to system calls. This function is called trap().

Read and understand the code for trap() in threados-kern.c. Concentrate on the simplest system call, namely sys_getthreadid/TRAP_SYS_GETTHREADID. Understand how the sys_getthreadid application function (in threados-app.h) and the TRAP_SYS_GETTHREADID clause in trap() (in threados-kern.c) interact.

Exercise 1. Answer the following question: What would happen if you replaced the run(current); in the TRAP_SYS_GETTHREADID clause with schedule();? Would the sys_getthreadid() system call still return the correct value to the calling thread?

You may have noticed, though, that the sys_newthread() system call isn't working! Your job is to write the code that actually creates a new thread.

Exercise 2. Fill out the do_newthread() function in threados-kern.c.

Congratulations, you've written code to create a thread -- it's not that hard, no? (Our version is under 20 lines of code, including comments.) Here's what you should see when you're done:

Exercise 3. Take a look at the code in threados-app.c that calls sys_join(). Now look at the TRAP_SYS_JOIN implementation in threados-kern.c. The current system call design uses a polling approach. Explain what about the current design indicates a polling approach, then critique this choice: what's wrong with a polling approach here?

Exercise 4. Change the implementation of TRAP_SYS_JOIN in threados-kern.c to use blocking instead of polling. In particular, when the caller tries to join on a thread that has not yet exited, that thread should block. When the other thread exits, the joining thread should unblock; the sys_join system call should return the other thread's exit status.

To implement Exercise 4, you will probably want to add a field to the thread descriptor structure. This field will indicate whether or not a thread is waiting on another thread. Essentially, this field is functioning like a wait queue! You will change TRAP_SYS_JOIN to add the calling thread to this "wait queue", and TRAP_SYS_EXIT to wake any threads that were on the "wait queue". There are several ways to do this; describe how you did it in answers.txt.

How can a single field be like a wait queue? Well, remember a wait queue's function: A wait queue holds a list of blocked threads that are waiting for a given event. That is exactly what this field is doing! This "queue" can hold at most one thread (since no more than one thread can join on any given thread), but that changes its implementation, not its basic function.

do {                              ==>     // do {
        status = sys_join(t);     ==>             status = sys_join(t);
} while (status == -2);           ==>     // } while (status == -2);

Now try running the other ThreadOS application with gmake run-threados2. You should see something like this (different threads generally print their lines in different colors):

The ThreadOS2 application, in threados-app2.c, tries to run 1024 child threads.

Read and understand threados-app2.c.

Unfortunately, your current kernel code doesn't seem able to run more than 15 total threads, ever! It looks like old, dead threads aren't being cleaned up, even after we call sys_join() on them. This is what we call a bug.

Exercise 5. Find and fix this bug.

When you've completed this exercise, gmake run-threados2 should look like this:

This completes the minilab. But here are some extra credit opportunities, if you're interested.

Extra-Credit Exercise 6. The current sys_newthread() system call takes only one parameter, the start function to run in the new thread. In real thread packages, the equivalent function takes at least two parameters, one the start function and another a parameter for that function. (man pthread_create for an example.) The extra credit problem is to make ThreadOS's sys_newthread() follow a similar design, where the start function takes a single argument. The comments in threados-app.h explains how to pass multiple arguments to the system call; but how can you make an argument available to the new thread function? On x86, you do this simply by pushing that argument onto the stack. You will store the argument in the memory space allocated for the thread's stack, and change the thread's stack pointer accordingly.

Extra-Credit Exercise 7. Introduce a sys_kill(threadid) system call, which forces thread threadid to exit. Use this system call to alter threados-app2.c's start_child_thread() function so that the even-numbered threads kill off all odd-numbered threads (except for thread 1). Running gmake run-threados2 should print out "Thread N lives" messages only for even-numbered values of N.