Assignment/Kernel

From WoxWiki

This is the first CS169-only project in the course. CS167 students should be working on UThreads instead. This project is divided into two parts. There are other pages specifically dedicated to part 1 and part 2.

Please remember to read the course Programming Guide for good style guidelines.

Introduction

In this assignment you will write the basic building blocks for the Weenix operating system: threads, processes, synchronization primitives, and device drivers. At this point you are writing only kernel code. You will not be interacting with the user level processes and threads until a later point. We have split this assignment into two parts, the first part deals with processes, threads, interrupts, and synchronization. The next part deals with device I/O and terminal drivers.

Writing an operating system can be complicated, so we provide much of the code to do basic kernel operations such as memory allocation and page table management. These are either beyond the scope of the course or too time consuming for too little value. You are free to modify the code that we provide as you see fit, although we suggest that you do not wander too far from the interfaces that we have provided (to make things easier on you and your TAs!). We want you to be able to take ownership of the code. Though you might not write the entire system from scratch, you should, by the end of the project, have a basic understanding of all the code involved. This will certainly be true by VM, which will involve the final touches on the kernel that you're beginning to write.

At the end of this assignment, you should have several threads and processes running, reading and writing to multiple terminals and to the disk. It is important to emphasize early on that test code is critical. Your mentor TA will make an appointment with you so that you can demo your basic kernel functionality. Without test code you have no demo! A substantial portion of the grade will be based on the quality of your test code, since the more that you can demo the more likely that your mentor TA will believe your operating system actually works. You should consider your appointments with your mentor TA as a time for you to prove that your operating system is working correctly, not a time for your mentor TA to test your operating system.

Before you start this assignment you will want to read the section of the Weenix/Guide/Hackers Guide on processes and threads. Remember, until you get to the VM project, all of your code will be operating in kernel mode, including your test code.

Coding Practices

In addition to the practices outlined in the Programming Guide, there is some kernel-specific information that you should be aware of. Proper use of these coding practices is essential in order to avoid wasting time trying to track down bugs. All of this information can be found on the Weenix debug system page:

• The panic macro should be used to halt your kernel with an error message if Weenix reaches an inconsistent state.
• The KASSERT macro should be used anytime you make an assumption about a piece of data or argument. If the assertion is false the macro will automatically panic the kernel with an appropriate error message. Good use of this macro will save you in the long run. Don't worry about the efficiency of having too many KASSERT statements.
• Debug messages should be used liberally throughout your code.

Test Code

This section describes some guidelines for writing test code for your kernel. It is your responsibility to think of boundary conditions which could potentially cause problems in your kernel. Test code is an important part of software development. If you cannot demonstrate that your kernel works properly, then we will assume that it does not. It is a good idea to show your test suite to your mentor before handing in to make sure that it is acceptable. To be in the best shape possible before moving on, you should be able to test all of the following situations:

• Run several threads and processes concurrently. Devise a way to show that multiple threads are running and that they are working properly.
• Stress test your terminal code. Have two threads read from the same terminal, which will cause each thread to read every other line. If you're not sure why this is, ask a TA. Make sure that you can input and output more data than can be held in the internal terminal buffer. Also, make sure that you can have two threads simultaneously writing to the same terminal. It is very important that you get this code working flawlessly, since it can be a constant source of headaches later on if you don't.
• Stress test your disk code. This will not be needed until the S5FS assignment, but it is a good idea to make sure it works now. Make sure that you can have multiple threads reading, writing, and verifying data from multiple disk blocks. Think of ways that you can write to a disk and display the data stored there. Good reading/writing drivers are crucial at the file system stage.
• Create several child processes and wait for them to terminate.
• Demonstrate that threads and processes exit cleanly, and that the synchronization primitives work.
• Note that kernels are hard to test so you might want to actually sit down and design your test code well so that it tests everything and in a relatively nice way (e.g. you might want to make it interactive so that you can have an easier time testing everything). You may find writing test function after test function in a central spot works well, but this can lead to ugly and long commented-out blocks when you're editing and testing your code. Writing some sort of limited shell is a good idea.

Installing, Building, and Running

Mercurial (Installing)

To install the source for this assignment, simply clone the code repository:

 $ cd <your cs169 course directory>
 $ hg clone /course/cs169/asgn/wox

hg is the command line front-end for the Mercurial version control system, you can use it to do your own local version control.

Running these commands will give you all of the support code for Wox, you should not need to get any more support code from the TAs unless we provide bug fixes or comment updates. See the information on the Makefile for information on what changes you need to make when to make sure you compile the projects you are working on. The default will compile only kern 1.

While you will have all of Weenix in your working directory from the very beginning you should find that each part of this project is concentrated in its own sub-directory and you can almost ignore directories for future projects (there are times you will need to go back and make updates or fix bugs you find). By the time you get to the end you will have slowly built up an understanding of all of Weenix's sub-systems. Note that all header files are kept in the include/ sub-directory. You will not be required to modify header files, but you will definitely want to refer to them for the definitions of the various structures used in Weenix. (You will probably most concerned with those header files in the include/weenix/ directory).

Building and Running

Once the assignment is installed, browse the supplied code carefully (paying close attention to comments) before attempting to write any code. Sections of code which you need to implement are marked with the NOT_YET_IMPLEMENTED macro.

You might also want to look into using Cscope to browse the code as you edit it.

You can compile and run your kernel with the command:

 $ make gdb

which will compile your code, send your binary to a remote Xen server, which will boot your operating system as its own "domain" (this is how Xen refers to operating systems). Your machine will then open an xterm which will connect to the output of your domain. There is also another command you can run:

 $ make run

which will do the same as the previous command except it will not start a gdb debugging session for you. We suggest you stick to using the gdb target because it will force your kernel to wait until your gdb session connects to your domain before booting. (If you use the run target your kernel will boot immediately and might even crash and burn before the terminals get a chance to connect and read the error messages, leading to cryptic errors like "Domain does not exist".)

Once your gdb session connects go ahead and type "continue" (or just "c") into the gdb window and watch your kernel go. Included in the output you should see the messages:

Not yet implemented: proctbl_init, file kernel/proc.c, line 35
Not yet implemented: bootstrap, file kernel/main.c, line 232
Connection to (machine name) closed.

Where (machine name) is the name of the Xen machine on which your code was executed. The "Not yet implemented" statements indicate functions which you need to write which were called during the execution of Weenix (you need to write more than the few functions, this is just the list of functions which was called before the kernel died). Note that bootstrap is listed as one of these functions, and since this is a very important function which "bootstraps" Weenix, Weenix never really started and just exited immediately. As you fill in functions Weenix will be able to execute more.

To see other commands you can run with make check out the page about our Makefile.

Handing In

To hand in your kernel, run:

 $ make clean
 $ /course/cs169/bin/cs169_handin kern?

Where kern? is either kern1 or kern2.

Once again we stress, make sure you have adequate test code! Without test code, you have no demo, without a demo you fail. Consult your mentor TA to make sure that your test code is sufficient to test the functionality of your kernel.