Assignment/VFS (167)

From WoxWiki

This is the second CS167-only project. CS169 students have a separate version of the same project here.

Please remember to read the course Programming Guide for good style guidelines.

Welcome to Weenix

For the final two projects of this semester (VFS and S5FS), you will be working with Weenix, the course's simple unix-like operating system. This is the same code base used in CS169, so you might find it useful to look at some of the CS169 documentation around the wiki (the most useful would probably be the Hackers Guide).

Mercurial

To install the support code for this assignment run the following commands to clone the source code repository:

    $ cd <your CS167 course directory>
    $ hg clone /course/cs167/asgn/vfs

hg is the front-end command to Mercurial, a distributed version control system. While you probably won't be sharing your changes with others, we might need to provide updates to the assignment (like bug fixes or comment changes) via Mercurial. We also recommend (but do not require) that you use it to keep track of your own changes, in case you need to revert back in your own version history.

You will probably want to look into using cscope to navigate the code base.

Building

Once the assignment is installed, take some time to look through the supplied code base. Sections of code which you need to implement are marked with the NOT_YET_IMPLEMENTED macro. If your kernel hits one of these sections while executing it will print out a warning message.

To compile and run your code you can do either:

    $ make run

or:

    $ make gdb

We suggest you use gdb since it will halt your domain until you connect a gdb session. If you do not use gdb and you have errors early on in your code your Weenix may crash before you get a chance to see any error messages.

Make sure to read the page on the Debug System and use it to debug your code.

VFS Introduction

The Virtual File System (VFS) provides a link between the operating system kernel and the various file systems. The VFS makes it simple to add many different file systems to your kernel (though Weenix only supports one) and access them through the same UNIX-style interface.

In this assignment, as before, we will be giving you a bunch of header files and you will supply much of the code. You will be working in the vfs/ directory of your source tree. You will be creating the internal data structures (files and their associated vnodes), and writing the operations which can be performed on the vnodes (vnode_ops). We will be providing you with a simple in-memory file system for testing (testfs), and you will need to write the special files to interact with devices. Finally, you will implement support for the system calls that operate on files: open, close, read, write, lseek, mknod, mkdir, rmdir, dup, dup2, link, unlink, rename, chdir, and getdents.

VFS Notes

In general, you should be able to find all the information about the structure of Weenix's VFS in the Hackers Guide, but there are a few extra notes on mounting which are specific to this assignment.

Mounting

Before a file can be accessed, the file system containing the file must be mounted. In standard UNIX a superuser can use the mount(2) system call to do this; in Weenix, mounting is performed internally at bootstrap time.

The first kernel thread to start executing in Weenix makes a call to vfs_init(), which calls mountproc() to mount the file system of type VFS_ROOTFS_TYPE. When you finish the root file system will be s5fs, but since you have not implemented s5fs yet, for now you will be using the in-memory file system testfs which supports all normal file system operations except for link and getpage.

The mounted file system is represented by a fs_t that is dynamically allocated at mount time.

The Assignment

The following is a check-list of the features which you will be adding to your implementation of Weenix for this assignment:

• Setting up the file system: vfs.c vnode.c file.c
• The testfs file system: testfs.c (mostly provided)
• Name to vnode translation: namev.c
• Opening files: open.c
• VFS system call implementation: vfs_syscall.c

Code Overview

This is a function-by-function breakdown of the assignment.

Setting up the File System

Functions you should take note of in vfs/vfs.c:

    int vfs_init (void);

Functions you should take note of in vfs/vnode.c:

    void     vnode_init (void);
    vnode_t *vget       (struct fs *fs, int vnum);
    void     vput       (struct vnode *vn);
    void     vref       (struct vnode *vn);

Functions you need to write in vfs/vnode.c:

    int special_file_read  (vnode_t *file, int offset, void *buf, int count);
    int special_file_write (vnode_t *file, int offset, void *buf, int count);

Functions you should take note of in vfs/file.c:

    void    file_init (void);
    file_t *fget      (int fd);
    void    fput      (file_t *f);

The testfs File System

The testfs file system is an extremely simple file system that provides basic implementation of all the file system operations except for link and getpage. Of course, there is no need for getpage without VM (a CS169-only project) and link is not implemented because it would require reference counting. We decided not to implement reference counting because it reduces the complexity of the code. This makes it easier to understand and makes the one function you will need to write simpler.

Of course this simplicity comes at a cost. For instance, testfs does not support multiple processes very well. Since there are no reference counts, one process can delete a file that another process has open without any complaint from the kernel. Keep this in mind while testing.

Functions you will need to write in vfs/testfs.c:

    int testfs_readdir (vnode_t *dir, int offset, struct dirent *d);

Name to vnode Translation

At this point you have a file system mounted, but still have no way to convert a path name into the vnode_t with the file at that path. This is where the namev.c functions come into play.

There is a series of functions which implement all of your name-to-vnode converting needs. We will give you an overview of their various purposes in the source code comments. Keeping track of vnode_t reference counts can get hairy here. Copious commenting and use of the Debug System will serve you fairly well.

Functions you will need to write in vfs/namev.c:

    int lookup     (vnode_t *dir, const char *name, int len, vnode_t **result);
    int dir_namev  (const char *pathname, int *namelen, const char **name, vnode_t *base, vnode_t **res_vnode);
    int open_namev (const char *pathname, int flag, int mode, vnode_t **res_vnode, vnode_t *base);

Opening Files

The functions herein will allow you to actually open files in a process. When you have open files you should have some sort of protection mechanism so that one process cannot delete a file that another process is using. You can either do that here or in the S5 layer, but we will be doing it in the S5 layer since the implementation is simpler.

Functions you will need to write in vfs/open.c:

    int do_open (const char *filename, int oflags);

VFS Syscall Implementation

At this point you have mounted your testfs; now you want to write the code that will allow you to test your code vigorously. The system call functions act as the link between your user mode and kernel mode^[1]. When a user program makes a call to read(3), your do_read() function will eventually be called. Thus you must be vigilant in checking for any and all types of errors that might occur along the way and return appropriate error codes.

Note: for each of these functions it is understood that you will handle errors by returning -errno if there is an error. A return value less than zero is assumed to be an error. You should read the man page corresponding to each of these functions to find the appropriate error numbers to return and hints about how to implement the functions. Read the man page for errno(3) for a specific breakdown of what the error numbers mean. This may look like a lot, but once you get one of them, it is easy to figure out the rest. Pay special attention to the comments and use the Debug System.

Functions you will need to write in vfs/vfs_syscall.c:

    int do_read    (int fd, void *buf, size_t nbytes);
    int do_write   (int fd, const void *buf, size_t nbytes);
    int do_close   (int fd);
    int do_dup     (int fd);
    int do_dup2    (int ofd, int nfd);
    int do_mknod   (const char *path, int mode, unsigned devid);
    int do_mkdir   (const char *path);
    int do_rmdir   (const char *path);
    int do_unlink  (const char *path);
    int do_link    (const char *from, const char *to);
    int do_rename  (const char *oldname, const char *newname);
    int do_chdir   (const char *path);
    int do_getdent (int fd, struct dirent *dirp);
    int do_lseek   (int fd, int offset, int whence);
    int do_stat    (const char *path, struct stat *buf);

Test Code

You should have lots of good test code. What does this mean? It means you should be able to demonstrate fairly clearly (via console I/O) that you have successfully implemented everything you can without having S5FS yet. Moreover, you want to be sure that your reference counts are correct. That is, when your Weenix shuts down (after you return from the testing hook in hook.c^[2]), vfs_shutdown() will check the reference counts to make sure they are correct and panic the kernel if they are not.

To further help you test your code, we have included a kernel mode version of CS169's VFS test suite. You can find it in tests/vfs_privtest.c Privtest is not a compelete test suite. If you are able to run it and have your file system unmount and shut down, you're off to a good start, but privtest alone is not sufficient.

Handing In

Hand in using the following command:

    $ make clean
    $ /course/cs167/bin/cs167_handin vfs

Notes

1. ↑ Actually only CS169 students will be implementing a user mode, so everything you do is in kernel mode, but you can still call these functions as if you were calling them from user mode.
2. ↑ Be very careful to make sure that when you return from the hook any processes which you created in order to test your code have exited and been waited on using do_wait, otherwise bad things may happen.