CS 111
Scribe Notes for Lecture 6
Thursday April 15th, 2010
by Sendie Hudaya and Jiaojiao Yang
File Descriptors
What can go wrong with file descriptors?
1 int main(void) {
2 write(963, "hello", 5);
3 return 0;
4 }
Opening a file id of 963 is a very unlikely condition because it is rare to have 963 files open at a time.
In the code below, the kernel will return a fd of 6 because it is the smallest free fd.
open("/ab/cd/",O_RDONLY); // if fds 0,,2,3,4,5,9,11,13,29 are open this returns 6
1 int main(void) {
2 if (write (1, "hello\n",6) != 6){
3 write (2, "Ouch!\n",6);
4 return 1;
5 }
6 else return 0;
7 }
By convention: 0 is STDIN, 1 is STDOUT, and 2 is STDERR
To check if the code above outputs "Ouch!" execute the following command:
1 /a.out > /dev/full
Note that /dev/full is always full, and a write to this file will always fail and a read will return 0's.
1 int f = open(...){
2 if( 0<=f){
3 close(g); //say g= f
4 h = open()// h can equal to f
5 }
6 write(f, ...);
Problem with line six is that it will return an EBADF if the file does not exist or is already closed.
But if it is indeed open (such as when fd h equals f) then f is writing on the wrong file.
1 for(int i=1 ; i < argv; i++){
2 int fd = open(argv[i],O_RDONLY);
3 if(0<=fd){
4 read(fd,buf_size,buf);
5 process(buff);
6 close(fd); // a fix to file descriptor leak
7 }
8 }
Problem with above code is not closing fd. There is a limit to the number of file descriptor, and in
Linux invoking ulimit -a
will show you that 1024 is the limit number for open files.
1 fd = open("/dev/usb2",O_RDONLY)// device exists
2 //unplugs USB key
3 read(fd,...);
The above code will fail to read, but how to report this problem? Turns out errno returned will depend on the driver of USB.
1 file /etc/passwd contains 713 bytes
2 fd=open("etc/passwd",O_RDONLY);
3 read(fd,buf,1024); // should return 713
4 read(fd,buf,1024); // should return 0
Why does the second read not return -1 with errno = EEOF? Because it is not an error for a file to end.
fd = open("etc/passwd",O_RDONLY);
lseek(fd,193726,0); //returns 0;
or
lseek (fd,-1,0);
The lseek above returns an error because it does not accept negative offset.
1 int fd= open("/dev/keyboard",O_RDONLY);
2 if(0<=fd){
3 char c;
4 read(fd,& c,1);
5 }
The above code opens a keyboard and waits for user input. But if a user never use the keyboard, what should the kernel do?
There are two options: return -1 EAGAIN, or wait.
Syscall Kill
int kill(pid_t,int); //2nd argument is signal number
The above command returns a 0 if signal sent, -1 if unsuccessful. So another option for the above problem is for
some process to kill keyboard process. Note that a process can send a signal to processes it owns.
(cmd1 &
cmd2 & ) > /tmp/log
cmd1 write(1,"hello",5);
cmd2 write(1,"there",5);
Race condition
The above command can produce race condition in which two processes tries to write to STDOUT at the same time. The final
result is undefined, and may be one of the following:
hellothere
therehello
htehlelroe
The third output is not possible in unix since all writes are guaranteed to be atomic if it is "small enough."
(cmd1 > /tmp/log & cmd2 > /tmp/log &)
In the above case both commands can overwrite each other.
Sorting
Suppose data is large and we can't sort all at once, we can create a temporary file for 'sort'
1 int fd=open("/tmp/sorttmp",O_RDWR|O_CREAT,0600); //need to check exit status
2 write to fd
3 read from fd
4 close (fd) // also need to check status because some O.S. delay writing and will report during close for any delayed I/O error
5 unlink("/tmp/sorttmp");
Potential problem since both sorts uses same file /tmp/sorttmp. How do we fix this?
1 sort f>g &
2 sort h>I &
Ideas: append pid -> won't work in network files because two processors over the network can share the same pid
-
1 if(fd=open(fbuff,O_RDONLY)>=0){
2 close(fd);
3 continue;
4 }
5 open(fbuff,O_RDWR|O_CREAT,0600);
There is a race condition in the above code since some other process might open file with same buffname during continue.
Better idea:
1 if(fd=open(fbuff,O_RDWR|O_CREAT|O_EXCL,0600) < 0){
2 if(errno==EEXIST){
3 continue;
4 return -1;
5 }
6 } // if code gets here, open succeeded
With this solution, we only need one system call which better handle race condition.
In unix, READ/WRITE to opened file that is unlinked is allowed, but it becomes invisible
Better way to create temp files:
1) fd = tempfile(); (library function)
2) new syscall opentmp(O_RDWR, ...) this syscall will create a nameless file
- problem a file can use up all disk space but is invisible
+ OS can choose file location
+ OS can free files on close automatically
How about locking?
acquire_lock("/tmp");
//generates a file name
//try it
//create file
release_lock("/tmp");
Unix file locks exist!
fcntl(fd,F_SETLK,p)
Structure of lock
struct flock const *
ltype lock type
l_whence interprets l_start
l_start starting offset for lock
l_length number of bytes to lock
l_pid pid of process blocking our lock
Lock types:
- F_RDLOCK -> read
- F_WR -> write
- F_UN -> getrid
F_SETLK if returns fail then fail
F_SETLKW if returns fail then wait
This solves race condition, but creates bottleneck in performance
Problem Deadlocking
2 processes waits for one another, in UNIX if F_SETLKW is set, returns an error during deadlock
Locks are purely advisory => a readlock does not prevent a write
If locks were mandatory => security is better but root can't modify or update, say a shell, if someone has readlock.
Pipes
Nameless and bounded sizes (say 8kb) => useful as conduit from one process to another.
Solve subset problems of tempfiles (but not all)
Circular buffer => reads and writes hangs if there is no data or full
1 (du& ls &) | sort -n => race condition on writing to pipe
2 du | /sort -n > /tmp/1& sort >/tmp/2 => creates race condition on read to pipe
How to create pipe? There is a syscall for that.
1 syscall => in pipe(int *fd);
This system call returns a -1 if the kernel runs out of fd or memory, otherwise returns a 0 with two file descriptors.
Say we want to implement du|sort -n we need 3 processes
du process du process sort
p1 p2 p3
two forks p0->p1->p2 or p0->p2->p1 or p1<- p0 -> p2
Whichever choice, pipe must work
- call fork() twice
- call pipe() once
ex p1->p2
in shell
1 pid_t p1=fork();
2 if(p1==0){
3 int fd[2];
4 pipe(fd); // need to check return value
5 pd_t p2 = fork();
6 if (p2==0){
7 dup2(fd[1],0); // goes to stdin
8 close(fd[1]); // if (fd[1] != 0) else don't close
9 close(fd[0]);
10 execvp("sort", ...); //need to check exit status
11 //dancing pipes for p1