Redirection

Redirection redux

In Bash, how can we redirect a program’s stdout?

What about stdin?

Where do the stdio file descriptors come from?

We can redirect stdout with the > operator, e.g.,

ls > out.txt

And we can redirect stdin with the < operator, e.g.,

grep define < /usr/include/stdio.h

The shell (Bash) opens the stdio file descriptors (stdin, stdout, and stderr) on behalf of new processes.

When Bash starts, it opens the three stdio file descriptors. When we run a command in Bash (e.g., ls), Bash first creates a child process for executing the program with the fork(). As we saw in the previous lecture, a child process created with fork() receives a copy of its parent’s open file descriptors. So the child simply inherits these descriptors from the parent, Bash. Bash then invokes exec to replace the child process with the given command.

Let’s walk through the first part of the board notes (from [1] to [3]). Bash starts with all three stdio file descriptors open. Each descriptor points to an entry (also called a /description/) in the system’s open file table. Each description in turn points to the I-node in the system’s I-node table, which stores the actual location of stdin, stdout, and stderr.

When Bash fork()s, the child process receives copies of Bash’s open file descriptors, including stdio.

How does Bash implement redirection?

By changing stdio after fork()

Let’s see an example: ls >out.txt

Let’s continue with the board notes, starting at [4]. Before invoking fork(), Bash opens a file called out.txt [4]. After invoking fork(), the child receives a copy of the open file descriptor for out.txt [5]. Bash can then change the child’s stdout file descriptor to point to the open file entry for out.txt [6] (the redirected arrow is shown in orange). Now that the child’s stdout has been redirected to out.txt, it’s good practice to close the original file descriptor in the child pointing to out.txt. So we also delete the arrow starting at row 3 in the child’s file description table.

But how does Bash change stdio?

With a system call (of course)

Redirecting stdio with `dup()`

Accepts one argument, oldfd
Returns a newly-opened file descriptor pointing to the file entry pointed to by oldfd
Simply uses the lowest available descriptor

`dup_1.c`

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

int main(void) {
    char buf[] = "Hello, world!\n";
    size_t count = strlen(buf);
    int newfd = dup(STDOUT_FILENO);

    if (-1 == newfd) { perror("dup"); exit(1); }

    // Close unused descriptor.
    close(STDOUT_FILENO);
    // What and where will this print?
    write(newfd, buf, count);
    // Close the opened file descriptor.
    close(newfd);

    return 0;
}

The above code uses dup() to copy stdout’s file descriptor, STDOUT_FILENO, and then writes to. The result still appears on the terminal because the file descriptor newfd points to the same entry in the open file table as STDOUT_FILENO. What do you think the value of newfd is?

`dup_2.c`

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

int main(void) {
    char buf[] = "Hello, world!\n";
    size_t count = strlen(buf);
    int newfd = dup(STDOUT_FILENO);

    if (-1 == newfd) { perror("dup"); exit(1); }

    // What do you think this will print?
    printf("Value of newfd: %d\n", newfd);
    // Close the opened file descriptor.
    close(newfd);

    return 0;
}

The value of newfd is three because the lowest available file descriptor is 3. What are 0, 1, and 2 being used by? stdin, stdout, and stderr.

Try modifying the code above to close these file descriptors before calling dup(). What happens? The value of newfd should change.

Now let’s see how we can use dup() to redirect stdout to a file.

`dup_3.c`

#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

int main(void) {
    int new_fd;
    char buf[] = "Hello, world!\n";
    size_t count = strlen(buf);
    int fd = open("out.txt",
                  O_WRONLY | O_CREAT | O_TRUNC,
                  0644);
    close(STDOUT_FILENO);
    new_fd = dup(fd);
    printf("Hello, world!\n");
    close(fd);

    return 0;
}

The above code shows how, with careful management, we can redirect stdout to an opened file with dup(). First we open a file out.txt for writing. Then we close STDOUT_FILENO, making it the lowest-available file descriptor. This makes it so that when we invoke dup() on the following line, it re-opens file descriptor 1 (stdout) to point to the file entry for out.txt. The end result is that when we call printf(), the output is redirected to out.txt.

What’s the problem with this approach? We have to juggle all the open file descriptors manually in order to perform redirection. This is awkward, confusing, and error-prone.

But there is a better way.

`dup2()`

Accepts two arguments, oldfd and newfd
Closes newfd first, then re-opens it point to the same file entry that oldfd points to
Easier to use

`dup2_1.c`

#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

int main(void) {
    int fd = open("out.txt",
                  O_WRONLY | O_CREAT | O_TRUNC,
                  0644);

    if (-1 == dup2(fd, STDOUT_FILENO)) {
        perror("dup2");
        exit(1);
    }
    printf("Hello, world!\n");

    return 0;
}

The above example uses dup2() to redirect stdout to out.txt. We don’t need to close STDOUT_FILENO first before invoking dup2(), because dup2() does this for us. dup2() then re-opens STDOUT_FILENO to point the file entry pointed to by fd (out.txt). Now, subsequent calls to printf() will be redirected to out.txt.

What about redirecting stdin?

`dup2_2.c`

#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

int main(void) {
    char s[1028];
    int size = 1028;
    int fd = open("hello.txt", O_RDONLY);

    dup2(fd, STDIN_FILENO);
    close(fd);
    fgets(s, size, stdin);
    printf("%s", s);

    return 0;
}

The above example uses dup2() to redirect stdin to hello.txt. dup2() closes STDIN_FILNO first, then re-opens STDOUT_FILENO to point the file entry pointed to by fd (hello.txt). Now, the subsequent call to fgets() (which reads from stdin) will be redirected from hello.txt.

How can we redirect the output of a child process while leaving the parent’s intact?

Putting it all together

Now let’s combine dup2(), fork(), and exec to imitate Bash redirection

`dup2_fork.c`

#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>

int main(void) {
    int fd = open("out.txt",
                  O_WRONLY | O_CREAT | O_TRUNC,
                  0644);
    switch (fork()) {
        case -1:
            perror("fork");
            exit(EXIT_FAILURE);
        case 0:
            dup2(fd, STDOUT_FILENO);
            close(fd);
            printf("Child process\n");
            fflush(stdout);
            _exit(0);
        default:
            wait(NULL);
            printf("Parent process\n");
            exit(0);
    }
}

The above example opens a file out.txt for writing in the parent process, then calls fork() to create a child process. The child receives a copy of the open file descriptor for out.txt, and uses dup2() to redirect its output to out.txt. Meanwhile, since the parent does not redirect stdout, its call to printf() will still print to the terminal.

In the child, we need to call fflush() to flush the child’s buffered stdout output before ending the process with _exit(). Writes to stdout aren’t immediate and are first buffered. Buffered outputs are only sent when a flush occurs. exit() flushes buffered outputs, but _exit() doesn’t, so we need to flush manually with fflush().

Now let’s add exec to see how Bash redirects the output of child processes.

`dup2_fork_exec.c`

#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>

int main(void) {
    int fd = open("out.txt",
                  O_WRONLY | O_CREAT | O_TRUNC,
                  0644);
    switch (fork()) {
        case -1:
            perror("fork");
            exit(EXIT_FAILURE);
        case 0:
            dup2(fd, STDOUT_FILENO);
            close(fd);
            char *argv[] = { "ls", NULL };
            execvp("ls", argv);
            perror("execlp");
            _exit(1);
        default:
            wait(NULL);
            printf("Parent process\n");
            exit(0);
    }
}

The above code is mostly the same as dup2_flush.c, with the addition that it invokes execvp() after redirecting the child’s stdout to out.txt with dup2(). The result is that the child runs ls, with its output is redirected to out.txt. Meanwhile, the parent’s output still goes to the terminal. Why? Because the parent never redirects stdout, only the child does.

Notice also that in this example we don’t need to call fflush() in the child. This is because, after invoking execvp(), the child will be replaced with ls, which will exit with exit(), flushing any buffers before terminating. It’s safe for the child to do this because execvp() will replace any exit handlers the parent process may have set up with those of the replacing process, meaning that ls’s call to exit() will not trigger any exit handlers in the parent. You can check man 2 execve for more details.

Redirection redux

How does Bash implement redirection?

But how does Bash change stdio?

Redirecting stdio with dup()

dup_1.c

dup_2.c

dup_3.c

dup2()

dup2_1.c

dup2_2.c