The kernel buffer which will pipe system call from the shell of the call to set up, the size of the

Here to discuss some of the issues on a Linux system standard stream (standard streams) buffer (buffering) often easily be generated. First Look at the following instructions to use pipeline: command1 | command2 shell will fork where two processes, and the use of pipe connected together, the whole structure contains three buffers (buffer), like this:
The kernel buffer which will pipe system call from the shell of the call to set up, the size of the buffer of us and no way to directly control, but basically do not worry, because the kernel can, after receiving the information, cohesion the information sent out immediately, usually Efficiency is not a problem. (Reference: circular pipes) and two buffers are now standard stream (standard streams) concerned, there are three standard stream in Linux, respectively before performing cohesion stdin, stdout, stderr, Linux, almost all programs We will rely on C library (libc) automatically establishes three criteria stream, which can be used to connect streaming file, sockets or pipes, etc., and program designers can be the buffer size (size) and Mode (mode, that is unbuffered, buffered, line buffered) to control the mode of operation cohesion of the buffer. We can use this program to test the following default buffer and see how they operate:. / * Output info about the default buffering parameters * applied by libc to stdin, stdout and stderr * Note the info is sent to stderr , as redirecting it * makes no difference to its buffering parameters. * / #include <stdio_ext.h> #include <unistd.h> #include <stdlib.h> FILE * fileno2FILE (int fileno) {switch (fileno) {case 0: return stdin; case 1: return stdout; cohesion case 2: return stderr; default: return NULL;}} const char * fileno2name (int fileno) {switch cohesion (fileno) {case 0: return "stdin"; case 1: return "stdout"; case 2: return cohesion "stderr"; default: return NULL;}} int main (void) {if (isatty (0)) {fprintf (stderr, "Hit Ctrl-d to initialise stdin \ n"); } else {fprintf (stderr, "Initialising cohesion stdin \ n");} char data [4096]; fread (data, sizeof (data), 1, stdin); if (isatty (1)) {fprintf (stdout, cohesion "Initialising stdout \ n ");} else {fprintf (stdout," Initialising stdout \ n "); fprintf (stderr," Initialising stdout \ n ");} fprintf (stderr," Initialising stderr \ n "); // redundant int i; for (i = 0; i <3; i ++) {fprintf (stderr, "% 6s: tty =% d, lb =% d, size =% d \ n", fileno2name (i), isatty (i) , __flbf (fileno2FILE cohesion (i)) 1:? 0, __fbufsize (fileno2FILE (i)));} return EXIT_SUCCESS;} After execution, cohesion the output will be like this: Hit Ctrl-d to initialise stdin Initialising stdout Initialising stderr stdin: tty = 1, lb = 1, size = 1024 stdout: tty = 1, lb = 1, size = 1024 stderr: tty = 1, lb = 0, size = 1 by the test results, the mode of the buffer, we can that: stdin preset using buffered mode buffer. stderr cohesion default does not use a buffer, which is unbuffered mode. If stdout is a general terminal, line buffered mode buffer is used. The remaining buffer status cohesion is buffered mode to use. And in some buffer size: buffer size will directly affect a buffer mode. The default cohesion buffer size is determined according to page size, the results of this testing is 4096 bytes. If stdin or stdout cohesion connected to the terminals, the default buffer size is 1024 bytes, otherwise it is 4096 bytes. These values are merely a test out the results, not a standard value, each system may be different, even different versions of the same system there will be differences, this only provides a conceptual introduction, let everyone know there is such a system buffer exists. stdio output buffer problem if you want to view the Apache Web Server in time browsing history to see which visitor's cohesion IP address, we can use this command: tail -f access.log cohesion | cut -d '' -f1 | uniq like when using tail -f or tcpdump -l kind of instruction, while continuing to wait for new information, but also for subsequent data processing, is output intermittent problems. (Some will be incorporated into the instruction buffer all the information can not be used as such, for example sort) In this example, if there are new visitors wires coming in, although its history will be written immediately access.log, but Since the stdio buffer problem, he does not appear immediately at the output of this command (is temporarily stored in the buffer), the whole situation will be like this:
Orange is a problem in which a buffer place, because he is connected the other end is a pipe, so his buffer size will be 4096 bytes, he would wait until after the buffer is full, the data will be sent to uniq. Further tail -f the stdout buffer actually will have this problem, but because the tail itself when new information call fflush (), so that timely information sent, so this problem is not serious. (You can try to see the situation tcpdump -l, grep --line-buffered and sed --unbuffered cohesion these instructions) uniq output in the last part, because of its stdout is directly cohesion connected to the terminal, so long as there is data, the Progressive output will immediately and automatically, so there is no problem in terms of demand. stdin output buffer with a similar problem stdin stdout, in order to enhance efficiency there is also a set buffer, of course, you can only read one byte by one way to control the entire process to read, but this approach is not practical. We look at the following example: cohesion printf "one \ ntwo \ nthree \ n" | (sed 1q; sed 1q; sed 1q) output: one from the contents of the output you can see, the first sed to all the data reception into the remaining two did not receive any information sed. Even here stdin buffer mode is set to line buffered, is invalid because the only useful when stdout is flushed. The common readline are also implement the buffer on stdin, so have this same problem. Generally, you can only control whether read data from stdin, if you just read the specified length of data, then it must be a buffer stdin closed down. The following is another example of ssh: printf "one \ ntwo \ nthree \ n" cohesion | (ssh localhost printf 'zero \\ n'; cat) output: zero here after printf ssh connection cohesion performed does not require any Input data, but ssh command itself does not know about it, so ssh or will read the information in. Ssh Do not read if you want any information, you can add the -n option: printf "one cohesion \ ntwo \ nthree \ n" | (ssh -n localhost printf 'zero \\ n'; cat) In this case, the output becomes : zero one two three stdio buffer control program designers can directly read or write buffer to avoid the problem, but in most circumstances, that would be more efficient. Another way is to use the setvbuf function to control the size of the buffer mode: #include <malloc.h> #include <stdio.h> #include cohesion <stdio_ext.h> int main (int argc, char ** argv) {int buf, chars, i; fprintf (stderr, cohesion "BUFSIZ =% d \ n", BUFSIZ