Linux Named Pipes - FIFO

Linux pipes or named pipes or FIFO(first in first out) are a special type of file that do not store data on the filesystem. They can be used to connect two data streams from two different processes which can't communicate through an anonymous pipe.

Here is a simple example using mkfifo to create the named pipe and echo and cat to input and output data:

╰─ mkfifo named_pipe ╰─ ls -l prw-r--r-- 1 tony tony 0 nov 19 10:02 named_pipe

As we can see in the first column of output of ls -l there is a "p" which stands for pipe. It is one of the many types of open files in linux systems, the rest you can see in file descriptors.

╰─ echo "Some data" > named_pipe & cat named_pipe [1] 12536 Some data [1] + 12536 done echo "Some data" > named_pipe

The first line represents the PID of the new process that was created, the second line is our actual data input from echo and the 3rd line is telling as that PID 12536 has been closed as data stream has finished for our input command also printing it again.

Please note that the reason I've run both commands with & is that pipes are actually inter-process communication mechanisms, they are to be used between processes running concurrently. If for example I would have run only the echo part, shell would hang because it's waiting for some process to read the data sent to the pipe.

Another way to use named pipes is with file descriptors:

#!/bin/bash # Open FD 3 in read/write mod to write to pipe exec 3<>named_pipe # Create data stream of 100 bytes and feed it to FD 3. # If FD 3 is not open read/write using "<>" it will block tr -dc "A-Za-z0-9" < /dev/urandom | head -c 100 >&3 # Open FD 4 to read from pipe exec 4&- # Read data from FD 4 cat <&4 #Close FD 4 exec 4<&- <

I've used /dev/random to generate random data and tr to only select alphanumeric characters and then head to select the first 100 characters of the output as /dev/random doesn't stop. Other commands are described well enough by comments in this miniscript

╰─ ./myscript.sh hirksnTuUWXOrUPhQbDw5lPfqFiPvnqBuBG8Y1iAqlZKC7NoDgcwZqfZBj8GBhZwYxqkZV1ET1FnlDuRt6i017GYLhCn5qkLrE65%

An important note here is that since Linux 2.6.11, the pipe capacity is 16 pages (65,536) bytes in a system with a page size of 4096 bytes which is also my case. If I change the number of characters I want to write to the pipe from 100 to 65537 which is 1 byte above the limit on my system, script will hang. Reason for this is that as writing exceeds max buffer size, pipe waits for some process to read what was written and the line where reading is performed is only executed after the line of writing finishes, so the script will hang.

In the following example I will use netcat to establish a connection to a listening port which I will open using also netcat and a named pipe to buffer data. This can be done without a named pipe but in order to work you first need to open the listening port and then generate data and send it to listening port but in this example I will buffer my data which is to be sent to port 1234

That's about it for this tutorial. A lot more can be done using named pipes but they are mainly used by processes that can't communicate through unnamed pipes. Named pipes have three main advantages over the unnamed ones and these are: