UNIX pipes are a monumental step in the development of UNIX and UNIX-like operating systems. It allowed users to perform complex computing tasks by linking together the input and output of basic programs. This article extends on that by showing you how to use a UNIX pipe in Linux over a network with the SSH protocol.
- Understanding the Unix Pipeline
- Automatically Transfer Compressed Directories
- Pushing and Retrieving Files from Remote Hosts
- Backing Up and Restoring Drives Remotely
- Redirecting Audio Input to a Remote Machine
- Streaming Live Video from a Remote Webcam
- Printing Text on a Remote Console
- Piping Remote Data to a Local Clipboard
Tip: If you need a refresher, learn how pipe and redirection work in Linux.
Understanding the Unix Pipeline
Pipes on Unix (and by extension, Linux) are used to chain programs together and make them work together. For example, using cat
, you can show the contents of a file, but if you used a pipe (|
), you could chain the cat
command to the more
command to make the file easier to read through.
cat file1 | more
The basic idea here is this: program1 fileX | program2
. It’s not just limited to one file and two programs, though. Piping can get about as advanced as you need it to be with as many modifiers as you can think of.
Here are some ways to make good use of pipe (|
) in SSH situations.
On a side note: learn some of the best tricks to secure your SSH server.
Automatically Transfer Compressed Directories
One of the most common ways of using UNIX pipes is for storing a program’s output to a file somewhere in the local system. For example, running echo "Hello, MakeTechEasier!" | tee Hello
will run the program echo
while also storing the string “Hello, MakeTechEasier!” inside the file “Hello.”
That said, you can use this idea to transfer directories across two Linux hosts. To do that, read the folder that you want to send using tar
, then pipe that to your SSH daemon:
tar czf - "~/Documents/myfolder" | ssh ramces@remote.host "tar xzf - -C ~/Documents/"
This command will bundle up your folder into a tar archive and send it to the command’s standard output. The UNIX pipe will then read that data and send it to your remote Linux host using SSH.
You can also reverse this command to get your files out of remote hosts:
ssh ramces@remote.host "tar czf - ~/Documents/myfolder" | tar xzf - -C "~/Documents/"
Good to know: learn how you can create encrypted cloud backups using rclone.
Pushing and Retrieving Files from Remote Hosts
You can also use pipes and SSH to send individual files over the network. This works by using cat
as a way to load the contents of a file and send that through SSH:
cat < my.local.file | ssh ramces@remote.host "cat > my.remote.file"
The remote host will receive the output stream from the local cat process and reconstruct the file as is.
To retrieve a file from a remote host, you need to reverse the order of the command and provide the path for your remote file:
ssh ramces@remote.host "cat < my.remote.file" | cat > my.local.file
Backing Up and Restoring Drives Remotely
Similar to sending files and directories, it is possible to back up entire drives in Linux remotely with a UNIX pipe and SSH. This can be useful if you want to create quick offsite backups and you don’t have a spare physical drive at the moment.
To backup an entire drive, run dd
with its “if=” variable set to the drive that you want to backup, then pipe that to your SSH daemon:
sudo dd if=/dev/sda | ssh ramces@remote.host "dd of=sda.img"
Reversing this command also allows you to restore a disk image from a remote machine to a physical disk:
ssh ramces@remote.host "dd if=sda.img" | sudo dd of=/dev/sda
Further, this SSH pipe syntax will also work with discrete disk partitions. For example, if your system has a /home partition in “/dev/sda4” you can run the following command to create a backup of it:
sudo dd if=/dev/sda4 | ssh ramces@remote.host "dd of=home.img"
FYI: learn more about the history and you can use the dd command to create disk images.
Redirecting Audio Input to a Remote Machine
One of the benefits of SSH pipes is that they allow you to interact with remote machines as if they are a local resource. This includes the ability to tap into device files such as the audio input of a system.
To do this, run a remote ALSA subshell using SSH and send its output to your local ALSA daemon:
ssh ramces@remote.host "arecord -f cd" | aplay
This will listen to the default audio input device on the remote machine and play what it is hearing on your system. That said, flipping the commands around will send your local machine’s audio input to your remote host’s audio output:
arecord -f cd | ssh ramces@remote.host "aplay"
The ALSA SSH pipe will also work when you combine it with other audio playback tools. For instance, you can send the arecord output from an SSH pipe to ffmpeg:
ssh ramces@remote.host "arecord -f cd" | ffplay -nodisp -
Streaming Live Video from a Remote Webcam
Another great use of SSH pipes in Linux is by streaming live video webcam feeds. Just like with audio, this allows you to take advantage of a remote host’s device and render its output on your local machine.
To stream from a remote host’s webcam, run SSH with an ffmpeg subshell then pipe it to a video playback client on your local machine:
ssh ramces@remote.host "ffmpeg -r 14 -s 640x480 -f video4linux2 -i /dev/video0 -f matroska -" | mpv --demuxer=mkv /dev/stdin
This command will stream the raw video output from the first webcam on your remote machine.
It is also possible to record the footage from your remote webcam to a separate file. You can do this by sending the data from the SSH pipe to tee before redirecting it to your video player:
ssh ramces@remote.host "ffmpeg -r 14 -s 640x480 -f video4linux2 -i /dev/video0 -f matroska -" | tee my_recording.mkv | mpv --demuxer=mkv /dev/stdin
Printing Text on a Remote Console
Aside from audio and video, you can also use SSH pipes to send raw text on a remote machine’s TTY. This is helpful if you want to send status messages to a system with no GUI.
To start, create a FIFO pipe on your local machine:
mkfifo my-fifo
Run a listening tail
command with your FIFO and send its output to an SSH daemon:
tail -f my-fifo | ssh root@remote.host "cat > /dev/tty0"
Test whether your new FIFO pipe is working over the network by sending text data using the echo command:
echo "Hello, MakeTechEasier!" > my-fifo
Note: Sending text to your machine’s TTY will only work if you are logged in as that machine’s root account.
Piping Remote Data to a Local Clipboard
The biggest downside of a system clipboard is that it only works with the local machine. This is a problem if you’re working with multiple computers and you want to transfer data without creating temporary files.
One way to fix this is to create an SSH pipe that can read and send a remote file directly to your local system clipboard:
ssh ramces@remote.host "cat < ~/ramces.txt" | xclip -sel clipboard
This command will connect to your remote machine, run the cat utility then start reading the “ramces.txt” file. Once done, it will send the remote data back to your local machine and redirect that to your system clipboard.
You can also push your system’s current clipboard as a file on your remote machine by using the following command:
xclip -sel clipboard -o | ssh ramces@remote.host "cat > ~/clip.txt"
Learning how to send data across a network using UNIX pipes and SSH is just the first step in understanding how computer networks work. Find out more about your network by tracking where your packets go using Traceroute.
Image credit: Mikhail Fesenko via Unsplash. All alterations and screenshots by Ramces Red.
Our latest tutorials delivered straight to your inbox