How to share audio/video on GNU/Linux
Table of Contents
- Overview
- Requirements
- Use
tigerVNC
andssh
send video through a chain of forwarded ports - Use
pulseaudio
,ffmpeg
,ssh
, andsocat
to share audio- Necessary programs
- Example of audio streaming through a common access point
ocsenave
: begin capturing audio withffmpeg
andpulseaudio
ocsenave
: usesocat
to transport UDP datagrams via TCPocsenave
: usessh
to forward audio to the common serverr
: usessh
to receive audio from the common serverr
: usesocat
to translate back from TCP to UDPr
: listen toocsenave's
audio
stream.pl
aurellem ☉
Overview
This guide will show you how to stream audio, video, and keypresses from one arch-linux computer to another arch-linux computer through a common internet facing server to which both the sender and receiver have ssh access.
You'll find a series of specific examples of how to do the necessary port
forwarding for both audio and video. At the end of this page there is a
short script (stream.pl
) which generalizes the specific examples and
provides a solution for streaming which was good enough to satisfy my
particular needs.
After reading this you will understand how to forward ports with ssh
,
remotely control a computer with tigerVNC
, transform UDP into TCP and
back with socat
, and how to use ffmpeg
in an unexpected way. Have fun
playing games with your friends!
Requirements
Recently, I wanted to watch my friend play a game on his computer. Unfortunately, he lives several thousand miles away, so I needed him to stream the game to me over the internet. I also wanted a more interactive experience where I could also control the game my friend is playing. However, both of us are behind NATs (Network Address Translantors) and don't have public IPs. I hate configuring my router and wanted something that could work anywhere (like a coffee shop). Fortunately for us, we both have access to aurellem.org (the computer hosting this webpage), so it ought to be possible to use aurellem.org as a common point of connection to transfer sound, video and keypresses.
In short my requirements for my streaming solution are:
- Work without any additional router configuration.
- Transfer video and audio from one computer to another, while not interfering with the experience of the person sending video/audio.
- Allow the person reveiving the video audio to also control the game through their keyboard.
- The sender and reveiver trust each other, but all connections over the internet must be secure and encrypted.
- Low bandwidth.
In all of the examples in this guide, my friend's name is ocsenave
, my
name is r
, and the internet-facing server we are using as a relay point
is aurellem.org
. ocsenave
is the one who wants to play the game, and
r
is the one who wants to watch the game (and occasionally press some
buttons). The game runs on ocsenave's
computer and ocsenave
streams
it to r
. Commands that start with ocsenave>
are executed ocsenave's
computer, and commands that start with r>
are executed on r's
computer.
Use tigerVNC
and ssh
send video through a chain of forwarded ports
Sending video was actually not too difficult. I chose tigerVNC and found it to be quite adequate for my purposes. It automatically compresses video and allows me to reomtely control a computer, meeting all of my streaming requirements except for audio, which tigerVNC does not support.
But how can I actually connect to my friend's computer, which again is
behind a NAT? The answer is to forward the ports that tigerVNC uses
through aurellem.org using ssh
. Then, I can access my friend's computer
by pointing tigerVNC at one of my own local ports.
Required programs for video/keypresses
Install tigerVNC
and ssh
if you don't already have them. On
arch-linux you can do this with:
pacman -S openssh tigervnc
Example of tigerVNC through a common access point
ocsenave
: Create vnc server
First, ocsenave
starts a tigerVNC server on his machine. If this is
the first time, then he'll be prompted to set up the password that
others will need to know to remotely control his computer. Since we're
ultimately going to send this through ssh
, this password does not
have to be very strong.
ocsenave> vncserver -geometry 1440x900 -dpi 96 -alwaysshared \
-localhost -rfbport 5000 :1
-geometry 1440x900
,-dpi 96
- These configure the size of the VNC window and its dpi. If you are having problems with choppy video, you can decrease the size of the vnc window in order to reduce bandwidth.
-localhost
- People can only access this server through ports on their local machine. This prevents people from connecting other than through forwarded ports, and is why we don't need to care about the vnc password.
-rfbport 5000
- Instructs tigerVNC to listen on TCP
port 5000. This is the port
ocsenave
will need to forward toaurellem.org
in order to provide access to his computer tor
. -alwaysshared
- allows multiple connections to
vncserver
. This is necessary to allow bothr
andocsenave
to both watch the same game!
ocsenave
: Publish the vnc server's port on the common access point
Now, ocsenave
makes his local TCP port 5000 (where tigerVNC is
listening) available on aurellem.org
using a "reverse ssh tunnel", so
that other people with ssh
access to aurellem.org
can access his
tigerVNC program.
ocsenave> ssh -N ocsenave@aurellem.org -R localhost:5001:localhost:5000 &
-N
- allows you to send the
ssh
to the background (using the "&" at the end of the command).ssh
will continue to keep the connection open (and the port forwarded) until it is killed. ocsenave@aurellem.org
- Specifies what usser and hostname to
forward ports to.
ocsenave
needs to be able to login without a password toaurellem.org
in order for the-N
option above to work. -R localhost:5001:localhost:5000
- sets up a "reverse" ssh tunnel
with
aurellem.org
. The first "localhost" means "aurellem.org" and the second "localhost" meansocsenave's
computer. Anything sent toaurellem.org's
TCP port 5001 will be sent toocsenave's
TCP port 5000, where the tigerVNC server is listening. This allows someont to connect toocsenave's
vnc server by usingaurellem.org:5001
.
After this is done, ocsenave
has successfully provided a gateway to
his computer through aurellem.org
.
ocsenave
: log in to the vnc server
While he's waiting for r
to connect, ocsenave
views his own
tigerVNC session locally. Then when r
connects, they will both be
looking at the same game.
vncviewer localhost:5000
r
: link the server's port to a local port
Now that ocsenave's
connection is available through aurellem.org
,
r
(who also has ssh
access to aurellem.org
) uses ssh
to
"forward" one of his local ports to aurellem.org
. This completes the
connection from one of r's
local ports to ocsenave's
tigerVNC
server.
ssh -N r@aurellem.org -L localhost:5002:localhost:5001 &
-N
- as above, allows
r
to put ssh into the background while keepig the port forwarding active. r@aurellem.org
r
also has an account ataurellem.org
and has passwordless login enabled.-L localhost:5002:localhost:5001
- sets up "port
forwarding". Anything sent to
r's
TCP port 5002 will be forwarded toaurellem.org's
TCP port 5001. Because of the previousssh -R
command, the information sent toaurellem.org's
TCP port 5001 will be further forwarded toocsenave's
TCP port 5000 on which the tigerVNC server is listening.
r
: connect to ocsenave's
computer
r
runs his own vncclient
pointed at the forwarded port:
vncviewer localhost:5002
r
enters the password ocsenave
configured earlier and now both r
and ocsenave
can view the same game and both can send keypresses to
control the game.
Note that the three different port numbers mentioned (5000,5001,5002) could have just as easily all been 5000 since all these ports are on different computers.
Use pulseaudio
, ffmpeg
, ssh
, and socat
to share audio
Sharing audio is very similar to video but with more of a "pushing" feel
than a "pulling" feel. With video you "invite" someone to share your
screen while with audio you "broadcast" your sound and someone else
listens. This will be reflected in how we use ssh
to set up port
forwarding for audio.
The general idea is to use pulseaudio
and ffmpeg
to gather all the
audio playing on the system and send it through a local port, which is
similar to the way tigerVNC made video available through a local port.
Then we again use a series of ssh
commands to tunnel the audio form
ocsenave
to r
. The only difference is that ssh
can only forward TCP
ports while ffmpeg
broadcasts to UDP ports. So we use socat
to
transform UDP to TCP on ocsenave's
end, transport to r's
computer
with ssh
, then again use socat
on r's
end to transform back from
TCP to UDP. Then r
can receive the audio stream with ffplay
.
Necessary programs
pacman -S socat ffmpeg openssh pulseaudio pavucontrol
Example of audio streaming through a common access point
ocsenave
: begin capturing audio with ffmpeg
and pulseaudio
ocsenave
starts capturing all the audio on his system and streaming
it to his local UDP port 6000.
ffmpeg -re -ar 20000 -f alsa -i pulse -c:a mp3 \
-f rtp rtp://127.0.0.1:6000 > ~/config.sdp &
-re
- "realtime" capture – this basically ensures that the audio will be captured and transported at the same rate as the video.
-ar 20000
- this is the bitrate of the audio stream. The higher the number is, the "better" the audio will sound. If you start getting popping or static after executing this command, reduce the bitrate until it goes away!
-f alsa
- this says that ffmpeg is going to be gathering raw sound data from the system, and to expect it to be in whatever format the system default is.
-i pulse
- defines the "input file" for
ffmpeg
, which is often a file but in this case is whatever pulseaudio wants to send toffmpeg
. You will be able to usepavucontrol
to send all system sounds toffmpeg
, just send some sounds, or mute the input entirely. -c:a mp3
- use the mp3 codec to encode audio. This option helps to
conserve bandwidth while streaming audio over the internet. For
example, if you're transmitting silence via mp3, it hardly takes
any bandwidth at all! If you were using an uncompressed format
like
wav
, you would consume quite a bit of bandwidth even while broadcasting silence. -f rtp
- This tells
ffmpeg
that the output format is actually going to be a streaming broadcast using thertp
protocol. rtp://127.0.0.1:6000
- This tells
ffmpeg
to broadcast sound to the local UDP port 6000. Something likeffplay
will be able to connect to this local UDP port and play or record the sound. > ~/config.sdp
This directs standard out to the file
/home/r/config.sdp
. When transporting sound over rtp, it's necessary to include some information describing what sort of sound data is being sent. Theconfig.sdp
file will look like this:SDP: v=0 o=- 0 0 IN IP4 127.0.0.1 s=No Name c=IN IP4 127.0.0.1 t=0 0 a=tool:libavformat 56.40.101 m=audio 6000 RTP/AVP 14
The most important part of this file is the line:
m=audio 6000 RTP/AVP 14
at the end of the file. This says to listen at port 6000 for sound.
&
- put
ffmpeg
into the background where it will continue to stream sound until is is killed.
Note that just as r
had to get the tigerVNC through some channel, he
will also need to have this sdp
file on his computer as well.
ocsenave
: use socat
to transport UDP datagrams via TCP
Since ssh
can only deal with TCP ports, ocsenave
uses socat
to
transport information form UDP 6000 to TCP 6000.
socat UDP4-RECVFROM:6000,fork TCP:127.0.0.1:6000 &
UDP4-RECVFROM:6000,fork
- listen to UDP port 6000 without blocking the port from other processes
TCP:127.0.0.1:6000
- send everything to TCP port 6000 on
ocsenave's
computer.
ocsenave
: use ssh
to forward audio to the common server
ssh -N ocsenave@aurellem.org -L localhost:6000:localhost:6001 &
Just as with video, this ssh
invocation sends everything from the
local TCP port 6000 to aurellem.org's
TCP port 6001. Note that this
is the opposite direction (-L
instead of -R
) than what we used to
share video.
r
: use ssh
to receive audio from the common server
ssh -N r@aurellem.org -R localhost:6001:localhost:6000 &
r
uses ssh
to bind aurellem.org's
TCP port 6001 to his local TCP
port 6000. Now sound will be streaming to ocsenave's
UDP port 6000,
through socat
to ocsenave's
TCP port 6000, through aurellem.org
via TCP port 6001, and will come out of r's
TCP port 6000. The only
thing left to do is to translate back to UDP on r's
end and play the
audio through the speakers.
r
: use socat
to translate back from TCP to UDP
socat tcp4-listen:6000,reuseaddr,fork UDP:localhost:6000 &
tcp4-listen:6000,reuseaddr,fork
- get data from TCP port 6000 without blocking…
UDP:localhost:6000
- …and forward it all to UDP 6000!
r
: listen to ocsenave's
audio
Again r
will need the config.sdp file generated by ocsenave's
invocation of ffmpeg
to make sense of the data that's now coming form
r
UDP port. This file doesn't change, although if r
has chosen to
ultimately map audio to a different UDP port than the one in the SDP
file, he will have to modify that file to reference the correct
port. Once everything's taken care of, r
executes:
ffplay -i ~/config.sdp
-i ~/config.sdp
- play audio as described by the SDP file at
/home/r/config.sdp
.
To begin listening to audio on ocesnave's
computer.
If r
wants to also COPY all the audio he's getting from ocsenave's
computer, he can also execute:
ffmpeg -i ~/config.sdp -c:a copy out.mp3
-c:a copy
- don't do any processing on the audio but instead copy it directly to a file.
out.mp3
- filename where audio will be recorded. Press
q
to stop recording.
stream.pl
This guide has examples to show how you might send and receive audio. I
abstracted them into a script, stream.pl that does everything at once
so that ocsenave
and I can get on with our lives and play our game!
#!/bin/perl use Getopt::Long; use Switch; $mode = shift @ARGV; # Default Options switch ($mode){ case "send" {$user = "ocsenave"} case "receive" {$user = "r"} } $server = "aurellem.org"; $local_video_port="5000"; $local_audio_port="6000"; $public_video_port=$local_video_port; $public_audio_port=$local_audio_port; GetOptions ("server=s" => \$server, "user=s" => \$user, "local-audio-port=i" => \$local_audio_port, "public-audio-port=i" => \$public_audio_port, "local-video-port=i" => \$local_video_port, "public-video-port=i" => \$public_video_port) or die("Error in command line arguments\n$usage\n"); ################################################## ## Video Sharing # ################################################## if ($mode eq "send"){ $start_vnc_server= "vncserver -geometry 1440x900 -alwaysshared \\ -dpi 96 -localhost -rfbport $local_video_port :1 &"; print("$start_vnc_server\n"); system($start_vnc_server); $publish_vnc_port= "ssh -N $user\@$server -R \\ localhost:$public_video_port:localhost:$local_video_port &"; print("$publish_vnc_port\n"); system($publish_vnc_port); } elsif ($mode eq "receive"){ $link_to_server= "ssh -N $user\@$server -L \\ localhost:$local_video_port:localhost:$public_video_port &"; print("$link_to_server\n"); system($link_to_server); `sleep 4`; $start_vnc_client= "vncviewer localhost:$local_video_port &"; print("$start_vnc_client\n"); system($start_vnc_client); `sleep 10`; } ################################################## ## Audio Sharing # ################################################## if ($mode eq "send"){ $udp_broadcast_port = $local_audio_port; $tcp_broadcast_port = $local_audio_port; $server_relay_port = $public_audio_port; $bind_udp_to_tcp = "socat UDP4-RECVFROM:$udp_broadcast_port,fork \\ TCP:127.0.0.1:$tcp_broadcast_port &"; system($bind_udp_to_tcp); $forward_tcp_to_server = "ssh -N $user\@$server -L \\ localhost:$tcp_broadcast_port:localhost:$server_relay_port &"; system($forward_tcp_to_server); $broadcast_sound = "ffmpeg -re -ar 20000 -f alsa -i pulse -c:a mp3 \\ -f rtp rtp://127.0.0.1:$udp_broadcast_port &"; system($broadcast_sound); } elsif ($mode eq "receive"){ $receiving_port = $local_audio_port; $server_relay_port = $public_audio_port; $spd_file = `mktemp`; chomp($spd_file); $SPD = "\\ SDP: v=0 o=- 0 0 IN IP4 127.0.0.1 s=No Name c=IN IP4 127.0.0.1 t=0 0 a=tool:libavformat 56.40.101 m=audio $receiving_port RTP/AVP 14 "; open SPD, ">", $spd_file or die $!; print SPD $SPD; $connect_to_server= "ssh -N $user\@$server -R \\ localhost:$server_relay_port:localhost:$receiving_port &"; print("$connect_to_server\n"); system($connect_to_server); $bind_tcp_to_udp= "socat tcp4-listen:$receiving_port,reuseaddr,fork \\ UDP:localhost:$receiving_port &"; print("$bind_tcp_to_udp\n"); system($bind_tcp_to_udp); $play_sound = "ffplay -i $spd_file &"; print("$play_sound\n"); system($play_sound); }
- stream.pl
- a program for sharing video, audio, and keypresses between friends.
All required programs:
sudo pacman -S base-devel openssh perl perl-switch \ tigervnc socat ffmpeg pulseaudio pavucontrol \ wine wine-mono wine_gecko winetricks
kill everything and start over
sudo killall ssh socat ffmpeg ffplay ssh Xvnc