How to set up your own Linux media server
In previous tutorials we looked at setting up and using web servers and how to make Fetchmail, Procmail and Dovecot deliver email.
While these are undoubtedly useful, some of you may consider them a little geeky. In this tutorial we are concentrating more on the home, with a media server (although there are definitely commercial and public uses for this). You could set up a standard file server using NFS or Samba, which we'll cover in the future, and simply make all your media files available to applications on each computer, but there are a number of advantages to a dedicated media server, including:
- Simultaneous access by several users viewing different files.
- Having all your files on one box makes them easier to find.
- Backing up is simpler.
- Indexing is possible.
- Media files can be played on devices other than full-blown computers - iPods, for example.
In this tutorial we're going to show you how to set up two types of media server: an on-demand server for all media types, and a streaming server for audio files.
As soon as you have more than one computer, you run into 'the file I want is on the other computer' syndrome. Keeping multiple copies of each file is a pain to keep up to date and a huge waste of disk space, especially with large media files.
What you need is a server that keeps all your files in one place and lets you play them on any computer. MediaTomb is a UPnP streaming media server, which means it will work with any UPnP enabled software (see UPnP box, below). The server contains all the files, and any UPnP program or device can connect to it to play those files.
So, what is UPnP?
UPnP (Universal Plug and Play) is a set of protocols to let networked devices communicate peer-to-peer with no prior configuration. It provides a means for devices to query and inform one another about themselves and their capabilities.
The specific part of UPnP that applies here is UPnP AV, which covers things like media servers and clients. It enables clients to discover servers on the network, request details of the files they can serve, and stream selected files and media. It also provides for the exchange of remote commands between clients and servers.
UPnP is designed to work with NAT devices, like most broadband routers, and works over HTTP so it requires no specialised firewall settings. If the firewall permits web browsing, it will pass UPnP traffic too.
UPnP programs are available on various platforms. Linux has MythTV (which can also work as a server), VLC (Video LAN Client) and even a UPnP filesystem so that the contents of media servers appears as local files. You can also use Windows Media Player or even a web browser to browse and play your media.
VLC can detect and play all files on any UPnP servers on the network.
MediaTomb is not only useful in a home environment - any organisation that needs to make a number of multimedia files available over its network, such as educational videos for a school, would be a prime candidate for using this software.
If your distro does not include MediaTomb in its repositories, you can download a selection of binary packages or the source code from www.mediatomb.cc. If your distro is not supported, there's a static binary package that should run on just about any x86 system. Of course, you can also download and compile the source code, which uses the standard method of unpacking the tarball and running this:
./configure && make && sudo make install
Once installed, the first decision you have to take is how you want MediaTomb to store its data, you have a choice of SQLite or MySQL, although your distro's packager may have already made the choice for you. The first time you start MediaTomb, it will create a default config file at ~/.mediatomb/config.xml.
Edit this to set either sqlite3_enabled or mysql_enabled to 'yes' and the other one to 'no'. Some distros use a global configuration file in /etc/mediatomb/config.xml, and this is the best option if you are starting the server as a boot service.
If you choose to use SQLite, MediaTomb will create the database file the first time you run it, but a MySQL server would probably be better for a larger collection, especially if you already use MySQL. To use MySQL, you need to create and populate a database called mediatomb, for which all the commands are provided in a batch file, so run this:
mysql -p mediatomb </usr/share/mediatomb/mysql.sql
Unless your distro has already taken care of it, you'll also need to create a user with a password and read/write privileges for the mediatomb database. You can do this using the MySQL command line client if you are familiar with it, or take the easy way out and use PHPMyAdmin, which all self-respecting admins use when no-one is looking. Edit the config file mentioned above to set the correct username and password.
Start MediaTomb and load http://localhost:49152 into Firefox (it doesn't currently work with Konqueror) If you're administering this from another PC on the network, use the address of the server instead. You'll see a very bare window to start with, because there is no content, so click on the + button to start adding content. When you're adding a video file, set the type to Item, the title to the name you want shown, and the location to the absolute path to the file.
The Mimetype setting is important; one way to determine the correct value is to use the file command on the video file, like this:
file -i /path/to/video.avi
You'll soon realise that adding a lot of files this way could take a while, but there is a command line shortcut. It is not necessary to shut down the running MediaTomb to do this, and you can add an item with this command:
mediatomb -a /path/to/video
This assumes you're running MediaTomb as the normal user with the configuration in ~/.mediatomb. However, if you're running it as a boot service, with the config file in /etc, you will not only need to specify the config file but switch to root to be able to read it.
sudo mediatomb -c /etc/mediatomb/config.xml /path/to/video
The path you give can be either a single file to add or a directory, in which case it will be scanned and all files added, which is a great time saver. You can then use the web interface to edit these entries if you need to.
Using the filesystem view, you can add directories to the server. You can then edit the properties of these directories in the database view to enable autoscanning. This means you can have one or more directories that will automatically make available any content dropped into them.
MediaTomb's web interface may look a little spartan (and grey!), but it's only there for administration.
You may have noticed that you were able to connect to the server then add and delete media files with no apparent form of security or authentication. That's because the default setup is completely open, and suitable only for use on a LAN where you fully trust every user.
If you are making files available over the Internet, you should set ui_enabled to 'no' in the configuration file. This will completely disable the web user interface, meaning media is only available using the direct URL for each file.
Even on a LAN, you should set accounts_enabled to 'yes' and define one or more users with a stanza something like this:
<accounts enabled=”no” session-timeout=”30”> <account user=”user1” password=”password1”/> <account user=”user2” password=”password2”/> </accounts>
The config file contains the passwords in plain text, so set its permissions to 600 to prevent anyone unauthorised from reading it. The default ownership should be the user that the server runs under, usually a user with no login shell, so only the program and the root user can read the configuration file.
While the web interface is useful for administration, it is not intended for browsing content. For that you need a UPnP client. There are several UPnP programs that can be used to browse and play the contents of your server. One good choice is Video LAN Client (www.videolan.org/vlc).
Run the app, go to the playlist menu and select Additional Sources > Universal Plug'n'Play Discovery, then open the playlist window and your media will appear under devices. With a Linux client computer, you can also make the content available to any program. Install DJmount from http://djmount.sf.net (you will need the Fuse filesystem in place to use this). Create a suitable mount point, say, /mnt/media, and mount it with this command:
Then /mnt/media will contain a MediaTomb directory, plus one for any other UPnP server on the network, with subdirectories for each type of content, such as /mnt/media/MediaTomb/Video.
MediaTomb provides on-demand content - you pick what you want to watch and when. The other way of serving media is to provide real-time audio content, like internet radio. icecast is an open source streaming media project. Rather confusingly, the name applies to both the entire project and the server component, which is what we're using here.
Icecast is similar in function to and partly compatible with the proprietary Shoutcast server. It is used to provide real-time audio streams over the network - internet broadcasting if you like - using either pre-recorded files or live input from a soundcard as its source.
You should be able to install Icecast from your distro's package manager. After installation, you need to edit its config file, icecast.xml, which is usually located in /etc/icecast2. Most of the settings work fine at their defaults but there are a few you must set.
Set hostname to the name or IP address of the server (the default of 'localhost' will cause playlist generation to fail unless you are connecting from the same computer). Each source client (more on these shortly) needs to authenticate before Icecast will accept a stream from it, and 'source-password' is the default password to use for this. 'Admin-password' is used to authenticate the administrator. The listen-socket is the port that Icecast uses - 8000 by default. The rest can be left at their defaults for now. Go into your services manager and set Icecast to start at boot, or fire it up for now with this:
Look at /var/log/icecast/error.log after you start Icecast and you should see a 'Started' message. If not, something has gone wrong, and you should look in the logfile to get an idea of what and how to fix it.
Now that Icecast is set up and running, are we ready to stream some audio? Well, no. Icecast is basically a middleman; it accepts one or more audio streams and makes them available over the network, so we need another program to provide these audio streams. Icecast calls these programs source clients, and they provide one of their own, IceS.
IceS streams audio in Ogg Vorbis format, taking it from a variety of sources and encoding it into a suitable format and bitrate on the fly. Its configuration file is found in /etc/ices2/ices.xml. There is no default configuration, but a number of sample files for different types of input.
IceS can stream audio from a live input (using either ALSA or OSS) a PCM audio stream on standard input or from a playlist of music files on your hard disk. To use the playlist, copy the ices-playlist.xml file to /etc/ices2/ices.xml and edit it. The main changes you will want to make are to set loglevel to 3, which produces more detailed logs and is useful when you're setting up. Later, you could set this to 1 to show only error messages.
Then you define one or more streams in the <stream> sections. Each stream section contains one <input> section that defines where the audio comes from, and one or more <instance> stanzas, which define the outgoing stream. The reason for multiple-instance stanzas is so you can define more than one output for an input, say high and low bandwidth versions.
The input section starts with a module option, which can be one of ALSA, OSS, stdinpcm or playlist. The first two options cover grabbing live input from a sound card using ALSA or OSS; the third accepts PCM audio data in standard input, enabling IceS to use audio provided by another program, such as a CD player. The final choice is to use a playlist of audio files on your disk.
The module option is followed by a number of parameters; for example, if you are using a playlist you'll want to see something like this:
<input> <module>playlist</module> <param name=”type”>basic</param> <param name=”file”>/mnt/audio/ices.m3u</param> <param name=”random”>0</param> <param name=”restart-after-reread”>0</param> <param name=”once”>0</param> </input>
The type parameter can be basic, for a standard playlist file, or script, which calls a script to generate the list of files to play. The file parameter obviously gives the full path to the playlist file. This should contain a list of full paths to Ogg Vorbis or MP3 files, one per line.
Blank lines and those starting with a # are ignored, so you can structure and comment the playlist if you wish. The remaining parameters set whether IceS should play the files in a random order, restart at the beginning of the list if it is modified or exit when all songs have been played once. All options are turned off in this example.
Many of the programs in this tutorial use XML for config files. If the app fails to start, check the log files - you probably forgot a closing element or quote.
There are four options you have to set in the 'instance' stanza, like this:
<hostname>localhost</hostname> <port>8000</port> <password>sourcepassword</password> <mount>/test.ogg</mount>
The hostname and port are for the Icecast server you've just set up. This can be on a separate machine, but it is more common to have the stream source and server on the same box, so you would use localhost here. The password is the source-password set in the Icecast config, and 'mount', which must start with a /, is the name by which the stream will be identified in Icecast.
Ogg Vorbis streams should end with .ogg, because some less intelligent players and browsers use the extension to determine the type of audio being sent. The rest of the settings can be left as set in the example file.
Now you can start IceS, either as a user with this:
ices /path/to config/file.xml
...or by starting it as a runtime service from your distro's services manager. Check the logfile in /var/log/ices/ices.log to make sure it has started.
Point your browser to http://localhost:8000/admin/status.xml and you should see a brief message indicating that Icecast is running and showing the stream it is playing. This is not prettily formatted, but it does show things are working. If that works, you can find more details at http://localhost:8000/admin/status.xsl.
Now point the browser at http://localhost:8000/test.ogg.m3u, that is, the URL of the server followed by the mount point of the stream with .m3u appended, and you should hear beautiful music (or not depending on your playlist content). You can also play music with one of the many media players available, such as this:
You can even play it in Windows Media Player or Internet Explorer if you install the Ogg codecs from www.xiph.org, although WinAmp will play it out of the box.
While many acknowledge the superiority of Ogg Vorbis over MP3, the fact remains that more people are familiar with MP3 and far more computers and players play that format. The current IceS version only generates Ogg Vorbis streams, so you need a different source client to feed MP3 streams to Icecast.
One option is EZStream, which feeds Ogg Vorbis and MP3 streams. Unlike IceS, which re-encodes your audio to suitable bitrates for the stream, EZStream's standard method of operation is to pass the audio data through unchanged. This is good if you're setting up a dedicated operation, as you can encode all your source material to suitable bitrates beforehand, reducing the load on the server during transmission.
This is particularly useful if you're using relatively low-powered hardware for the server, as it means that you can encode in advance on a more powerful computer. As with IceS, it uses an XML configuration file, and there's no default included with the program. Copy the ezstream_mp3.xml from the distribution, probably installed in /usr/share/doc/ezstream-x.y.z/examples, somewhere sensible and edit it.
Change the <sourcepassword> entry and set the <filename> to the name of a playlist file that contains the files to be played, one per line. The <url> should be set to the URL of your server, plus the port number, followed by the stream name something like this:
With MP3 streams, no extension is necessary, unlike Ogg Vorbis. You may also want to add
Then run EZStream with
ezstream -c /path/to/config.xml
While EZStream is running, you have a small amount of control over it. If you edit the playlist, you can force EZStream to re-read it without stopping with
killall -HUP ezstream
You can also force it to skip immediately to the next track with
killall -USR1 ezstream
EZStream can re-encode streams by calling external programs, but if you are using prerecorded material, it makes more sense to encode it into the correct format beforehand. If you want to work with live audio from a soundcard, encoded into MP3, there are better options, like DarkIce (http://darkice.sourceforge.net) or Muse (http://muse.dyne.org) which also has the luxury of a GUI.
We have really only scratched the surface here. The documentation on the respective programs' websites and wikis provides a lot more information, but this is enough to get you started and give you a base for your own experiments, have fun!
Serving media often requires it be in the correct format, this often means transcoding video or audio to suit the output you need. While some of the servers, like the various source clients for Icecast, will transcode on the fly, it is generally better to have the source material already prepared, and necessary with some programs. Here are some of the programs most commonly used to transcode media formats.
- DVD::Rip Rips DVD tracks to MPEG or AVI files www.exit1.org/dvdrip
- Transcode General video transcoder www.transcoding.org/cgi-bin/transcode
- Mencoder The transcoding/encoding part of the MPlayer package www.mplayerhq.hu
- Lame MP3 encoder http://lame.sourceforge.net
- Vorbis Tools Ogg Vorbis encoding and decoding tools www.vorbis.com
- Grip GTK CD ripper and encoder www.nostatic.org/grip
- abcde CLI CD ripper and encoder www.hispalinux.es/~data/abcde.php
First published in Linux Format magazine