How to build your own Linux distro
Since Manchester University's Owen Le Blanc released MCC Interim Linux (generally agreed to have been the first Linux distribution), way back in 1992, there have been hundreds of ways to get the world's favourite free software operating system on to a computer. The diversity of alternatives reflects the diversity in the development community, with distros split along technical, functional, linguistic and even ideological lines.
There have been large distros, tiny ones, bleeding edge and rock-solid stable distros. Easy for the newbie to install, or downright impenetrable to the uninitiated. Created exclusively with free software as a badge of pride, or so proprietary in attitude that not even the toolchain was fully GNU (hello Red Flag Server 4.1, built with the Intel compiler in 2004).
So with all the variety that's already out there, why would anyone want to create their own distro? Well, that's down to you. But we want to show you how to get started the easy way, as well as giving you some advice along the way to help make your distro stand out in the crowd. Read on!
Everyone's motivation is different, and for many that motivation isn't strong enough to make it past the first months of planning. But the ones that do survive – the distros that prove they have lasting power – usually have a unique selling point that captures the interest of other people.
You might look at the Ubuntu project and think you couldn't possibly compete, but you shouldn't forget that Ubuntu itself started life as a simple Debian fork with the goal of releasing stable versions more frequently than the Debian project. Mandriva started out as a Red Hat fork, and the early SUSE releases were simply German translations of Slackware!
What this amounts to is that it doesn't take much to warrant a new distro. If people tell you, "there are already enough distros out there; join one of those!" don't succumb to the pressure – it doesn't matter whether you're creating a distro for your home country (we think Cymru Linux is well overdue!) or just changing the colour scheme, because what matters is that if other people find it useful/attractive/cool, they'll use it, and you'll be helping the free software world.
And if for some reason your idea doesn't take off, then you've lost nothing and actually gained quite a few skills along the way! Not everyone has the cash reserves of Ubuntu's founder and space tourist Mark Shuttleworth, so do your best and see where it takes you.
Few new distros are created from scratch, although it is possible – see the Linux From Scratch HOWTOs at www.linuxfromscratch.org if you want to try it. Apart from getting a head start, there are other reasons to hang on the coat-tails of another distro, and probably the biggest is package compatibility. Consider that there are over 18,000 free software packages considered good enough to be in Debian, and many more that aren't on that project's radar.
You don't want to be compiling those yourself, particularly since dozens of releases happen every day. Also, being based on a better-known distro helps attract new users who might be familiar with that distro's way of doing things. Not least, you get to be part of the community that has formed around the base distro. Then again, free software means never having to ask permission before you create your fork.
We gave Debian as an example, because along with Red Hat it's one of the most popular distributions for forking. Part of the appeal is Debian's incredible commitment to free software, which means that if you base your distro on Debian you're guaranteed no patent, licensing or trademark clashes. Sure, you can go ahead and customise your distribution to include non-free stuff, but the Debian base means you always know where you start. And that ability to customise is one of the other great reasons for using Debian – the project is happy for people to fork as often and as completely as they want, which means you can make as many changes as you want and not worry.
The rise of Ubuntu has opened up one further possiblity here, which is to fork from a fork. Previously, distros that forked from Debian had to cope with the outdated software problem themselves, but that was usually worth the bother because it meant you got to take advantage of Debian's huge software repository and unparallelled community testing. But Ubuntu does all that work already – it's based on Debian (even giving easy access to all of Debian's universe and multiverse repositories), then updates the software to newer versions and patches/fixes it all appropriately.
The result is that you get all the size of Debian with all the latest features of Ubuntu. From your perspective, that means you can choose whether you want to base off Ubuntu or Debian, or indeed create a blend of both. One of the extra advantages to basing off Ubuntu is that you benefit at least a little from the Ubuntu mega-brand, but on the flip side you run the risk of people thinking you've made Yet Another Ubuntu Clone.
Beyond Debian, Red Hat/Fedora is another good candidate for distro base, largely because Red Hat puts in a lot of effort to ensure that all its packages and patches are open for the world to see and re-use if they want. The most prominent example of this is CentOS, the community rebuild of Red Hat Enterprise Linux. RHEL is where Red Hat makes most of its money, and yet the company produces the distro in such a way that anyone can rebuild it from scratch for free without too much hassle.
The simple choices
Before you get started, there are a few basic choices you need to make that will define your distro. Every distro makes these choices, so none of this counts as a unique selling point – you still need some spark of an idea!
- Free or non-free? By that we mean, do you want your finished distro to be 100% free software, or do you want to pre-install things such as Adobe Reader or Flash Player, or perhaps even some of the most common drivers?
- Live CD, installable, or both? Some distros (such as Fedora) opt for special Live respins, whereas others (like Ubuntu) have the Live CD and installable version on the same disc.
- CD or DVD? This is less of an issue if you'll be distributing your creation by hand, but if you intend people to download a whole DVD you'll need to start thinking about bandwidth.
- KDE, Gnome, or something else? The desktop is the first thing people will see when they start up, so it will really influence how people feel about your distro. Remember, a lot of people are diehard fans of one desktop, so you might consider following Ubuntu's lead and providing some choice!
Use the source
You might think that a small distro project, based on another well-known distro, wouldn't have to spend much time on GNU GPL compliance. After all, the source code is probably available from some upstream server, right? You'd be wrong. The GPL requires the distributor (that's you) to make matching source code available for every binary of each software package that you obtain under the licence. This has proved an onerous undertaking for those distro projects that didn't think much about it, until someone pointed out that they were violating the GPL.
In terms of this key licence, even the most well-intentioned distro project that doesn't provide source code is no better than those problematic hardware OEMs that like to ship modified free software exclusively in binary form. So if you're going to distribute binaries in an install or Live image, plan ahead and think about where your users are going to get matching source code from. GPL version 2 referred to 'a medium customarily used for software interchange', which most people have interpreted to mean a CD-R or DVD-R.
GPL version 3 explicitly refers to downloading corresponding source code from a server or peer-to-peer network, which need not be in the same location that the object code (ie the binaries) are. However, you have to provide 'clear directions' so that a person downloading binaries can find the source code without having to go on a detective adventure across the internet.
Online distro building
Wouldn't it be great if you could just click your way to a custom distro? After all, most of the packages you will need are sitting on a well-connected web server somewhere, so it makes sense to build ISO images and repositories directly on that server. And since that server has a HTTP interface, why not make the distro building software into a web application? This is the principle behind www.instalinux.com, created by Chris Slater. It's based on the SystemDesigner CGI scripts from the Linux Common Operating Environment project (http://linuxcoe.sourceforge.net), originally developed as a tool for internal use at HP, and now released under the GNU GPL.
According to HP's Jeffrey Wade, quoted last year at the time of the LinuxCOE 4.0 release (www.itjungle.com/tlb/tlb060507-story07.html), the company has installed over 15,000 of its own desktops and servers using LinuxCOE. It also uses the software to install Linux on the servers it sells, and as part of its services offering too.
For the new distro creator, it certainly beats having to download all the source code over a domestic broadband connection, then figure out how to build it. The setup is very straightforward, and it supports a range of base distros. The only questions you have to answer in SystemDesigner are the same ones you would have to in a standard install. It's also very useful for a more experienced system administrator who has to apply an identical, pre-seeded configuration to a large number of machines. In the following example, we're going to take a look at creating a 64-bit Ubuntu Hardy desktop distro with additional educational packages from Edubuntu and support for Japanese – just because we can.
To create your custom distro, click on the Designer link on the Instalinux homepage and select your base distro. You currently have a choice of CentOS 4 or 5, Debian Etch, Fedora up to version 8, OpenSUSE up to 10.3, Scientific Linux up to 5.0, SUSE 9.3 or Ubuntu up to Hardy Heron. All base distros are available in both 32-bit and 64-bit Intel/AMD versions. Apart from these considerations, your only other option on the first page of the System Designer is to set the hostname, or set the target machine to grab a hostname via DHCP.
Check your sums
There's also an MD5SUM provided, which you should take a note of now – it's best to save the web page to your local machine, along with the preseed.txt file offered. Once your image download is complete, test it with the md5sum command. You should have the md5sum package already installed in any decent Linux distro, but if not, it's a standard package and should certainly be available from your current distro's online repository. Simply type md5sum in a terminal and append the name of the ISO image, like so:
daniel@64studio:~$ md5sum iso8574.iso 41a0b070302bae85ec8d5a0159969eb6 iso8574.iso
The output of the md5sum command should be exactly the same as the one shown on the final Instalinux page. Your check sum will almost certainly be different from mine, unless you specified exactly the same options. Having verified that your download is OK (because it would be really embarrassing to distribute a corrupted install image), you can burn it to CD using your preferred application. Remember to back up any personal data before testing the disc on a PC or laptop, especially if you specified automatic hard disc formatting during the setup process; as the boot screen warns, the contents of some or all drives may be erased.
Despite the preseed file containing language settings, in manual mode, the LinuxCOE version of the Ubuntu installer still asked us to set the install language and dialect. The rest of the install wasn't exactly bug-free either – at one point, the install halted with a 'Loading apt-mirror-setup failed for unknown reasons' message. We had to select Continue with the Tab and Enter keys before the downloading of packages from the Ubuntu apt mirror could continue. Once the installation was complete and the test laptop rebooted into GDM, we found the Japanese language support packages had not been fully installed, and so when changing the language setting in Gnome, additional packages had to be downloaded. But considering the breadth of Linux distros that SystemDesigner attempts to support, it's still an impressive tool. The other system settings were correctly pre-seeded, including the user and network details.
64-bit Edubuntu in Japanese, which is what we wanted. We don't speak Japanese, but we always have the freedom to learn!
If you're now happy with your custom Ubuntu install, then it's worth having a look at that preseed.txt file to understand how automated installs work. Although Ubuntu is normally installed with a GUI from a Live CD, automated installs are usually done with Debian's text-mode installer, known as d-i, because it's much more scriptable than a point and click environment. The preseed file is all commented in English, so it's easy to understand. For example, here's the part about the base distro's package repository:
# Where are we pulling bits from? d-i mirror/http/hostname string us.archive.ubuntu.com d-i mirror/http/directory string /ubuntu/
And here's the part about our initial user account:
# Mortal User d-i passwd/user-fullname string Dave Smith d-i passwd/username string dave d-i passwd/user-password-crypted passwd $1$QOimuo6b$4/wHzeVqXbQjpclCYDtbK/
No need to attempt to decrypt that user's password; we can save you the bother, and tell you that it's just 'dave'. Of course, in real life, you would never set such an easily guessed default password, would you? In fact, Ubuntu's user and groups tool wouldn't let you.
If you find the Instalinux site useful, you have the option to create and manage your profiles for SystemDesigner. Just click Profiles on the Instalinux web page, select your base distro, version and architecture of choice, then the 'Create a new LinuxCOE profile' button to get started. By using the Display button, you can recall profiles that you, or other Instalinux users, have already created.
Save your SystemDesigner profiles for another time, or share them.
Re-spinning Fedora and Debian Live
If you're a fan of the Fedora Project, you'll be pleased to discover that the Fedora Unity team has created a GUI application for making 'Re-Spins' of install and live media. Known as Revisor (http://revisor.fedoraunity.org), it's a front-end to programs including Pungi (https://fedorahosted.org/pungi) and live-cdtools (http://git.fedoraproject.org/git/?p=livecd).
On Fedora 7 or later, simply type in a terminal (as root):
# yum install revisor
Using Revisor is similar to running SystemDesigner, except that the action is taking place on the local machine. Revisor has the advantage over the online tool in that it can also make Live CDs, or images for live booting from USB sticks, but it's very much a Fedora-only affair. The nearest equivalent to Revisor for Debian-based distros is live-magic (http://packages.debian.org/lenny/live-magic) which is a GUI front-end for the live-helper scripts. As the names of these packages suggest, the Debian tools were originally meant just for the creation of Live images. Having said that, there is now support for including the Debian installer as an alternative entry in the bootloader of the Debian Live CD, which copies the live filesystem to the target hard disk. See http://alioth.debian.org/~lamby-guest/live-manual/html/common-tasks.html for details.
Revisor is a very user-friendly tool, but there's no sign of support for distros other than Fedora.
Implications for support
The major limitations of SystemDesigner, Revisor or live-magic are that you can only select packages and meta-packages that are already featured in the base distro, and that there isn't any mechanism for tweaking those packages (beyond pre-seeding installation options, or running a post-install script). These limitations are actually helpful, if you want to ensure that commercial support services for all the packages in the custom distro will be available from the organisation behind the parent distro, such as Novell or Canonical.
Most service agreements for Linux offerings indicate that you are perfectly free to modify the code, but if you do so, you cannot expect support for the modified packages. That's fair enough, because you might have screwed them up royally. Even if the packages in question still work, it's a bit much to expect the parent distro to support an indeterminate number of modifications are floating around deployment sites.
If you're not constrained by the need to keep individual packages unmodified from the base distro, and you want more customisation than that afforded by SystemDesigner, then there is a tool that can help: PDK. More on that soon!
11 distro design tips
Now that you have the first inkling of how your distro coming together, it's time to start thinking about the tricksy little unique selling point matter. In short, what can you do to stand out from the hundreds of other Linux distros that already exist?
There are a number of angles you can explore for this – choose one, two or all of them if you want!
Be cutting edge
At its height, Mandriva was famed for including the very latest software, but the management decided to focus less on features and more on stability, which meant that older, "tested" software became the norm, and many users left to find their cutting-edge fix elsewhere. Similarly, Ubuntu launched as an up-to-date version of Debian, but has also fallen prey to the same problem – Ubuntu 8.10 didn't include Mono 2.0, for example, despite it being released a month before the distro ships.
We're well aware of feature freezes and other stability-enhancing techniques, but if you want to capture the market for cutting-edge software then you need to forgo such niceties and focus on getting the software into your distro as fast you can. Your users will thank you for it!
KDE 4 brings a wealth of new cosmetic features, most notably pervasive SVG vector graphics support.
Be super stable
The opposite of being on the cutting-edge is being super stable, which means choosing your selection of apps from the sturdiest, the most tested and the most reliable options available to you.
Yes, this does mean shipping OpenOffice.org 2.x rather than 3.x, but it also means you should have fewer bug reports because the software you provided has already been through years of testing and fixing. The downside to this plan is that stable distros require years of support – Ubuntu LTS comes with five years of support on the desktop, and Red Hat Enterprise Linux comes with seven. If you're aiming to compete in this market, you need to be ready to provide lots of backports of security updates and the like.
One of the most impressive features of open source software is its apparent ability to outpace Intel's attempts to ramp up the performance of our computers. By that we mean that some of the most popular free software applications have a reputation for being slow, RAM-hungry or otherwise resource intensive, which is why there is such a market for thin-and-light distros.
The idea here is simple: rather than choosing Application A for your distro, which uses 100MB of RAM, choose Application B, which uses 10MB of RAM. Repeat that for just about everything in your distro (excluding any apps you honestly feel you can't live without), and suddenly even a Pentium II is starting to look quick.
Again, there's a lot of competition in this space, but all too often you either get a fully functional and fat desktop, or you get a slim-and-light desktop with a look and feel on a par with Windows 3.1. Perhaps there's room for you to innovate somewhere inbetween?
Focus on a group of users
If you want to build a loyal group of users, give them some love: make a distro focused on the needs of a specific kind of user, and you'll find that they stick with you through thick and thin. So, rather than create yet another do-everything distro that covers all bases, why not create a distro focused on developers? Or artists? Or gamers? This allows you to slim down your choice of packages to the essentials for your target market, then lets you pre-set all the options they most want to see. For example, here at TuxRadar HQ we're big fans of programming with Mono. Why not create a distro with all the Mono libraries and documentation pre-installed, along with all the best Mono-based apps for inspiration?
In Shuttleworth's words, "pretty is a feature". If average users were to choose between two distros that were identical apart from their look and feel, of course they would choose the one that looks the best. That said, "best" is clearly highly subjective – we're not particular fans of Sabayon's inky black interface, and yet many other people think it looks smart and sophisticated. Similarly, Ubuntu's brown can look warm and human or dull and 1970s depending on your background. So, when it comes to your own distro, please give some thought to the look and feel – don't just expect people to customise the design to suit their tastes, because if your initial design looks bad then people will just give up.
There are two options for you here: create your own theme for your desktop, or go for the defaults. It's easy to create a theme, and you can borrow the pieces from other places if Gimp isn't your forte. But there's a lot of value in going for the default look and theme for your desktop, because so many distros insist on branding things with their logo. So rather than having a K button for KDE you get a gecko, or you get an Ubuntu logo rather than the Gnome foot in the menu bar.
Ubuntu espouses the pretty is a feature policy. Well, if you like brown, that is.
Aim for ease of use
This one is becoming a bit tired these days – Mandriva was the first distro to really make it work, but now multiple distros compete for the title of being most easy to use. And the downsides don't end there: being easy to use invariably means having to create your own graphical tools for basic system admin, and you should expect lots of flak from some of the more aggressive community members for "dumbing down" Linux. Still, if you succeed in making Linux easy to use, then you're targeting the most important market: people who are Windows/Mac users and want to jump ship to Linux.
The alternative is to play the Slackware card, which means creating absolutely no ease-of-use tools and letting users get on with it. That might sound bad on paper, but lots of more advanced users really dislike the way that some distros overwrite their customised settings by trying to be "easy", which means that something like Slackware has a lot of appeal.
Focus on a language or country
This is the easiest option, both for creating a distro and for attracting an instant user base. The downside, rather predictably, is that it rather requires you to come from/live in a country with weak free software support, but you'd be surprised at just how successful you can be if you try. Remember, if your language is in the minority in your country (think Scottish Gaelic in Scotland), you may be able to scrounge up some government funding for helping promote its survival and propagation.
Solve a problem
This option sounds easy, but it really isn't: find a problem that lots of people have with their computer or with Linux, then solve it. It sounds easy because everyone has problems with their computer, but it's not easy because the mass-market computing industry has been striving for improvement for the last 20 years now and those problems still exist despite the best efforts of the world's largest software companies. The smart money is on tackling a small problem, or at least breaking a larger one up into small chunks, so that you can make progress easily.
Build a community
Some distros, most notably PCLinuxOS, owe a large amount of their popularity to their friendly and open community, and it's not hard to replicate their success. The key to getting a large community is to help users help themselves and each other as early as possible. That is, if the only people who can answer questions about your distro are experts, then only a few people will be able to help build the community. But if your distro is designed in such a way that even people with only a few days of learning can already start answering forum questions, then you'll find people join in much faster and the whole community springs up at lightning speeds.
Target some hardware
This is a tempting but tricky option: tempting because if you can get a good Linux port on to a specific device then you'll find it easy to gain users and may even gain support or funding from the manufacturer; tricky because you need to do an awful lot of the legwork yourself – if the term "cross-compiling" means nothing to you, then this is definitely something to leave alone.
And finally: copy, copy, copy!
The whole point of free software is that we're sharing ideas, sharing code and sharing innovations among ourselves to benefit all computer users. So go and try some of the top distributions. Try PCLinuxOS. Try Linux Mint. Try Sabayon. Write down a list of three things from each one that you like, then steal them for your own distro. Not-Invented-Here syndrome doesn't exist on Linux, so use that to your advantage and copy stuff!
Tips from the pros
Every issue of Linux Format magazine comes with at least one (and sometimes up to a dozen!) Linux distributions on the DVD. Given that we print 13 issues a year, this alone is testament to just how many Linux distros there are out there!
Because we want our readers to get the most from our discs, we don't just drop some ISOs on to a DVD image and send it off to the printers. Instead, our full-time disc editor spends his days trying to add features, customise boot scripts, encourage distros to multi-boot and more, all so that you spend less time wondering how to get stuff working and more time actually using your new distro.
One of the most famous LXF discs is known informally as 'Mikebuntu' – it's a Mike Saunders-customised edition of Ubuntu that comes with a lot of extra software pre-installed so that you don't need to spend time and money downloading it later. If you've ever fancied trying to create your own Mikebuntu-alike (perhaps with even more software!), we asked Mike to reveal just how it's done, and why Ubuntu is so commonly used in the first place. Here's what he said:
"We often rebuild Ubuntu releases to put on the coverdisc, adding extra software beyond the stock installation. Ubuntu makes this relatively straightforward by including a compressed filesystem image on the CD. There are no complicated lists, dependencies and package interactions to fathom out. Ubuntu stores the compressed image in SquashFS format, so we loopback mount it and copy the contents into a temporary directory.
Next, we use chroot to switch into the expanded filesystem as if it was a standalone Linux installation, then apt-get extra packages. Once we're happy with the software selection, we exit the chroot environment and then rebuild the SquashFS filesystem image. Finally, we copy the new image onto our DVD, and incorporate the Ubuntu bootloader and scripts around it."
So, now you know how the magic happens. If you're still not 100% sure how it's done, post a message on our forums (www.linuxformat.com/forums) with your specific questions and Mike will try to help you out.
Ubuntu's design makes it possible for mere mortals to rebuild the distro and include non-standard software such as Scribus.
PDK - the hard way
Daniel James from 64 Studio proudly shows off PDK - the Platform Development Kit.
Back in the year 2000, Debian project founder Ian Murdock launched a start-up with John Hartman called Progeny Linux Systems. Its original mission was to provide a more commercial version of Debian with a Gnome desktop and an easy-to-use GUI installer; very much like Ubuntu became, but perhaps Progeny was ahead of its time. By 2003, the company had shifted focus to what it called Componentised Linux. The reasoning behind this service offering was that standard Linux distributions were too monolithic and top-down in design. Progeny's business model was based on customers ordering bespoke distros that the company would then assemble from ready-made components.
In 2005, Progeny announced it was working on software called the Platform Development Kit, PDK for short, which would allow version control of the distro building process. Several well-received releases of Progeny Debian were made using the Componentised Linux framework, but the idea didn't really catch on. Murdock left the company to work for the Linux Foundation, and ultimately Sun Microsystems. Progeny subsequently closed its doors in 2007.
PDK might have been lost altogether, since although the tool was released as free software, it was not widely understood or used outside of Progeny. Fortunately, we'd been using PDK at 64 Studio since early 2006, and our CTO Free Ekanayaka had already contributed improvements to the Python codebase, including automatic resolution of dependencies. We continue to maintain PDK, and make it available under the GNU GPL via our site at http://trac.64studio.com/pdk.
PDK's chief benefit is that it automates the building and maintenance of custom Debian- and RPM-based distros. This automation becomes particularly useful if a development team has to maintain a variety of different distros. In the component-based approach, the common parts of the various distros only have to be maintained once, which not only saves a great deal of time, but of course makes quality control a lot easier to get right. The components to be included in a particular distro are specified by a series of XML files, which reference component files and other custom settings, like installation pre-seed values. Links to component files are found in the file picax.xml, like so:
<contents> <component>system/all.xml</component> <component>linux/2.6.26.xml</component> <component>gnome/key.xml</component> <component>graphics/gimp.xml</component> <component>64studio/custom.xml</component> <component>64studio/closure.xml</component> </contents>
In the example above, you can see that the components are broken up by section – there's a directory path for the system base components, another for the Linux kernel packages, and one for the desktop environment, which in this case is Gnome. There's also an example of a component for Gimp. This gimp.xml file not only contains references to the various binary packages needed in different architectures and the Gimp source package, but also the supplementary packages that the distro maintainer believes the user is going to want, such as gimp-help (documentation) or Gutenprint (printer drivers).
Next, there's a custom.xml file that specifies packages which aren't found in the parent distro, Debian in this example. Finally, there's a closure.xml file which takes care of dependencies.
PDK not only produces install media, but binary and source repositories too. This enables you to roll out updates to users of your custom distro without them having to obtain new physical media or re-install. That can be a problem with some custom distros, particularly those based on Live CDs where there is no update mechanism. Generating source repositories also helps you meet you obligations as a free software redistributor under the GNU GPL and other licences.
In addition, PDK has an API that you can hook into to produce all kinds of reports, for example in a format suitable for a security audit on package version numbers.
At 64 Studio we use PDK in combination with the Trac source code and project management tool (http://trac.edgewall.org), because developers then have a source browser on the web, together with all the other integrated ticketing and wiki features that Trac provides. Trac was originally intended for use with the Subversion (SVN) revision control system, but there is a plugin for Trac 0.10.x that enables Git support, available from http://nanosleep.org/proj/trac-git-plugin. We're hoping to integrate Trac with PDK to provide a more seamless interface for custom distro production in future.
First published in Linux Format magazine