Simon! I know how you feel; I struggled with this part of learning Linux, too. Based on my own experiences, I wrote a tutorial about some of the items that you address (mostly as a reference for myself!): http://easyaspy.blogspot.com/2008/12/buildinginstalling-application-from.html. I think you'll appreciate my note about how simple Python applications are to build/install. :)
Hope that this helps! And happy compiling.
Tim Jones
Building/Installing an application from source in Ubuntu Linux
While the Ubuntu repositories are chock full of great applications, at one time or another you are bound to come across that "must-have" tool that isn't in the repositories (or doesn't have a Debian package) or you need a newer version than in the repositories. What do you do? Well, you have to build the application from source! Don't worry, it's really not as complicated as it sounds. Here are some tips, based on my experiences of going from being a rank amateur! (While I'm using Ubuntu for this example, the general concepts should be applicable to most any Unix/Linux distribution, such as Fedora, and even the Cygwin platform on Windows.)
The basic process of building (compiling) most applications from source follows this sequence: configure --> compile --> install. The typical Unix/Linux commands to do these things are: config
--> make
--> make install
. In some cases, you'll even find web pages that show that all of these can be combined into a single command:
$ config && make && make install
Of course, this command assumes that there are no problems in any of these steps. This is where the fun comes in!
Getting Started
If you haven't compiled an application from source on your system before, you will probably need to set it up with some general development tools, such as the gcc
compiler suite, some common header files (think of this as code that has already been written by someone else that is used by the program you are installing), and the make tool. Fortunately, in Ubuntu, there is a metapackage called build-essential
that will install of this. To install it (or just make sure that you already have it!), run this command in the terminal:
$ sudo apt-get install build-essential
Now that you have the basic setup, download the application source files and save them to a directory for which you have read/write permissions, such as your "home" directory. Typically, these will be in an archive file with file extension of either .tar.gz
or .tar.bz2
. The .tar
simply means that it's a "tape archive", which is a grouping of files that preserves their relative directory structure. The .gz
stands for gzip (GNU zip), which is a popular Unix/Linux compression format. Similarly, the .bz2
stands for bzip2, which is a newer compression format that provides higher compression (smaller compressed file size) than gzip.
After you've downloaded the source file, open a terminal window (System Terminal from the Ubuntu menu) and change to the directory where you saved your file. (I'll use ~/download
in this example. Here, '~' is a shortcut to your "home" directory.) Use the tar command to extract the files from the downloaded archive file:
If your file is a gzip archive (e.g., ends with .tar.gz
), use the command:
$ tar -zxvf filename.tar.gz
If your file is a bzip2 archive (e.g., ends with .tar.bz2
), use the command:
$ tar -jxvf filename.tar.gz
Tip: If you don't want to have to
remember all of the command line
switches for extracting archives, I
recommend getting one (or both) of
these utilities: dtrx (my favorite!)
or deco (more popular). With either of
these utilities, you just enter the
name of the utility (dtrx or deco) and
the filename, it it does all of the
rest. Both of these "know" how to
handle most any archive format that
you are likely to run across and they
have great error handling.
When building from source, there are two common types of errors that you are likely to encounter:
- Configuration errors occur when you run the configuration script (usually named config or configure) to create a makefile that is specific to your setup.
- Compiler errors happen when you run the make command (after the makefile has been generated) and the compiler is unable to find some code that it needs.
We'll look at each of these and discuss how to resolve them.
Configuration and Configuration Errors
After you've extracted the source code archive file, in the terminal, you should change to the directory that contains the extracted files. Typically, this directory name will be the same as the name of the file (without the .tar.gz
or .tar.bz2
extension). However, sometimes the directory name is just the name of the application, without any version information.
In the source directory look for a README
file and/or an INSTALL
file (or something with similar names). These files typically contain useful information about how to build/compile the application and to install it, including information about dependencies. "Dependencies" are just a fancy name for other components or libraries that are required to successfully compile.
After you've read the README
and/or INSTALL
file (and, hopefully looked at any relevant online documentation for the application), look for an executable (has the "x" permission set on the file) file named config
or configure
. Sometimes the file might have an an extension, such as .sh
(e.g., config.sh
). This is usually a shell script that runs some other utilities to confirm that you have a "sane" environment for compiling. In other words, it will check to ensure that you have everything installed that you need.
Tip: If this is a Python-based
application, instead of a config file,
you should find a file named setup.py
.
Python applications are typically very
simple to install. To install this
application, as root (e.g., put sudo
in front of the following command
under Ubuntu), run this command:
$ python setup.py install
That should be all that you need to
do. You can skip the remainder of this
tutorial and proceed directly to using
and enjoying your application.
Run the configuration script in the terminal. Typically, you can (and should!) run your configuration script with your regular user account.
$ ./config
The script will display some messages to give you an idea of what it is doing. Often, the script will give you an indication of whether it succeeded or failed and, if it failed, some information about the cause of the failure. If you don't get any error messages, then you can usually assume that everything went fine.
If you don't find any script that looks like a configuration script, then it typically means that the application is a very simple one and it is platform independent. This means that you can simply skip to the build/compile step below, because the provided Makefile
should work on any system.
An Example
In this tutorial, I'm going to use the text-based RSS reader called Newsbeuter as an example for the types of errors that you may encounter when building your application. For Newsbeuter, the name of the configuration script is config.sh
. On my system, when I run config.sh
, the following errors occur:
tester@sitlabcpu22:~/download/newsbeuter-1.3$ ./config.sh
Checking for package sqlite3... not found
You need package sqlite3 in order to compile this program.
Please make sure it is installed.
Upon doing some research, I found that, in fact, the sqlite3
application was installed. However, since I'm trying to build from source, this is a tip that what config.sh
is actually looking for are the development libraries (headers) for sqlite3
. In Ubuntu, most packages have an associated development counterpart package that ends in -dev
. (Other platforms, such as Fedora, often use a package suffix of -devel
for the development packages.)
To find the appropriate package for the sqlite3
development package, we can use the apt-cache
utility in Ubuntu (and, similarly, the yum
utility in Fedora):
tester@sitlabcpu22:~/download/newsbeuter-1.3$ sudo apt-cache search sqlite
This command returns quite a big list of results, so we have to do a bit of detective work to determine which is the appropriate package. In this case, the appropriate package turns out to be libsqlite3-dev
. Notice that sometimes the package we are looking for will have the lib
prefix, instead of just the same package name plus -dev
. This is because sometimes we are just looking for a shared library that may be used by many different applications. To install libsqlite3-dev
, run the typical apt-get install command in the terminal:
tester@sitlabcpu22:~/download/newsbeuter-1.3$ sudo apt-get install libsqlite3-dev
Now, we have to run config.sh
again to make sure that we've resolved this dependency problem and that we don't have any more dependency problems. (While I won't show it here, in the case of Newsbeuter, I also had to install the libcurl4-openssl-dev
package, as well.) Also, if you install a development package (like libsqlite3-dev
) and the associated application package (e.g., sqlite3
) is not already installed, most systems will automatically install the associated application package at the same time.
When the configuration runs successfully, the result will be that it will create one or more make files. These files are typically named Makefile
(remember that file name case matters in Unix/Linux!). If the build package includes sub-directories, such as src
, etc., each of these sub-directories will contain a Makefile
, as well.
Building and Compilation Errors
Now, we are ready to actually compile the application. This is often called building and the name is borrowed from the real-world process of constructing something. The various "pieces" of the application, which are typically multiple source code files, are combined together to form the overall application. The make utility manages the build process and calls other applications, such as the compiler and linker, to actually do the work. In most cases, you simply run make (with your regular user account) from directory where you ran the configuration. (In a few cases, such as compiling applications written with the Qt library, you will need to run another "wrapper" application like qmake instead. Again, always check the README
and/or INSTALL
documents for details.)
As with the configuration script above, when you run make (or the similar utility) in the terminal, it will display some messages about what is executing and any warnings and errors. You can typically ignore warnings, as they are mainly for the developers of the application and are telling them that there are some standard practices that are being violated. Usually, these warnings do not affect the application function. On the other hand, compiler errors must be dealt with. With Newsbeuter, when I ran make, things went fine for a while, but then I got an error:
tester@sitlabcpu22:~/download/newsbeuter-1.3$ make
...
c++ -ggdb -I/sw/include -I./include -I./stfl -I./filter -I. -I./xmlrss -Wall -Wextra -DLOCALEDIR=\"/usr/local/share/locale\" -o src/configparser.o -c src/configparser.cpp
c++ -ggdb -I/sw/include -I./include -I./stfl -I./filter -I. -I./xmlrss -Wall -Wextra -DLOCALEDIR=\"/usr/local/share/locale\" -o src/colormanager.o -c src/colormanager.cpp
In file included from ./include/pb_view.h:5,
from src/colormanager.cpp:4:
./include/stflpp.h:5:18: error: stfl.h: No such file or directory
In file included from ./include/pb_view.h:5,
from src/colormanager.cpp:4:
./include/stflpp.h:33: error: ISO C++ forbids declaration of \u2018stfl_form\u2019 with no type
./include/stflpp.h:33: error: expected \u2018;\u2019 before \u2018*\u2019 token
./include/stflpp.h:34: error: ISO C++ forbids declaration of \u2018stfl_ipool\u2019 with no type
./include/stflpp.h:34: error: expected \u2018;\u2019 before \u2018*\u2019 token
make: *** [src/colormanager.o] Error 1
The make process will stop as soon as the first error is encountered. Handling compiler errors can sometimes be tricky business. You have to look at the errors for some clues about the problem. Typically, the problem is that some header files, which usually have extension of .h
or .hpp
, are missing. In the case of the error above, it is (or should be!) clear that the problem is that stfl.h
header file cannot be found. As this example shows, you want to look at the first lines of the error message and work your way down to find the underlying cause of the problem.
After looking at the Newsbeuter documentation (which I should have done before I started, but then this part of the tutorial wouldn't be very meaningful!), I found that it requires a 3rd-party library called STFL. So what do we do in this case? Well, we essentially repeat this exact same process for that required library: obtain the library and execute the configure-build-install process for it and, then, resume building the desired application. For example, in the case of STFL, I had to install the libncursesw5-dev
package for it to build properly. (Usually, it is not necessary to redo the configuration step on our original application after installing another required application, but it never hurts either.)
After successfully installing STFL toolkit, the make process for Newsbeuter ran successfully. The make process typically picks up where it leaves off (at the point of the error). Thus, any files that had already compiled successfully will not be recompiled. If you want to recompile everything, you can run make clean all to remove any compiled objects and then run make again.
Installing
After the build process completes successfully, you are ready to install the application. In most cases, to install the application to the common areas of the file system (e.g., /usr/bin
or /usr/share/bin
, etc.), you will need to run the installation as root. Installing is really the simplest step in the whole process. To install, in the terminal run:
$ make install
Check the output of this process for any errors. If everything was successful, you should be able to run the command name in the terminal and it will launch. (Append & to the end of the command line, if it's a GUI application, or you won't be able to use the terminal session until the application finishes running.)
When you build an application from source, it won't typically add an icon or shortcut to the GUI menus in Ubuntu. You will need to add this manually.
And that is basically the process, albeit potentially iterative, to building and installing an application from source in Ubuntu. After you've done this just a few times, it'll become second nature to you!