fedora /usr/lib vs /usr/lib64

Question

It seems i have two python2.6 folders located in /usr/lib vs /usr/lib64 respectively. Most python stuff (source) is in /usr/lib64/python2.6 but when in installed packages they have been put into /usr/lib/python2.6

How the system decides which directory to go when python is requested, and how it finds the packaged i installed?

Answer 1

Any source being installed to /usr/lib64 should be from installing src or devel packages which, by default, are picked by your architecture.

/usr/lib should only have 32bit libraries - likewise /usr/lib64 should be 64bit version. I've found that yum will occasionally install both 32 & 64 bit versions of some libraries, and there are some libraries yet to be ported to 64 bit so if you're particular app or library has installed to /usr/lib odds are its either just to satisfy some 32bit only apps dependency or yum messed up.

Answer 2

This simple answer is that packages built without any C/native extensions should end up under lib, any ones with native extensions will end up in lib64 on multilib systems. As to how it finds the packages is contained in sys.path - this is from a x86_64 F-11 system:

>>> import sys
>>> for pth in sys.path: print pth
... 

/usr/lib64/python26.zip
/usr/lib64/python2.6
/usr/lib64/python2.6/plat-linux2
/usr/lib64/python2.6/lib-tk
/usr/lib64/python2.6/lib-old
/usr/lib64/python2.6/lib-dynload
/usr/lib64/python2.6/site-packages
/usr/lib64/python2.6/site-packages/Numeric
/usr/lib64/python2.6/site-packages/gst-0.10
/usr/lib64/python2.6/site-packages/gtk-2.0
/usr/lib/python2.6/site-packages

The more detailed answer on how packages get there requires a bit of understanding about how python works in terms of it's own layout. What we're interested in is part of the standard library called distutils. This is the workhorse, note there are also tools built on top of this (setuptools) and a fork called distribute at the moment trying to improve python packaging.

There is an important patch that fedora applies to talk about here which makes all this work:

• python-2.6-lib64.patch

This patch is conditionally applied in the RPM spec for python on architectures where the lib dir is lib64:

If we look at how this patches distutils:

diff -up Python-2.6/Lib/distutils/sysconfig.py.lib64 Python-2.6/Lib/distutils/sysconfig.py
--- Python-2.6/Lib/distutils/sysconfig.py.lib64 2008-06-05 08:58:24.000000000 -0400
+++ Python-2.6/Lib/distutils/sysconfig.py   2008-11-24 02:34:04.000000000 -0500
@@ -115,8 +115,12 @@ def get_python_lib(plat_specific=0, stan
         prefix = plat_specific and EXEC_PREFIX or PREFIX

     if os.name == "posix":
+        if plat_specific or standard_lib:
+            lib = "lib64"
+        else:
+            lib = "lib"
         libpython = os.path.join(prefix,
-                                 "lib", "python" + get_python_version())
+                                 lib, "python" + get_python_version())
         if standard_lib:
             return libpython
         else:

We now have a condition on distutils that now changes what distutils.sysconfig.get_python_lib() returns in the cases when we are asking about platform specific or system packages. You can experiment with calling this with various options in a python interpreter:

This function gets used within distutils - we can see from the doc string what it does:

Docstring:
    Return the directory containing the Python library (standard or
    site additions).

    If 'plat_specific' is true, return the directory containing
    platform-specific modules, i.e. any module from a non-pure-Python
    module distribution; otherwise, return the platform-shared library
    directory.  If 'standard_lib' is true, return the directory
    containing standard Python library modules; otherwise, return the
    directory for site-specific modules.

    If 'prefix' is supplied, use it instead of sys.prefix or
    sys.exec_prefix -- i.e., ignore 'plat_specific'.

So when using building a python package using distutils (or the layers built on top of it) we will at some point ask the system configuration where the right place to put the files are, depending on if it is a system or platform lib it'll go in lib64 else it'll go in lib.

If you look at the Fedora Python Packaging documentation or use the fedora rpmdev tool to create a skeletal python spec rpmdev-newspec python-foo you'll see detailed comments of how fedora sets variables for the rpm build based on calling this function.