How to Install CRF++ on Server Without Sudo Permisson

CRF++ is a API that implements Conditional random field technique, it also has a Python API. It is used in one of my projects and I need to install in on server without sudo permission, the author didn’t specify how to do that in detail. So I think it might be helpful to share my experience, here is how to do this:

1. Install Python in home directory

With sudo permission one can easily install Python using one line command, but without that, Python can be installed to, just a little bit trickier, first download the specific version of Python under Python Source Releases, here I chose Python 2.7.10:

wget https://www.python.org/ftp/python/2.7.10/Python-2.7.10.tar.xz
tar -xvf Python-2.7.10.tar.xz
cd Python-2.7.10

There is a configure script under the unzipped directory and it helps generate make files and check dependencies. If execute:

./configure --help

There will be information like this:

Installation directories:
  --prefix=PREFIX         install architecture-independent files in PREFIX
                          [/usr/local]
  --exec-prefix=EPREFIX   install architecture-dependent files in EPREFIX
                          [PREFIX]

By default, 'make install' will install all the files in
'/usr/local/bin', '/usr/local/lib' etc.  You can specify
an installation prefix other than '/usr/local' using '--prefix',
for instance '--prefix=$HOME'.

Because without sudo permission one usually can’t write to /usr/local/ dir, so here I changed the prefix, notice you need to mkdir yourself, configure won’t do that for you:

./configure --prefix=$HOME/local/python

Then compile python with make files generated:

make
make install

After the installation, check if success:

$HOME/local/python/bin/python --version
>> Python 2.7.10

If want to set it as the default python version, one need to change the import path in profile(my default profile is .bash_profile) by add line below:

export PATH="$HOME/local/bin/python:$PATH"

Then check if success:

source .bash_profile
python --version
>> Python 2.7.10
which python
>> /your_home_dir/local/python/bin/python

2. Install CRF++

First download CRF++ source file from here, I’m using version 0.58. Then go into the unzipped file, install with prefix just as described above:

./configure --prefix=$HOME/local/CRF++
make
make install

$HOME/local/CRF++/bin is where command crf_learn and crf_test in, so don’t forget to add local CRF++ bin path in profile file:

export PATH="$HOME/local/CRF++/bin:$PATH"

Then test if CRF++ is correctly installed:

source .bash_profile
which crf_learn
>> /your_home_dir/local/CRF++/bin/crf_learn

3. Install CRF++ python API

After both Python and CRF++ is successfully installed, now we can install the python API, first cd in the python dir under CRF++ install package, and there are files like these:

CRFPP.py
CRFPP_wrap.cxx
README
setup.py
test.py

After reading the file setup.py script, apparently the auther is using distutils.core.Extension to add the C-Extension files in python, the original code is like this:

#!/usr/bin/env python
from distutils.core import setup,Extension,os
import string

setup(name = "mecab-python",
      py_modules=["CRFPP"],
      ext_modules = [Extension("_CRFPP",
                               ["CRFPP_wrap.cxx",],
                               libraries=["crfpp", "pthread"])
                     ])

However from the document Building C and C++ Extensions with distutils, the default dirs for this setup to find C include files and lib files are /usr/local/include and /usr/local/lib so we need to make change the setup.py file like this:

#!/usr/bin/env python

from distutils.core import setup,Extension,os
import string

setup(name = "mecab-python",
      py_modules=["CRFPP"],
      ext_modules = [Extension("_CRFPP",
                               ["CRFPP_wrap.cxx",],
                               include_dirs = ['/your_home_dir/local/CRF++/include'],
                               library_dirs = ['/your_home_dir/local/CRF++/lib'],
                               libraries=["crfpp", "pthread"])
                     ])

Then we can run in the directory:

python setup.py build
python setup.py install

Then for the final step, we need to add our local CRF++ lib to the system shared path so Python can find it, the detailed explanation is in here, but basically, we just need to add this line to our profile:

export LD_LIBRARY_PATH="$HOME/local/CRF++/lib:$LD_LIBRARY_PATH"

Then again:

source .bash_profile

To test if the python API is successfully installed:

python
>> import CRFPP
>> CRFPP
<module 'CRFPP' from '/your_home_dir/local/python/lib/python2.7/site-packages/CRFPP.pyc'

That’s it, then you can just explore the usage of CRFPP happily. :)