Sunday, August 12, 2012

Two hours I'll never get back

Today I had to install node.js on a CentOS 5.7 box.
As usual with system administration you never know how long it will take to complete. But since I managed to do it and found it rather painful I wanted to share the moment with you :)
This is a commented copy/past of the operation log.



$ ssh orion@xxxxx
orion@xxxxx password: 
Last login: Tue Jul 10 12:01:23 2012 from xxxxx
CentOS release 5.7 (Final)
Linux xxxxx.ovh.net 2.6.34-xxxx-std-ipv4-32 #2 SMP Thu Jun 17 07:12:38 UTC 2010 i686 i686 i386 GNU/Linux
This server is hosting many web sites and I really don't want to mess the things up so I'm going to be extra-careful and keep all my activity in a sandbox.

When I am on a production server I usually create a "inst" folder in my $HOME directory where I put all the sources I need to compile.

$ cd inst
Then get the sources. At this point I really don't bother with potential dependencies since I know the problem will come soon enougth... IYKWIM.

$ sudo wget http://nodejs.org/dist/v0.8.6/node-v0.8.6.tar.gz
--2012-08-12 15:44:07--  http://nodejs.org/dist/v0.8.6/node-v0.8.6.tar.gz
Resolving nodejs.org... 8.12.44.238
Connecting to nodejs.org|8.12.44.238|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11468801 (11M) [application/octet-stream]
Saving to: `node-v0.8.6.tar.gz'

100%[======================================>] 11,468,801  3.50M/s   in 3.1s    

2012-08-12 15:44:11 (3.50 MB/s) - `node-v0.8.6.tar.gz' saved [11468801/11468801]

$ tar zxvf node-v0.8.6.tar.gz 

Ok, now, ... let's meet the problems

$ ./configure
  File "./configure", line 325
    o['default_configuration'] = 'Debug' if options.debug else 'Release'
                                          ^
SyntaxError: invalid syntax

I told you...
A bit of search in g**gle tells me the problem must come from an outdated python package bundled with the CentOS 5.7. mmmmm... am I gonna like this distro or not...

$ python --version
Unknown option: --
usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
Try `python -h' for more information.

Outdated it seems indeed. Okay I've never used Python. It doesn't comply well with the GNU command line long options though :)

$ python -V
Python 2.4.3

Many comments on the web say that python < 2.6 cannot build node.js sources so I need a newer python. Latest stable is (at the time I am writing these lines) 2.7.3.

$ sudo wget http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tgz
$ tar zxvf Python-2.7.3.tgz 

I will install all of the binaries in a safe place and will prefix every configure. Since the /var/www is mount of a disk with a lot of free space. I made a folder named mcc inside of it for all my stuff.

$ ./configure --prefix /var/www/mcc/
...
configure: creating ./config.status
config.status: creating Makefile.pre
config.status: creating Modules/Setup.config
config.status: creating Misc/python.pc
config.status: creating Modules/ld_so_aix
config.status: creating pyconfig.h
config.status: pyconfig.h is unchanged
creating Modules/Setup
creating Modules/Setup.local
creating Makefile

$ make

At this point you would believe just like me that it's going to be perfect. How naive we are. make did finish without any error. which is great.

There seems to be a consensus in the community of people installing python from sources on linux distros that "make altinstall" is safer (especially if you prefixed your configure). I take it.

$ sudo make altinstall
$ /var/www/mcc/bin/python2.7 -V
Python 2.7.3

Now Python 2.7.3 is installed in /var/www/mcc/bin. Thrilled! Let's go on with node.js not without having put the new python path in the $PATH environment variable and symlink python.

$ pushd /var/www/mcc/bin
$ sudo ln -s python2.7 python
$ export PATH=/var/www/mcc/bin:$PATH
$ popd

Same player, try again.

$ ./configure --prefix /var/www/mcc/
{ 'target_defaults': { 'cflags': [],
                       'default_configuration': 'Release',
                       'defines': [],
                       'include_dirs': [],
                       'libraries': []},
  'variables': { 'host_arch': 'ia32',
                 'node_install_npm': 'true',
                 'node_install_waf': 'true',
                 'node_prefix': '/var/www/mcc/',
                 'node_shared_openssl': 'false',
                 'node_shared_v8': 'false',
                 'node_shared_zlib': 'false',
                 'node_use_dtrace': 'false',
                 'node_use_etw': 'false',
                 'node_use_openssl': 'true',
                 'target_arch': 'ia32',
                 'v8_no_strict_aliasing': 1,
                 'v8_use_snapshot': 'true'}}
creating  ./config.gypi
creating  ./config.mk

Yesss. but then...
$ make
...

Traceback (most recent call last):
  File "../../tools/js2c.py", line 36, in 
    import bz2
ImportError: No module named bz2
make[1]: *** [/home/orion/inst/node-v0.8.6/out/Release/obj/gen/libraries.cc] Error 1
make[1]: Leaving directory `/home/orion/inst/node-v0.8.6/out'
make: *** [node] Error 2

Ishhhk! It misses the bz2 support in python... I hate snakes. Bzzzzz!
To be fair, if I had read the whole Python build log (around line #21245675554) I would have known that it didn't build the bz2 support.
Okay so to have the bz2 support it must have detected the bz2 library or something. Let's check that with yum:

$ yum list | grep bz
bzip2.i386                            1.0.3-6.el5_5                   installed 
bzip2-libs.i386                       1.0.3-6.el5_5                   installed 
bzip2-devel.i386                      1.0.3-6.el5_5                   base    

Then, get it:

$ sudo yum install bzip2-devel
...
Complete!

And now you may be saying that I did touch the system and installed bzip2-devel system wide. You are right but it's a standard package for CentOS not a custom made from a custom repository. And here we go again, let's rebuild python once more.
$ ./configure --prefix /var/www/mcc/ && make clean && make
$ sudo make altinstall

To check that python has bz2 support:
$ python -c "import bz2; print bz2.__doc__"
The python bz2 module provides a comprehensive interface for
the bz2 compression library. It implements a complete file
interface, one shot (de)compression functions, and types for
sequential (de)compression.

Okay, now let's build node ... again.
$ cd ../node-v0.8.6
$ make

And Yes, It built BUT...
$ sudo make install
make -C out BUILDTYPE=Release V=1
make[1]: Entering directory `/home/orion/inst/node-v0.8.6/out'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/home/orion/inst/node-v0.8.6/out'
ln -fs out/Release/node node
python tools/install.py install 
  File "tools/install.py", line 219
    cmd = args[1] if len(args) > 1 else 'install'
                   ^
SyntaxError: invalid syntax
make: *** [install] Error 1

F*CK!

$ python -V
Python 2.7.3

....(12 seconds of thinking)...

$ sudo python tools/install.py install
installing /var/www/mcc/include/node/ares.h
installing /var/www/mcc/include/node/ares_version.h
installing /var/www/mcc/include/node/uv.h
...
installing /var/www/mcc/lib/node_modules/npm/node_modules/which/bin/which
symlinking ../lib/node_modules/npm/bin/npm-cli.js -> /var/www/mcc/bin/npm
updating shebang of /var/www/mcc/bin/npm to /var/www/mcc/bin/node

Houra! It worked. Although I don't understand why make install failed at once. Maybe a hard coded python path or an exported PATH env variable in the makefile did override my setup. I should report this to the developers. Now let's enjoy this with a great node.js program:

$ /var/www/mcc/bin/node -e 'console.log("That all folks!")'

EDIT:
Concerning the "sudo python tools/install.py" trick. The thing here is that root may not have the right (newly updated) python path because I did not change anything in /etc/profile, right? What is a bit confusing here is that I did install the whole bunch of new tools' versions in a private directory (/var/www/mcc). And logically I should own this directory and thus not use any "sudo" to install stuff in it. Lesson learned ... ;)

Oh and also, don't forget to set your $NODE_PATH to the right node_modules directory path. In my case I have set it to /var/www/mcc/lib/node_modules

13 comments:

  1. Great findings! Thank you very much.

    I had the same issue... and probably found why.

    I did almost the same path... Update Python2.4 -> 2.7. Download and untar node-v0.8.8. ./configure, make ... Things look good.

    Then, make install failed with the same error.

    I checked $PATH
    >echo $PATH, and found in addition to my main executable /usr/local/bin/python, /usr/bin/python is also pointing to old 2.4. Once I fixed it, no problem, I was able to "make install"

    ReplyDelete
    Replies
    1. Good to know you succeeded.
      I will double check the $PATH of root next time. Thanks for the feedback!

      Delete
  2. Awesome, Dude! You helped me figured it out. THANK YOU!

    ReplyDelete
  3. Thanks! I had figured out all the steps myself, except for that last one.

    $ sudo python tools/install.py install

    Why is that necessary? Anyway, thanks a lot!!

    ReplyDelete
    Replies
    1. Hey Greg, sorry for the late answer. I have added a little "EDIT" at the end of the post for this matter. Glad you found it useful.

      Delete
  4. hello thierry,

    thank you for this. your info here has gotten me aaaaaaaalmost there. my one stumbling block is getting the "python tools/install.py install" line to work.

    when i attempt that, i get the following error:

    IOError: [Errno 2] No such file or directory: 'out/Release/node'

    [full output & error here —> http://pastebin.com/L9jqf5PX]

    i've tried searching for that error in the node install of python tools with google and such, but all i get are errors that came up for people using home-brew to install on mac os x — i'm ssh'ing into a CentOS 5.9 linux server VPS host.

    any help you can offer with this would be appreciated. thank you.

    best,

    — faddah
    portland, oregon, u.s.

    ReplyDelete
    Replies
    1. Hello Faddah,

      First of all, I'm glad you did find some useful info here.
      Next, concerning the python tool/install.py install. As explained in the Edit notes, you shouldn't do it by hand like so. I did it like so because at that time I was not aware of the PATH env issue of the root account as spotted by "Mo Mo". Anyways, it's a bad thing to do, mostly because it does not take into account all the context settings the Makefile may provide.

      Now, what you could try is to check that your root account has the right PATH which leads to the newly installed python version ("which python" as root will help you figure it out) and then "sudo make install" if you indeed need to install node in a root-owned directory.

      Also keep in mind that perhaps this tutorial should be updated for CentOS 5.9.

      Cheers

      Delete
  5. python tools/install.py install - helped me
    Thanks !

    ReplyDelete
  6. CentOS 5 has a few different python versions available.

    I simply ran the configure script using one of the suitable versions:

    python26 ./configure

    ReplyDelete