ImportError when compiled with MPI support

[schmuker]

Hi,

when using nrnpython with MPI support (--with-paranrn), I get an ImportError when doing

Code: Select all

from neuron import h
Traceback (most recent call last):
[...]
ImportError: neuron.hoc module not found.

 Are you perhaps importing neuron in
 the nrnpython source directory?
 Please move out of this directory and try again,
 or remove nrnpython from your import path.
>>>                                                         

Actually the error message is misleading here, because the original message is not retained when the ImportError is first caught. Trying the import in a debugger reveals that some MPI symbol is not found:

Code: Select all

ImportError: '/home/micha/sims/nrn/nrn/x86_64/lib/libsundials.so.0: undefined symbol: ompi_mpi_op_min'

The error goes away when I initialize MPI before import neuron:

Code: Select all

>>> from mpi4py import MPI
>>> from neuron import h
>>>

The error also does not appear when I use nrniv -python.

Taken together, this means that as long as I use an MPI-enabled nrnpython I have to initialize MPI also if I don't want to use it. Is there a way around this?

Cheers,

Michael

[hines]

I can't reproduce that problem with openmpi, python2.5, and the most recent alpha version. You should get a first line of
$ python
...
>>> from neuron import h
MPI_Initialized==false, disabling MPI functionality.
...
I do have to have
$ export PYTHONPATH=$PYTHONPATH:/home/hines/lib/python
export LD_LIBRARY_PATH=/home/hines/openmpi-1.3.3/i686/lib:$LD_LIBRARY_PATH
I know that nrniv contains all the names of the libraries that needed to be loaded and
I don't know why in your case that loading libsundials.so.0 does not cause the openmpi library
to be loaded.

[hines]

I notice that
ldd $HOME/lib/python/neuron/hoc.so
which lists the libraries to be loaded contains not only
libsundials.so.0 => /home/hines/neuron/nrnompi/i686/lib/libsundials.so.0
but also a bunch of openmpi libraries such as
libmpi.so.0 => /home/hines/openmpi-1.3.3/i686/lib/libmpi.so.0 (0xb7867000)

What is the situation in your environment?

[schmuker]

Code: Select all

ldd mypython/lib64/python2.5/site-packages/neuron/hoc.so

gives no MPI libs here. I tried on a SUSE system (which has a somewhat awkward way to set up openmpi) and on debian 4.0. I'm on 64 bit, if that's important.

On SUSE I need to set MPICC and MPICXX environment vars when configuring paranrn, because the compilers are not in my path, while on the debian system I use self-compiled openmpi libs and set LD_LIBRARY_PATH accordingly. Can this affect the linker?

In any case, maybe it would help forwarding the original ImportError message, e.g. by changing __init__.py starting from line 77 to:

Code: Select all

try:
    import hoc
except ImportError, e:
    raise ImportError, """
Can't import neuron.hoc module. 

In case you are importing neuron from within 
the nrnpython source directory, please 
move out of this directory and try again, 
or remove nrnpython from your import path. 

The original error message was: 

%s

"""%e.message

[hines]

Thanks for the suggestion about printing the original exception info.

gives no MPI libs here

So that seems to be the problem. On my development machine (x86_64) with mpich2 it shows
libmpich.so.1.1 => /home/hines/mpich2/lib/libmpich.so.1.1 (0x00002aff8ddea000)
I wonder if something is wrong with nrn/src/nrnpython/setup.py.in in the fragment

Code: Select all

mpicc_bin = "@CC@"
mpicxx_bin = "@CXX@"
import os
os.environ["CC"]=mpicc_bin
os.environ["CXX"]=mpicxx_bin

On my machine, config.status shows:

s,@CXX@,mpic++,;t t
s,@CC@,mpicc,;t t
and so
python setup.py install --home=$HOME
uses the correct mpi compilers.
Is that similar to what is happening in your environment.

[schmuker]

I found the error. src/nrnpython/setup.py and config.status both used the right compilers. But there was an old nrnpython/build directory from a non-MPI build still hanging around which make clean did not make clean :) So I guess setup.py used the old libraries and installed them, instead of compiling new ones using mpicc/mpicxx.

Removing src/nrnpython/build and calling "setup.py install" again solved the problem.

Maybe "make clean" should also remove the build directory?

Thanks a lot for your help!

Cheers,

Michael

[hines]

Maybe "make clean" should also remove the build directory?

I agree. After a reconfiguration, setup.py is newer than the build directory but that does not cause a full rebuild.
I'll have to add it to the Makefile.
I occasionally toy with ideas about how to avoid the extra 'python setup.py install ...' step but it just seems like
too much understanding too early is required to specify the installation choices during configure. ie.
do not want installation because the desire is only to launch nrniv,
or what should be the analogy to --prefix for it.