NWChem 6.1.1 and Intel MPI with MKL


Click here for full thread
Clicked A Few Times
Compiling Information
I followed the advice for compiling with Intel Cluster Studio found here: ARMCI Errors with MKL ScaLAPACK[1].

I compiled NWChem in my local user directory, and then moved the executables and data files to the share application space here "/share/apps/NWChem" per the instructions in the INSTALL file.
Here are the environment variables I used to compile
export NWCHEM_TOP="/pathtouserhome/csoft/nwchem-6.1.1-src"
export TARGET=LINUX64
export LARGE_FILES=TRUE
export ENABLE_COMPONENT=yes
export TCGRSH=/usr/bin/ssh
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export LIB_DEFINES="-DDFLT_TOT_MEM=16777216"

export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y

export IB_HOME=/usr
export IB_INCLUDE=$IB_HOME/include/infiniband
export IB_LIB=$IB_HOME/lib64
export IB_LIB_NAME="-libumad -libverbs -lpthread -lrt"
export ARMCI_NETWORK=OPENIB
#export ARMCI_NETWORK=MELLANOX

export MKLROOT="/opt/intel/mkl"
export MKL_INCLUDE=$MKLROOT/include/intel64/ilp64

export BLAS_LIB="-L$MKLROOT/lib/intel64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lpthread -lm"
export BLASOPT="$BLAS_LIB"
export BLAS_SIZE=8
export SCALAPACK_SIZE=8
export SCALAPACK="-L$MKLROOT/lib/intel64 -lmkl_scalapack_ilp64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lmkl_blacs_intelmpi_ilp64 -lpthread -lm"
export SCALAPACK_LIB="$SCALAPACK"
export USE_SCALAPACK=y

export MPI_HOME=/opt/intel/impi/4.1.0
export MPI_LOC=$MPI_HOME
export MPI_LIB=$MPI_LOC/lib64
export MPI_INCLUDE=$MPI_LOC/include64
export LIBMPI="-lmpigf -lmpigi -lmpi_ilp64 -lmpi"

export CXX=/opt/intel/bin/icpc
export CC=/opt/intel/bin/icc
export FC=/opt/intel/bin/ifort

export PYTHONPATH="/usr"
export PYTHONHOME="/usr"
export PYTHONVERSION="2.6"
export USE_PYTHON64=y
export PYTHONLIBTYPE=so

export MPICXX=$MPILOC/bin/mpiicpc
export MPICC=$MPILOC/bin/mpiicc
export MPIF77=$MPILOC/bin/mpiifort
export MPIEXEC="$MPILOC/bin/mpirun -machinefile=/pathtouserhome/csoft/scripts/testmf.txt -np 8"


I have the NWCHEM_CONFIG and the make.log files if that would be helpful for troubleshooting.