compile nwchem-6.3 with mkl (blas e lapack) sequential
Clicked A Few Times
1:02:37 PM PDT - Wed, Mar 19th 2014
Hi all,

My build script is:

  1. /bin/bash
export NWCHEM_TOP="/opt/nwchem"
export NWCHEM_TARGET="LINUX64"
export ARMCI_NETWORK="SOCKETS"

  1. export USE_MPI="y"
  2. export USE_MPIF="y"
  3. export USE_MPIF4="y"
  4. export MPI_LIB="/usr/local/lib"
  5. export MPI_INCLUDE="/usr/local/include"
  6. export LIBMPI="-lmpi_f90 -lmpi_f77 -lmpi -ldl -Wl,--export-dynamic -lnsl -lutil"
  7. export TCGRSH="ssh"

export NWCHEM_MODULES="all python"
export LARGE_FILES="TRUE"
export USE_NOFSCHECK="TRUE"
export LIB_DEFINES="-DDFLT_TOT_MEM=16777216"
export MRCC_THEORY="TRUE"
export NWCHEM_MODULES="all"

export PYTHONHOME="/usr"
export PYTHONVERSION="2.7"
export USE_PYTHON64="y"

export MKLROOT="/opt/intel/mkl"
export MKL_LIB="/opt/intel/mkl/lib/intel64"
export MKL_INCLUDE="$MKLROOT/include/intel64/ilp64"
export HAS_BLAS="yes"
export BLAS_SIZE=8
export BLAS_LIB=" $MKLROOT/lib/intel64/libmkl_blas95_ilp64.a -Wl,--start-group $MKLROOT/lib/intel64/libmkl_intel_ilp64.a $MKLROOT/lib/intel64/libmkl_core.a $MKLROOT/lib/intel64/libmkl_sequential.a -Wl,--end-group -lpthread -lm"
export BLASOPT="$BLAS_LIB"
export LAPACK_SIZE=8
export LAPACK=" $MKLROOT/lib/intel64/libmkl_lapack95_ilp64.a -Wl,--start-group $MKLROOT/lib/intel64/libmkl_intel_ilp64.a $MKLROOT/lib/intel64/libmkl_core.a $MKLROOT/lib/intel64/libmkl_sequential.a -Wl,--end-group -lpthread -lm"
export LAPACK_LIB="$LAPACK"

  1. export USE_SCALAPACK="y"
  2. export SCALAPACK_SIZE=8
  3. export SCALAPACK=" $MKLROOT/lib/intel64/libmkl_scalapack_ilp64.a -Wl,--start-group $MKLROOT/lib/intel64/libmkl_cdft_core.a $MKLROOT/lib/intel64/libmkl_intel_ilp64.a $MKLROOT/lib/intel64/libmkl_core.a $MKLROOT/lib/intel64/libmkl_sequential.a -Wl,--end-group $MKLROOT/lib/intel64/libmkl_blacs_intelmpi_ilp64.a -lpthread -lm"
  4. export SCALAPACK_LIB="$SCALAPACK"

export FC=ifort
export CFLAGS=" -i8 -I$(MKLROOT)/include/intel64/ilp64 -I$(MKLROOT)/include"

export CC=icc
export CCFLAGS=" -DMKL_ILP64 -I$MKLROOT/include/intel64/ilp64 -I$MKLROOT/include"

cd /opt/nwchem/src
make clean
make nwchem_config
make -j8

Whats wrong in this config?

Thanks
André

Forum Regular
11:49:06 AM PDT - Mon, Apr 7th 2014
Andre,
I am guessing your machine has MPI installed, you want to use the MKL libraries for BLAS, LAPACK and ScaLAPACK, and you have versions of those libraries with 64-bit integers. You also want to use PYTHON, and all modules including the multi-reference coupled cluster methods. Is that right?
Huub

Forum Regular
10:37:38 AM PDT - Tue, Apr 8th 2014
Andre,
Please get the latest version of the build_nwchem script (e.g. Google "build_nwchem site:pnl.gov" should find it)(the script usually lives in nwchem/contrib/distro-tools). Then unset everything. Instead set:

    export MKLROOT="/opt/intel/mkl"

    export BLASOPT=" $MKLROOT/lib/intel64/libmkl_scalapack_ilp64.a $MKLROOT/lib/intel64/libmkl_lapack95_ilp64.a $MKLROOT/lib/intel64/libmkl_blas95_ilp64.a -Wl,--start-group $MKLROOT/lib/intel64/libmkl_cdft_core.a $MKLROOT/lib/intel64/libmkl_intel_ilp64.a $MKLROOT/lib/intel64/libmkl_core.a $MKLROOT/lib/intel64/libmkl_sequential.a -Wl,--end-group $MKLROOT/lib/intel64/libmkl_blacs_intelmpi_ilp64.a -lpthread -lm"

    export LIB_DEFINES="-DDFLT_TOT_MEM=16777216"

    export MRCC_METHODS="TRUE"

Next check that MPI is in your path. I.e. as you are using the Intel compilers the command 

    which mpiifort

should return something sensible. Then

    cd /opt/nwchem

    ./contrib/distro-tools/build_nwchem | tee build_nwchem.log

That should do it. If it does not them please send me the build_nwchem.log file (hubertus dot vandam at pnnl dot gov) and I'll take a look at it.

I hope this helps,

Huub

Clicked A Few Times
2:31:48 PM PDT - Wed, Sep 24th 2014
Huub:

I'm not sure what became of the original poster, but I've been trying to get this working using your build_nwchem script, the exact BLASOPT line you recommended above (using my own path to MKL of course), and the 6.5 release.

It seems that GA 5.3 uses a simple 'ld -lscalapack' to determine whether or not to build its scalapack interfaces. On a system with no scalapack shared library installed at all (e.g. no RedHat scalapack package installed), the check fails. This results in the situation where nwchem tries to link with scalapack but GA has no scalapack support. Thus the link of nwchem fails.

Any recommendations on how to convince GA to not try to find scalapack when we are explicitly specifying it?

Thanks,

Gabe


Forum >> NWChem's corner >> Compiling NWChem