| 2:42:55 PM PST - Tue, Dec 3rd 2019 |
|
I've been having some trouble with this for a while. I managed to get NWChem to compile and run using the instructions at http://nwchemgit.github.io/index.php/Compiling_NWChem. I noticed my performance was five to ten times worse than what I had gotten from a compilation on a shared platform elsewhere. All nodes are Intel Xeon processors. I thought I would recompile with BLAS/SCALAPACK/LAPACK taken from Intel's MKL (which I installed in my home directory). I also made some other changes to try and include threading and other tweaks to get better performance. The compilation now fails and can't find the routines from MKL or the ELPA libraries. Any thoughts?
My script is below (thought it appears my sha-bang is having some syntax issues in the preview):
- !/bin/bash -l
source /home/epj/intel/parallel_studio_xe_2019.5.075/psxevars.sh intel64
source /home/epj/intel/bin/compilervars.sh intel64
source /home/epj/intel/mkl/bin/mklvars.sh intel64
module purge
module load intel/2019
module load mpi/mpich-3.0-x86_64
export NWCHEM_TOP=/home/epj/thirdcomp/nwchem-6.8.1-release
export USE_MPI=y
export USE_MPIF=y
export USE_MPI4=y
export NWCHEM_TARGET=LINUX64
export NWCHEM_LONG_PATHS=y
export USE_NOFSCHECK=y
export TCGRSH=/usr/bin/ssh
export LARGE_FILES=y
export USE_PYTHONCONFIG=y
export PYTHONVERSION=2.7
export PYTHONHOME=/usr
export USE_PYTHON64=y
export USE_64TO32=y
export HAS_BLAS=y
export USE_SCALAPACK=y
export BLAS_SIZE=8
export LAPACK_SIZE=8
export SCALAPACK_SIZE=8
export USE_OPENMP=1
export USE_F90_ALLOCATABLE=T
export ARMCI_NETWORK=OPENIB
export IB_HOME=/usr
export IB_INCLUDE=/usr/include/infiniband
export IB_LIB=/usr/lib64
export IB_LIB_NAME="-libumad -libverbs -lpthread"
export MPI_LOC="/usr/lib64/mpich"
export MPI_INCLUDE="/usr/include/mpich-x86_64 -I/usr/include/mpich-x86_64"
export MPI_LIB="/usr/lib64/mpich/lib"
export LIBMPI="-lmpichf90 -lmpich -lopa -lmpl -lrt -lpthread"
export MKLLIB="/home/epj/intel/mkl/lib/intel64"
export MKLINC="/home/epj/intel/mkl/include"
export BLASOPT="-L$MKLLIB -lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_sequential -lmkl_core -lmkl_def -lmkl_vml_mc2 -lmkl_scalapack_ilp64 -lmkl_blacs_intelmpi_ilp64 -lmklblas95_ilp64 -lmkl_lapack95_ilp64 -liomp5 -lpthread -lm -ldl"
export LAPACK_LIB="$BLASOPT"
export LAPACK_LIBS="$BLASOPT"
export LAPACKOPT="$BLASOPT"
export SCALAPACK="-L$MKLLIB -lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_sequential -lmkl_core -lmkl_def -lmkl_vml_mc2 -lmkl_scalapack_ilp64 -lmkl_blacs_intelmpi_ilp64 -lmklblas95_ilp64 -lmkl_lapack95_ilp64 -liomp5 -lpthread -lm -ldl"
export SCALAPACK_LIB="$SCALAPACK"
export SCALAPACK_LIBS="$SCALAPACK"
export ELPA="-I/usr/lib64/gfortran/modules/openmpi-x86_64 -L/usr/lib64/openmpi/lib -lelpa"
export LD_LIBRARY_PATH=/usr/lib64/mpich/lib:$LD_LIBRARY_PATH
export PATH=/usr/lib64/mpich/bin:$PATH
cd $NWCHEM_TOP/src
make realclean
make nwchem_config NWCHEM_MODULES="all python" >& make_config1a.log
make 64_to_32 >& make_64_1a.log
make FC=ifort CC=icc >& make1a.log
|