Segmentation fault with Scalapack from MKL

Clicked A Few Times

9:06:29 AM PDT - Tue, Apr 14th 2015

I compiled NWChem version 6.5 on a local cluster with openmpi 1.6.2, intel compilers version 13.1.1 and MKL

My configuration file is:

export NWCHEM_TOP=/home/attaccalite/SOFTWARE/Nwchem-6.5 

export NWCHEM_TARGET=LINUX64

export ARMCI_NETWORK=MPI-TS

export USE_MPI=y

export USE_MPIF=y

export USE_MPIF4=y

export MPI_LOC=/opt/openmpi/

export MPI_LIB=$MPI_LOC/lib

export MPI_INCLUDE=$MPI_LOC/include

export LIBMPI='-lmpi_f90 -lmpi_f77 -lmpi -ldl -Wl,--export-dynamic -lnsl -lutil'

export NWCHEM_MODULES="all"

export USE_NOFSCHECK=TRUE

export LARGE_FILES=TRUE

export MKLROOT=/opt/intel/composer_xe_2013.3.163/mkl/lib/intel64

export BLASOPT="-L$MKLROOT -lmkl_core -lmkl_sequential -lmkl_intel_ilp64"

export CC=icc

export FC=ifort

export MPICC=/opt/openmpi/bin/mpicc

export BLAS_SIZE=8 

export LAPACK_SIZE=8

export FOPTIMIZE="-O3 -axSS2,SSE3,SSSE3 -i8"

export COPTIMIZE="-O3 -axSS2,SSE3,SSSE3"

#export SCALAPACK_SIZE=8

#export USE_SCALAPACK=y

#export SCALAPACK="-L$MKLROOT -lmkl_scalapack_ilp64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lmkl_blacs_intelmpi_ilp64 -lpthread -lm"

#export SCALAPACK_LIB="$SCALAPACK

when I compile nwchem works fine also in parallel, but if I try to use Scalpack from MKL I get "Segementation fault"

..............

 ao basis        = "ao basis"

 functions       =    25

 atoms           =     3

 closed shells   =     5

 open shells     =     0

 charge          =   0.00

 wavefunction    = RHF 

 input vectors   = atomic

 output vectors  = ./h2o.movecs

 use symmetry    = T

 symmetry adapt  = T

 Forming initial guess at       0.1s

 Segmentation fault

any hints? any ideas ?

regards
Claudio

Forum Vet

10:07:45 AM PDT - Tue, Apr 14th 2015
The following settings are a recipe for disaster export FOPTIMIZE="-O3 -axSS2,SSE3,SSSE3 -i8" export COPTIMIZE="-O3 -axSS2,SSE3,SSSE3" My suggestion is to recompile everything from scratch without redefining FOPTIMIZE and COPTIMIZE

Clicked A Few Times

2:13:17 AM PDT - Wed, Apr 15th 2015
Same problem
I tried to remove FOPTIMIZE and COPTIMIZE but it gives the same error. Anyway I'm obligated to use FOPTIMIZE and COPTIMIZE for production runs, because the nodes of the cluster are different from the computer were the code is compiled. I will try to recompile SCALPACK from sources and see it works. Claudio

Clicked A Few Times

5:46:00 AM PDT - Wed, Apr 15th 2015

Test with Scalpack from Netlib failed

Ok I tried to compile my own scalapack and then link them to nwchem

export NWCHEM_TOP=/home/attaccalite/SOFTWARE/Nwchem-6.5

export NWCHEM_TARGET=LINUX64

export ARMCI_NETWORK=MPI-TS

export USE_MPI=y

export USE_MPIF=y

export USE_MPIF4=y

export MPI_LOC=/opt/openmpi/

export MPI_LIB=$MPI_LOC/lib

export MPI_INCLUDE=$MPI_LOC/include

export LIBMPI='-lmpi_f90 -lmpi_f77 -lmpi -ldl -Wl,--export-dynamic -lnsl -lutil'

export NWCHEM_MODULES="all"

export USE_NOFSCHECK=TRUE

export LARGE_FILES=TRUE

export MKLROOT=/opt/intel/composer_xe_2013.3.163/mkl/lib/intel64

export BLASOPT="-L$MKLROOT -lmkl_core -lmkl_sequential -lmkl_intel_lp64"

export USE_64TO32=y

export CC=icc

export FC=ifort

export MPICC=/opt/openmpi/bin/mpicc

export BLAS_SIZE=4

export LAPACK_SIZE=4

export SCALAPACK_SIZE=4

export USE_SCALAPACK=y

export SCALAPACK="/home/attaccalite/SOFTWARE/scalapack-2.0.2/libscalapack.a"

export SCALAPACK_LIB="$SCALAPACK"

but I get a new error

Starting SCF solution at       0.7s

------------------------------------------------------------------------

ga_iter_lsolve: dgesv failed         -4294967296

------------------------------------------------------------------------

------------------------------------------------------------------------

 current input line : 

   26: task scf optimize

------------------------------------------------------------------------

ok, I can live without scalapack :-)
but if you have suggestion I can try differently to compile scalapack+nwchem

notice that I compiled them with 32bit integer because it is the unix standard and other code use them with this integer dimension

cla

Forum Vet

11:53:28 AM PDT - Wed, Apr 15th 2015
Did you execute the following command before compiling? `make 64_to_32` More details at http://nwchemgit.github.io/index.php/Compiling_NWChem#Optimized_math_libraries

Clicked A Few Times

12:57:43 AM PDT - Thu, Apr 16th 2015
It works
Thank you I completely forgot that point, now it works with Scalapack sincerely Claudio

Forum >> NWChem's corner >> Compiling NWChem