NWChem 6.6/CentOS 7 compile fails

Gets Around

4:09:05 PM PST - Sun, Nov 8th 2015
I am trying to compile NWChem 6.6 revision 27746-src-2015-10-20 on an updated CentOS 7 system following the "CentOS 7.1" How-To on the Documentation website. This system has an Intel Core i7-930 cpu with 24 GB memory. The following software is installed: OS:CentOS 7.x86_64 3.10.0-229.20.1.el7 python.x86_64 2.7.5-18.el7_1.1 python-devel.x86_64 2.7.5-18.el7_1.1 scalapack-openmpi.x86_64 2.0.2-5.el7 scalapack-openmpi-devel.x86_64 2.0.2-5.el7 tcsh.x86_64 6.18.01-7.el7 openblas.x86_64 0.2.14-1.el7 openblas-devel.x86_64 0.2.14-1.el7 openblas-openmp.x86_64 0.2.14-1.el7 openblas-openmp64.x86_64 0.2.14-1.el7 openblas-serial64.x86_64 0.2.14-1.el7 openblas-threads.x86_64 0.2.14-1.el7 openblas-threads64.x86_64 0.2.14-1.el7 gcc-gfortran.x86_64 4.8.3-9.el7 gcc.x86_64 4.8.3-9.el7 elpa-devel.noarch 2015.02.002-4.el7 elpa-openmpi.x86_64 2015.02.002-4.el7 elpa-openmpi-devel.x86_64 2015.02.002-4.el7 openmpi.x86_64 1.6.4-5.el7 openmpi-devel.x86_64 1.6.4-5.el7 make.x86_64 1:3.82-21.el7 perl.x86_64 4:5.16.3-285.el7 The environment variables were set as recommended: export USE_MPI=y export NWCHEM_TARGET=LINUX64 export USE_PYTHONCONFIG=y export PYTHONVERSION=2.7 export PYTHONHOME=/usr export USE_64TO32=y export BLAS_SIZE=4 export BLASOPT="-lopenblas -lpthread -lrt" export SCALAPACK_SIZE=4 export SCALAPACK="-L/usr/lib64/openmpi/lib -lscalapack -lmpiblacs" export ELPA="-I/usr/lib64/gfortran/modules/openmpi -L/usr/lib64/openmpi/lib -lelpa" export LD_LIBRARY_PATH=/usr/lib64/openmpi/lib/:$LD_LIBRARY_PATH export PATH=/usr/lib64/openmpi/bin/:$PATH export NWCHEM_TOP=/usr/local/nwchem-6.6 export NWCHEM_MODULES="all python" I ran make nwchem_config, make 64_to_32, then make (the latter went for over 1 h, then failed). The last 31 lines of the make output were: /usr/local/nwchem-6.6/src/tools/install/lib/libga.a(scalapack.o): In function `ga_evp_real_': scalapack.F:(.text+0x9bc0): undefined reference to `numroc_' scalapack.F:(.text+0x9bf1): undefined reference to `numroc_' scalapack.F:(.text+0x9c22): undefined reference to `numroc_' scalapack.F:(.text+0x9c53): undefined reference to `numroc_' scalapack.F:(.text+0x9e70): undefined reference to `descinit_' scalapack.F:(.text+0x9f11): undefined reference to `descinit_' scalapack.F:(.text+0x9f80): undefined reference to `__elpa1_MOD_get_elpa_row_col_comms' scalapack.F:(.text+0xa02d): undefined reference to `__elpa1_MOD_solve_evp_real' /usr/local/nwchem-6.6/src/tools/install/lib/libga.a(scalapack.o): In function `ga_pzheevd_': scalapack.F:(.text+0xa4e2): undefined reference to `numroc_' scalapack.F:(.text+0xa513): undefined reference to `numroc_' scalapack.F:(.text+0xa544): undefined reference to `numroc_' scalapack.F:(.text+0xa575): undefined reference to `numroc_' scalapack.F:(.text+0xa813): undefined reference to `descinit_' scalapack.F:(.text+0xa8b4): undefined reference to `descinit_' scalapack.F:(.text+0xaa17): undefined reference to `pzheevd_' scalapack.F:(.text+0xaccf): undefined reference to `pzheevd_' /usr/local/nwchem-6.6/src/tools/install/lib/libga.a(scalapack.o): In function `ga_pzheevr_': scalapack.F:(.text+0xb151): undefined reference to `numroc_' scalapack.F:(.text+0xb182): undefined reference to `numroc_' scalapack.F:(.text+0xb1b3): undefined reference to `numroc_' scalapack.F:(.text+0xb1e4): undefined reference to `numroc_' scalapack.F:(.text+0xb482): undefined reference to `descinit_' scalapack.F:(.text+0xb523): undefined reference to `descinit_' scalapack.F:(.text+0xb72e): undefined reference to `pzheevr_' scalapack.F:(.text+0xba55): undefined reference to `pzheevr_' /usr/local/nwchem-6.6/src/tools/install/lib/libga.a(scalapack.o): In function `pxerbla_': scalapack.F:(.text+0xbdd0): undefined reference to `blacs_gridinfo_' collect2: error: ld returned 1 exit status make: *** [all] Error 1 Any suggestions on where to go now?

Forum Vet

12:07:09 PM PST - Mon, Nov 9th 2015
Thanks for the feedback. A patch is needed to fix the problem. http://nwchemgit.github.io/download.php?f=Tools_lib64.patch.gz Here are the instructions for applying patches http://nwchemgit.github.io/index.php/Download#Patches_for_the_27746_revision_of_NWChem_6.6 For this specific case, here is the recipe cd $NWCHEM_TOP wget -O Tools_lib64.patch.gz http://nwchemgit.github.io/download.php?f=Tools_lib64.patch.gz gzip -d Tools_lib64.patch.gz patch -p0 < Tools_lib64.patch cd src make link

Gets Around

2:36:56 PM PST - Mon, Nov 9th 2015
Edo, The patch was applied, but I get the same result, with the same lines at the end of make.log. John K.

Forum Vet

3:11:00 PM PST - Mon, Nov 9th 2015
Did you try make or make link ?

Gets Around

4:02:27 PM PST - Mon, Nov 9th 2015
make. So the procedure is as follows? 1. apply the patch 2. cd $NWCHEM_TOP/src 3. make nwchem_config 4. make 64_to_32 5. make link

Forum Vet

4:14:30 PM PST - Mon, Nov 9th 2015
Please try the following cd $NWCHEM_TOP/src make link If you really want to start from the beginning, have the compilation analyze every single directory (Yes!) and wait quite a bit longer, you might try the following recipe cd $NWCHEM_TOP/src make _USE_SCALAPACK=y

Gets Around

7:10:09 PM PST - Mon, Nov 9th 2015
Hi Edo, I tried your recipe, but the last 31 lines of make _USE_SCALAPACK=y were exactly the same as shown in the first entry. John K.

Forum Vet

8:23:44 PM PST - Mon, Nov 9th 2015
Quote:Jwkeller Nov 9th 6:10 pm Hi Edo, I tried your recipe, but the last 31 lines of make _USE_SCALAPACK=y were exactly the same as shown in the first entry. John K. Not sure how to read this ... did you manage to get NWChem to build or not?

Gets Around

10:14:05 PM PST - Mon, Nov 9th 2015
Sorry for the cryptic answer. NWChem did not compile. Only the depend.x file appears in /bin/LINUX64 Some of the programs I have installed are not mentioned explicitly in the CentOS 7.1 How-To entry, such as openblas-threads.x86_64. Could these be interfering with proper compilation of nwchem 6.6? John Keller

Forum Vet

12:01:44 PM PST - Tue, Nov 10th 2015
Could you post the last few lines of your failed compilation(s)?

Gets Around

2:45:34 PM PST - Tue, Nov 10th 2015

The procedure was the same as shown in the top box, but with
export USE_SCALAPACK=y
was added to the environment variables, and the Tools_lib64 patch as applied.
Then I did
make nwchem_config
make 64_to_32
make >& make.log

The last 30 lines of the make.log file were exactly as shown in the top box.
Lines around 284-320 were as follows:

configure: Checks for BLAS,LAPACK,ScaLAPACK
configure:
configure: Attempting to locate BLAS library
checking for BLAS with user-supplied flags... yes
configure: Attempting to locate LAPACK library
configure: WARNING: LAPACK library not found, using internal LAPACK
configure: Attempting to locate SCALAPACK library
checking for SCALAPACK with user-supplied flags... no
checking for SCALAPACK in generic library... yes
checking whether SCALAPACK implements pdsyevr... yes
configure: Attempting to locate ELPA library
checking for ELPA with user-supplied flags... yes
configure:
configure: Checks for additional programs
configure:
checking for grep that handles long lines and -e... (cached) /usr/bin/grep
checking for a sed that does not truncate output... /usr/bin/sed
checking for python... /usr/bin/python
checking for python version... 2.7
checking for python platform... linux2
checking for python script directory... ${prefix}/lib/python2.7/site-packages
checking for python extension module directory... ${exec_prefix}/lib64/python2.7/site-packages
checking for m4... /usr/bin/m4
checking for perl... /usr/bin/perl
configure:
configure: Libtool setup
configure:
checking how to print strings... printf
checking for a sed that does not truncate output... (cached) /usr/bin/sed
checking for fgrep... /usr/bin/grep -F
checking for ld used by cc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes

Forum Vet

3:06:19 PM PST - Tue, Nov 10th 2015
Could you try the following cd $NWCHEM_TOP wget http://nwchemgit.github.io/images/Config_libs66.patch.gz gzip -d Config_libs66.patch.gz patch -p0 < Config_libs66.patch Then recompile

Gets Around

9:10:53 PM PST - Tue, Nov 10th 2015

Still no success. Here are the last few lines of make.log

ixamax.F:14: note: === vect_analyze_slp ===
ixamax.F:14: note: Failed to SLP the basic block.
ixamax.F:14: note: not vectorized: failed to find SLP opportunities in basic block.
make[2]: Leaving directory `/usr/local/nwchem-6.6/src/64to32blas'
make[2]: Entering directory `/usr/local/nwchem-6.6/src/64to32blas'
Got lock on /usr/local/nwchem-6.6/lib/LINUX64/lib64to32.lock
ar r /usr/local/nwchem-6.6/lib/LINUX64/lib64to32.a yscal.o ycopy.o ydot.o ygemm.o ygemv.o iyamax.o ygesv.o ygesvd.o yaxpy.o yspsvx.o yposv.o ysyev.o yger.o yrot.o ylaset.o yswap.o ygebak.o ygebal.o ynrm2.o ylacpy.o ylascl.o ylange.o ytrevc.o yhseqr.o yorghr.o ygehrd.o ysterf.o ylarnv.o ylagtf.o ylagts.o yasum.o ypotri.o ypotrf.o ysygv.o ygeev.o ygeevx.o ifily.o xscal.o xaxpy.o xgemm.o xheev.o xcopy.o xdotc.o ixamax.o
ar: creating /usr/local/nwchem-6.6/lib/LINUX64/lib64to32.a
echo /usr/local/nwchem-6.6/lib/LINUX64/lib64to32.a
/usr/local/nwchem-6.6/lib/LINUX64/lib64to32.a
make[2]: Leaving directory `/usr/local/nwchem-6.6/src/64to32blas'
make[1]: Leaving directory `/usr/local/nwchem-6.6/src/64to32blas'
make nwchem.o stubs.o
make[1]: Entering directory `/usr/local/nwchem-6.6/src'
gfortran -m64 -ffast-math  -Warray-bounds -fdefault-integer-8 -march=native -mtune=native -finline-functions -O2 -g -fno-aggressive-loop-optimizations -g -O   -I.  -I/usr/local/nwchem-6.6/src/include -I/usr/local/nwchem-6.6/src/tools/install/include -DGFORTRAN -DCHKUNDFLW -DGCC4 -DGCC46 -DEXT_INT -DLINUX -DLINUX64 -DSCALAPACK -DPARALLEL_DIAG -DCOMPILATION_DATE="'`date +%a_%b_%d_%H:%M:%S_%Y`'" -DCOMPILATION_DIR="'/usr/local/nwchem-6.6'" -DNWCHEM_BRANCH="'6.6'"  -c -o nwchem.o nwchem.F
gfortran -m64 -ffast-math  -Warray-bounds -fdefault-integer-8 -march=native -mtune=native -finline-functions -O2 -g -fno-aggressive-loop-optimizations -g -O   -I.  -I/usr/local/nwchem-6.6/src/include -I/usr/local/nwchem-6.6/src/tools/install/include -DGFORTRAN -DCHKUNDFLW -DGCC4 -DGCC46 -DEXT_INT -DLINUX -DLINUX64 -DSCALAPACK -DPARALLEL_DIAG -DCOMPILATION_DATE="'`date +%a_%b_%d_%H:%M:%S_%Y`'" -DCOMPILATION_DIR="'/usr/local/nwchem-6.6'" -DNWCHEM_BRANCH="'6.6'"  -c -o stubs.o stubs.F
make[1]: Leaving directory `/usr/local/nwchem-6.6/src'
gfortran  -Wl,--export-dynamic  -L/usr/local/nwchem-6.6/lib/LINUX64 -L/usr/local/nwchem-6.6/src/tools/install/lib   -o /usr/local/nwchem-6.6/bin/LINUX64/nwchem nwchem.o stubs.o -lnwctask -lccsd -lmcscf -lselci -lmp2 -lmoints -lstepper -ldriver -loptim -lnwdft -lgradients -lcphf -lesp -lddscf -ldangchang -lguess -lhessian -lvib -lnwcutil -lrimp2 -lproperty -lsolvation -lnwints -lprepar -lnwmd -lnwpw -lofpw -lpaw -lpspw -lband -lnwpwlib -lcafe -lspace -lanalyze -lqhop -lpfft -ldplot -lnwpython -ldrdy -lvscf -lqmmm -lqmd -letrans -lpspw -ltce -lbq -lmm -lcons -lperfm -ldntmc -lccca -lnwcutil -lga -larmci -lpeigs -lperfm -lcons -lbq -lnwcutil -I/usr/lib64/gfortran/modules/openmpi -L/usr/lib64/openmpi/lib -lelpa -L/usr/lib64/openmpi/lib -lscalapack -lmpiblacs -l64to32 -lopenblas -lpthread -lrt  -lnwclapack  -lnwcblas   -L/usr/lib64/openmpi/lib -lmpi_f90 -lmpi_f77 -lmpi     -lrt -lm  -lpthread  -lnwcutil -lpthread -ldl -lutil -lm -lpython2.7  -lpthread -ldl -lutil -lm -lpython2.7 -Xlinker -export-dynamic  
/usr/bin/ld: cannot find -lmpiblacs
collect2: error: ld returned 1 exit status
make: *** [all] Error 1

I'm giving up the battle for now: if someone can make this work, they will hopefully post more details about which version of CentOS, and which programs are installed, and in which directories.

Gets Around

12:03:11 AM PST - Wed, Nov 11th 2015

Edo-
OK it compiled. This time I followed the procedure for Fedora 22 on the Documentation webpage. Here are the details:

CPU: Intel Core i7-930
NWChem version: Nwchem-6.6.revision27746-src.2015-10-20.tar.gz
OS:CentOS 7.x86_64 3.10.0-229.20.1.el7
python.x86_64 2.7.5-18.el7_1.1
python-devel.x86_64 2.7.5-18.el7_1.1
tcsh.x86_64 6.18.01-7.el7
openblas.x86_64 0.2.14-1.el7
openblas-devel.x86_64 0.2.14-1.el7
openblas-openmp.x86_64 0.2.14-1.el7
openblas-openmp64.x86_64 0.2.14-1.el7
openblas-serial64.x86_64 0.2.14-1.el7
openblas-threads.x86_64 0.2.14-1.el7
openblas-threads64.x86_64 0.2.14-1.el7
gcc-gfortran.x86_64 4.8.3-9.el7
gcc.x86_64 4.8.3-9.el7
openmpi.x86_64 1.6.4-5.el7
openmpi-devel.x86_64 1.6.4-5.el7
make.x86_64 1:3.82-21.el7
perl.x86_64 4:5.16.3-285.el7

Setting environment variables:

export USE_MPI=y
export NWCHEM_TARGET=LINUX64
export USE_PYTHONCONFIG=y
export PYTHONVERSION=2.7
export PYTHONHOME=/usr
export BLAS_SIZE=8
export BLASOPT="-L/usr/lib64 -lopenblas64"
export LD_LIBRARY_PATH=/usr/lib64/openmpi/lib/:$LD_LIBRARY_PATH
export PATH=/usr/lib64/openmpi/bin/:$PATH
export USE_ARUR=y
export NWCHEM_TOP=/usr/local/nwchem-6.6
export NWCHEM_MODULES="all python"

Tools_lib64.patch and Config_lib66.patch were applied per the directions on Download page.

Finally, II did:
make nwchem_config
make >& make.log

It required about 30 min. and produced an 83.6 mb  nwchem executable.

Clicked A Few Times

9:50:13 PM PST - Mon, Dec 21st 2015
Hello, When you compiled it, where is the executable and what is it called (is it located in .. /..../nwchem-6.6/bin/LINUX64/nwchem ) I followed your instructions but do not know if it compiled correctly. My file is 79.8 mb , not 83.6 mb as yours.

Forum >> NWChem's corner >> Compiling NWChem