compiling on centos~~ can't find Python.h
Clicked A Few Times
12:28:29 PM PDT - Fri, Nov 4th 2016
Hi all,
I am having issues compiling nwchem-6.6 on HPC. I followed the Documentation regarding setting environment variables for Intel Xeon Phi processors.

These are the relevant environment variables I have declared in my ~/.bashrc profile:

export PATH="$JAVA_HOME/bin:usr/bin/cc:/usr/local/intel-2015.2/impi/5.0.3.048/bin64/mpif90:$PATH"

export NWCHEM_TOP=/work/zacharyw/nwchem-6.6
export NWCHEM_TARGET="LINUX64" ##changed from just LINUX
export NWCHEM_MODULES="all"
export ARMCI_NETWORK="OPENIB"

export USE_64TO32=y
export BLAS_SIZE=4
export BLASOPT="-mkl -openmp -lpthread -lm"
export SCALAPACK="-mkl -openmp -lmkl_scalapack_ilp64 -lmkl_blacs_intelmpi_ilp64 -lpthread -lm"
export OMP_NUM_THREADS=4
export MIC_USE_2MB_BUFFER=16k
export ARMCI_OPENIB_DEVICE=mlx4_0
export LD_LIBRARY_PATH=/usr/local/intel-2015.2/impi/5.0.3.048/bin64/mpif90:$LD_LIBRARY_PATH
export USE_INTERNALBLAS=y
export USE_MPI=y
export LIBMPI="-lmpich -lopa -lmpl -lpthread -lmpichf90 -lfmpich -lmpich"
export USE_MPIF=y
export USE_MPIF4=y
export USE_OPENMP=1
export USE_OFFLOAD=1
export USE_PYTHONCONFIG=y
export PYTHONVERSION=2.6
export PYTHONHOME=/usr

This is the corresponding error message I get after I attempt to compile:

Got lock on /work/zacharyw/nwchem-6.6/lib/LINUX64/libpfft.lock
xiar r /work/zacharyw/nwchem-6.6/lib/LINUX64/libpfft.a nwfft3d.o
xiar: executing 'ar'
echo /work/zacharyw/nwchem-6.6/lib/LINUX64/libpfft.a
/work/zacharyw/nwchem-6.6/lib/LINUX64/libpfft.a
make[3]: Leaving directory `/work/zacharyw/nwchem-6.6/src/fft/pfftwrap'
make[2]: Leaving directory `/work/zacharyw/nwchem-6.6/src/fft/pfftwrap'
make[2]: Entering directory `/work/zacharyw/nwchem-6.6/src/fft'
make[2]: Leaving directory `/work/zacharyw/nwchem-6.6/src/fft'
make[1]: Leaving directory `/work/zacharyw/nwchem-6.6/src/fft'
Making libraries in python
make[1]: Entering directory `/work/zacharyw/nwchem-6.6/src/python'
icc -c -I. -I/usr/include/python2.6 -I/usr/include -I/usr/Include -I/usr -I/work/zacharyw/nwchem-6.6/src/include -I/work/zacharyw/nwchem-6.6/src/tools/install/include -DEXT_INT -DLINUX -DLINUX64 -DUSE_OPENMP -DUSE_OFFLOAD -DINTEL_64ALIGN -DIFCV8 -DIFCLINUX -DPARALLEL_DIAG -qopenmp -qopt-report-phase=offload -xHOST -ftz -qopt-report-phase=vec -qopt-report-file=stderr -qopenmp -qopt-report-phase:openmp -g -o task_python.o task_python.c
icc -c -I. -I/usr/include/python2.6 -I/usr/include -I/usr/Include -I/usr -I/work/zacharyw/nwchem-6.6/src/include -I/work/zacharyw/nwchem-6.6/src/tools/install/include -DEXT_INT -DLINUX -DLINUX64 -DUSE_OPENMP -DUSE_OFFLOAD -DINTEL_64ALIGN -DIFCV8 -DIFCLINUX -DPARALLEL_DIAG -qopenmp -qopt-report-phase=offload -xHOST -ftz -qopt-report-phase=vec -qopt-report-file=stderr -qopenmp -qopt-report-phase:openmp -g -o nwchem_wrap.o nwchem_wrap.c
icc -c -I. -I/usr/include/python2.6 -I/usr/include -I/usr/Include -I/usr -I/work/zacharyw/nwchem-6.6/src/include -I/work/zacharyw/nwchem-6.6/src/tools/install/include -DEXT_INT -DLINUX -DLINUX64 -DUSE_OPENMP -DUSE_OFFLOAD -DINTEL_64ALIGN -DIFCV8 -DIFCLINUX -DPARALLEL_DIAG -qopenmp -qopt-report-phase=offload -xHOST -ftz -qopt-report-phase=vec -qopt-report-file=stderr -qopenmp -qopt-report-phase:openmp -g -o nw_inp_from_string.o nw_inp_from_string.c
task_python.c(4): catastrophic error: cannot open source file "Python.h"
 #include <Python.h>

                    ^

compilation aborted for task_python.c (code 4)
make[1]: *** [/work/zacharyw/nwchem-6.6/lib/LINUX64/libnwpython.a(task_python.o)] Error 4
make[1]: *** Waiting for unfinished jobs....
nwchem_wrap.c(4): catastrophic error: cannot open source file "Python.h"
 #include <Python.h>

                    ^

compilation aborted for nwchem_wrap.c (code 4)
make[1]: *** [/work/zacharyw/nwchem-6.6/lib/LINUX64/libnwpython.a(nwchem_wrap.o)] Error 4
make[1]: Leaving directory `/work/zacharyw/nwchem-6.6/src/python'
make: *** [libraries] Error 1

I try using the 'find' UNIX command to find this Python.h file. Unfortunately, I can't seem to find this file, so my attempts at solving this problem are proving unsuccessful. The results of 'which python' is /usr/bin/python. Python 2.6 is being used.

Additionally, I attempt to compile (while in /src folder) the make file using the syntax:

make FC=ifort CC=icc -j 8



I have found similar issues upon google searching but nothing that appears directly relevant to this problem I am facing. I am new to this, so please bear with me. If there are any questions please let me know and I would be happy to answer. Anything that could illuminate this issue would be much appreciated. Thanks!

Forum Regular
1:23:26 PM PDT - Fri, Nov 4th 2016
Is the package python-devel installed on your system?

Clicked A Few Times
4:41:15 PM PDT - Fri, Nov 4th 2016
Actually, I found Python.h under: "/usr/local/python/include/python2.7"

How does this change my .bashrc profile? Thanks for your prompt response!

Forum Regular
7:42:47 AM PST - Tue, Nov 8th 2016
If you have the header file for python 2.7, I am assuming that version 2.7 is installed on your system as well, probably somewhere in /usr/local based on the location of the header file. Try changing your pythonhome variable to point to that version rather than the one in /usr/bin

Forum Vet
10:45:44 AM PST - Tue, Nov 8th 2016
Quote:Zww9 Nov 4th 3:41 pm
Actually, I found Python.h under: "/usr/local/python/include/python2.7"

How does this change my .bashrc profile? Thanks for your prompt response!

You value of PYTHONHOME should be changed to /usr/local/python

export PYTHONHOME=/usr/local/python

Clicked A Few Times
1:29:11 PM PST - Tue, Jan 17th 2017
Hi,
I have made this change in my setup script, as well as a few others. I am attempting compiling under the Intel Xeon Phi processors. The following is my setup script:

!#/bin/bash
export NWCHEM_TARGET=LINUX64
export NWCHEM_TOP=/work/zacharyw/recompile_nwchem6_6/nwchem-6.6
export ARMCI_NETWORK=OPENIB
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export PATH=/usr/local/intel-2015.2/impi/5.0.3.048/bin64/mpif90:$PATH #location of 'which mpif90'
export PYTHONHOME=/usr/local/python #defined in manual online
export PYTHONVERSION=2.7
export USE_PYTHONCONFIG=y
export USE_64TO32=y


export ARMCI_OPENIB_DEVICE=mlx4_0

export IB_HOME="/usr"
export IB_INCLUDE="$IB_HOME/include/infiniband"
export IB_LIB="$IB_HOME/lib64"
export IB_LIB_NAME="-libverbs -libumad -lpthread -lrt"

export MIC_LIBRARY_PATH=/usr/linux-k1om-4.7/linux-k1om/lib64:$MIC_LIBRARY_PATH
export MIC_LD_LIBRARY_PATH=/usr/linux-k1om-4.7/linux-k1om/lib64:$MIC_LD_LIBRARY_PATH


export ARMCI_OPENIB_DEVICE=mlx4_0
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y



export MKLROOT=/usr/local/intel-2017/compilers_and_libraries_2017.0.098/linux/mkl

export USE_OPENMP=1
export USE_F90_ALLOCATABLE=T
export USE_FASTMEM=T
export SCALAPACK="-mkl -qopenmp -lmkl_scalapack_ilp64 -lmkl_blacs_intelmpi_ilp64"
export ELPA="-I/usr/lib64/gfortran/modules/openmpi -L/usr/lib64/openmpi/lib -lelpa"
export LD_LIBRARY_PATH=/usr/lib64/openmpi/lib/:$LD_LIBRARY_PATH
export PATH=/usr/lib64/openmpi/bin/:$PATH
export BLASOPT="-mkl -qopenmp"
export SCALAPACK_SIZE=4
export BLAS_SIZE=4
export OMP_NUM_THREADS=4
export MIC_USE_2MB_BUFFER=16k



export USE_INTERNALBLAS=y

cd $NWCHEM_TOP/src
make nwchem_config NWCHEM_MODULES="all python"
make 64_to_32
make FC=ifort CC=icc AR=xiar -j 8 |tee /work/zacharyw/recompile_nwchem6_6/nwchem-6.6/nwchem_buil.log


However, I get the following error message:

DLINUX64 -DUSE_OPENMP -DIFCV8 -DIFCLINUX -DSCALAPACK -DPARALLEL_DIAG -DUSE_F90_ALLOCATABLE -DDEBUG_PRINT icsd_t2_neword.F
ifort -c -i8 -align -fpp -qopt-report-file=stderr -qopenmp -qopt-report-phase=openmp -qno-openmp-offload -fimf-arch-consistency=true -finline-limit=250 -O3 -unroll -ip -xHost -I. -I../include -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/include -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/include -DEXT_INT -DLINUX -DLINUX64 -DUSE_OPENMP -DIFCV8 -DIFCLINUX -DSCALAPACK -DPARALLEL_DIAG -DUSE_F90_ALLOCATABLE -DDEBUG_PRINT ccsd_kernels.F

Begin optimization report for: T2_P8

   Report from: OpenMP optimizations [openmp]

OpenMP Construct at ccsd_kernels.F(37,8)
  remark #16200: OpenMP DEFINED LOOP WAS PARALLELIZED
OpenMP Construct at ccsd_kernels.F(14,8)
  remark #16200: OpenMP DEFINED LOOP WAS PARALLELIZED
ifort -c -i8 -align -fpp -qopt-report-file=stderr -qopenmp -qopt-report-phase=openmp -qno-openmp-offload -fimf-arch-consistency=true -finline-limit=250 -O3 -unroll -ip -xHost -I. -I../include -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/include -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/include -DEXT_INT -DLINUX -DLINUX64 -DUSE_OPENMP -DIFCV8 -DIFCLINUX -DSCALAPACK -DPARALLEL_DIAG -DUSE_F90_ALLOCATABLE -DDEBUG_PRINT ccsd_t2_8.F
ccsd_t2_8.F(489): error #6404: This name does not have a type, and must have an explicit type. [L_A]
     if (e_a) call errquit("MA pops a",l_a,MA_ERR)
^
ccsd_t2_8.F(490): error #6404: This name does not have a type, and must have an explicit type. [L_T]
     if (e_t) call errquit("MA pops t",l_t,MA_ERR)
^
compilation aborted for ccsd_t2_8.F (code 1)
make[3]: *** [/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/lib/LINUX64/libtce.a(ccsd_t2_8.o)] Error 1
make[3]: *** Waiting for unfinished jobs....
===============================================================
make[3]: Leaving directory `/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tce/ccsd'
make[2]: *** [optimized] Error 2
make[2]: Leaving directory `/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tce/ccsd'
make[1]: *** [subdirs] Error 1
make[1]: Leaving directory `/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tce'
make: *** [libraries] Error 1


I do not understand what this means. Additionally, I am confused as to why I have to include the 64_to32 bit in my setup script. My initial assumption is something is wrong with the SCALAPATH or BLASOPT environment variable. Could someone please lend me some assistance? Thanks

Forum Vet
1:47:15 PM PST - Tue, Jan 17th 2017
Please unset USE_F90_ALLOCATABLE and USE_FASTMEM (since they can be used only with the current development version of NWChem and not with version 6.6) and then recompile

unset USE_F90_ALLOCATABLE
unset USE_FASTMEM
rm $NWCHEM_TOP/lib/$NWCHEM_TARGET/libtce.a
cd $NWCHEM_TOP/src/tce
make FC=ifort
cd ..
make FC=ifort link

Clicked A Few Times
3:32:24 PM PST - Tue, Jan 17th 2017
Hi,
I made the changes you suggested, including manually deleting the libtce.a file. My setup bash script now reads:

!#/bin/bash
export NWCHEM_TARGET=LINUX64
export NWCHEM_TOP=/work/zacharyw/recompile_nwchem6_6/nwchem-6.6
export ARMCI_NETWORK=OPENIB
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export PATH=/usr/local/intel-2015.2/impi/5.0.3.048/bin64/mpif90:$PATH #location of 'which mpif90'
export PYTHONHOME=/usr/local/python #defined in manual online
export PYTHONVERSION=2.7
export USE_PYTHONCONFIG=y
export USE_64TO32=y


export ARMCI_OPENIB_DEVICE=mlx4_0

export IB_HOME="/usr"
export IB_INCLUDE="$IB_HOME/include/infiniband"
export IB_LIB="$IB_HOME/lib64"
export IB_LIB_NAME="-libverbs -libumad -lpthread -lrt"

export MIC_LIBRARY_PATH=/usr/linux-k1om-4.7/linux-k1om/lib64:$MIC_LIBRARY_PATH
export MIC_LD_LIBRARY_PATH=/usr/linux-k1om-4.7/linux-k1om/lib64:$MIC_LD_LIBRARY_PATH


export ARMCI_OPENIB_DEVICE=mlx4_0
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
  1. changed **

    1. !!!!!!
export MKLROOT=/usr/local/intel-2017/compilers_and_libraries_2017.0.098/linux/mkl

export USE_OPENMP=1
  1. export USE_F90_ALLOCATABLE=T
  2. export USE_FASTMEM=T
export SCALAPACK="-mkl -qopenmp -lmkl_scalapack_ilp64 -lmkl_blacs_intelmpi_ilp64"
export ELPA="-I/usr/lib64/gfortran/modules/openmpi -L/usr/lib64/openmpi/lib -lelpa"
export LD_LIBRARY_PATH=/usr/lib64/openmpi/lib/:$LD_LIBRARY_PATH
export PATH=/usr/lib64/openmpi/bin/:$PATH
export BLASOPT="-mkl -qopenmp"
export SCALAPACK_SIZE=4
export BLAS_SIZE=4
export OMP_NUM_THREADS=4
export MIC_USE_2MB_BUFFER=16k



export USE_INTERNALBLAS=y
unset USE_F90_ALLOCATABLE
unset USE_FASTMEM
cd $NWCHEM_TOP/src/tce
make FC=ifort
cd ..
make FC=ifort link

However, this time I am getting an error that reads:

===============================================================
ifort -c -i8 -align -fpp -qopt-report-file=stderr -qopenmp -qopt-report-phase=openmp -qno-openmp-offload -fimf-arch-consistency=true -finline-limit=250 -O3 -unroll -ip -xHost -I. -I../include -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/include -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/include -DEXT_INT -DLINUX -DLINUX64 -DUSE_OPENMP -DIFCV8 -DIFCLINUX -DSCALAPACK -DPARALLEL_DIAG new_sort6.F
ifort -c -i8 -align -fpp -qopt-report-file=stderr -qopenmp -qopt-report-phase=openmp -qno-openmp-offload -fimf-arch-consistency=true -finline-limit=250 -O3 -unroll -ip -xHost -I. -I../include -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/include -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/include -DEXT_INT -DLINUX -DLINUX64 -DUSE_OPENMP -DIFCV8 -DIFCLINUX -DSCALAPACK -DPARALLEL_DIAG tce_sort8.F
ifort -c -i8 -align -fpp -qopt-report-file=stderr -qopenmp -qopt-report-phase=openmp -qno-openmp-offload -fimf-arch-consistency=true -finline-limit=250 -O3 -unroll -ip -xHost -I. -I../include -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/include -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/include -DEXT_INT -DLINUX -DLINUX64 -DUSE_OPENMP -DIFCV8 -DIFCLINUX -DSCALAPACK -DPARALLEL_DIAG tce_sortacc8.F
ifort -c -i8 -align -fpp -qopt-report-file=stderr -qopenmp -qopt-report-phase=openmp -qno-openmp-offload -fimf-arch-consistency=true -finline-limit=250 -O3 -unroll -ip -xHost tce_sort1.f
make[2]: Leaving directory `/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tce/sort'
make[2]: Entering directory `/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tce/sort'
Got lock on /work/zacharyw/recompile_nwchem6_6/nwchem-6.6/lib/LINUX64/libtce.lock
ar r /work/zacharyw/recompile_nwchem6_6/nwchem-6.6/lib/LINUX64/libtce.a tce_sort_4kg.o tce_sort0.o tce_sortacc0.o new_sort2.o new_sort4.o new_sort6.o tce_sort8.o tce_sortacc8.o tce_sort1.o
echo /work/zacharyw/recompile_nwchem6_6/nwchem-6.6/lib/LINUX64/libtce.a
/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/lib/LINUX64/libtce.a
make[2]: Leaving directory `/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tce/sort'
make[1]: Leaving directory `/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tce/sort'
make[1]: Entering directory `/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tce'
Got lock on /work/zacharyw/recompile_nwchem6_6/nwchem-6.6/lib/LINUX64/libtce.lock
ar r /work/zacharyw/recompile_nwchem6_6/nwchem-6.6/lib/LINUX64/libtce.a tce_input.o tce_init.o tce_energy.o tce_energy_fragment.o tce_tile.o tce_ao1e.o tce_mo1e.o tce_ao2e.o tce_filename.o createfile.o deletefile.o reconcilefile.o dratoga.o gatodra.o gatoeaf.o get_block.o put_block.o add_block.o tce_tidy.o sf_test.o hbar.o tce_mo2e.o tce_mo1e_offset.o tce_mo2e_offset.o tce_e_offset.o tce_guess_t1.o tce_guess_t2.o tce_zero.o tce_print_x1.o tce_print_x2.o tce_residual_t1.o tce_residual_t2.o tce_residual_t3.o tce_residual_t4.o ma_pack.o sf_print.o ma_print_compact.o ma_scale.o copyfile.o ddotfile.o daxfile.o daxpyfile.o dscalfile.o tce_aod1.o tce_nud1.o tce_mod1.o tce_mod1_offset.o sym_abelian_axis.o tce_mod1_new.o tce_mod1_offset_new.o tce_eom_init.o tce_eom_tidy.o tce_eom_xdiagon.o tce_eom_ydiagon.o tce_t1_offset.o tce_t2_offset.o tce_t3_offset.o tce_t4_offset.o tce_x1_offset.o tce_x2_offset.o tce_x3_offset.o tce_x4_offset.o tce_y1_offset.o tce_y2_offset.o tce_y3_offset.o tce_y4_offset.o tce_mbpt2.o tce_restricted.o tce_hash.o get_hash_block.o put_hash_block.o add_hash_block.o tce_residual_t3a.o tce_t3a_offset.o tce_x3a_offset.o tce_cholesky.o tce_ao2e_cholesky.o tce_ao2e_cholesky_2d.o tce_ao2e_cholesky_4d.o e2_offset_2_offset.o get_block_ind.o get_block_ind_sf.o tce_hash_ind.o tce_mo2e_offset_intorb.o tce_mo2e_offset_size.o tce_mo2e_trans.o atpart_nalength.o tce_mo2e_zones_4a_disk_ga.o tce_mo2e_zones_4a_disk_ga_chop.o tce_mo2e_zones_4a_disk_2s_new.o tce_restart.o tce_restart_triples.o tce_restart_quadruples.o tce_dens_hh_offset.o tce_dens_hp_offset.o tce_dens_pp_offset.o tce_dens_ph_offset.o get_mo_rdm_hh.o get_mo_rdm_hp.o get_mo_rdm_ph.o get_mo_rdm_pp.o ao_rdm_write.o tce_prop_start.o tce_property.o tce_diagnose_t1.o tce_fourindex_driver.o tce_fock_create.o tce_multipole_create.o tce_reference_check.o tce_fourindex_cleanup.o tce_fock_destroy.o tce_multipole_destroy.o tce_create_t1.o tce_create_t2.o tce_create_t3.o tce_create_t4.o tce_create_x1.o tce_create_x2.o tce_create_x3.o tce_create_x4.o tce_create_y1.o tce_create_y2.o tce_create_y3.o tce_create_y4.o tce_clone_x1.o tce_clone_x2.o tce_clone_x3.o tce_clone_x4.o tce_clone_y1.o tce_clone_y2.o tce_clone_y3.o tce_clone_y4.o tce_t1_offset_new.o tce_t2_offset_new.o tce_t3_offset_new.o tce_t4_offset_new.o tce_x1_offset_new.o tce_x2_offset_new.o tce_x3_offset_new.o tce_x4_offset_new.o tce_y1_offset_new.o tce_y2_offset_new.o tce_y3_offset_new.o tce_y4_offset_new.o tce_ccsd_driver.o tce_ccsdt_driver.o tce_ccsdtq_driver.o tce_ccsd_perturbative.o tce_ccsd_lambda_perturbative.o tce_ccsd_lambda_driver.o tce_ccsdt_lambda_driver.o tce_ccsdtq_lambda_driver.o tce_ccsd_response_driver.o tce_ccsdt_response_driver.o tce_ccsdtq_response_driver.o tce_ccsd_symm_polar.o tce_ccsdt_symm_polar.o tce_ccsdtq_symm_polar.o tce_ccsd_lambda_response_driver.o tce_ccsd_asym_polar.o tce_ccsd_hyperpolar.o tce_ao2e_new.o tce_mo2e_new.o tce_tile_new.o tce_init_new.o tce_mo2e_zones_4a_disk_4s_new_patch.o tce_mo2e_zones_4a_disk_4s_new_patch_m2.o tce_mo2e_zones_4a_disk_4s_new_patch_m3.o tce_ao2e_disk.o tce_mo2e_disk.o tce_mo2e_disk_2eorb.o tce_mo2e_incore_2eorb.o tce_mo2e_incore_2eorb_split.o tce_mo2e_hybrid_2eorb_split.o tce_mo2e_offset_intorb_hybrid.o tce_mo2e_2emet3.o tce_mo2e_2emet3_n5.o tce_mo2e_2emet4.o tce_mo2e_2emet4_n5.o get_hash_block_ma.o ccsd_energy_loc.o cc2_energy.o ccsd_lambda.o ccsd_imaginary.o ccsd_lr_alpha.o ccsd_qr_beta.o tce_dgeev.o tce_schmidt.o tensor_read_write.o tce_diis_act.o tce_mo2e_zones_4a_disk_ga_act.o tce_mo2e_zones_4a_disk_ga_chop_act.o tce_t2a_offset.o tce_x2a_offset.o new_ga4ind_N5.o tce_mo2e_zones_4a_disk_ga_N5.o tce_mo2e_zones_4a_disk_ga_chop_N5.o tce_eom_xguess.o tce_eom_yguess.o ma_transpose.o
echo /work/zacharyw/recompile_nwchem6_6/nwchem-6.6/lib/LINUX64/libtce.a
/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/lib/LINUX64/libtce.a
make[1]: Leaving directory `/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tce'
+ cd ..
+ make FC=ifort link
make nwchem.o stubs.o
make[1]: Entering directory `/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src'
ifort -i8 -align -fpp -qopt-report-file=stderr -qopenmp -qopt-report-phase=openmp -qno-openmp-offload -fimf-arch-consistency=true -finline-limit=250 -O2 -g -fp-model source -I. -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/include -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/include -DEXT_INT -DLINUX -DLINUX64 -DUSE_OPENMP -DIFCV8 -DIFCLINUX -DSCALAPACK -DPARALLEL_DIAG -DCOMPILATION_DATE="'`date +%a_%b_%d_%H:%M:%S_%Y`'" -DCOMPILATION_DIR="'/work/zacharyw/recompile_nwchem6_6/nwchem-6.6'" -DNWCHEM_BRANCH="'6.6'" -c -o nwchem.o nwchem.F
ifort -i8 -align -fpp -qopt-report-file=stderr -qopenmp -qopt-report-phase=openmp -qno-openmp-offload -fimf-arch-consistency=true -finline-limit=250 -O2 -g -fp-model source -I. -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/include -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/include -DEXT_INT -DLINUX -DLINUX64 -DUSE_OPENMP -DIFCV8 -DIFCLINUX -DSCALAPACK -DPARALLEL_DIAG -DCOMPILATION_DATE="'`date +%a_%b_%d_%H:%M:%S_%Y`'" -DCOMPILATION_DIR="'/work/zacharyw/recompile_nwchem6_6/nwchem-6.6'" -DNWCHEM_BRANCH="'6.6'" -c -o stubs.o stubs.F
make[1]: Leaving directory `/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src'
ifort -i8 -align -fpp -qopt-report-file=stderr -qopenmp -qopt-report-phase=openmp -qno-openmp-offload -fimf-arch-consistency=true -finline-limit=250 -O2 -g -fp-model source -Wl,--export-dynamic -L/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/lib/LINUX64 -L/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/lib -o /work/zacharyw/recompile_nwchem6_6/nwchem-6.6/bin/LINUX64/nwchem nwchem.o stubs.o -lnwctask -lccsd -lmcscf -lselci -lmp2 -lmoints -lstepper -ldriver -loptim -lnwdft -lgradients -lcphf -lesp -lddscf -ldangchang -lguess -lhessian -lvib -lnwcutil -lrimp2 -lproperty -lsolvation -lnwints -lprepar -lnwmd -lnwpw -lofpw -lpaw -lpspw -lband -lnwpwlib -lcafe -lspace -lanalyze -lqhop -lpfft -ldplot -lnwpython -ldrdy -lvscf -lqmmm -lqmd -letrans -lpspw -ltce -lbq -lmm -lcons -lperfm -ldntmc -lccca -lnwcutil -lga -larmci -lpeigs -lperfm -lcons -lbq -lnwcutil -I/usr/lib64/gfortran/modules/openmpi -L/usr/lib64/openmpi/lib -lelpa -mkl -qopenmp -lmkl_scalapack_ilp64 -lmkl_blacs_intelmpi_ilp64 -l64to32 -mkl -qopenmp -lnwclapack -lnwcblas -L/usr/local/intel-2015.2//impi/5.0.3.048/intel64/lib/release_mt -L/usr/local/intel-2015.2//impi/5.0.3.048/intel64/lib -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread -libverbs -libumad -lpthread -lrt -L/usr/lib64 -lrt -lpthread -lnwcutil -lpthread -ldl -lutil -lm -lpython2.7 -lpthread -ldl -lutil -lm -lpython2.7 -Xlinker -export-dynamic
ld: cannot find -ldangchang
make: *** [link] Error 1


Any ideas, I am again at a loss? Thanks for your understanding and patience of these seemingly trivial questions!

Clicked A Few Times
8:04:32 AM PST - Wed, Jan 18th 2017
I tried starting over by completely removing the nwchem-6.6 directory, untarring the nwchem tar file and recompiling just to see if I would get the same error message. Instead I get another error message shown in the following:

ifort -i8 -align -fpp -qopt-report-file=stderr -qopenmp -qopt-report-phase=openmp -qno-openmp-offload -fimf-arch-consistency=true -finline-limit=250 -O2 -g -fp-model source -I. -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/include -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/include -DEXT_INT -DLINUX -DLINUX64 -DUSE_OPENMP -DIFCV8 -DIFCLINUX -DSCALAPACK -DPARALLEL_DIAG -DCOMPILATION_DATE="'`date +%a_%b_%d_%H:%M:%S_%Y`'" -DCOMPILATION_DIR="'/work/zacharyw/recompile_nwchem6_6/nwchem-6.6'" -DNWCHEM_BRANCH="'6.6'" -c -o stubs.o stubs.F
make[1]: Leaving directory `/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src'
ifort -i8 -align -fpp -qopt-report-file=stderr -qopenmp -qopt-report-phase=openmp -qno-openmp-offload -fimf-arch-consistency=true -finline-limit=250 -O2 -g -fp-model source -Wl,--export-dynamic -L/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/lib/LINUX64 -L/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/lib -o /work/zacharyw/recompile_nwchem6_6/nwchem-6.6/bin/LINUX64/nwchem nwchem.o stubs.o -lnwctask -lccsd -lmcscf -lselci -lmp2 -lmoints -lstepper -ldriver -loptim -lnwdft -lgradients -lcphf -lesp -lddscf -ldangchang -lguess -lhessian -lvib -lnwcutil -lrimp2 -lproperty -lsolvation -lnwints -lprepar -lnwmd -lnwpw -lofpw -lpaw -lpspw -lband -lnwpwlib -lcafe -lspace -lanalyze -lqhop -lpfft -ldplot -lnwpython -ldrdy -lvscf -lqmmm -lqmd -letrans -lpspw -ltce -lbq -lmm -lcons -lperfm -ldntmc -lccca -lnwcutil -lga -larmci -lpeigs -lperfm -lcons -lbq -lnwcutil -l64to32 -mkl -qopenmp -lnwclapack -lnwcblas -L/usr/local/intel-2015.2//impi/5.0.3.048/intel64/lib/release_mt -L/usr/local/intel-2015.2//impi/5.0.3.048/intel64/lib -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread -lnwcutil -lpthread -ldl -lutil -lm -lpython2.7 -lpthread -ldl -lutil -lm -lpython2.7 -Xlinker -export-dynamic
ld: cannot find -lpython2.7
make: *** [all] Error 1

I notice that this linker file 'L/usr/local/intel-2015.2//impi/5.0.3.048/intel64/lib' shows up in both error messages. I assume there is an issue here, and I feel like perhaps python is an issue? There is a similar path '/usr/local/intel-2017/impi/2017.0.098/intel64/lib' that exists. My question is: how do I know which path to include in my setup script? It seems like the HPC I am attempting to compile this in puts me in this predicament a lot--- another example is that python2.7 and python3.4 are in my /usr/local path, but only python2.6 exists in my /usr/bin path. How do I know what are the correct versions of mpi and python to reference in my setup script?

Forum Vet
10:16:51 AM PST - Wed, Jan 18th 2017
Since I am not sure yet how to deal with your Python environment, may I suggest you the following: first, try to build a NWChem binary and test it and then try to create a build with python?

In order to exclude Python from the build, please execute the following commands:

cd $NWCHEM_TOP/src
make nwchem_config NWCHEM_MODULES="all"
make FC=ifort link

PS#1 I see mention of OpenMPI in your env. settings and -- at the same time -- your are using Intel MPI, this is likely to cause problems. Please remove anything mention of OpenMPI in your settings

PS#2 You defined BLAS_SIZE=4, while your definition of SCALAPACK uses ILP64, a.k.a 64-bit integers, therefore you should set BLAS_SIZE=8. I would suggest you the following

export BLAS_SIZE=8
cd $NWCHEM_TOP/src/64to32blas
make clean
make FC=ifort
cd ../tools
make clean
make FC=ifort
cd ..
make FC=ifort link

Clicked A Few Times
12:36:18 PM PST - Wed, Jan 18th 2017
I have made the changes you have suggested, and everything runs smoothly (i.e. no discernable error messages). Here is the setup script I used, with your modifications for clarification purposes:

!#/bin/bash
export NWCHEM_TARGET=LINUX64
export NWCHEM_TOP=/work/zacharyw/recompile_nwchem6_6/nwchem-6.6
export ARMCI_NETWORK=OPENIB
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export PATH=/usr/local/intel-2015.2/impi/5.0.3.048/bin64/mpif90:$PATH #location of 'which mpif90'
export PYTHONHOME=/usr/local/python #defined in manual online
export PYTHONVERSION=2.7
export USE_PYTHONCONFIG=y
export USE_64TO32=y


export ARMCI_OPENIB_DEVICE=mlx4_0

export IB_HOME="/usr"
export IB_INCLUDE="$IB_HOME/include/infiniband"
export IB_LIB="$IB_HOME/lib64"
export IB_LIB_NAME="-libverbs -libumad -lpthread -lrt"

export MIC_LIBRARY_PATH=/usr/linux-k1om-4.7/linux-k1om/lib64:$MIC_LIBRARY_PATH
export MIC_LD_LIBRARY_PATH=/usr/linux-k1om-4.7/linux-k1om/lib64:$MIC_LD_LIBRARY_PATH


export ARMCI_OPENIB_DEVICE=mlx4_0
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y

export MKLROOT=/usr/local/intel-2017/compilers_and_libraries_2017.0.098/linux/mkl

export USE_OPENMP=1
export SCALAPACK="-mkl -qopenmp -lmkl_scalapack_ilp64 -lmkl_blacs_intelmpi_ilp64"
export BLASOPT="-mkl -qopenmp"
export SCALAPACK_SIZE=4
export BLAS_SIZE=8
export OMP_NUM_THREADS=4
export MIC_USE_2MB_BUFFER=16k



export USE_INTERNALBLAS=y
unset USE_F90_ALLOCATABLE
unset USE_FASTMEM

cd $NWCHEM_TOP/src
make nwchem_config NWCHEM_MODULES="all"
make FC=ifort link
cd $NWCHEM_TOP/src/64to32blas
make clean
make FC=ifort
cd ../tools
make clean
make FC=ifort
cd ..
make FC=ifort link

The last few lines of output looks something like:

Libraries have been installed in:
  /work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
  - add LIBDIR to the `LD_LIBRARY_PATH' environment variable

    during execution

  - add LIBDIR to the `LD_RUN_PATH' environment variable

    during linking

  - use the `-Wl,-rpath -Wl,LIBDIR' linker flag

  - have your system administrator add LIBDIR to `/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.

/bin/mkdir -p '/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/bin'

 /bin/sh ./libtool   --mode=install /usr/bin/install -c global/trace/adjust.x global/trace/collisions.x '/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/bin'
libtool: install: /usr/bin/install -c global/trace/adjust.x /work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/bin/adjust.x
libtool: install: /usr/bin/install -c global/trace/collisions.x /work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/bin/collisions.x
/bin/mkdir -p '/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/bin'

/usr/bin/install -c tools/ga-config '/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/bin'

/bin/mkdir -p '/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/include'

/usr/bin/install -c -m 644 ../ga-5-4/ma/error.h ../ga-5-4/ma/macdecls.h ../ga-5-4/ma/macommon.h ../ga-5-4/ma/maf2c.fh ma/mafdecls.fh ma/matypes.h ../ga-5-4/ma/scope.h ../ga-5-4/ma/string-util.h ../ga-5-4/ma/table.h ../ga-5-4/global/src/gacommon.h ../ga-5-4/global/src/ga.h global/src/global.fh ../ga-5-4/global/src/ga-papi.h ../ga-5-4/global/src/ga-mpi.h ../ga-5-4/global/src/ga-mpi.fh ../ga-5-4/pario/dra/dra.fh ../ga-5-4/pario/dra/dra.h ../ga-5-4/pario/eaf/eaf.fh ../ga-5-4/pario/eaf/eaf.h ../ga-5-4/pario/elio/chemio.h ../ga-5-4/pario/elio/elio.h ../ga-5-4/pario/sf/coms.h ../ga-5-4/pario/sf/sf.fh ../ga-5-4/pario/sf/sf.h gaf2c/typesf2c.h gaf2c/farg.h ../ga-5-4/tcgmsg/tcgmsg.fh ../ga-5-4/tcgmsg/tcgmsg.h ../ga-5-4/tcgmsg/tcgmsg-mpi/msgtypesc.h ../ga-5-4/tcgmsg/tcgmsg-mpi/msgtypesf.h '/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/include'

/bin/mkdir -p '/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/include'

/usr/bin/install -c -m 644 global/src/ga-wapi.h '/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/include'
make[4]: Leaving directory `/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/build'
make[3]: Leaving directory `/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/build'
make[2]: Leaving directory `/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/build'
make[1]: Leaving directory `/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/build'
+ cd ..
+ make FC=ifort link
make nwchem.o stubs.o
make[1]: Entering directory `/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src'
ifort -i8 -align -fpp -qopt-report-file=stderr -qopenmp -qopt-report-phase=openmp -qno-openmp-offload -fimf-arch-consistency=true -finline-limit=250 -O2 -g -fp-model source -I. -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/include -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/include -DEXT_INT -DLINUX -DLINUX64 -DUSE_OPENMP -DIFCV8 -DIFCLINUX -DSCALAPACK -DPARALLEL_DIAG -DCOMPILATION_DATE="'`date +%a_%b_%d_%H:%M:%S_%Y`'" -DCOMPILATION_DIR="'/work/zacharyw/recompile_nwchem6_6/nwchem-6.6'" -DNWCHEM_BRANCH="'6.6'" -c -o nwchem.o nwchem.F
ifort -i8 -align -fpp -qopt-report-file=stderr -qopenmp -qopt-report-phase=openmp -qno-openmp-offload -fimf-arch-consistency=true -finline-limit=250 -O2 -g -fp-model source -I. -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/include -I/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/include -DEXT_INT -DLINUX -DLINUX64 -DUSE_OPENMP -DIFCV8 -DIFCLINUX -DSCALAPACK -DPARALLEL_DIAG -DCOMPILATION_DATE="'`date +%a_%b_%d_%H:%M:%S_%Y`'" -DCOMPILATION_DIR="'/work/zacharyw/recompile_nwchem6_6/nwchem-6.6'" -DNWCHEM_BRANCH="'6.6'" -c -o stubs.o stubs.F
make[1]: Leaving directory `/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src'
ifort -i8 -align -fpp -qopt-report-file=stderr -qopenmp -qopt-report-phase=openmp -qno-openmp-offload -fimf-arch-consistency=true -finline-limit=250 -O2 -g -fp-model source -L/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/lib/LINUX64 -L/work/zacharyw/recompile_nwchem6_6/nwchem-6.6/src/tools/install/lib -o /work/zacharyw/recompile_nwchem6_6/nwchem-6.6/bin/LINUX64/nwchem nwchem.o stubs.o -lnwctask -lccsd -lmcscf -lselci -lmp2 -lmoints -lstepper -ldriver -loptim -lnwdft -lgradients -lcphf -lesp -lddscf -ldangchang -lguess -lhessian -lvib -lnwcutil -lrimp2 -lproperty -lsolvation -lnwints -lprepar -lnwmd -lnwpw -lofpw -lpaw -lpspw -lband -lnwpwlib -lcafe -lspace -lanalyze -lqhop -lpfft -ldplot -ldrdy -lvscf -lqmmm -lqmd -letrans -lpspw -ltce -lbq -lmm -lcons -lperfm -ldntmc -lccca -lnwcutil -lga -larmci -lpeigs -lperfm -lcons -lbq -lnwcutil -mkl -qopenmp -lmkl_scalapack_ilp64 -lmkl_blacs_intelmpi_ilp64 -l64to32 -mkl -qopenmp -lnwclapack -lnwcblas -L/usr/local/intel-2015.2//impi/5.0.3.048/intel64/lib/release_mt -L/usr/local/intel-2015.2//impi/5.0.3.048/intel64/lib -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread -libverbs -libumad -lpthread -lrt -L/usr/lib64 -lrt -lm -lpthread

This leads us to assume that, providing my corrections to 'BLAS_SIZE' env variable and omitting references to openMPI, the python env variables I used in earlier setup scripts are the issue. How should I proceed?

Curiously, there is no specific reference to nwchem-6.6 in my /usr/local directory. I was going to see if the executable linked successfully using 'ldd' command; but as the directory is not present this is not possible. There is, however, a directory to nwchem-6.5 (containing an executable that is not completely linked), as well as a 'nwchem' symbolic link to the path '/usr/local/server/x86_64-RedHat6/nwchem' which also contains an executable that is not completely linked. I find this odd; any thoughts?

Thanks


Forum >> NWChem's corner >> Compiling NWChem