NWChem-6.3 (2013-05-28) on SUSE Linux

Just Got Here

7:00:12 AM PDT - Sun, Jun 16th 2013

Actually I did recompile nwchem with MPICH2 and the compilation went fine. Probably some of my variables were not set properly for openmpi.

Here is my env.

PATH=/soft/mpich2-1.4.1p1/install/bin:$PATH ; export PATH
LD_LIBRABRY_PATH=${LD_LIBRARY_PATH}:/soft/mpich2-1.4.1p1/install/lib64
export LD_LIBRABRY_PATH

export NWCHEM_TOP=/varna/tiordano/nwchem-6.3-src.2013-05-28

export NWCHEM_TARGET=LINUX64

export NWCHEM_MODULES="all python"

export ARMCI_NETWORK=SOCKETS

export USE_MPI=yes

export USE_MPIF=yes

export USE_MPI4=yes

export MPI_LOC=/soft/mpich2-1.4.1p1/install

export MPI_LIB=/soft/mpich2-1.4.1p1/install/lib64

export MPI_INCLUDE=/soft/mpich2-1.4.1p1/install/include

export LIBMPI="-lmpich -lopa -lmpl -lrt -lpthread"

export MPIF77=/soft/mpich2-1.4.1p1/install/bin/mpif77

export MPICC=/soft/mpich2-1.4.1p1/install/bin/mpicc

export MPICXX=/soft/mpich2-1.4.1p1/install/bin/mpicxx

export PYTHONHOME=${NWCHEM_TOP}/python

export PYTHONVERSION=2.7

export USE_PYTHON64=yes

cd $NWCHEM_TOP/src
make nwchem_config
make FC=mpif90 -f GNUmakefile

Forum Regular

10:04:56 AM PDT - Tue, Jun 18th 2013
I see something interesting in your environment variables. You are specifying that the code be build using MPI but you request for the ARMCI_NETWORK that SOCKETS be used. That combination makes no sense. What your ARMCI_NETWORK should be set to depends on your platform and in particular the interconnect. If you have no special interconnect you will to set ARMCI_NETWORK to one of the MPI variants. This can be either MPI_TS or MPI-MT. The first option uses MPI twosided communication this should always work but introduces additional synchronization. The second option uses multi-threading by which the Global Arrays create a special thread in every process to handle remote communication requests. This is faster but requires multi-threading support in your MPI library. You can either try this out by hand or use the build_nwchem script (please unset all you NWCHEM environment variables before trying that) see http://nwchemgit.github.io/index.php/Compiling_NWChem#How-to:_Build_nwchem_script for details. The script will automatically check for any high performance interconnects on your machine as well as the capabilities of your MPI library.

Just Got Here

9:02:18 AM PDT - Wed, Jun 19th 2013
Thank you! The compilation went fine this time. Here is the NWCHEM environment: NWCHEM_TOP = /varna/tiordano/nwchem-6.3-src.2013-05-28 NWCHEM_TARGET = LINUX64 NWCHEM_MODULES = all USE_MPI = y USE_MPIF = y USE_MPIF4 = y MPI_INCLUDE = -I/soft/mpich2-1.4.1p1/install/include -I/soft/mpich2-1.4.1p1/install/include MPI_LIB = -L/soft/mpich2-1.4.1p1/install/lib64 LIBMPI = -lmpichf90 -lmpichf90 -lmpich -lopa -lmpl -lrt -lpthreadMPI_F90 = gfortran MPI_CC = gcc MPI_CXX = c++ARMCI_NETWORK = MPI-MTMSG_COMMS = MPI BLAS_LIB = -lblas integer-size = 4LAPACK_LIB = -llapack integer-size = 4 However if I try to run nwchem the jobs are terminated. Here is the error message. Screening Tolerance Information ------------------------------- Density screening/tol_rho: 1.00D-10 AO Gaussian exp screening on grid/accAOfunc: 14 CD Gaussian exp screening on grid/accCDfunc: 20 XC Gaussian exp screening on grid/accXCfunc: 20 Schwarz screening/accCoul: 1.00D-08 0:0:ga_diag:peigs not interfaced:: -1 1:1:ga_diag:peigs not interfaced:: -1 (rank:1 hostname:ws31 pid:31904):ARMCI DASSERT fail. src/common/armci.c:ARMCI_Error():208 cond:0 2:2:ga_diag:peigs not interfaced:: -1 (rank:2 hostname:ws31 pid:31905):ARMCI DASSERT fail. src/common/armci.c:ARMCI_Error():208 cond:0 3:3:ga_diag:peigs not interfaced:: -1 (rank:3 hostname:ws31 pid:31906):ARMCI DASSERT fail. src/common/armci.c:ARMCI_Error():208 cond:0 Last System Error Message from Task 1:: No such file or directory Last System Error Message from Task 2:: No such file or directory application called MPI_Abort(comm=0x84000004, -1) - process 2 Last System Error Message from Task 3:: No such file or directory application called MPI_Abort(comm=0x84000004, -1) - process 3 (rank:0 hostname:ws31 pid:31903):ARMCI DASSERT fail. src/common/armci.c:ARMCI_Error():208 cond:0 application called MPI_Abort(comm=0x84000004, -1) - process 1 Last System Error Message from Task 0:: Inappropriate ioctl for device application called MPI_Abort(comm=0x84000007, -1) - process 0

Forum Vet

5:28:17 PM PDT - Wed, Jun 19th 2013
Is this last build a clean build? I.e. did you do a "make realclean"?

Just Got Here

1:57:12 AM PDT - Thu, Jun 20th 2013

I have started from a new Nwchem****.tar

% cd <my path>/nwchem

% ./contrib/distro-tools/build_nwchem | tee build_nwchem.log

I thought that the script should point to the wrappers instead it points to the basename of the compilers:

MPI_F90 = gfortran
MPI_CC = gcc
MPI_CXX = c++

I have mpich2 in my PATH:

PATH=/soft/mpich2-1.4.1p1/install/bin:$PATH

export PATH

LD_LIBRABRY_PATH=${LD_LIBRARY_PATH}:/soft/mpich2-1.4.1p1/install/lib64

export LD_LIBRABRY_PATH

I am afraid that it takes amci and ga libraries from the root directory. Apparently, in previous attempts I did not have the correct setting ( had ARMCI_NETWORK=SOCKETS) and I did:

as user:

% cd nwchem-6.3-src.2013-05-28/src/tools/ga-5-2
% ./configure
% make
% make check

as root:
% make install

as user:
% make installcheck

Do I have to remove any libraries from the root directories?

Just Got Here

9:19:14 PM PDT - Fri, Jun 21st 2013

Ok, when GA is compiled the Makefile overwrites my environment and sets ARMCI_NETWORK=SOCKETS.

I did recompile also with openmpi and then compiled nwchem from scratch. However when I try to run the executable I have the following error:

...

      Schwarz screening/accCoul: 1.00D-08

1:1:ga_diag:peigs not interfaced:: -1
(rank:1 hostname:ws31 pid:1973):ARMCI DASSERT fail. src/common/armci.c:ARMCI_Error():208 cond:0
2:2:ga_diag:peigs not interfaced:: -1
(rank:2 hostname:ws31 pid:1974):ARMCI DASSERT fail. src/common/armci.c:ARMCI_Error():208 cond:0
3:3:ga_diag:peigs not interfaced:: -1
(rank:3 hostname:ws31 pid:1975):ARMCI DASSERT fail. src/common/armci.c:ARMCI_Error():208 cond:0
0:0:ga_diag:peigs not interfaced:: -1
Last System Error Message from Task 3:: No such file or directory
Last System Error Message from Task 2:: No such file or directory
Last System Error Message from Task 1:: No such file or directory

MPI_ABORT was invoked on rank 3 in communicator MPI COMMUNICATOR 4 DUP FROM 0
with errorcode -1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.

(rank:0 hostname:ws31 pid:1972):ARMCI DASSERT fail. src/common/armci.c:ARMCI_Error():208 cond:0
Last System Error Message from Task 0:: Inappropriate ioctl for device

mpirun has exited due to process rank 1 with PID 1973 on
node ws31 exiting without calling "finalize". This may
have caused other processes in the application to be
terminated by signals sent by mpirun (as reported here).

[ws31:01971] 3 more processes have sent help message help-mpi-api.txt / mpi-abort
[ws31:01971] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages

                                                                                                                                                           626,1         Bot

Forum >> NWChem's corner >> Compiling NWChem