nwchem hangs before dft scf
Clicked A Few Times
3:24:14 PM PDT - Thu, Jul 28th 2011
Hi all

I have recently attempted to install nwchem using openmpi and openib on our cluster on campus. All the QA tests for the modules I have installed seem to be OK.

Small molecules run fine (<20 atoms), but now I have moved onto some slightly larger metal-organic molcules (~60 molecules) and the program appears to hang while determining the best guess for the first dft scf iteration. It takes more than an hour to complete the guess using 6-31g*, even when I am importing vectors from a previous 3-21g calculation. Once it makes a first guess the DIIS scf iterations go fairly quickly.

I am not sure if I am doing something wrong or not, but I feel that it should not hang here while using 24 processors.

Any help you could provide would be greatly appreciated!

More info below:

Clicked A Few Times
3:26:26 PM PDT - Thu, Jul 28th 2011

Build Variables: 

export LARGE_FILES=TRUE
export NWCHEM_TOP=/projects/damrauer/nwchem-6.0
export NWCHEM_TARGET=LINUX64
export TCGRSH=/usr/bin/ssh
export ENABLE_COMPONENT=yes
export NWCHEM_MODULES=all
export USE_MPI=y
export USE_MPIF=y
export MPI_LOC=/curc/tools/free/redhat_5_x86_64/openmpi-1.4.3_intel-12.0_ib
export MPI_LIB=$MPI_LOC/lib
export MPI_INCLUDE=$MPI_LOC/include
export LIBMPI="-L $MPI_LIB -lmpi -lopen-pal -lopen-rte -lmpi_f90 -lmpi_f77"
export ARMCI_NETWORK=OPENIB

export IB_HOME=/usr
export IB_INCLUDE=$IB_HOME/include/infiniband
export IB_LIB=$IB_HOME/lib64
export IB_LIB_NAME="-libumad -libverbs -lpthread"
export FOPTIMIZE="-O3 -xSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 -no-prec-div -funroll-loops -unroll-aggressive"
export COPTIMIZE="-O3 -xSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 -no-prec-div -funroll-loops"

export FC=ifort
export CC=icc
export MPI_CC=icc


Input file: 

START
SCRATCH_DIR /scratch/stmp00/vallett
PERMANENT_DIR /home/vallett/nwchem_output
ECHO
MEMORY total 2000 mb

TITLE "RuBpy"

INCLUDE RuBpy.geom

set "geometry" "initial RuBpy"

charge 2

basis "all 321g basis"
	* library 3-21g #except Ru
end

basis "all 631g* except Ru basis"
	* library 6-31g* except Ru
	Ru library 3-21g
end

set "ao basis" "all 631g* except Ru basis"

dft
	mult 1 
	xc pbe0
	iterations 600
	vectors input project "all 321g basis" RuBpy.movecs output RuBpy_631g.movecs
end

task dft


Output:

everything seems ok in the geometry and basis set output, until the job actually starts to run:


 Filename           = /home/vallett/nwchem_output/RuBpy.movecs
 Old title          = "RuBpy"
 Old basis size     =   740
 Old no. of sets    =     1
 Old no. of MOs     =   740
 Old basis name     = all 321g basis
 Basis name on file = all 321g basis

   Time after variat. SCF:     16.7
   Time prior to 1st pass:     16.7

 Integral file          = /scratch/stmp00/vallett/RuBpy.aoints.00
 Record size in doubles =  65536        No. of integs per rec  =  32766
 Max. records in memory =      0        Max. records in file   =  64930
 No. of bits per label  =     16        No. of bits per value  =     64


And then it stalls here for an hour or more.

Forum Vet
4:52:09 PM PDT - Tue, Aug 2nd 2011
It looks like it is stalling when it tries to store integrals to disk. Try running this calculation with setting "direct" in the "scf" block.

Bert


Quote:Pvallett Jul 28th 10:26 pm
== Build Variables: == 
export LARGE_FILES=TRUE
export NWCHEM_TOP=/projects/damrauer/nwchem-6.0
export NWCHEM_TARGET=LINUX64
export TCGRSH=/usr/bin/ssh
export ENABLE_COMPONENT=yes
export NWCHEM_MODULES=all
export USE_MPI=y
export USE_MPIF=y
export MPI_LOC=/curc/tools/free/redhat_5_x86_64/openmpi-1.4.3_intel-12.0_ib
export MPI_LIB=$MPI_LOC/lib
export MPI_INCLUDE=$MPI_LOC/include
export LIBMPI="-L $MPI_LIB -lmpi -lopen-pal -lopen-rte -lmpi_f90 -lmpi_f77"
export ARMCI_NETWORK=OPENIB

export IB_HOME=/usr
export IB_INCLUDE=$IB_HOME/include/infiniband
export IB_LIB=$IB_HOME/lib64
export IB_LIB_NAME="-libumad -libverbs -lpthread"
export FOPTIMIZE="-O3 -xSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 -no-prec-div -funroll-loops -unroll-aggressive"
export COPTIMIZE="-O3 -xSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 -no-prec-div -funroll-loops"

export FC=ifort
export CC=icc
export MPI_CC=icc


Input file: 

START
SCRATCH_DIR /scratch/stmp00/vallett
PERMANENT_DIR /home/vallett/nwchem_output
ECHO
MEMORY total 2000 mb

TITLE "RuBpy"

INCLUDE RuBpy.geom

set "geometry" "initial RuBpy"

charge 2

basis "all 321g basis"
	* library 3-21g #except Ru
end

basis "all 631g* except Ru basis"
	* library 6-31g* except Ru
	Ru library 3-21g
end

set "ao basis" "all 631g* except Ru basis"

dft
	mult 1 
	xc pbe0
	iterations 600
	vectors input project "all 321g basis" RuBpy.movecs output RuBpy_631g.movecs
end

task dft


Output:

everything seems ok in the geometry and basis set output, until the job actually starts to run:


 Filename           = /home/vallett/nwchem_output/RuBpy.movecs
 Old title          = "RuBpy"
 Old basis size     =   740
 Old no. of sets    =     1
 Old no. of MOs     =   740
 Old basis name     = all 321g basis
 Basis name on file = all 321g basis

   Time after variat. SCF:     16.7
   Time prior to 1st pass:     16.7

 Integral file          = /scratch/stmp00/vallett/RuBpy.aoints.00
 Record size in doubles =  65536        No. of integs per rec  =  32766
 Max. records in memory =      0        Max. records in file   =  64930
 No. of bits per label  =     16        No. of bits per value  =     64


And then it stalls here for an hour or more.


Forum >> NWChem's corner >> General Topics