armci malloc:malloc 1 failed

Gets Around

8:33:17 PM PDT - Sat, May 31st 2014

Hi,

I am using this code to calculate MR-CCSD energy of cubane transition state.

title "cubane MK-CCSD(2,2)/cc-pVDZ calculation"

scratch_dir /mnt/scratch

memory stack 100 mb heap 100 mb global 11000 mb

geometry
 symmetry C2v
 H         -1.32680        2.05167       -0.03231
 C         -0.78899        1.09343        0.03178
 H          1.32680        2.05167       -0.03231
 C          0.78899        1.09343        0.03178
 H         -1.48781       -0.00000        2.07216
 C         -1.20592        0.00000        1.01567
 H          1.48781       -0.00000        2.07216
 C          1.20592       -0.00000        1.01567
 H         -1.44937        0.00000       -1.98042
 C         -0.77946        0.00000       -1.10963
 H          1.44937        0.00000       -1.98042
 C          0.77946        0.00000       -1.10963
 H         -1.32680       -2.05167       -0.03231
 C         -0.78899       -1.09343        0.03178
 H          1.32680       -2.05167       -0.03231
 C          0.78899       -1.09343        0.03178
end

scf
 direct
end

basis spherical
 H library cc-pVDZ
 C library cc-pVDZ
end

tce
 mkccsd
 2emet 1
 freeze atomic
end

mrccdata
 root 1
 nref 2
 22222222222222222222222222220
 22222222222222222222222222202
end

task tce energy

I am using Intel(R) Core(TM) i5-4670 CPU @ 3.40GHz with 8GB memory + 120Gb swap.
SHMMAX set to 16GB (echo 16384000000 > /proc/sys/kernel/shmmax)
With openblas parallelization I'm running calculation in single process.
mpirun -np 1 nwchem N8.nw > N8.nwo

But I've got issue in computation of 2-e integrals:

MRCC tiling completed in             0.0            0.0
tce_ao1e_fock2e       36.28000       36.36079
F:     1 in bytes =                87040
tce_mo1e        0.03200        0.06773
eone,etwo,enrep,energy  -1126.356944754460    460.249162408168    358.838446347621   -307.269335998670
mrcc_uhf_energy        8.78800        8.78590
tce_ao1e_fock2e       35.68800       35.74738
F:     2 in bytes =                87040
tce_mo1e        0.02400        0.02567
eone,etwo,enrep,energy  -1125.936417042576    459.900381432998    358.838446347621   -307.197589261957
mrcc_uhf_energy        9.31200        9.33206
2-e(intermediate) /mnt/scratch/cubane. in bytes=         8159223808
Ref.   1 Half 2-e         915.42        1115.33
V 2-e /mnt/scratch/cubane. in bytes=         1437934592
0:armci_malloc:malloc 1 failed: 1437934600
(rank:0 hostname:kbob-G41MT-S2 pid:23581):ARMCI DASSERT fail. ../../ga-5-2/armci/src/memory/memory.c:PARMCI_Malloc():880 cond:0

I tried to set the environment variable ARMCI_DEFAULT_SHMMAX in the different values (4096, 16000, 16384) but nothing has changed.
No additional errors reported with 16000 value.

I tried not to use GA IO-scheme.
In this case calculation of the 2-e integrals have been successfully completed, but MRCC iterations itself fails without GA initialization.

I am using development snapshot May 03, 2014 Nwchem-6.3.revision25564-src.2014-05-03 with patch http://nwchemgit.github.io/Special_AWCforum/sp/id4530.

Can anyone help to figure out what might be the problem and show workarounds?
Thanks!

Forum Vet

3:14:12 PM PDT - Mon, Jun 2nd 2014
Vladimir What value of ARMCI_NETWORK have you used for your installation? Have you left it undefined? Thanks, Edo

Gets Around

6:55:09 PM PDT - Mon, Jun 2nd 2014

my build file

#!/bin/sh

#sudo apt-get install python2.7-dev zlib1g-dev libssl-dev gfortran
#Edit src/config/makefile.h and add "-lz -lssl" to the end of line 2094 (needed by python)

export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export PYTHONVERSION=2.7
export PYTHONHOME=/usr

export BLASOPT="-L/usr/lib/openblas-base -lopenblas"
export LIBRARY_PATH=$LIBRARY_PATH:/usr/lib/openblas-base

#sudo apt-get install libopenmpi-dev openmpi-bin
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MRCC_METHODS=y
export MPI_LOC=/usr/lib/openmpi/lib
export MPI_INCLUDE=/usr/lib/openmpi/include
export ARMCI_NETWORK=SOCKETS
export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"
export LIBRARY_PATH=$LIBRARY_PATH:/usr/lib/openmpi/lib
export FC=gfortran
cd $NWCHEM_TOP/src
make clean
make nwchem_config
make > make.log 2>&1
cd ../contrib
./getmem.nwchem

I found some note about ARMCI_NETWORK on this forum
http://nwchemgit.github.io/Special_AWCforum/sp/id1600
So in my case (with openblas parallelization I'm running mpirun -np 1 nwchem N8.nw).
Do I need to compile nwhem with mpi or can I compile without?
What value should I set for the variable ARMCI_NETWORK if I use only one node?

Forum Vet

9:15:48 AM PDT - Tue, Jun 3rd 2014
Vladimir, I suggest you to try first with ARMCI_NETWORK=MPI-TS (the default value). I have just tried you input and it works with the following memory and nproc=4 memory stack 400 mb heap 100 mb global 3100 mb The ARMCI_DEFAULT_SHMMAX story become irrelevant for ARMCI_NETWORK=MPI-TS

Gets Around

11:29:52 PM PDT - Tue, Jun 3rd 2014
Thank you very much, Edo. Your magical spell worked perfectly. But I still do not understand why the calculation do not work in a single thread.

Gets Around

8:12:37 AM PDT - Thu, Jun 5th 2014
I found another way to fix the issue. https://groups.google.com/forum/#!msg/hpctools/-bYstidUAYA/LqZ38W1f1ukJ just set -#define DEFAULT_MAX_NALLOC (41024102416) +#define DEFAULT_MAX_NALLOC (81024102416) Incidentally, nproc=1 with openblas parallelization is 7 times faster than nproc=4.

Forum Vet

9:37:25 AM PDT - Thu, Jun 5th 2014
Vladimir Thank you very much for your feedback. In order to reproduce your single processor find, I need a few more details about your setting. 1) memory line used in NWChem input file 2) value of ARMCI_NETWORK used Cheers, Edo

Gets Around

6:10:25 PM PDT - Thu, Jun 5th 2014
1. Memory line like in 1-st post: memory stack 400 mb heap 100 mb global 11000 mb 2. ARMCI_NETWORK not set (default).

Forum >> NWChem's corner >> Running NWChem