TCE ARMCI DASSERT fail in running NWChem-6.1


Click here for full thread
Just Got Here
Hello,

We have NWChem 6.1 built with intel-11.0.083 and Openmpi-1.4.2 on Infiniband (IB) cluster back in 2012. We did not see problem until recently a user tried to run TCE (I guess nobobdy use TCE in nwchem before) with large memory size (small test seems fine).

For large memory, the error message is


Last System Error Message from Task 0:: Inappropriate ioctl for device


MPI_ABORT was invoked on rank 0 in communicator MPI COMMUNICATOR 4 DUP FROM 0
with errorcode 1880870161.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.


0:0:createfile: failed ga_create size=:: 1880870161
(rank:0 hostname:hnd17 pid:25208):ARMCI DASSERT fail. ../../ga-5-1/armci/src/common/armci.c:ARMCI_Error():208 cond:0


The input file is
---
restart ni-c2h4 ccsd
 title "UCCSD calculation of NiC2H4"
echo

memory total 20000 Mb
geometry units angstroms
symmetry c2v
Ni 0.000000 0.000000 1.781951
C 0.000000 -0.700112 -0.461338
H 0.928257 1.259678 -0.576692
H -0.928257 -1.259678 -0.576692
end
BASIS spherical
(omit the basis sets here)

relativistic
 douglas-krollend
scf
 SINGLET  rohf
maxiter 300end
TCE
MAXITER 300 FREEZE core atomic
print debug SCF
diis 8 2eorb
2emet 13
split 2
CCSD(T)
END
TASK TCE ENERGY
---

Is this a TCE issue or a system IB bandwidth issue? Thanks.