Hello All,
I just started playing around with EOM-CCSD for excitation energies with NWChem 6.5. I have a smallish test system that just won't finish, apparently due to out-of-memory. Here's the input file I'm trying to run:
START phpyr
TITLE "PhPyr EOM-CCSD"
memory stack 3000 mb heap 600 mb global 3400 mb
GEOMETRY
C 0.00000 0.00000 -3.20391
C 1.20789 0.00000 -2.49014
C 1.21375 0.00000 -1.08902
C 0.00000 0.00000 -0.36623
C -1.21375 0.00000 -1.08902
C -1.20789 0.00000 -2.49014
N 0.00000 0.00000 1.05365
C 1.12627 0.00000 1.87026
C 0.71524 0.00000 3.19179
C -0.71524 0.00000 3.19179
C -1.12627 0.00000 1.87026
H 0.00000 0.00000 -4.30386
H 2.16819 0.00000 -3.02830
H 2.17635 0.00000 -0.56155
H -2.17635 0.00000 -0.56155
H -2.16819 0.00000 -3.02830
H 2.13532 0.00000 1.45205
H 1.37744 0.00000 4.06507
H -1.37744 0.00000 4.06507
H -2.13532 0.00000 1.45205
END
BASIS spherical
* library def2-SVP
END
SCF
SINGLET
RHF
thresh 1.0e-10
tol2e 1.0e-10
END
TCE
CCSD
2eorb
eomsol 2
diis 5
tilesize 10
attilesize 30
DIPOLE
FREEZE CORE ATOMIC
NROOTS 3
TARGETSYM a1
SYMMETRY a1
END
set tce:thresheom 1.0d-4
set tce:threshl 1.0d-3
TASK TCE ENERGY
I have played around with different settings for "tilesize" and the partitioning of the memory, without success. The calculation always crashes at some point. With the above, it goes the furthest, starting to solve for "EOM-CCSD left-hand side iterations", crashing after iteration 13 with:
Iteration 12 using 36 trial vectors
0.0003636340752 0.1986052891396 5.40433
0.0007455083161 0.2456298542250 6.68393
0.0019875161423 0.2545313238981 6.92615 45.9 46.7
Iteration 13 using 39 trial vectors
Application 2467260 exit codes: 134
Application 2467260 exit signals: Killed
Application 2467260 resources: utime ~2320s, stime ~49s, Rss ~38212, inblocks ~3087249, outblocks ~9166261
In stdout I get errors like:
PE 284 [Wed Feb 25 04:07:16 2015] [c6-0c1s3n0] [nid01228] LIBDMAPP ERROR: Unsuccessful memory registration.
nwchem: ../../ga-5-3/comex/src-dmapp/comex.c:890: _comex_malloc_local: Assertion `status == DMAPP_RC_SUCCESS' failed.
If someone who knows the code and module much better than me could confirm that the calcs really are supposed to need this much memory, I can stop trying to play around, because then any "real" systems will be way beyond current resources. Even better, any suggestions on how to get CCSD excitation energies with less memory would be great!
The hardware I have access to is a Cray XC40, and I can run on 42 nodes with 24 cores and 64 GB of memory per node. The above example was run on 42 nodes utilising only 8 cores per node in order to have 8 GB/core, but that is of course not optimal. I'd rather run these types of calcs on 1008 cores with 2.6 GB/core, if at all possible.
Any and all help appreciated.
|