SCF memory

Just Got Here

1:30:15 AM PST - Sat, Mar 3rd 2012

Dear all,

   I need to ab intio calculations with where no. of basis functions ~ 1000 at MP2 l.o. theory with basis sets such as aug-cc-pvtz.

My hardware is a beowulf cluster with 7x intel i7 @ 3.40GHz machines with 16gb memory each.
Nwchem is compiled with openmpi. The jobs run fine in parallel, and with ~500-600 basis fns the performance is good. But with larger systems, the scf takes too long. Both direct and semidirect.

The disk I/O is very slow so i need to to direct scf calculationsfor the integral computation or semidirect with very limited disk I/O.
But direct seems very expensive

f.e. water cluster of 11 units takes 2hrs for a S.P.E. at MP2/aug-cc-pvtz on the given hardware. Of this the SCF takes 45 mins!

1. Is the time reasonable?
2. Are there some optimal values for stack, heap and global memory (other than default to maximize chacing)?

Thanks in advance!

Jonathan

Some relevant info :

The scf options were :

scf

semidirect filesize 10000000

end

mp2
freeze atomic
end

task mp2

          Memory information

          ------------------

   heap     =  129602401 doubles =    988.8 Mbytes

   stack    =  129602401 doubles =    988.8 Mbytes

   global   =  259204800 doubles =   1977.6 Mbytes (distinct from heap & stack)

   total    =  518409602 doubles =   3955.2 Mbytes

   verify   = yes

   hardfail = no

     Superposition of Atomic Density Guess

     -------------------------------------

Sum of atomic energies:        -833.58996280

Integral file          = ./scr/test.aoints.00

Record size in doubles =  65536        No. of integs per rec  =  32766

Max. records in memory =      0        Max. records in file   =   2478

No. of bits per label  =     16        No. of bits per value  =     64

#quartets = 9.885D+08 #integrals = 3.954D+10 #direct = 94.2% #cached =  5.8%

Forum Vet

1:08:24 PM PST - Wed, Mar 7th 2012
In direct mode each two-electron integral is recomputed every time it is needed. In semidirect the SCF will try to store as much as it can, either in memory or on disk. If you do not want to use disk for caching you should use "semidirect filesize 0". The calculation will try to store as much in memory as it can. The memory it uses is the stack. I see you specify about 4 Gbyte of memory per core in your input. Standard, memory will be allocated 25% heap + 25% stack + 50% global. To store more integrals in memory, you want to maximize stack. I would try reducing the heap to maybe 100 mb and up the stack to 1900 mb. You may be able to reduce the global space and move that to stack too. Alternatively, adding more processors gives you more memory to work with. In all reality, recomputing these integrals every time is expensive. One could fiddle with the threshold for the integrals, to reduce the number of integrals being recomputed if you really want. Bert Quote:Jonathan Mar 3rd 8:30 am Dear all, I need to ab intio calculations with where no. of basis functions ~ 1000 at MP2 l.o. theory with basis sets such as aug-cc-pvtz. My hardware is a beowulf cluster with 7x intel i7 @ 3.40GHz machines with 16gb memory each. Nwchem is compiled with openmpi. The jobs run fine in parallel, and with ~500-600 basis fns the performance is good. But with larger systems, the scf takes too long. Both direct and semidirect. The disk I/O is very slow so i need to to direct scf calculationsfor the integral computation or semidirect with very limited disk I/O. But direct seems very expensive f.e. water cluster of 11 units takes 2hrs for a S.P.E. at MP2/aug-cc-pvtz on the given hardware. Of this the SCF takes 45 mins! 1. Is the time reasonable? 2. Are there some optimal values for stack, heap and global memory (other than default to maximize chacing)? Thanks in advance! Jonathan Some relevant info : The scf options were : scf semidirect filesize 10000000 end mp2 freeze atomic end task mp2 Memory information ------------------ heap = 129602401 doubles = 988.8 Mbytes stack = 129602401 doubles = 988.8 Mbytes global = 259204800 doubles = 1977.6 Mbytes (distinct from heap & stack) total = 518409602 doubles = 3955.2 Mbytes verify = yes hardfail = no Superposition of Atomic Density Guess ------------------------------------- Sum of atomic energies: -833.58996280 Integral file = ./scr/test.aoints.00 Record size in doubles = 65536 No. of integs per rec = 32766 Max. records in memory = 0 Max. records in file = 2478 No. of bits per label = 16 No. of bits per value = 64 #quartets = 9.885D+08 #integrals = 3.954D+10 #direct = 94.2% #cached = 5.8%[/quote]

Just Got Here

11:39:01 AM PDT - Sun, Mar 11th 2012
Bert, Thanks. Maximizing stack does help. Will try and fiddle with threshold

Forum >> NWChem's corner >> Running NWChem