How to define the optimum value of ARMCI DEFAULT SHMMAX?
Clicked A Few Times
10:12:49 PM PDT - Thu, Mar 24th 2011
Hi,

I compiled the nwchem-6.0 with Intel 11.x compilers, OpenMPI/1.4.2 and OPENIB flag to use it on the Infiniband em64t cluster. I found the script 'getmem.newchem' in nwchem-6.0/contrib subdirecotry, which was handy to define the preprocessor memory-related directive "-DDFLT_TOT_MEM=value".

However, neither installation nor users' guide has information on how to define the optimum value of ARMCI_DEFAULT_SHMMAX, which must be specified in run-time. Can somebody instruct me how to define the optimum value for ARMCI_DEFAULT_SHMAX ?

Regards,

Ivan Rostov
ANU Supercomputer Facility
Leonard Huxley Bldg #56
The Australian National University
Canberra ACT 0200 Australia

Forum Vet
2:01:33 PM PDT - Tue, Mar 29th 2011
Answer
You do not have to define it to run. It will use the default value. Global arrays use shared memory (which is what is used for the global portion of the memory for fast access to data as it is pinned. If the shared memory needed by the global arrays exceeds the shared block it will simply allocate another block. You can increase the initial block size (which is in megabytes) by setting ARMCI_DEFAULT_SHMMAX to 2048 or larger, but remember this memory will be pinned, used or not.

Bert

Clicked A Few Times
8:20:44 PM PDT - Tue, Mar 29th 2011
then a question what is the default value?
The doqmtests pass if I specify ARMCI_DEFAULT_SHMMAX=128, when running on one or several SMP nodes of 8 cores each (SMP RAM 24GB, though /proc/sys/kernel/shmmax value is 17179869184 or 16GB). Without the explicit specification, they fail.

Forum Vet
11:16:09 AM PDT - Fri, Apr 1st 2011
Depends on how it is compiled. What NWCHEM_TARGET are you using?

Quote:Rostov Mar 30th 3:20 am
The doqmtests pass if I specify ARMCI_DEFAULT_SHMMAX=128, when running on one or several SMP nodes of 8 cores each (SMP RAM 24GB, though /proc/sys/kernel/shmmax value is 17179869184 or 16GB). Without the explicit specification, they fail.


Forum >> NWChem's corner >> Compiling NWChem