I'm still stuck here, now at 16 ranks over 8 nodes. Each node has 64 GB, so I've set memory to
memory total 32000 mb global 30000 mb stack 500 mb
based on 8-rank usage with 1 rank/node, which was
heap 803 MB, stack 414 MB, global 13.2 GB, files 13.2 GB -> 27.6 GB/rank
I also set ARMCI_DEFAULT_SHMMAX to 2048. However, I'm still getting
0: error ival=4
(rank:0 hostname:n2149 pid:29429):ARMCI DASSERT fail. ../../ga-5.6.5/armci/src/devices/openib/openib.c:armci_call_data_server():2209 cond:(pdscr->status==IBV_WC_SUCCESS)
2: error ival=4
(rank:2 hostname:n2146 pid:19600):ARMCI DASSERT fail. ../../ga-5.6.5/armci/src/devices/openib/openib.c:armci_call_data_server():2209 cond:(pdscr->status==IBV_WC_SUCCESS)
...
FWIW, I don't understand this statement from the documentation:
Quote:
MEMORY
This is a start-up directive that allows the user to specify the amount of memory PER PROCESSOR CORE that NWChem can use for the job.
What if I'm using only 2 cores and no threads--then is it memory/rank, or still just total memory segment size divided by core count? Does threading affect this?
Any suggestions? Thanks
|