Compiling NWChem 6.3 on BlueGene/Q


Clicked A Few Times
Hi,

We have been having problems with compiling 6.3 on the BlueGene/Q platform using MPI-TS as advised on http://nwchemgit.github.io/index.php/Compiling_NWChem.

It appears to be a GA problem as simple MD runs that don't use GA run OK, but other tests fail along the lines of:

MA_verify_allocator_stuff: starting scan ...
stack block 'gai_diag_std:z', handle 1, address 0x11ae499bb8:
current right signature 1060908322 != proper right signature 1431655765
stack block '?Q¥Ò£Ó?GÍX?', handle 0, address 0x11ae4c4898:
current checksum 13660927947873495360 != stored checksum 4563766883317136985

We get this error with any number of processors including nproc=1.

The environment variables set before compilation were:

export NWCHEM_TOP=/gpfs/ .... /nwchem-6.3
export LARGE_FILES TRUE
export USE_NOFSCHECK TRUE
export LIB_DEFINES="-DDFLT_TOT_MEM=16777216"
export NWCHEM_TARGET=BGQ
export NWCHEM_MODULES="all"
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_INCLUDE="/bgsys/drivers/ppcfloor/comm/xl/include"
export LIBMPI=" "
export ARMCI_NETWORK=MPI-TS
export DISABLE_GAMIRROR=y

We have also tried using ARMCI-MPI instead of the built-in version but get similar problems.

Has anyone encountered an error of this kind before on BGQ?

Tom

Forum Vet
Tom
Are these all the env. variables you set? There are quite a few more we listed at the URL you quote.

Clicked A Few Times
Hi Edo

Thanks for looking into this. As far as I can see the only variables missing are those for using external BLAS libraries (BLASOPT, BLAS_LIB, BLAS_SIZE, USE_64TO32). Is it essential to use external BLAS on BGQ? We assumed it would be safer to start with the internal routines but perhaps that was a mistake.

Tom

Forum Vet
Tom
I am not 100% sure what happened in your installation.
It would be better if you can send me the output of the last compilation step (i.e. make link) and the files

$NWCHEM_TOP/src/tools/build.log

$NWCHEM_TOP/src/tools/armci/build.log

Clicked A Few Times
Hi Edo,

Just to update we had more success when linking to ESSL with 64TO32 as recommended. For whatever reason it seems the internal libraries are not compatible.

Thanks for your help,

Tom

Forum Regular
Just a comment. We have had problems in the past with the internal libraries and linkers that found something else. On the link line the internal libraries are specified as -lblas -llapack etc. In some cases this led the linker to find system BLAS and LAPACK libraries rather than the ones we ship with the code. If that happens that does indeed lead to compatibility problems.


Forum >> NWChem's corner >> Compiling NWChem