post installation problem "0:0:ga orthog: hard zero:: 1 and ARMCI DASSERT fail. armci.c:ARMCI Error
Just Got Here
6:50:31 AM PST - Sun, Nov 18th 2012
Dear all,
I am facing the following problem when I have installed the NWChem-6.0. on our workstation.

1. When I have included any MKL libraries (SCALAPACK, BLAS, BLACS) in the build file, although the make file is not printing any errors, I observed the following error during the run time of the executable.

ERROR FROM THE OUTPUT FILE STARTS HERE:

    Superposition of Atomic Density Guess

------------------------------------------------------------------------

Sum of atomic energies:         -75.75081731

------------------------------------------------------------------------

ga_orthog: hard zero        1

------------------------------------------------------------------------

------------------------------------------------------------------------

 current input line :

   17: task scf

------------------------------------------------------------------------

------------------------------------------------------------------------

This error has not yet been assigned to a category

------------------------------------------------------------------------

For more information see the NWChem manual at

http://nwchemgit.github.io/index.php/NWChem_Documentation




For further details see manual section:

No section for this category



0:0:ga_orthog: hard zero:: 1
(rank:0 hostname:varahi.intranet.jncasr.ac.in pid:13050):ARMCI DASSERT fail. armci.c:ARMCI_Error():260 cond:0

ERROR FROM THE OUTPUT FILE ENDS HERE

2. By seeing the other related posts I have included the following command in the input file

set lindep:n_dep 0

but still I was facing with the same problem.

3. Surprisingly, the same version when compiled with out any MKL libraries don't create any of these problems. I have checked the paths of all the libraries and include files and everything is perfect (which could be the reason, I feel, for zero errors in the make.log file even when compiled with MKL libraries).

I am copying the build file (for the case 3, I have not included the MKL variables) and the input file.


Please suggest me what is going wrong.

Thanking you in advance,
Chaitanya.

INPUT FILE:
echo

title h2o_preliminary_calculations
start h2o_prelim
geometry
O                 -1.09473683    1.05263156    0.00000000

H                 -0.13473683    1.05263156    0.00000000

H                 -1.41519142    1.95756739    0.00000000
end
basis
* library 6-31g*
end
task scf


BUILD FILE:
export NWCHEM_TOP=/home/sharma/Softwares/nwchem-6.0_no_optimize_all_intel
export LARGE_FILES=TRUE
export ENABLE_COMPONENT=yes
export TCGRSH=/usr/bin/ssh
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES=all

export CC=/opt/intel/composer_xe_2011_sp1.10.319/bin/intel64/icc
export FC=/opt/intel/composer_xe_2011_sp1.10.319/bin/intel64/ifort

export USE_MPI=y
export USE_MPIF=y
export MPI_HOME=/opt/intel/impi/4.0.2
export MPI_LOC=$MPI_HOME
export MPI_LIB=$MPI_LOC/lib64
export MPI_INCLUDE=$MPI_LOC/include64
export LIBMPI='-lmpigf -lmpigi -lmpi_ilp64 -lmpi'MKL_HOME="/opt/intel/mkl"

MKL_LIB="$MKL_HOME/lib/intel64"
MKL_INCLUDE="$MKL_HOME/include/intel64/ilp64"
export USE_64TO32=yes
export HAS_BLAS=yes
export BLASOPT="-L$MKL_LIB -lmkl_blas95_ilp64" # the other existing blas library is libmkl_blas95_lp64
export USE_LAPACK=yes
export USE_SCALAPACK=yes
export SCALAPACK="-i8 -I$MKL_INCLUDE -L$MKL_LIB -lmkl_scalapack_ilp64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lmkl_blacs_intelmpi_ilp64 -lpthread -lm"
export LAPACK="-i8 -I$MKL_INCLUDE -L$MKL_LIB -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lpthread -lm"

Forum Vet
8:27:41 AM PST - Sun, Nov 18th 2012
The easy solution is not to use MKL libraries.

If you do want to use MKL libraries, please read the compiling section on http://nwchemgit.github.io/index.php/Compiling_NWChem#Adding_optional_environmental_variable... related to extra libraries and especially the warning note related to math libraries on 64-bit platforms. An approach is described there to use these libraries.

Bert


Quote:Chaitanya Nov 18th 1:50 pm
Dear all,
I am facing the following problem when I have installed the NWChem-6.0. on our workstation.

1. When I have included any MKL libraries (SCALAPACK, BLAS, BLACS) in the build file, although the make file is not printing any errors, I observed the following error during the run time of the executable.

ERROR FROM THE OUTPUT FILE STARTS HERE:

    Superposition of Atomic Density Guess

------------------------------------------------------------------------

Sum of atomic energies:         -75.75081731

------------------------------------------------------------------------

ga_orthog: hard zero        1

------------------------------------------------------------------------

------------------------------------------------------------------------

 current input line :

   17: task scf

------------------------------------------------------------------------

------------------------------------------------------------------------

This error has not yet been assigned to a category

------------------------------------------------------------------------

For more information see the NWChem manual at

http://nwchemgit.github.io/index.php/NWChem_Documentation




For further details see manual section:

No section for this category



0:0:ga_orthog: hard zero:: 1
(rank:0 hostname:varahi.intranet.jncasr.ac.in pid:13050):ARMCI DASSERT fail. armci.c:ARMCI_Error():260 cond:0

ERROR FROM THE OUTPUT FILE ENDS HERE

2. By seeing the other related posts I have included the following command in the input file

set lindep:n_dep 0

but still I was facing with the same problem.

3. Surprisingly, the same version when compiled with out any MKL libraries don't create any of these problems. I have checked the paths of all the libraries and include files and everything is perfect (which could be the reason, I feel, for zero errors in the make.log file even when compiled with MKL libraries).

I am copying the build file (for the case 3, I have not included the MKL variables) and the input file.


Please suggest me what is going wrong.

Thanking you in advance,
Chaitanya.

INPUT FILE:
echo

title h2o_preliminary_calculations
start h2o_prelim
geometry
O                 -1.09473683    1.05263156    0.00000000

H                 -0.13473683    1.05263156    0.00000000

H                 -1.41519142    1.95756739    0.00000000
end
basis
* library 6-31g*
end
task scf


BUILD FILE:
export NWCHEM_TOP=/home/sharma/Softwares/nwchem-6.0_no_optimize_all_intel
export LARGE_FILES=TRUE
export ENABLE_COMPONENT=yes
export TCGRSH=/usr/bin/ssh
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES=all

export CC=/opt/intel/composer_xe_2011_sp1.10.319/bin/intel64/icc
export FC=/opt/intel/composer_xe_2011_sp1.10.319/bin/intel64/ifort

export USE_MPI=y
export USE_MPIF=y
export MPI_HOME=/opt/intel/impi/4.0.2
export MPI_LOC=$MPI_HOME
export MPI_LIB=$MPI_LOC/lib64
export MPI_INCLUDE=$MPI_LOC/include64
export LIBMPI='-lmpigf -lmpigi -lmpi_ilp64 -lmpi'MKL_HOME="/opt/intel/mkl"

MKL_LIB="$MKL_HOME/lib/intel64"
MKL_INCLUDE="$MKL_HOME/include/intel64/ilp64"
export USE_64TO32=yes
export HAS_BLAS=yes
export BLASOPT="-L$MKL_LIB -lmkl_blas95_ilp64" # the other existing blas library is libmkl_blas95_lp64
export USE_LAPACK=yes
export USE_SCALAPACK=yes
export SCALAPACK="-i8 -I$MKL_INCLUDE -L$MKL_LIB -lmkl_scalapack_ilp64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lmkl_blacs_intelmpi_ilp64 -lpthread -lm"
export LAPACK="-i8 -I$MKL_INCLUDE -L$MKL_LIB -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lpthread -lm"

Just Got Here
11:43:36 PM PST - Sun, Nov 18th 2012
Dear Dr. Bert,

Thanks for such a quick reply.

Sorry to bother, but, I do want to use the MKL libraries (mainly to improve the performance) and I have followed the instructions given. I have used the environment variable "export USE_64to32=yes" as mentioned in the INSTALL file. (Also, I had built without it). Unfortunately, the results are same when I kept this environmental variable either YES or NO. I am not getting any other idea to fix this problem.

Kindly suggest some method through which I can solve this.

Thanking you a lot,
Sincerely,
Chaitanya Sharma Y.

P.S: Yesterday, I also tried of installing newest GA to see any better performance, but I was not able to install GA successfully (So, I dropped that Idea!).

Forum Regular
10:37:01 AM PST - Mon, Nov 19th 2012
Hi Chaitanya,

I think the answer is that setting some environment variables by itself is not enough. The environment variable USE_64TO32 only controls whether a library with wrapper routines is compiled or not. It does not ensure that the code is actually going to use those wrappers. To ensure the latter you will need to do an extra step in your compilation. So the general sequence of make commands becomes:

  make nwchem_config

  make 64_to_32

  make

Of course you still need to provide whatever flags you usually provide make with, but the extra "make 64_to_32" step is crucial. That step converts all BLAS and LAPACK calls to calling the wrapper library.

I hope this helps,

Huub

Just Got Here
10:55:10 AM PST - Mon, Dec 3rd 2012
Dear Dr.Huub,

Firstly, sorry for such a late reply.

I compiled as per your instructions. But, I have encountered the following error.

"Last System Error Message from Task 0:: No such file or directory
application called MPI_Abort(MPI_COMM_WORLD, 0) - process 0"

Then following the link "http://nwchemgit.github.io/Special_AWCforum/st/id301/" I have added the environmental variable
USE_MPIF4=y as suggested by Dr. Birt.

As the error was persisting even after this change, I added the "make clean" command after the "make 64_to_32" command as suggested in the link "http://www.emsl.pnl.gov/docs/nwchem/nwchem-support/2007/12/0051.Re:_NWCHEM_problem_encount..."

Even after this my problem is continuing and I couldn't run the executable without errors whenever I am including the MKL libraries.

Below, I am giving the error message printed in the output file

0:Segmentation Violation error, status=: 11
(rank:0 hostname:varahi.intranet.jncasr.ac.in pid:3466):ARMCI DASSERT fail. signaltrap.c:SigSegvHandler():301 cond:0

Please, help me.

Thanks,
Yours sincerely,
Chaitanya Sharma Y.

Just Got Here
11:15:48 PM PST - Mon, Mar 4th 2013
Dear All

I am searching every forum topic and applying the compilation options/patches but it is not taking me anywhere..

Appreciate your help in this

Himanshu

Forum Vet
9:38:52 AM PST - Tue, Mar 5th 2013
Have you followed the build instructions outlined on the webpage?

If so, please provide information about:

1. Machine you are trying to compile on.
2. Environment variables set.

Bert

Just Got Here
3:37:41 AM PST - Wed, Mar 6th 2013
Yes I have followed the steps mentioned on webpage
But i did not use "make 64_32" as while compiling it stared giving undefined reference to "ycopy_" and related functions.

I have compiled on the following machines and am experiencing same error

1) CPU: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
OS: Red Hat Enterprise Linux Server release 6.2, Linux kernel 2.6.32-220.el6.x86_64

2) CPU: Intel(R) Xeon(R) CPU X5670 @ 2.93GHz
O.S: Red Hat Enterprise Linux Server release 6.1, Linux Kernel 2.6.32-131.0.15.el6.x86_64

3) Intel MPI Version 4.0.3

4) Intel(R) Math Kernel Library Version 11.0.1 Product Build 20121009 for Intel(R) 64 architecture applications
Major version: 11
Minor version: 0
Update version: 1
Product status: Product
Build: 20121009
Processor optimization: Intel(R) Advanced Vector Extensions (Intel(R) AVX) Enabled Processor
====================================================

I have used the following configuration script

export MPI_HOME=/opt/intel/impi/4.0.3
export NWCHEM_TOP=/gpfs/fs1/bench/himanshu/NWCHEM/TMP/nwchem-6.1.1-src

export LARGE_FILES=TRUE
export ENABLE_COMPONENT=yes
export TCGRSH=/usr/bin/ssh

export NWCHEM_TARGET=LINUX64

export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y

export USE_64to32=yes
export ARMCI_NETWORK=OPENIB
export ARMCI_DEFAULT_SHMMAX=8192
export NWCHEM_MODULES=all

export MPI_LOC=$MPI_HOME
export MPI_LIB=$MPI_LOC/lib64
export MPI_INCLUDE=$MPI_LOC/include64
export LIBMPI="-O3 -xHOST -no-prec-div -funroll-loops -unroll-aggressive -lmpigf -lmpigi -lmpi_ilp64 -lmpi"


export BLASOPT="-L/opt/intel/composerxe/mkl/lib/intel64 -lmkl_blas95_ilp64"
export SCALAPACK="-i8 -I/opt/intel/composerxe/mkl/include/intel64/ilp64 -L/opt/intel/composerxe/mkl/lib/intel64 -lmkl_scalapack_ilp64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lmkl_blacs_intelmpi_ilp64 -lpthread -lm"

Forum Regular
11:24:36 AM PST - Wed, Mar 6th 2013
Himanshu,

I think your problem is easy to solve. You have set the following environment variable:

    export USE_64to32=yes

and you should have set

   export USE_64TO32

That should take care of the undefined ycopy references and such like.

Best wishes, Huub

Forum Regular
11:26:20 AM PST - Wed, Mar 6th 2013
Himanshi,

A correction (I should really read stuff before hitting submit): You should set:

    export USE_64TO32=yes

I.e. the "to" should be written in capitals.

Huub

Forum Regular
11:42:38 AM PST - Wed, Mar 6th 2013
Chaitanya,

I think your problem might be caused by issues understanding what the Intel libraries that you use provide. I found the following page on the Intel web site: http://software.intel.com/sites/products/documentation/hpc/mkl/mkl_userguide_lnx/GUID-8782... (I hope this link is generic).

This document suggests that the ILP64 libraries provide 64-bit integer interfaces whereas the LP64 libraries provide 32-bit integer interfaces. You are building the code as if you are using 32-bit integer libraries but you are actually linking the 64-bit integer libraries. So in your case I would suggest:
- Get rid of the USE_64TO32 environment variable
- Run make 32_to_64 (or if you start from a clean version of the code don't do "make 64_to_32")
- Link to the ILP64 versions of the Intel libraries
and you should be fine.

Huub


Forum >> NWChem's corner >> General Topics