there must be at least two ranks per node
Just Got Here
12:56:58 AM PDT - Fri, Oct 12th 2018
Hi,

I setup a system for testing NW, I used NW6.6 on ubuntu18.04 with cuda on single platform. the env setting list as follow.

export USE_MPI=y
export USE_MPIF=y

  1. export MPI_LOC=/opt/mpich
  2. export MPI_LIB=$MPI_LOC/lib
  3. export MPI_INCLUDE=$MPI_LOC/include
  4. export LIBMPI="-lmpich -lopa -lmpl -lrt -lpthread"


export NWCHEM_TOP="/home/test/nwchem-6.6"
export NWCHEM_TARGET="LINUX64"
export ARMCI_NETWORK="MPI-PR"
export NWCHEM_MODULES="all"
export LARGE_FILES="TRUE"
export USE_NOFSCHECK="TRUE"
export FC="gfortran"
export USE_64TO32=y
export BLAS_SIZE=4
export LAPACK_SIZE=4
  1. export SCALAPACK_SIZE=4
  2. export SCALAPACK=-lsci_pgi_mp
export BLASOPT="-L/usr/lib/openblas-base -lopenblas"

export TCE_CUDA=Y
export CUDA_LIBS="-L/usr/local/cuda-10.0/lib64 -lcudart -lcublas -lstdc++"
export CUDA_FLAGS="-arch sm_50"
export CUDA_INCLUDE="-I. -I/usr/local/cuda-10.0/include"
export CUDA="nvcc"

Now, I get the error no matter what cmd I used to run sample.

cmd:
mpirun -np 2 $NWCHEM_TOP/bin/$NWCHEM_TARGET/nwchem input.nw or $NWCHEM_TOP/bin/$NWCHEM_TARGET/nwchem input.nw

error log:
[0] Received an Error in Communication: (1) there must be at least two ranks per node
application called MPI_Abort(comm=0x84000000, 1) - process 0
[unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1


Any insight for it?

Thanks

Just Got Here
1:06:26 AM PDT - Fri, Oct 12th 2018
some MPI env has commented out when build NW, I installed MPI from apt install.

Thanks

Forum Vet
9:52:17 AM PDT - Fri, Oct 12th 2018
Jim
Plesase send the output of the following commands

which mpirun
ldd $NWCHEM_TOP/bin/$NWCHEM_TARGET/nwchem
mpif90 -show

PS We do have precompiled packages for Ubuntu 18.04
https://github.com/nwchemgit/nwchem/releases/download/6.8.1-release/nwchem_6.8.1+133+gitge...
https://github.com/nwchemgit/nwchem/releases/download/6.8.1-release/nwchem-data_6.8.1+133+...

Instructions for installing them:
curl -O https://github.com/nwchemgit/nwchem/releases/download/6.8.1-release/nwchem-data_6.8.1+133+gitge032219-2_all.ubuntu_bionic.deb
curl -O https://github.com/nwchemgit/nwchem/releases/download/6.8.1-release/nwchem-data_6.8.1+133+gitge032219-2_all.ubuntu_bionic.deb
sudo apt get libscalapack-mpi-dev libopenblas-dev 
sudo dpkg -i nwchem-*6.8.1*deb

Just Got Here
1:36:14 AM PDT - Mon, Oct 15th 2018
Quote:Edoapra Oct 12th 9:52 am
Jim
Plesase send the output of the following commands

which mpirun
ldd $NWCHEM_TOP/bin/$NWCHEM_TARGET/nwchem
mpif90 -show

PS We do have precompiled packages for Ubuntu 18.04
https://github.com/nwchemgit/nwchem/releases/download/6.8.1-release/nwchem_6.8.1+133+gitge...
https://github.com/nwchemgit/nwchem/releases/download/6.8.1-release/nwchem-data_6.8.1+133+...

Instructions for installing them:
curl -O https://github.com/nwchemgit/nwchem/releases/download/6.8.1-release/nwchem-data_6.8.1+133+gitge032219-2_all.ubuntu_bionic.deb
curl -O https://github.com/nwchemgit/nwchem/releases/download/6.8.1-release/nwchem-data_6.8.1+133+gitge032219-2_all.ubuntu_bionic.deb
sudo apt get libscalapack-mpi-dev libopenblas-dev 
sudo dpkg -i nwchem-*6.8.1*deb



$ which mpirun
/usr/bin/mpirun
$ ldd $NWCHEM_TOP/bin/$NWCHEM_TARGET/nwchem
linux-vdso.so.1 (0x00007ffd3abc9000)
libopenblas.so.0 => /usr/lib/x86_64-linux-gnu/libopenblas.so.0 (0x00007fdfb4783000)
libmpichfort.so.0 => /usr/lib/x86_64-linux-gnu/libmpichfort.so.0 (0x00007fdfb454b000)
libmpich.so.0 => /usr/lib/x86_64-linux-gnu/libmpich.so.0 (0x00007fdfb4095000)
libcudart.so.10.0 => /usr/local/cuda-10.0/lib64/libcudart.so.10.0 (0x00007fdfb3e1b000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fdfb3a8d000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fdfb3885000)
libgfortran.so.4 => /usr/lib/x86_64-linux-gnu/libgfortran.so.4 (0x00007fdfb34a6000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fdfb3108000)
libmvec.so.1 => /lib/x86_64-linux-gnu/libmvec.so.1 (0x00007fdfb2ede000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fdfb2cbf000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fdfb2aa7000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fdfb26b6000)
libcr.so.0 => /usr/lib/libcr.so.0 (0x00007fdfb24ab000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fdfb22a7000)
/lib64/ld-linux-x86-64.so.2 (0x00007fdfc59f0000)
libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007fdfb2067000)
$ mpif90 -show
gfortran -I/usr/lib/x86_64-linux-gnu/openmpi/include -pthread -I/usr/lib/x86_64-linux-gnu/openmpi/lib -L/usr//lib -L/usr/lib/x86_64-linux-gnu/openmpi/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi

Forum Vet
12:44:28 PM PDT - Mon, Oct 15th 2018
wrong mpirun
Jim, since you have compiled NWChem with the MPI library installed under /opt/mpich, you must use the mpirun provided with it (most likely in /opt/mpich/bin) and not the one in /usr/bin


Forum >> NWChem's corner >> Running NWChem