Junk.xxx files keep rewriting


Clicked A Few Times
Hi,

I have some problem with the junk.xxx files in scratch directory.
I am using NWChem 6.6 with all patches applied for a NEB calculation. Even though I specified USE_NOFSCHECK=TRUE, there are still a lot of junk.xxx files writing/rewriting, which put a heavy I/O burden on cluster. Is it possible I can get rid of them?

    • environment setup**
export NWCHEM_TOP=/gpfs/home/lhan/software/nwchem-6.6-handy
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all"
export NWCHEM_MPIF_WRAP="/gpfs/software/intel_2016_update1/impi/5.1.2.150/intel64/bin/mpiifort"
export NWCHEM_MPIC_WRAP="/gpfs/software/intel_2016_update1/impi/5.1.2.150/intel64/bin/mpiicc"
export NWCHEM_MPICXX_WRAP="/gpfs/software/intel_2016_update1/impi/5.1.2.150/intel64/bin/mpiicpc"
export NWCHEM_LONG_PATHS=Y
export USE_NOFSCHECK=TRUE
export USE_NOIO=TRUE
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y

export FC=ifort
export CC=icc
export CXX=icpc
export ARMCI_NETWORK=OPENIB
export MSG_COMMS=MPI
export IB_INCLUDE="/usr/include/infiniband"
export IB_LIB="/usr/lib64"
export IB_LIB_NAME="-libverbs"

export INTEL_MKL_DIR=/gpfs/software/intel_2016_update1/mkl/lib/intel64
export MKLINC="/gpfs/software/intel_2016_update1/mkl/include/intel64/ilp64 /gpfs/software/intel_2016_update1/mkl/include"
export BLASOPT="$INTEL_MKL_DIR/libmkl_lapack95_ilp64.a $INTEL_MKL_DIR/libmkl_scalapack_ilp64.a -Wl,--start-group $INTEL_MKL_DIR/libmkl_intel_ilp64.a $INTEL_MKL_DIR/libmkl_core.a $INTEL_MKL_DIR/libmkl_intel_thread.a $INTEL_MKL_DIR/libmkl_blacs_intelmpi_ilp64.a -Wl,--end-group /gpfs/software/intel_2016_update1/compilers_and_libraries/linux/lib/intel64/libiomp5.a -lpthread -lm"
export BLAS_SIZE=8
export BLAS_LIB="-Wl,--start-group $INTEL_MKL_DIR/libmkl_intel_ilp64.a $INTEL_MKL_DIR/libmkl_core.a $INTEL_MKL_DIR/libmkl_intel_thread.a -Wl,--end-group -lpthread -lm"
export LAPACK_SIZE=8
export LAPACK_LIB="$INTEL_MKL_DIR/libmkl_lapack95_ilp64.a"
export USE_SCALAPACK=y
export SCALAPACK_SIZE=8
export SCALAPACK_LIB="$INTEL_MKL_DIR/libmkl_scalapack_ilp64.a $INTEL_MKL_DIR/libmkl_blacs_intelmpi_ilp64.a"

The task script: (This is actually for benchmark, because my configuration contains ~100 atoms)
start h2o-neb
scratch_dir /gpfs/projects/lhan/nwchem/test/scratch/
memory total 2000 mb

geometry nocenter noautosym noautoz
O 0.00000000 -0.02293938 0.00000000
H 0.00000000 0.55046969 0.75406534
H 0.00000000 0.55046969 -0.75406534
end

geometry endgeom nocenter noautosym noautoz
O 0.00000000 0.02293938 0.00000000
H 0.00000000 -0.55046969 0.75406534
H 0.00000000 -0.55046969 -0.75406534
end

        1. Gaussian DFT ####
basis
  • library 6-31G
end
set dft:smear_sigma double 0.001d0
dft
 direct
xc pbe0
maxiter 5000
convergence damp 35 ncydp 0 dampon 1d9 dampoff 1d-4 lshift 0
end

neb
 nbeads 20
kbeads 1.0
maxiter 60
stepsize 0.10
algorithm 0
print_shift 1
end
task dft neb ignore


Thank you so much!

Longtao

Forum Vet
Compiling the util/ directory with USE_NOIO=TRUE should make any junk.xxx file to not appear.
Are you sure that NWChem was compiled with USE_NOIO=TRUE?

Clicked A Few Times
Quote:Edoapra Feb 17th 3:03 pm
Compiling the util/ directory with USE_NOIO=TRUE should make any junk.xxx file to not appear.
Are you sure that NWChem was compiled with USE_NOIO=TRUE?


I didn't do anything specific in util/ directory, just under src/:
make nwchem_config
make
then use ./getmem.nwchem

I put "export USE_NOIO=TRUE" and "export USE_NOFSCHECK=TRUE" in my .bashrc file.

Forum Vet
The only recommendation I am able to give you is to modify the value of scratch_dir to a filesystem that is not shared across the cluster nodes (e.g. /tmp/ or /run/shm)

Clicked A Few Times
Quote:Edoapra Feb 17th 5:54 pm
The only recommendation I am able to give you is to modify the value of scratch_dir to a filesystem that is not shared across the cluster nodes (e.g. /tmp/ or /run/shm)


Hi Edo,

That doesn't work for me. But I managed to suppress those junk files by adding "grid nodisk" in the dft task block. And at the same time, there is a great speedup in both CPU time and of course wall time.

Still, thank you for your help!

Longtao


Forum >> NWChem's corner >> Running NWChem