Frequency Analysis Grid Density Problem
Clicked A Few Times
8:12:00 AM PDT - Mon, Aug 27th 2018
Dear NWChem developers and users,

We are currently investigating tin gadolinium clusters and have run into troubles regarding the frequency analysis.
The clusters were previousyl geometry optimized and converged properly, we are thus able to use the movecs files as input for the frequency analysis. However the calculations print the following in the output file:

  convergence    iter        energy       DeltaE   RMS-Dens  Diis-err    time
---------------- ----- ----------------- --------- --------- ---------  ------
d=70,ls=0.0,diis     1  -3766.7449285324 -1.09D+04  9.84D-06  1.60D-06   157.4
                                                     1.17D-05  2.25D-06
Grid integrated density:     343.999999006691
Requested integration accuracy:   0.10E-13
d= 0,ls=0.0,diis     2  -3766.7449284587  7.37D-08  6.89D-06  1.79D-06   300.8
                                                     8.15D-06  2.40D-06
Grid integrated density:     343.999999006660
Requested integration accuracy:   0.10E-13
d= 0,ls=0.0,diis     3  -3766.7449283781  8.06D-08  4.56D-07  2.39D-06   444.6
                                                     2.82D-06  3.16D-06
Grid integrated density:     343.999999006664
Requested integration accuracy:   0.10E-13
d=70,ls=0.0,diis     4  -3766.7449285428 -1.65D-07  2.14D-07  1.35D-06   587.5
                                                     2.43D-07  1.74D-06
Grid integrated density:     343.999999006664
Requested integration accuracy:   0.10E-13
d= 0,ls=0.0,diis     5  -3766.7449285465 -3.72D-09  1.47D-07  1.34D-06   730.3
                                                     1.43D-07  1.73D-06
Grid integrated density:     343.999999006663
Requested integration accuracy:   0.10E-13
d= 0,ls=0.0,diis     6  -3766.7449285501 -3.66D-09  3.81D-08  1.33D-06   873.1
                                                     7.22D-08  1.73D-06
Grid integrated density:     343.999999006664
Requested integration accuracy:   0.10E-13
d= 0,ls=0.0,diis     7  -3766.7449285505 -3.62D-10  2.00D-08  1.33D-06  1015.8
                                                     2.39D-08  1.72D-06
Grid integrated density:     343.999999006664
Requested integration accuracy:   0.10E-13
d= 0,ls=0.0,diis     8  -3766.7449285506 -7.82D-11  4.14D-09  1.33D-06  1158.5
                                                     5.10D-09  1.72D-06
Grid integrated density:     343.999999006664
Requested integration accuracy:   0.10E-13
d= 0,ls=0.0,diis     9  -3766.7449285506 -5.46D-12  1.56D-09  1.33D-06  1301.6
                                                     2.00D-09  1.72D-06
Grid integrated density:     343.999999006664
Requested integration accuracy:   0.10E-13
d= 0,ls=0.0,diis    10  -3766.7449285506 -5.46D-12  8.20D-10  1.33D-06  1444.4
                                                     1.57D-09  1.72D-06
Grid integrated density:     343.999999006664
Requested integration accuracy:   0.10E-13
d= 0,ls=0.0,diis    11  -3766.7449285506 -1.27D-11  1.23D-10  1.33D-06  1587.1
                                                     2.72D-10  1.72D-06
Grid integrated density:     343.999999006664
Requested integration accuracy:   0.10E-13
  Singularity in Pulay matrix. Error and Fock matrices removed.
  Singularity in Pulay matrix. Error and Fock matrices removed.
d= 0,ls=0.0,diis    12  -3766.7449285506  2.55D-11  5.96D-09  1.33D-06  1729.8
                                                     1.02D-08  1.72D-06
Grid integrated density:     343.999999006664
Requested integration accuracy:   0.10E-13
  Singularity in Pulay matrix. Error and Fock matrices removed.
  Singularity in Pulay matrix. Error and Fock matrices removed.
  Singularity in Pulay matrix. Error and Fock matrices removed.
  Singularity in Pulay matrix. Error and Fock matrices removed.
d= 0,ls=0.0,diis    13  -3766.7449285506 -3.09D-11  1.71D-09  1.33D-06  1872.6
                                                     1.79D-09  1.72D-06
Grid integrated density:     343.999999006664
Requested integration accuracy:   0.10E-13
  Singularity in Pulay matrix. Error and Fock matrices removed.
d= 0,ls=0.0,diis    14  -3766.7449285506  3.82D-11  2.90D-09  1.33D-06  2015.3
                                                     3.13D-09  1.72D-06
Grid integrated density:     343.999999006664
Requested integration accuracy:   0.10E-13
d= 0,ls=0.0,diis    15  -3766.7449285506 -1.82D-11  1.80D-09  1.33D-06  2158.0
                                                     2.81D-09  1.72D-06
Grid integrated density:     343.999999006664
Requested integration accuracy:   0.10E-13
d= 0,ls=0.0,diis    16  -3766.7449285506 -2.36D-11  6.90D-10  1.33D-06  2300.8
                                                     6.52D-10  1.72D-06
Grid integrated density:     343.999999006664
Requested integration accuracy:   0.10E-13
d= 0,ls=0.0,diis    17  -3766.7449285506  3.82D-11  4.16D-10  1.33D-06  2443.6
                                                     5.13D-10  1.72D-06


From what I read on other posts, this does not seem to be an error per se. However since the calculations do not converge at any point, it is not possible for us to calculate the frequencies.

We tried using heigher grid values similar to the trouble with negative frequencies:

grid lebedev sn 250 16 gd 750 16



We also tried using hartree fock as initial guess. What is the reason for this output? Is there a way to fix this?

Thank you in advance!

Forum Vet
10:49:35 AM PDT - Mon, Aug 27th 2018
You might want to try to tighten the accuracy on the 2-el integrals, i.e.

tolerances acccoul 15

https://github.com/nwchemgit/nwchem/wiki/Density-Functional-Theory-for-Molecules#tolerance...

Clicked A Few Times
5:04:07 AM PDT - Tue, Aug 28th 2018
Hello and thank you for your help. I tried the following combinations in the dft block:

grid lebedev sn 250 16 gd 750 16
tolerances acccoul 15




grid lebedev sn 250 16 gd 750 16
tolerances acccoul 30


I also submitted jobs without the enhanced grid, however all calculations show the same output and do not converge.

I noticed that the numerical frequency calculations of the same structure sometimes do not show this issue, but in some cases they produce negative frequencies. Since the structures are the results of global optimizations, that is probably again a grid issue. Are there other options I can try?

Forum Vet
10:15:10 AM PDT - Tue, Aug 28th 2018
If you can provide one of the troublesome input file, it would be easier to analyze this problem

Forum Vet
10:15:16 AM PDT - Tue, Aug 28th 2018
If you can provide one of the troublesome input files, it would be easier to analyze this problem

Clicked A Few Times
3:01:22 PM PDT - Tue, Aug 28th 2018
Hello, yes glady! This the inputfile:

start Sn14Gd_1SepFrequencies

memory total 32000 mb
charge 0

geometry units angstrom noautoz noautosym
 Sn                    0.39245521    -2.25209614     2.53405778
 Sn                    0.98748424     0.79467877     3.01355923
 Sn                   -2.00760994    -0.26633228     2.56870216
 Sn                   -1.39800294     2.51972248     1.62154797
 Sn                    2.85372281    -1.11918236     0.90389702
 Sn                    0.87255849    -3.28865651    -0.26593901
 Sn                    1.51553310     3.00928744     0.78461644
 Gd                    0.18911067     0.04829527     0.01460642
 Sn                   -1.98561955    -2.59918451     0.62175492
 Sn                   -2.80882737     0.58933352    -0.70036555
 Sn                    1.99846385     1.96892603    -2.02390227
 Sn                   -0.78781388     3.09272843    -1.28070329
 Sn                    1.91117113    -1.19780389    -2.36690131
 Sn                   -1.18026671    -2.05209611    -2.25676053
 Sn                   -0.57389734     0.74605051    -3.17030322
end

basis

Sn library def2-tzvpp

Gd S
  70672.982000      0.13500000000E-03
  10580.420000      0.10060000000E-02
  2467.2196000      0.46050000000E-02
  710.30550000      0.13577000000E-01
  223.69340000      0.20915000000E-01
Gd S
  42.884500000       1.0000000000
Gd S
  30.595200000       1.0000000000
Gd S
  15.792300000       1.0000000000
Gd S
  3.9598000000       1.0000000000
Gd S
  2.0077000000       1.0000000000
Gd S
 0.74940000000       1.0000000000
Gd S
 0.32410000000       1.0000000000
Gd S
 0.57800000000E-01   1.0000000000
Gd S
 0.23700000000E-01   1.0000000000
Gd P
  1547.5176828     -0.27484377768E-03
  341.04355363     -0.22027518871E-02
  100.44808811     -0.74993330072E-02
  25.638724200     -0.75632911948E-01
  15.897630316      0.24448929000
  3.5643308647     -0.26400163004
Gd P
  8.3793352896     -0.11016848999
  4.7567465018     -0.24310255997
  2.1155437701     -0.18589755998
Gd P
  1.2378242468       1.0000000000
Gd P
 0.57874986425       1.0000000000
Gd P
 0.25295715243       1.0000000000
Gd P
 0.78345762145E-01   1.0000000000
Gd P
 0.28325755449E-01   1.0000000000
Gd D
  406.09940000      0.65500000000E-03
  122.42050000      0.53410000000E-02
  46.367900000      0.22180000000E-01
  20.728900000      0.29630000000E-01
  7.9773000000      0.28827000000
  4.1175000000      0.44967800000
Gd D
  2.0783000000       1.0000000000
Gd D
 0.98880000000       1.0000000000
Gd D
 0.33890000000       1.0000000000
Gd D
 0.10370000000       1.0000000000
Gd F
  128.70520000      0.39700000000E-02
  46.564700000      0.38735000000E-01
  20.888200000      0.13395800000
  9.5233000000      0.26283500000
  4.3476000000      0.35391100000
Gd F
  1.9241000000       1.0000000000
Gd F
 0.79260000000       1.0000000000
Gd F
 0.28550000000       1.0000000000
Gd G
  12.060950000      0.12125000000
  2.7404500000      0.60918000000
Gd G
 0.47570000000       1.0000000000
end

ecp

Sn library def2-ecp

 Gd nelec 28
Gd ul
    2          1.00000000    0.00000000   
Gd S 
    2         24.60215100  637.20086900   
Gd P
    2         16.88925000  261.68960100   
Gd D
    2         13.64335800  106.85653300   
Gd F
    2         24.12691700  -50.68359000   
Gd G
    2         22.13188700  -27.57963000   
end

dft
 vectors input test.movecs output dft.movecs
 tolerances acccoul 30
 grid lebedev sn 250 16 gd 750 16
 xc pbe0
 direct grid xfine
 mult 7
 vectors input test.movecs output dft.movecs
 convergence ncydp 0 dampon 1d99 dampoff 1d-7 damp 70
 iterations 1750
#print "final evals" "final vectors"
 noio
#smear 0.001
end

set int:txs:limxmem               32000000
task dft frequencies

Forum Vet
9:35:48 AM PDT - Wed, Aug 29th 2018
This input gives all positive frequencies in 6.8.1
start Sn14Gd_1SepFrequencies

memory stack 4250 mb heap 100 mb global 3625 mb noverify

geometry units angstrom 
 Sn                    0.39245521    -2.25209614     2.53405778
 Sn                    0.98748424     0.79467877     3.01355923
 Sn                   -2.00760994    -0.26633228     2.56870216
 Sn                   -1.39800294     2.51972248     1.62154797
 Sn                    2.85372281    -1.11918236     0.90389702
 Sn                    0.87255849    -3.28865651    -0.26593901
 Sn                    1.51553310     3.00928744     0.78461644
 Gd                    0.18911067     0.04829527     0.01460642
 Sn                   -1.98561955    -2.59918451     0.62175492
 Sn                   -2.80882737     0.58933352    -0.70036555
 Sn                    1.99846385     1.96892603    -2.02390227
 Sn                   -0.78781388     3.09272843    -1.28070329
 Sn                    1.91117113    -1.19780389    -2.36690131
 Sn                   -1.18026671    -2.05209611    -2.25676053
 Sn                   -0.57389734     0.74605051    -3.17030322
end

basis spherical

Sn library def2-tzvpp

Gd S
  70672.982000      0.13500000000E-03
  10580.420000      0.10060000000E-02
  2467.2196000      0.46050000000E-02
  710.30550000      0.13577000000E-01
  223.69340000      0.20915000000E-01
Gd S
  42.884500000       1.0000000000
Gd S
  30.595200000       1.0000000000
Gd S
  15.792300000       1.0000000000
Gd S
  3.9598000000       1.0000000000
Gd S
  2.0077000000       1.0000000000
Gd S
 0.74940000000       1.0000000000
Gd S
 0.32410000000       1.0000000000
Gd S
 0.57800000000E-01   1.0000000000
Gd S
 0.23700000000E-01   1.0000000000
Gd P
  1547.5176828     -0.27484377768E-03
  341.04355363     -0.22027518871E-02
  100.44808811     -0.74993330072E-02
  25.638724200     -0.75632911948E-01
  15.897630316      0.24448929000
  3.5643308647     -0.26400163004
Gd P
  8.3793352896     -0.11016848999
  4.7567465018     -0.24310255997
  2.1155437701     -0.18589755998
Gd P
  1.2378242468       1.0000000000
Gd P
 0.57874986425       1.0000000000
Gd P
 0.25295715243       1.0000000000
Gd P
 0.78345762145E-01   1.0000000000
Gd P
 0.28325755449E-01   1.0000000000
Gd D
  406.09940000      0.65500000000E-03
  122.42050000      0.53410000000E-02
  46.367900000      0.22180000000E-01
  20.728900000      0.29630000000E-01
  7.9773000000      0.28827000000
  4.1175000000      0.44967800000
Gd D
  2.0783000000       1.0000000000
Gd D
 0.98880000000       1.0000000000
Gd D
 0.33890000000       1.0000000000
Gd D
 0.10370000000       1.0000000000
Gd F
  128.70520000      0.39700000000E-02
  46.564700000      0.38735000000E-01
  20.888200000      0.13395800000
  9.5233000000      0.26283500000
  4.3476000000      0.35391100000
Gd F
  1.9241000000       1.0000000000
Gd F
 0.79260000000       1.0000000000
Gd F
 0.28550000000       1.0000000000
Gd G
  12.060950000      0.12125000000
  2.7404500000      0.60918000000
Gd G
 0.47570000000       1.0000000000
end

ecp

Sn library def2-ecp

 Gd nelec 28
Gd ul
    2          1.00000000    0.00000000   
Gd S 
    2         24.60215100  637.20086900   
Gd P
    2         16.88925000  261.68960100   
Gd D
    2         13.64335800  106.85653300   
Gd F
    2         24.12691700  -50.68359000   
Gd G
    2         22.13188700  -27.57963000   
end

dft
 vectors input test.movecs output dft.movecs
 tolerances acccoul 14
 grid fine
 xc pbe0
 grid fine
 mult 7
 vectors input atomic output gdft.movecs
 convergence energy 1d-7 ncydp 0 dampon 1d99 dampoff 1d-4 damp 45
 iterations 1750
#print "final evals" "final vectors"
 noio
smear 
end
set int:txs:limxmem               32000000
task dft optimize
dft
grid huge
end

task dft frequencies

Clicked A Few Times
4:24:59 AM PDT - Fri, Aug 31st 2018
Thank you for your help, I am currently running the calculation on my test HPC. It may take some while.
I will report back once the calculation is finished.

Clicked A Few Times
3:00:44 AM PDT - Thu, Sep 13th 2018
Sorry for the late reply, I tested the input you provided and it results in the following error message:

[/code]
    --------------------------

    Expectation value of S2:  

    --------------------------

     <S2> =     20.0760 (Exact =    20.0000)





center of mass

--------------

x =  -0.00121958 y =   0.00066495 z =   0.00000000



moments of inertia (a.u.)

------------------

      47526.270544724408          40.461379525740           0.000000000000

         40.461379525740       39191.347848665857           0.000000000000

          0.000000000000           0.000000000000       41696.548163056184



    Multipole analysis of the density

    ---------------------------------



    L   x y z        total         alpha         beta         nuclear

    -   - - -        -----         -----         ----         -------

    0   0 0 0      0.000000   -176.000000   -168.000000    344.000000



    1   1 0 0      0.147169     -2.096083      1.105096      1.138157

    1   0 1 0      0.027464      0.145898     -0.050116     -0.068317

    1   0 0 1      0.000000      0.000000      0.000000      0.000000



    2   2 0 0   -280.729033  -1705.654518  -1688.015021   3112.940505

    2   1 1 0      0.873793      1.573177      6.920267     -7.619651

    2   1 0 1      0.000000      0.000000      0.000000      0.000000

    2   0 2 0   -285.708611  -2484.369813  -2469.187327   4667.848528

    2   0 1 1      0.000000      0.000000      0.000000      0.000000

    2   0 0 2   -284.451180  -2240.240963  -2244.585211   4200.374995



 texas integral default override: limxmem =              32000000

 texas integral default override: limxmem =              32000000



HESSIAN: the one electron contributions are done in     197.9s



 texas integral default override: limxmem =              32000000

 texas integral default override: limxmem =              32000000

 texas integral default override: limxmem =              32000000



HESSIAN: 2-el 1st deriv. term done in                 14454.3s



 texas integral default override: limxmem =              32000000



HESSIAN: 2-el 2nd deriv. term done in                 33663.5s



 stpr_wrt_fd_from_sq: overwrite of existing file:./Sn14Gd_1SepFrequencies.hess

stpr_wrt_fd_dipole: overwrite of existing file

./Sn14Gd_1SepFrequencies.fd_ddipole



HESSIAN: the two electron contributions are done in   54094.7s



 texas integral default override: limxmem =              32000000

 texas integral default override: limxmem =              32000000

                               NWChem CPHF Module

                               ------------------





 texas integral default override: limxmem =              32000000

 scftype          =     UHF 

 nalpha           =      175

 nbeta            =      169

 variables        =   216702

 # of vectors     =       45

 tolerance        = 0.10D-03

 level shift      = 0.00D+00

 max iterations   =       50

 max subspace     =      450



Grid integrated density:     344.000000224982

Requested integration accuracy:   0.10E-13

SCF residual:   0.115998573963724     

------------------------------------------------------------------------

cphf_solve2:SCF residual greater than 1d-2                   0

------------------------------------------------------------------------

cphf_solve2:SCF residual greater than 1d-2                   0

cphf_solve2:SCF residual greater than 1d-2                   0

cphf_solve2:SCF residual greater than 1d-2                   0

cphf_solve2:SCF residual greater than 1d-2                   0

------------------------------------------------------------------------

cphf_solve2:SCF residual greater than 1d-2                   0

------------------------------------------------------------------------

------------------------------------------------------------------------

cphf_solve2:SCF residual greater than 1d-2                   0

------------------------------------------------------------------------

cphf_solve2:SCF residual greater than 1d-2                   0

------------------------------------------------------------------------

------------------------------------------------------------------------

------------------------------------------------------------------------

------------------------------------------------------------------------

 current input line : 

 current input line : 

 current input line : 

------------------------------------------------------------------------

 current input line : 

    0: 

    0: 

    0: 

------------------------------------------------------------------------

    0: 

------------------------------------------------------------------------

------------------------------------------------------------------------

------------------------------------------------------------------------

------------------------------------------------------------------------

------------------------------------------------------------------------

------------------------------------------------------------------------

------------------------------------------------------------------------

This error has not yet been assigned to a category

This error has not yet been assigned to a category

This error has not yet been assigned to a category

This error has not yet been assigned to a category

------------------------------------------------------------------------

------------------------------------------------------------------------

For more information see the NWChem manual at 

------------------------------------------------------------------------

For more information see the NWChem manual at 

------------------------------------------------------------------------

------------------------------------------------------------------------

For more information see the NWChem manual at 

For more information see the NWChem manual at 

http://nwchemgit.github.io/index.php/NWChem_Documentation

http://nwchemgit.github.io/index.php/NWChem_Documentation



http://nwchemgit.github.io/index.php/NWChem_Documentation



http://nwchemgit.github.io/index.php/NWChem_Documentation








For further details see manual section: 

For further details see manual section: 

No section for this category                                                   

For further details see manual section: 

No section for this category                                                   

No section for this category                                                   

                                                                               

For further details see manual section: 

No section for this category                                                   

                                                                               

 current input line :
4:4:cphf_solve2:SCF residual greater than 1d-2:: -1
                  

    0:
6:6:cphf_solve2:SCF residual greater than 1d-2:: -1
5:5:cphf_solve2:SCF residual greater than 1d-2:: -1
(rank:4 hostname:node103 pid:1131):ARMCI DASSERT fail. ../../ga-5-4/armci/src/common/armci.c:ARMCI_Error():208 cond:0
(rank:6 hostname:node103 pid:1133):ARMCI DASSERT fail. ../../ga-5-4/armci/src/common/armci.c:ARMCI_Error():208 cond:0
(rank:5 hostname:node103 pid:1132):ARMCI DASSERT fail. ../../ga-5-4/armci/src/common/armci.c:ARMCI_Error():208 cond:0
7:7:cphf_solve2:SCF residual greater than 1d-2:: -1
(rank:7 hostname:node103 pid:1134):ARMCI DASSERT fail. ../../ga-5-4/armci/src/common/armci.c:ARMCI_Error():208 cond:0
------------------------------------------------------------------------

------------------------------------------------------------------------

------------------------------------------------------------------------

------------------------------------------------------------------------

This error has not yet been assigned to a category

------------------------------------------------------------------------

 current input line : 

For more information see the NWChem manual at 

    0: 

http://nwchemgit.github.io/index.php/NWChem_Documentation

------------------------------------------------------------------------

------------------------------------------------------------------------



------------------------------------------------------------------------



For further details see manual section: 

This error has not yet been assigned to a category

------------------------------------------------------------------------

------------------------------------------------------------------------

For more information see the NWChem manual at 

No section for this category                                                   

                                                                               

                                                                               

http://nwchemgit.github.io/index.php/NWChem_Documentation

                  




 current input line : 

For further details see manual section:
3:3:cphf_solve2:SCF residual greater than 1d-2:: -1
No section for this category
(rank:3 hostname:node103 pid:1129):ARMCI DASSERT fail. ../../ga-5-4/armci/src/common/armci.c:ARMCI_Error():208 cond:0
                                                                               

    0: 

                                                                               

                  

------------------------------------------------------------------------
2:2:cphf_solve2:SCF residual greater than 1d-2:: -1
(rank:2 hostname:node103 pid:1130):ARMCI DASSERT fail. ../../ga-5-4/armci/src/common/armci.c:ARMCI_Error():208 cond:0
------------------------------------------------------------------------

This error has not yet been assigned to a category

------------------------------------------------------------------------

For more information see the NWChem manual at 

http://nwchemgit.github.io/index.php/NWChem_Documentation




For further details see manual section: 

No section for this category
1:1:cphf_solve2:SCF residual greater than 1d-2:: -1
------------------------------------------------------------------------

 current input line : 

  145: task dft frequencies

------------------------------------------------------------------------
(rank:1 hostname:node103 pid:1128):ARMCI DASSERT fail. ../../ga-5-4/armci/src/common/armci.c:ARMCI_Error():208 cond:0
------------------------------------------------------------------------

This error has not yet been assigned to a category

------------------------------------------------------------------------

For more information see the NWChem manual at 

http://nwchemgit.github.io/index.php/NWChem_Documentation




For further details see manual section: 

No section for this category
0:0:cphf_solve2:SCF residual greater than 1d-2:: -1
(rank:0 hostname:node103 pid:1135):ARMCI DASSERT fail. ../../ga-5-4/armci/src/common/armci.c:ARMCI_Error():208 cond:0
rank 4 in job 1 node103.cluster_41087 caused collective abort of all ranks
 exit status of rank 4: killed by signal 9
[code]

The test HPC only has acess to NWChem 6.6 sofar, is it possible that the error message is related to that version?

Forum Vet
9:59:35 AM PDT - Thu, Sep 13th 2018
6.8.1
Please use version 6.8.1. 6.8.1 contains important changes for hessian calculations

Clicked A Few Times
4:53:40 AM PST - Wed, Feb 13th 2019
Hello again, please excuse the late reply. I had troubles running the input and did some extensive testing on different molecules and hardware.

This specific error seems to appear for a range of SnGd molecules and the input file works for smaller molecules (which previously produced the same problematic output). However, starting with 12 atoms I was not able to run the calculations and with that neither the Sn14Gd input. I tested the input on different HPC, in each case the slurm system reports that the RAM demand is too high. I tested this for different hardware:

-1 node with 250 GB RAM per node
-1-5 nodes with 26 GB per node


I additionally tried to add the direct keyword in front of the grid keyword, but in each case the error appears unchanged. I also tried to measure the RAM costs with the slurm command: sacct -j SLURMID--format=jobid,elapsed,ncpus,ntasks,state,MaxVMSize,AveVMSize
The command shows that the jobs consume far lower RAM than provided, but still the jobs crash with RAM error.
Is this really a RAM issue or something else appearing as RAM error? How much RAM should these jobs actually consume? Is there away to reduce the required RAM?

Thank you in advance!

Forum Vet
9:56:24 AM PST - Wed, Feb 13th 2019
Please post complete input, output and error files to a public website

Clicked A Few Times
5:10:39 AM PST - Thu, Feb 14th 2019
I uploaded the data to this file-hoster:
https://ufile.io/rka4o

The error in the Lichtenberg folder is identical for 1-5 nodes, no matter how much RAM I give.
I can attach the data for the 3-5 nodes calculations as well if needed.

The slurm system reports for Sn11Gd_10Nonett_Lichtenberg_2nodes:
sacct -j 10937389 --format=jobid,elapsed,ncpus,ntasks,state,MaxVMSize,AveVMSize


      JobID    Elapsed      NCPUS   NTasks      State  MaxVMSize  AveVMSize
---------- ---------- -------- ---------- ---------- ----------
10937389 03:30:07 48 COMPLETED
10937389.ba+ 03:30:07 24 1 COMPLETED 403400K 403400K
10937389.ex+ 03:30:08 48 2 COMPLETED 159772K 4360K
10937389.0 03:30:02 48 48 CANCELLED+ 11740388K 242943.75K

Forum Vet
2:40:29 PM PST - Thu, Feb 14th 2019
Reduce memory usage
I have analyzed only the Lichtenberg folder. Quick browse on the other two folders seems to indicate that your job ran out of time.
The job reported in the Lichtenber folder was killed because it ran out of memory. If you look at the error file slurm-*.out, you will find the line
slurmstepd: error: Step 10937389.0 exceeded memory limit (60372784 > 59392000), being killed

Since your input file might be able to use up to ~8GB and you has 24 processes/node, you might end up using 24*8 =192GB.
In reality, your system allows to use only up to 60GB, therefore you need to
either drastically reduce your input memory line
or use fewer processes on each node (say 8 or less)

Clicked A Few Times
4:54:43 AM PST - Thu, Feb 21st 2019
Calculations work but produce imaginary modes.
Hello again,

I think the timeout was caused by the immense I/O generated without the direct keyword in the input-file. I did not know that the memory specifications are per process and not per job.

We extensively tested your input on a range of structures. The calculations work with less CPU and full RAM of the nodes while reducing the RAM per process and empoying all of the CPUs per nodes does not work. However the example structure Sn14Gd_1Se (the original structure from above) and some of the other structures still generate imaginary modes. I uploaded the input and output of the calculations: https://ufile.io/39f8m

Thank you in advance,
best wishes


Forum >> NWChem's corner >> Running NWChem