int/Integer argument type mismatch inside peigs is causing segmentation fault


Click here for full thread
Clicked A Few Times
Most of peigs calls function reduce_maps without a complete prototype, and in many cases with a literal zero for argument nC.

src/c/pdcomplex.c:  extern void     pdiff(), xstop_(), pgexit(), mapdif_(), reduce_maps_();
src/c/pdcomplex.c:   reduce_maps( msize, mapZ, *n, mapZ, 0, mapZ, &nn_proc, proclist );
src/c/pdspeval.c:    extern void     pdiff(), xstop_(), pgexit(), mapdif_(), reduce_maps_();
src/c/pdspeval.c:    reduce_maps( *n, mapA, *n, mapZ, 0, mapZ, &nn_proc, proclist );
src/c/pdspevx.c:    extern void     pdiff(), xstop_(), pgexit(), mapdif_(), reduce_maps();
src/c/pdspevx.c:    reduce_maps( *n, mapA, *n, mapZ, 0, mapZ, &nn_proc, proclist );
src/c/pdspev_c.c:  extern void     pdiff(), xstop_(), pgexit(), mapdif_(), reduce_maps();
src/c/pdspev_c.c:   reduce_maps( *n, mapA, *n, mapZ, 0, mapZ, &nn_proc, proclist );
src/c/pdspgvx.c:  extern void     reduce_maps();
src/c/pdspgvx.c:  reduce_maps( *n, mapA, *n, mapB, *n, mapZ, &nn_proc, proclist );
src/c/pdspgv_c.c:    extern void     reduce_maps();
src/c/pdspgv_c.c:    reduce_maps( *n, mapA, *n, mapB, *n, mapZ, &nn_proc, proclist );
src/c/pstein3.c:  extern void     reduce_maps_();
src/c/pstein3.c:  reduce_maps( *meigval, mapZ, 0, mapZ, 0, mapZ, &nn_proc, proclist );
src/c/pstein4.c:  extern void reduce_maps();
src/c/pstein4.c:  reduce_maps( *meigval, mapZ, 0, mapZ, 0, mapZ, &nn_proc, proclist );
src/c/pstein44.c:  extern void     reduce_maps_();
src/c/pstein44.c:  reduce_maps( *meigval, mapZ, 0, mapZ, 0, mapZ, &nn_proc, proclist );
src/c/pstein5.c:   extern void     reduce_maps();
src/c/pstein5.c:  reduce_maps( *meigval, mapZ, 0, mapZ, 0, mapZ, &nn_proc, proclist );


Such a function call with unspecified argument list causes the C compiler to determine argument types from "the usual promotions". The type deduced for a literal 0 is "int".

But the actual type of this argument in the function's definition is "Integer".

void reduce_maps( nA, mapA, nB, mapB, nC, mapC, nproc, proclist )

  Integer      nA, nB, nC;
  Integer      mapA[], mapB[], mapC[], *nproc, proclist[];


and "Integer" is not necessarily the same type as "int".

#ifdef  STD_INT
typedef int    Integer;
typedef unsigned int unInteger;
#else
typedef long Integer;
typedef unsigned long unInteger;
#endif


In short, there is undefined behavior whenever STD_INT is not defined, and this resulted in actual bad code generation on CYGWIN64, because the compiler used 64-bit alignment for the parameters, but only zeroed the bottom 32-bits, and the high bits caused reduce_maps to see a very large value of nC and led to a segmentation fault.

From the build instructions, it sounds as if there have been problems building 64-bit nwchem on recent cygwin, possibly linked to newer compiler versions. This type mismatch would cause exactly those symptoms.

I've attempted a fix by adding a full prototype to globalp.c.h and it did cure the segmentation fault. (Unfortunately, other failures still exist)

Besides this, that "typedef long Integer" is useful on ILP32 and LP64, but still causes problems on LLP64 platforms such as Win64, where sizeof (long) < sizeof (void*). So that typedef should change, possibly to use intptr_t from inttypes.h.