Orphan processes in killed NWChem 6.1.1 job

What happens if you try to kill the left-over processes? Are they in "D" state?

We are running NWChem 6.1.1 (Jan 2012) on CentOS-6.3-x86_64 using openmpi 1.5.4 on a 16-core node. I notice that after a multi-processor job is killed, the system still claims the cpu activity and the %idle does not return to 100% as it does after a completed job, according to the output of a sar command. These cores are then not available for subsequent multi-processor jobs. The only way I know to reclaim these cores is to restart the node.

They are in the "R" state. Here is the "ps aux" output (keller6 is the NWChem user.)

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
keller6 6001 99.8 0.2 1262432 140092 ? R 19:25 29:12 /usr/local/nwchem/bin/nwchem input.inp
keller6 6002 99.9 0.1 1261552 125244 ? R 19:25 29:13 /usr/local/nwchem/bin/nwchem input.inp
keller6 6003 99.9 0.1 1261540 128448 ? R 19:25 29:13 /usr/local/nwchem/bin/nwchem input.inp
keller6 6004 99.9 0.1 1261540 122332 ? R 19:25 29:13 /usr/local/nwchem/bin/nwchem input.inp

I can kill the process with "kill -9 6001"

Do you still need to reboot the cluster nodes after having manually killed the leftover processes?

No - I tried several runs, and I can indeed recover full funationality of the node by finding the process numbers and then issuing n "kill -9 [process number} n times. Hopefully this can be done automatically, or prevented in the first place.
John K.