
Utilities to measure memory usage.

Included is a C++ function get_heap_usage() that measure current memory 
heap usage.  To use it, include "get_heap_usage.h" in your file, call
heap_size = get_heap_usage(), and compile with mpic++.  The get_heap_usage
function is defined for linux systems (including tbird) and mac systems.
Code exists for suns, but I couldn't make it work on seamus.

Programs commdup.c and commsplit.c perform repeated MPI_Comm_dup and 
MPI_Comm_split operations, followed by MPI_Comm_free, 
all surrounded by calls to get_heap_usage().  They print stats showing 
the heap usage before and after each MPI call.  They are used to determine
whether there are memory leaks resulting from MPI_Comm_dup and MPI_Comm_split.

In general, the first call to MPI_Comm_dup and MPI_Comm_split causes
memory leaks on every platform.  On macs and linux boxes running LAM MPI,
subsequent calls to MPI_Comm_dup and MPI_Comm_split do not cause leaks.

On tbird, however, with mpich and openmpi, subsequent calls to 
MPI_Comm_dup and MPI_Comm_split continue to cause memory leaks.  This
problem is significant for our RCB and PHG codes.  For RCB, using parameter
TFLOPS_SPECIAL=1 circumvents the problem.  For PHG, we have no work-around.

tbird environments tested:
  Default environment on log-in:
    > module list
      Currently Loaded Modulefiles:
        1) /opt/modules/oscar-modulefiles/default-manpath/1.0.1
        2) compilers/intel-9.1
        3) misc/env-openmpi
        4) mpi/openmpi-1.1.2-ofed-intel-9.1
        5) libraries/intel-mkl

  Mike Glass's environment:
    > module list
      Currently Loaded Modulefiles:
        1) compilers/gcc-3.4.6                
        2) misc/env-mvapich-devel
        3) mpi/mvapich-0.9.8-ofed-gcc-3.4.6



Program rcblike.c performs MPI_Comm_dup and MPI_Comm_split in a manner
similar to that in RCB (i.e., recursively splitting the communicator).
It shows similar results.
