Installing parallel I/O libraries¶
High-resolution simulations (e.g. modeling the entire Greenland ice sheet using the 900 m resolution and a grid of 1700 by 3000 points) can produce huge amounts of data and I/O can become a bottleneck.
PISM supports several parallel I/O approaches that take advantage of parallel NetCDF, PnetCDF, and NCAR ParallelIO. The administrators of your HPC system should be able to help you install these libraries, but it may be easier to install them in the “home” directory instead.
This section describes the steps needed to build
NetCDF with parallel I/O based on HDF5 (needed to use PISM’s option
-o_format netcdf4_parallel
),PNetCDF (needed to use PISM’s option
-o_format pnetfdf
),ParallelIO (needed to use options
-o_format pio_netcdf4p
,-o_format pio_netcdf4c
,-o_format pio_pnetcdf
,-o_format pio_netcdf
).
Scripts below install libraries in ~/local/library_name
, using
~/local/build/library_name
to build them.
Section Building PISM with libraries in non-standard locations explains how build PISM with these libraries.
Section PISM’s I/O performance explains how to use them in PISM.
Installing HDF5-based parallel NetCDF¶
Installing HDF5¶
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | # Install HDF5 1.12.0 with parallel I/O in ~/local/hdf5,
# using ~/local/build/hdf5 as the build directory.
version=1.12.0
prefix=$HOME/local/hdf5
build_dir=~/local/build/hdf5
hdf5_site=https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.12
url=${hdf5_site}/hdf5-${version}/src/hdf5-${version}.tar.gz
mkdir -p ${build_dir}
pushd ${build_dir}
wget -nc ${url}
tar xzf hdf5-${version}.tar.gz
pushd hdf5-${version}
CC=mpicc CFLAGS=-w ./configure \
--disable-static \
--enable-parallel \
--prefix=${prefix} 2>&1 | tee hdf5_configure.log
make all 2>&1 | tee hdf5_compile.log
make install 2>&1 | tee hdf5_install.log
popd
popd
|
To compile parallel HDF5 one should use the MPI compiler wrapper mpicc
and run
configure
with the option --enable-parallel
. The flag -w
is not important: it
hides numerous compiler warnings emitted when building HDF5.
Installing NetCDF¶
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | # Install parallel NetCDF using parallel HDF5 in ~/local/hdf5 and
# ~/local/build/netcdf as a build directory.
hdf5=~/local/hdf5
version=4.7.4
prefix=$HOME/local/netcdf
build_dir=~/local/build/netcdf
url=https://github.com/Unidata/netcdf-c/archive/refs/tags/v${version}.tar.gz
mkdir -p ${build_dir}
pushd ${build_dir}
wget -nc ${url}
tar zxf v${version}.tar.gz
pushd netcdf-c-${version}
export CFLAGS="-g -O0"
unset CFLAGS
CC=mpicc CPPFLAGS=-I${hdf5}/include LDFLAGS=-L${hdf5}/lib ./configure \
--enable-netcdf4 \
--disable-dap \
--prefix=${prefix} 2>&1 | tee netcdf_configure.log
make all 2>&1 | tee netcdf_compile.log
make install 2>&1 | tee netcdf_install.log
popd
popd
|
Here we use the same compiler wrapper and set CPPFLAGS
and LDFLAGS
to select the
parallel HDF5 library installed earlier. The option --enable-netcdf4
is required for
parallel I/O; --disable-dap
is not required (it disables a NetCDF feature not used by
PISM).
Installing PnetCDF¶
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | # Install PnetCDF 1.12.1 in ~/local/pnetcdf,
# using ~/local/build/pnetcdf as a build directory.
version=1.12.1
prefix=$HOME/local/pnetcdf
build_dir=~/local/build/pnetcdf/
url=https://parallel-netcdf.github.io/Release/pnetcdf-${version}.tar.gz
mkdir -p ${build_dir}
pushd ${build_dir}
wget -nc ${url}
tar xzf pnetcdf-${version}.tar.gz
pushd pnetcdf-${version}
./configure \
--prefix=${prefix} \
--enable-shared \
--disable-static \
--disable-cxx \
--disable-fortran
make all
make install
popd
popd
|
Here we disable PnetCDF’s C++ and Fortran APIs and build the shared library.
Installing NCAR ParallelIO¶
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | # Install NCAR ParallelIO in ~/local/parallelio
# using parallel NetCDF and PnetCDF installed in ~/local/netcdf
# and ~/local/pnetcdf. Uses ~/local/build/parallelio
# as a build directory.
netcdf_prefix=~/local/netcdf
pnetcdf_prefix=~/local/pnetcdf
url=https://github.com/NCAR/ParallelIO.git
build=~/local/build/parallelio
prefix=$HOME/local/parallelio
rm -rf ${build}
mkdir -p ${build}/build ${build}/sources
git clone ${url} ${build}/sources
pushd ${build}/sources
git checkout -b 2_5_7 pio2_5_7
popd
pushd ${build}/build
CC=mpicc cmake \
-DCMAKE_C_FLAGS="-fPIC" \
-DCMAKE_INSTALL_PREFIX=${prefix} \
-DNetCDF_PATH=${netcdf_prefix} \
-DPnetCDF_PATH=${pnetcdf_prefix} \
-DPIO_ENABLE_FORTRAN=0 \
-DPIO_ENABLE_TIMING=0 \
${build}/sources
make install
popd
|
Here we use CMake’s variable CMAKE_FIND_ROOT_PATH
to tell CMake to use libraries in
~/local/netcdf
and ~/local/pnetcdf
, to install in ~/local/parallelio
, and to
disable ParallelIO features not used by PISM.
Previous | Up | Next |