|
H5Part
1.6.6
|
Modules | |
| Setting up the Data Model | |
| Reading and Writing Datasets | |
| Reading and Writing Attributes | |
This package includes C code for writing and reading 3D datasets intended for distributed memory applications. The underlying file format is H5Part, a simple data storage schema and API derived from HDF5. In particular, we use the H5Block subset of H5Part for managing 3D field data. To achieve large contiguous reads and writes, the datasets use HDF5's chunking mechanism to reorder the layout on disk such that local subfields (or "blocks") of the 3D field are stored contiguously (as opposed to storing an entire column contiguously, as in the Fortran column-major layout for an array). We refer to this chunked layout as "multiblock" and the column-major layout as "uniblock." Multiblocked datasets exhibit the following constraints:
All blocks must be the same size (a requirement of the HDF5 chunking mechanism).
The block dimensions must divide the field dimensions, to prevent fringe data that will lower performance by interfering with contiguous I/O operations.
For load-balancing purposes, the total number of blocks that the field is decomposed into must be a multiple of the number of nodes in the distributed system. (For ease of implementation, we require that the number of blocks equal the number of nodes.)