SELF_DataPool Module

  1. SELF_DataPool.f90
  2. SELF_DataPool

High-water-mark storage for the mesh-sized arrays of the data classes (AMR Stage 6b).

Why this exists

Adaptive mesh refinement changes the element count almost every epoch, and the adaptive loop responded by freeing and re-initializing every mesh-sized field: seven in the model plus six in the geometry. Profiling one MI300X showed that cycle to be the single largest component of an adaptation - larger than the solution transfer and larger than geometry regeneration - and that the cost is NOT the device allocator (hipMalloc plus hipFree together are only ~8% of an adaptation) but the host-side work that accompanies a fresh allocation: allocating, zeroing ~140 MB of arrays per epoch at modest resolution, and then uploading those zeros to the device only for the solution transfer to overwrite them.

How it avoids the stride problem

A naive high-water-mark scheme allocates the element dimension larger than the logical element count. That does not work here: the device kernels take nElem as BOTH the launch bound and the array stride (see SC_2D_INDEX in src/gpu/SELF_GPU_Macros.h), the variable index is the slowest-varying dimension so a logical element prefix is not a contiguous prefix of the buffer, HDF5 hyperslabs are taken from shape(), and several routines assert on nElem equality.

Instead, each array is backed by a rank-1 pool that is grown monotonically, and the array itself is a pointer remapped onto the leading part of that pool at the EXACT logical shape. nElem therefore keeps meaning the logical element count everywhere, every stride, bound, shape() and equality assertion stays correct with no change, and no padding is ever computed over or transferred. Only the pool is oversized, and nothing indexes the pool directly.

The device side needs no shape at all - a device pointer is an address - so a device buffer is reused whenever the bytes required fit the bytes already allocated.

The pools are POINTER rather than ALLOCATABLE components deliberately: a derived-type component cannot carry the TARGET attribute, and pointing at a component of an object that is not itself a target is not conforming. An allocated pointer, by contrast, is always a valid target, so the public arrays can be remapped onto it safely regardless of how the owning object was declared.

Growth policy

poolGrowth is the over-allocation factor applied when a pool must grow. 1.0 reproduces exact sizing, i.e. the pre-6b behaviour, and is the escape hatch if capacity reuse is ever suspected of misbehaving: set it to 1.0 and every Resize reallocates exactly as Init used to. Above 1.0, a monotonically growing element count (the usual case while a wavefront expands) stops reallocating almost immediately.

This mirrors the amortized-capacity pattern already used for the quadtree forest in EnsureCapacity_QuadTreeMesh2D (src/SELF_QuadTreeMesh_2D.f90).


Uses

  • module~~self_datapool~~UsesGraph module~self_datapool SELF_DataPool module~self_constants SELF_Constants module~self_datapool->module~self_constants iso_c_binding iso_c_binding module~self_constants->iso_c_binding iso_fortran_env iso_fortran_env module~self_constants->iso_fortran_env

Used by

  • module~~self_datapool~~UsedByGraph module~self_datapool SELF_DataPool module~self_scalar_2d_t SELF_Scalar_2D_t module~self_scalar_2d_t->module~self_datapool module~self_tensor_2d_t SELF_Tensor_2D_t module~self_tensor_2d_t->module~self_datapool module~self_vector_2d_t SELF_Vector_2D_t module~self_vector_2d_t->module~self_datapool module~self_scalar_2d SELF_Scalar_2D module~self_scalar_2d->module~self_scalar_2d_t module~self_tensor_2d SELF_Tensor_2D module~self_tensor_2d->module~self_tensor_2d_t module~self_tensor_2d~2 SELF_Tensor_2D module~self_tensor_2d~2->module~self_tensor_2d_t module~self_vector_2d SELF_Vector_2D module~self_vector_2d->module~self_vector_2d_t module~self_scalar_2d~2 SELF_Scalar_2D module~self_scalar_2d~2->module~self_scalar_2d_t module~self_vector_2d~2 SELF_Vector_2D module~self_vector_2d~2->module~self_vector_2d_t module~self_geometry_2d SELF_Geometry_2D module~self_geometry_2d->module~self_scalar_2d module~self_geometry_2d->module~self_tensor_2d~2 module~self_geometry_2d->module~self_vector_2d module~self_refinementindicator_2d_t SELF_RefinementIndicator_2D_t module~self_refinementindicator_2d_t->module~self_scalar_2d module~self_mappedscalar_2d~2 SELF_MappedScalar_2D module~self_mappedscalar_2d~2->module~self_scalar_2d module~self_mappedscalar_2d_t SELF_MappedScalar_2D_t module~self_mappedscalar_2d~2->module~self_mappedscalar_2d_t module~self_refinementindicator_2d SELF_RefinementIndicator_2D module~self_refinementindicator_2d->module~self_scalar_2d module~self_refinementindicator_2d->module~self_refinementindicator_2d_t module~self_mappedscalar_2d_t->module~self_scalar_2d module~self_mappedscalar_2d_t->module~self_tensor_2d~2 module~self_mappedscalar_2d_t->module~self_geometry_2d module~self_mappedvector_2d_t SELF_MappedVector_2D_t module~self_mappedvector_2d_t->module~self_tensor_2d~2 module~self_mappedvector_2d_t->module~self_vector_2d module~self_mappedvector_2d_t->module~self_geometry_2d module~self_mappedvector_2d SELF_MappedVector_2D module~self_mappedvector_2d->module~self_vector_2d module~self_mappedvector_2d->module~self_mappedvector_2d_t module~self_dgmodel2d_t SELF_DGModel2D_t module~self_dgmodel2d_t->module~self_geometry_2d module~self_mappedscalar_2d SELF_MappedScalar_2D module~self_dgmodel2d_t->module~self_mappedscalar_2d module~self_mappedvector_2d~2 SELF_MappedVector_2D module~self_dgmodel2d_t->module~self_mappedvector_2d~2 module~self_mappedtwopointvector_2d_t SELF_MappedTwoPointVector_2D_t module~self_mappedtwopointvector_2d_t->module~self_geometry_2d module~self_points_t SELF_Points_t module~self_points_t->module~self_geometry_2d module~self_points_t->module~self_mappedscalar_2d module~self_points SELF_Points module~self_points->module~self_geometry_2d module~self_points->module~self_points_t module~self_points->module~self_mappedscalar_2d module~self_amrcontroller_2d SELF_AMRController_2D module~self_amrcontroller_2d->module~self_geometry_2d module~self_amrcontroller_2d->module~self_dgmodel2d_t module~self_refinementindicator_2d~2 SELF_RefinementIndicator_2D module~self_amrcontroller_2d->module~self_refinementindicator_2d~2 module~self_dgmodel2d~2 SELF_DGModel2D module~self_dgmodel2d~2->module~self_geometry_2d module~self_dgmodel2d~2->module~self_dgmodel2d_t module~self_esatmo2d SELF_ESAtmo2D module~self_esatmo2d->module~self_geometry_2d module~self_esatmo2d_t SELF_ESAtmo2D_t module~self_esatmo2d->module~self_esatmo2d_t module~self_ecdgmodel2d_t SELF_ECDGModel2D_t module~self_esatmo2d->module~self_ecdgmodel2d_t module~self_ecadvection2d~2 SELF_ECAdvection2D module~self_ecadvection2d~2->module~self_geometry_2d module~self_ecadvection2d~2->module~self_ecdgmodel2d_t module~self_ecadvection2d_t SELF_ECAdvection2D_t module~self_ecadvection2d~2->module~self_ecadvection2d_t module~self_refinementindicator_2d~2->module~self_refinementindicator_2d_t module~self_mappedscalar_2d->module~self_mappedscalar_2d_t module~self_mappedvector_2d~2->module~self_mappedvector_2d_t module~self_dgmodel2d SELF_DGModel2D module~self_dgmodel2d->module~self_dgmodel2d_t module~self_points~2 SELF_Points module~self_points~2->module~self_points_t module~self_mappedtwopointvector_2d~2 SELF_MappedTwoPointVector_2D module~self_mappedtwopointvector_2d~2->module~self_mappedtwopointvector_2d_t module~self_lineareuler2d_pml_t self_LinearEuler2D_PML_t module~self_lineareuler2d_pml_t->module~self_mappedscalar_2d module~self_lineareuler2d_pml_t->module~self_dgmodel2d module~self_lineareuler2d_t self_LinearEuler2D_t module~self_lineareuler2d_pml_t->module~self_lineareuler2d_t module~self_mappedtwopointvector_2d SELF_MappedTwoPointVector_2D module~self_mappedtwopointvector_2d->module~self_mappedtwopointvector_2d_t module~self_esatmo2d_t->module~self_mappedscalar_2d module~self_ecdgmodel2d SELF_ECDGModel2D module~self_esatmo2d_t->module~self_ecdgmodel2d module~self_ecdgmodel2d_t->module~self_dgmodel2d module~self_ecdgmodel2d_t->module~self_mappedtwopointvector_2d module~self_linearshallowwater2d_t self_LinearShallowWater2D_t module~self_linearshallowwater2d_t->module~self_dgmodel2d module~self_nulldgmodel2d_t self_NullDGModel2D_t module~self_nulldgmodel2d_t->module~self_dgmodel2d module~self_lineareuler2d_t->module~self_dgmodel2d module~self_advection_diffusion_2d_t self_advection_diffusion_2d_t module~self_advection_diffusion_2d_t->module~self_dgmodel2d module~self_lineareuler2d_pml self_LinearEuler2D_PML module~self_lineareuler2d_pml->module~self_lineareuler2d_pml_t module~self_lineareuler2d_pml~2 self_LinearEuler2D_PML module~self_lineareuler2d_pml~2->module~self_lineareuler2d_pml_t module~self_esatmo2d~2 SELF_ESAtmo2D module~self_esatmo2d~2->module~self_esatmo2d_t module~self_ecdgmodel2d->module~self_ecdgmodel2d_t module~self_linearshallowwater2d~2 self_LinearShallowWater2D module~self_linearshallowwater2d~2->module~self_linearshallowwater2d_t module~self_nulldgmodel2d self_NullDGModel2D module~self_nulldgmodel2d->module~self_nulldgmodel2d_t module~self_ecdgmodel2d~2 SELF_ECDGModel2D module~self_ecdgmodel2d~2->module~self_ecdgmodel2d_t module~self_linearshallowwater2d self_LinearShallowWater2D module~self_linearshallowwater2d->module~self_linearshallowwater2d_t module~self_nulldgmodel2d~2 self_NullDGModel2D module~self_nulldgmodel2d~2->module~self_nulldgmodel2d_t module~self_lineareuler2d~2 self_LinearEuler2D module~self_lineareuler2d~2->module~self_lineareuler2d_t module~self_lineareuler2d self_LinearEuler2D module~self_lineareuler2d->module~self_lineareuler2d_t module~self_advection_diffusion_2d self_advection_diffusion_2d module~self_advection_diffusion_2d->module~self_advection_diffusion_2d_t module~self_advection_diffusion_2d~2 self_advection_diffusion_2d module~self_advection_diffusion_2d~2->module~self_advection_diffusion_2d_t module~self_ecadvection2d_t->module~self_ecdgmodel2d module~self_ecadvection2d SELF_ECAdvection2D module~self_ecadvection2d->module~self_ecadvection2d_t

Contents

Variables

Subroutines


Variables

TypeVisibilityAttributesNameInitial
real(kind=prec), public, parameter:: poolGrowth =1.25_prec

Over-allocation factor for pool growth; 1.0 disables capacity reuse (see above).

Subroutines

public subroutine EnsurePool(pool, needed)

Guarantee that pool has at least needed elements, growing it by poolGrowth when it must reallocate. Existing contents are NOT preserved: every caller re-establishes the array contents after resizing (the solution through the AMR transfer, the geometry through GenerateFromMesh, everything else by being written before it is read), so copying the old data would be wasted bandwidth.

Arguments

TypeIntentOptionalAttributesName
real(kind=prec), intent(inout), pointer, contiguous:: pool(:)
integer, intent(in) :: needed