Cabana::Distributor< MemorySpace > Class Template Reference

A communication plan for migrating data from one uniquely-owned decomposition to another uniquely owned decomposition. More...

#include <Cabana_Distributor.hpp>

Inheritance diagram for Cabana::Distributor< MemorySpace >:

Collaboration diagram for Cabana::Distributor< MemorySpace >:

Public Member Functions
template<class ViewType>
	Distributor (MPI_Comm comm, const ViewType &element_export_ranks, const std::vector< int > &neighbor_ranks)
	Topology and export rank constructor. Use this when you already know which ranks neighbor each other (i.e. every rank already knows who they will be sending and receiving from) as it will be more efficient. In this case you already know the topology of the point-to-point communication but not how much data to send to and receive from the neighbors.
template<class ViewType>
	Distributor (MPI_Comm comm, const ViewType &element_export_ranks)
	Export rank constructor. Use this when you don't know who you will be receiving from - only who you are sending to. This is less efficient than if we already knew who our neighbors were because we have to determine the topology of the point-to-point communication first.
Public Member Functions inherited from Cabana::CommunicationPlan< MemorySpace >
	CommunicationPlan (MPI_Comm comm)
	Constructor.
MPI_Comm	comm () const
	Get the MPI communicator.
int	numNeighbor () const
	Get the number of neighbor ranks that this rank will communicate with.
int	neighborRank (const int neighbor) const
	Given a local neighbor id get its rank in the MPI communicator.
std::size_t	numExport (const int neighbor) const
	Get the number of elements this rank will export to a given neighbor.
std::size_t	totalNumExport () const
	Get the total number of exports this rank will do.
std::size_t	numImport (const int neighbor) const
	Get the number of elements this rank will import from a given neighbor.
std::size_t	totalNumImport () const
	Get the total number of imports this rank will do.
std::size_t	exportSize () const
	Get the number of export elements.
Kokkos::View< std::size_t *, memory_space >	getExportSteering () const
	Get the steering vector for the exports.
template<class ExecutionSpace, class RankViewType>
Kokkos::View< size_type *, memory_space >	createWithTopology (ExecutionSpace exec_space, Export, const RankViewType &element_export_ranks, const std::vector< int > &neighbor_ranks)
	Neighbor and export rank creator. Use this when you already know which ranks neighbor each other (i.e. every rank already knows who they will be sending and receiving from) as it will be more efficient. In this case you already know the topology of the point-to-point communication but not how much data to send to and receive from the neighbors.
template<class RankViewType>
Kokkos::View< size_type *, memory_space >	createWithTopology (Export, const RankViewType &element_export_ranks, const std::vector< int > &neighbor_ranks)
	Neighbor and export rank creator. Use this when you already know which ranks neighbor each other (i.e. every rank already knows who they will be sending and receiving from) as it will be more efficient. In this case you already know the topology of the point-to-point communication but not how much data to send to and receive from the neighbors.
template<class ExecutionSpace, class RankViewType>
Kokkos::View< size_type *, memory_space >	createWithoutTopology (ExecutionSpace exec_space, Export, const RankViewType &element_export_ranks)
	Export rank creator. Use this when you don't know who you will receiving from - only who you are sending to. This is less efficient than if we already knew who our neighbors were because we have to determine the topology of the point-to-point communication first.
template<class RankViewType>
Kokkos::View< size_type *, memory_space >	createWithoutTopology (Export, const RankViewType &element_export_ranks)
	Export rank creator. Use this when you don't know who you will receiving from - only who you are sending to. This is less efficient than if we already knew who our neighbors were because we have to determine the topology of the point-to-point communication first.
template<class ExecutionSpace, class RankViewType, class IdViewType>
auto	createWithTopology (ExecutionSpace exec_space, Import, const RankViewType &element_import_ranks, const IdViewType &element_import_ids, const std::vector< int > &neighbor_ranks) -> std::tuple< Kokkos::View< typename RankViewType::size_type *, typename RankViewType::memory_space >, Kokkos::View< int *, typename RankViewType::memory_space >, Kokkos::View< int *, typename IdViewType::memory_space > >
	Neighbor and import rank creator. Use this when you already know which ranks neighbor each other (i.e. every rank already knows who they will be sending and receiving from) as it will be more efficient. In this case you already know the topology of the point-to-point communication but not how much data to send to and receive from the neighbors.
template<class RankViewType, class IdViewType>
auto	createWithTopology (Import, const RankViewType &element_import_ranks, const IdViewType &element_import_ids, const std::vector< int > &neighbor_ranks)
	Neighbor and import rank creator. Use this when you already know which ranks neighbor each other (i.e. every rank already knows who they will be sending and receiving from) as it will be more efficient. In this case you already know the topology of the point-to-point communication but not how much data to send to and receive from the neighbors.
template<class ExecutionSpace, class RankViewType, class IdViewType>
auto	createWithoutTopology (ExecutionSpace exec_space, Import, const RankViewType &element_import_ranks, const IdViewType &element_import_ids) -> std::tuple< Kokkos::View< typename RankViewType::size_type *, typename RankViewType::memory_space >, Kokkos::View< int *, typename RankViewType::memory_space >, Kokkos::View< int *, typename IdViewType::memory_space > >
	Import rank creator. Use this when you don't know who you will be receiving from - only who you are importing from. This is less efficient than if we already knew who our neighbors were because we have to determine the topology of the point-to-point communication first.
template<class RankViewType, class IdViewType>
auto	createWithoutTopology (Import, const RankViewType &element_import_ranks, const IdViewType &element_import_ids)
	Import rank creator. Use this when you don't know who you will be receiving from - only who you are importing from. This is less efficient than if we already knew who our neighbors were because we have to determine the topology of the point-to-point communication first.
template<class PackViewType, class RankViewType>
void	createExportSteering (const PackViewType &neighbor_ids, const RankViewType &element_export_ranks)
	Create the export steering vector.
template<class PackViewType, class RankViewType, class IdViewType>
void	createExportSteering (const PackViewType &neighbor_ids, const RankViewType &element_export_ranks, const IdViewType &element_export_ids)
	Create the export steering vector.

Additional Inherited Members
Public Types inherited from Cabana::CommunicationPlan< MemorySpace >
using	memory_space = MemorySpace
	Kokkos memory space.
using	execution_space = typename memory_space::execution_space
	Default execution space.
using	size_type = typename memory_space::memory_space::size_type
	Size type.

Detailed Description

template<class MemorySpace>
class Cabana::Distributor< MemorySpace >

A communication plan for migrating data from one uniquely-owned decomposition to another uniquely owned decomposition.

Template Parameters

MemorySpace

Kokkos memory space in which data for this class will be allocated.

The Distributor allows data to be migrated to an entirely new decomposition. Only uniquely-owned decompositions are handled (i.e. each local element in the source rank has a single unique destination rank).

Some nomenclature:

Export - the data we uniquely own that we will be sending to other ranks.

Import - the data we uniquely own that we will be receiving from other ranks.

Note

We can migrate data to the same rank. In this case a copy will occur instead of communication.

To get the number of elements this rank will be receiving from migration in the forward communication plan, call totalNumImport() on the distributor. This will be needed when in-place migration is not used and a user must allocate their own destination data structure.

Constructor & Destructor Documentation

Distributor() [1/2]

template<class MemorySpace>

template<class ViewType>

Cabana::Distributor< MemorySpace >::Distributor ( MPI_Comm comm, const ViewType & element_export_ranks, const std::vector< int > & neighbor_ranks )

inline

Topology and export rank constructor. Use this when you already know which ranks neighbor each other (i.e. every rank already knows who they will be sending and receiving from) as it will be more efficient. In this case you already know the topology of the point-to-point communication but not how much data to send to and receive from the neighbors.

Template Parameters

ViewType

The container type for the export element ranks. This container type can be either a Kokkos View or a Cabana Slice.

Parameters

comm	The MPI communicator over which the distributor is defined.
element_export_ranks	The destination rank in the target decomposition of each locally owned element in the source decomposition. Each element will have one unique destination to which it will be exported. This export rank may be any one of the listed neighbor ranks which can include the calling rank. An export rank of -1 will signal that this element is not to be exported and will be ignored in the data migration. The input is expected to be a Kokkos view or Cabana slice in the same memory space as the distributor.
neighbor_ranks	List of ranks this rank will send to and receive from. This list can include the calling rank. This is effectively a description of the topology of the point-to-point communication plan. The elements in this list must be unique.

Note

For elements that you do not wish to export, use an export rank of -1 to signal that this element is not to be exported and will be ignored in the data migration. In other words, this element will be completely* removed in the new decomposition. If the data is staying on this rank, just use this rank as the export destination and the data will be efficiently migrated.

Distributor() [2/2]

template<class MemorySpace>

template<class ViewType>

Cabana::Distributor< MemorySpace >::Distributor ( MPI_Comm comm, const ViewType & element_export_ranks )

inline

Export rank constructor. Use this when you don't know who you will be receiving from - only who you are sending to. This is less efficient than if we already knew who our neighbors were because we have to determine the topology of the point-to-point communication first.

Template Parameters

ViewType

The container type for the export element ranks. This container type can be either a Kokkos View or a Cabana Slice.

Parameters

comm

The MPI communicator over which the distributor is defined.

element_export_ranks

The destination rank in the target decomposition of each locally owned element in the source decomposition. Each element will have one unique destination to which it will be exported. This export rank may any one of the listed neighbor ranks which can include the calling rank. An export rank of -1 will signal that this element is not to be exported and will be ignored in the data migration. The input is expected to be a Kokkos view or Cabana slice in the same memory space as the distributor.

Note

For elements that you do not wish to export, use an export rank of -1 to signal that this element is not to be exported and will be ignored in the data migration. In other words, this element will be completely* removed in the new decomposition. If the data is staying on this rank, just use this rank as the export destination and the data will be efficiently migrated.

---

The documentation for this class was generated from the following file:

• core/src/Cabana_Distributor.hpp