Communication plan class. Uses vanilla MPI as the communication backend. More...

#include <Cabana_CommunicationPlan_Mpi.hpp>

Inheritance diagram for Cabana::CommunicationPlan< MemorySpace, Mpi >:

[legend]

Collaboration diagram for Cabana::CommunicationPlan< MemorySpace, Mpi >:

Public Member Functions
template<class ExecutionSpace, class RankViewType>
Kokkos::View< size_type *, memory_space >	createWithTopology (ExecutionSpace exec_space, Export, const RankViewType &element_export_ranks, const std::vector< int > &neighbor_ranks)
	Neighbor and export rank creator. Use this when you already know which ranks neighbor each other (i.e. every rank already knows who they will be sending and receiving from) as it will be more efficient. In this case you already know the topology of the point-to-point communication but not how much data to send to and receive from the neighbors.

template<class RankViewType>
Kokkos::View< size_type *, memory_space >	createWithTopology (Export, const RankViewType &element_export_ranks, const std::vector< int > &neighbor_ranks)
	Neighbor and export rank creator. Use this when you already know which ranks neighbor each other (i.e. every rank already knows who they will be sending and receiving from) as it will be more efficient. In this case you already know the topology of the point-to-point communication but not how much data to send to and receive from the neighbors.

template<class ExecutionSpace, class RankViewType>
Kokkos::View< size_type *, memory_space >	createWithoutTopology (ExecutionSpace exec_space, Export, const RankViewType &element_export_ranks)
	Export rank creator. Use this when you don't know who you will receiving from - only who you are sending to. This is less efficient than if we already knew who our neighbors were because we have to determine the topology of the point-to-point communication first.

template<class RankViewType>
Kokkos::View< size_type *, memory_space >	createWithoutTopology (Export, const RankViewType &element_export_ranks)
	Export rank creator. Use this when you don't know who you will receiving from - only who you are sending to. This is less efficient than if we already knew who our neighbors were because we have to determine the topology of the point-to-point communication first.

template<class ExecutionSpace, class RankViewType, class IdViewType>
auto	createWithTopology (ExecutionSpace exec_space, Import, const RankViewType &element_import_ranks, const IdViewType &element_import_ids, const std::vector< int > &neighbor_ranks) -> std::tuple< Kokkos::View< typename RankViewType::size_type , typename RankViewType::memory_space >, Kokkos::View< int , typename RankViewType::memory_space >, Kokkos::View< int *, typename IdViewType::memory_space > >
	Neighbor and import rank creator. Use this when you already know which ranks neighbor each other (i.e. every rank already knows who they will be sending and receiving from) as it will be more efficient. In this case you already know the topology of the point-to-point communication but not how much data to send to and receive from the neighbors.

template<class RankViewType, class IdViewType>
auto	createWithTopology (Import, const RankViewType &element_import_ranks, const IdViewType &element_import_ids, const std::vector< int > &neighbor_ranks)
	Neighbor and import rank creator. Use this when you already know which ranks neighbor each other (i.e. every rank already knows who they will be sending and receiving from) as it will be more efficient. In this case you already know the topology of the point-to-point communication but not how much data to send to and receive from the neighbors.

template<class ExecutionSpace, class RankViewType, class IdViewType>
auto	createWithoutTopology (ExecutionSpace exec_space, Import, const RankViewType &element_import_ranks, const IdViewType &element_import_ids) -> std::tuple< Kokkos::View< typename RankViewType::size_type , typename RankViewType::memory_space >, Kokkos::View< int , typename RankViewType::memory_space >, Kokkos::View< int *, typename IdViewType::memory_space > >
	Import rank creator. Use this when you don't know who you will be receiving from - only who you are importing from. This is less efficient than if we already knew who our neighbors were because we have to determine the topology of the point-to-point communication first.

template<class RankViewType, class IdViewType>
auto	createWithoutTopology (Import, const RankViewType &element_import_ranks, const IdViewType &element_import_ids)
	Import rank creator. Use this when you don't know who you will be receiving from - only who you are importing from. This is less efficient than if we already knew who our neighbors were because we have to determine the topology of the point-to-point communication first.

Public Member Functions inherited from Cabana::CommunicationPlanBase< MemorySpace >
MPI_Comm	comm () const
	Get the MPI communicator.

int	numNeighbor () const
	Get the number of neighbor ranks that this rank will communicate with.

int	neighborRank (const int neighbor) const
	Given a local neighbor id get its rank in the MPI communicator.

std::size_t	numExport (const int neighbor) const
	Get the number of elements this rank will export to a given neighbor.

std::size_t	totalNumExport () const
	Get the total number of exports this rank will do.

std::size_t	numImport (const int neighbor) const
	Get the number of elements this rank will import from a given neighbor.

std::size_t	totalNumImport () const
	Get the total number of imports this rank will do.

std::size_t	exportSize () const
	Get the number of export elements.

Kokkos::View< std::size_t *, memory_space >	getExportSteering () const
	Get the steering vector for the exports.

template<class PackViewType, class RankViewType>
void	createExportSteering (const PackViewType &neighbor_ids, const RankViewType &element_export_ranks)
	Create the export steering vector.

template<class PackViewType, class RankViewType, class IdViewType>
void	createExportSteering (const PackViewType &neighbor_ids, const RankViewType &element_export_ranks, const IdViewType &element_export_ids)
	Create the export steering vector.

Protected Member Functions
	CommunicationPlan (MPI_Comm comm)
	Constructor.

Protected Member Functions inherited from Cabana::CommunicationPlanBase< MemorySpace >
	CommunicationPlanBase (MPI_Comm comm)
	Constructor.

Additional Inherited Members
Public Types inherited from Cabana::CommunicationPlanBase< MemorySpace >
using	memory_space = MemorySpace
	Kokkos memory space.

using	execution_space = typename memory_space::execution_space
	Default execution space.

using	size_type = typename memory_space::memory_space::size_type
	Size type.

Protected Attributes inherited from Cabana::CommunicationPlanBase< MemorySpace >
std::shared_ptr< MPI_Comm >	_comm_ptr
	Shared pointer to Mpi communicator.

std::vector< int >	_neighbors
	List of Mpi neighbors.

std::size_t	_total_num_export
	Number of elements exported.

std::size_t	_total_num_import
	Number of elements imported.

std::vector< std::size_t >	_num_export
	Number of elements exported to each neighbor.

std::vector< std::size_t >	_num_import
	Number of elements imported from each neighbor.

std::size_t	_num_export_element

Kokkos::View< std::size_t *, memory_space >	_export_steering
	Export steering vector.

Detailed Description

template<class MemorySpace>
class Cabana::CommunicationPlan< MemorySpace, Mpi >

Communication plan class. Uses vanilla MPI as the communication backend.

Constructor & Destructor Documentation

◆ CommunicationPlan()

template<class MemorySpace>

Cabana::CommunicationPlan< MemorySpace, Mpi >::CommunicationPlan ( MPI_Comm comm )

inlineprotected

Constructor.

Parameters

comm	The MPI communicator over which the CommunicationPlan is defined.

Member Function Documentation

◆ createWithoutTopology() [1/4]

template<class MemorySpace>

template<class ExecutionSpace, class RankViewType>

Kokkos::View< size_type *, memory_space > Cabana::CommunicationPlan< MemorySpace, Mpi >::createWithoutTopology	(	ExecutionSpace	exec_space,
		Export	,
		const RankViewType &	element_export_ranks )

inline

Export rank creator. Use this when you don't know who you will receiving from - only who you are sending to. This is less efficient than if we already knew who our neighbors were because we have to determine the topology of the point-to-point communication first.

Parameters

exec_space Kokkos execution space.

element_export_ranks The destination rank in the target decomposition of each locally owned element in the source decomposition. Each element will have one unique destination to which it will be exported. This export rank may any one of the listed neighbor ranks which can include the calling rank. An export rank of -1 will signal that this element is not to be exported and will be ignored in the data migration. The input is expected to be a Kokkos view or Cabana slice in the same memory space as the communication plan.

Returns: The location of each export element in the send buffer for its given neighbor.

Note: Calling this function completely updates the state of this object and invalidates the previous state.; For elements that you do not wish to export, use an export rank of -1 to signal that this element is not to be exported and will be ignored in the data migration. In other words, this element will be completely* removed in the new decomposition. If the data is staying on this rank, just use this rank as the export destination and the data will be efficiently migrated.

◆ createWithoutTopology() [2/4]

template<class MemorySpace>

template<class ExecutionSpace, class RankViewType, class IdViewType>

auto Cabana::CommunicationPlan< MemorySpace, Mpi >::createWithoutTopology	(	ExecutionSpace	exec_space,
		Import	,
		const RankViewType &	element_import_ranks,
		const IdViewType &	element_import_ids ) -> std::tuple<Kokkos::View<typename RankViewType::size_type, typename RankViewType::memory_space>, Kokkos::View<int, typename RankViewType::memory_space>, Kokkos::View<int*, typename IdViewType::memory_space>>

inline

Import rank creator. Use this when you don't know who you will be receiving from - only who you are importing from. This is less efficient than if we already knew who our neighbors were because we have to determine the topology of the point-to-point communication first.

Parameters

exec_space	Kokkos execution space.
element_import_ranks	The source rank in the target decomposition of each remotely owned element in element_import_ids. This import rank may be any one of the listed neighbor ranks which can include the calling rank. The input is expected to be a Kokkos view in the same memory space as the communication plan.
element_import_ids	The local IDs of remotely owned elements that are to be imported. These are local IDs on the remote rank. element_import_ids is mapped such that element_import_ids(i) lives on remote rank element_import_ranks(i).

Returns: A tuple of Kokkos views, where: Element 1: The location of each export element in the send buffer for its given neighbor. Element 2: The remote ranks this rank will export to Element 3: The local IDs this rank will export Elements 2 and 3 are mapped in the same way as element_import_ranks and element_import_ids

Note: Calling this function completely updates the state of this object and invalidates the previous state.; Unlike creating from exports, an import rank of -1 is not supported.

◆ createWithoutTopology() [3/4]

template<class MemorySpace>

template<class RankViewType>

Kokkos::View< size_type *, memory_space > Cabana::CommunicationPlan< MemorySpace, Mpi >::createWithoutTopology	(	Export	,
		const RankViewType &	element_export_ranks )

inline

Export rank creator. Use this when you don't know who you will receiving from - only who you are sending to. This is less efficient than if we already knew who our neighbors were because we have to determine the topology of the point-to-point communication first.

Parameters

element_export_ranks The destination rank in the target decomposition of each locally owned element in the source decomposition. Each element will have one unique destination to which it will be exported. This export rank may any one of the listed neighbor ranks which can include the calling rank. An export rank of -1 will signal that this element is not to be exported and will be ignored in the data migration. The input is expected to be a Kokkos view or Cabana slice in the same memory space as the communication plan.

Returns: The location of each export element in the send buffer for its given neighbor.

Note: Calling this function completely updates the state of this object and invalidates the previous state.; For elements that you do not wish to export, use an export rank of -1 to signal that this element is not to be exported and will be ignored in the data migration. In other words, this element will be completely* removed in the new decomposition. If the data is staying on this rank, just use this rank as the export destination and the data will be efficiently migrated.

◆ createWithoutTopology() [4/4]

template<class MemorySpace>

template<class RankViewType, class IdViewType>

auto Cabana::CommunicationPlan< MemorySpace, Mpi >::createWithoutTopology	(	Import	,
		const RankViewType &	element_import_ranks,
		const IdViewType &	element_import_ids )

inline

Import rank creator. Use this when you don't know who you will be receiving from - only who you are importing from. This is less efficient than if we already knew who our neighbors were because we have to determine the topology of the point-to-point communication first.

Parameters

element_import_ranks	The source rank in the target decomposition of each remotely owned element in element_import_ids. This import rank may be any one of the listed neighbor ranks which can include the calling rank. The input is expected to be a Kokkos view in the same memory space as the communication plan.
element_import_ids	The local IDs of remotely owned elements that are to be imported. These are local IDs on the remote rank. element_import_ids is mapped such that element_import_ids(i) lives on remote rank element_import_ranks(i).

Returns: A tuple of Kokkos views, where: Element 1: The location of each export element in the send buffer for its given neighbor. Element 2: The remote ranks this rank will export to Element 3: The local IDs this rank will export Elements 2 and 3 are mapped in the same way as element_import_ranks and element_import_ids

Note: Calling this function completely updates the state of this object and invalidates the previous state.; Unlike creating from exports, an import rank of -1 is not supported.

◆ createWithTopology() [1/4]

template<class MemorySpace>

template<class ExecutionSpace, class RankViewType>

Kokkos::View< size_type *, memory_space > Cabana::CommunicationPlan< MemorySpace, Mpi >::createWithTopology	(	ExecutionSpace	exec_space,
		Export	,
		const RankViewType &	element_export_ranks,
		const std::vector< int > &	neighbor_ranks )

inline

Neighbor and export rank creator. Use this when you already know which ranks neighbor each other (i.e. every rank already knows who they will be sending and receiving from) as it will be more efficient. In this case you already know the topology of the point-to-point communication but not how much data to send to and receive from the neighbors.

Parameters

exec_space	Kokkos execution space.
element_export_ranks	The destination rank in the target decomposition of each locally owned element in the source decomposition. Each element will have one unique destination to which it will be exported. This export rank may be any one of the listed neighbor ranks which can include the calling rank. An export rank of -1 will signal that this element is not to be exported and will be ignored in the data migration. The input is expected to be a Kokkos view or Cabana slice in the same memory space as the communication plan.
neighbor_ranks	List of ranks this rank will send to and receive from. This list can include the calling rank. This is effectively a description of the topology of the point-to-point communication plan. Only the unique elements in this list are used.

Returns: The location of each export element in the send buffer for its given neighbor.

Note: Calling this function completely updates the state of this object and invalidates the previous state.; For elements that you do not wish to export, use an export rank of -1 to signal that this element is not to be exported and will be ignored in the data migration. In other words, this element will be completely* removed in the new decomposition. If the data is staying on this rank, just use this rank as the export destination and the data will be efficiently migrated.

◆ createWithTopology() [2/4]

template<class MemorySpace>

template<class ExecutionSpace, class RankViewType, class IdViewType>

auto Cabana::CommunicationPlan< MemorySpace, Mpi >::createWithTopology	(	ExecutionSpace	exec_space,
		Import	,
		const RankViewType &	element_import_ranks,
		const IdViewType &	element_import_ids,
		const std::vector< int > &	neighbor_ranks ) -> std::tuple<Kokkos::View<typename RankViewType::size_type, typename RankViewType::memory_space>, Kokkos::View<int, typename RankViewType::memory_space>, Kokkos::View<int*, typename IdViewType::memory_space>>

inline

Neighbor and import rank creator. Use this when you already know which ranks neighbor each other (i.e. every rank already knows who they will be sending and receiving from) as it will be more efficient. In this case you already know the topology of the point-to-point communication but not how much data to send to and receive from the neighbors.

Parameters

exec_space	Kokkos execution space.
element_import_ranks	The source rank in the target decomposition of each remotely owned element in element_import_ids. This import rank may be any one of the listed neighbor ranks which can include the calling rank. The input is expected to be a Kokkos view in the same memory space as the communication plan.
element_import_ids	The local IDs of remotely owned elements that are to be imported. These are local IDs on the remote rank. element_import_ids is mapped such that element_import_ids(i) lives on remote rank element_import_ranks(i).
neighbor_ranks	List of ranks this rank will send to and receive from. This list can include the calling rank. This is effectively a description of the topology of the point-to-point communication plan. Only the unique elements in this list are used.

Returns: A tuple of Kokkos views, where: Element 1: The location of each export element in the send buffer for its given neighbor. Element 2: The remote ranks this rank will export to Element 3: The local IDs this rank will export Elements 2 and 3 are mapped in the same way as element_import_ranks and element_import_ids

Note: Calling this function completely updates the state of this object and invalidates the previous state.; Unlike creating from exports, an import rank of -1 is not supported.

◆ createWithTopology() [3/4]

template<class MemorySpace>

template<class RankViewType>

Kokkos::View< size_type *, memory_space > Cabana::CommunicationPlan< MemorySpace, Mpi >::createWithTopology	(	Export	,
		const RankViewType &	element_export_ranks,
		const std::vector< int > &	neighbor_ranks )

inline

Neighbor and export rank creator. Use this when you already know which ranks neighbor each other (i.e. every rank already knows who they will be sending and receiving from) as it will be more efficient. In this case you already know the topology of the point-to-point communication but not how much data to send to and receive from the neighbors.

Parameters

element_export_ranks The destination rank in the target decomposition of each locally owned element in the source decomposition. Each element will have one unique destination to which it will be exported. This export rank may be any one of the listed neighbor ranks which can include the calling rank. An export rank of -1 will signal that this element is not to be exported and will be ignored in the data migration. The input is expected to be a Kokkos view or Cabana slice in the same memory space as the communication plan.

neighbor_ranks List of ranks this rank will send to and receive from. This list can include the calling rank. This is effectively a description of the topology of the point-to-point communication plan. Only the unique elements in this list are used.

Returns: The location of each export element in the send buffer for its given neighbor.

Note: Calling this function completely updates the state of this object and invalidates the previous state.; For elements that you do not wish to export, use an export rank of -1 to signal that this element is not to be exported and will be ignored in the data migration. In other words, this element will be completely* removed in the new decomposition. If the data is staying on this rank, just use this rank as the export destination and the data will be efficiently migrated.

◆ createWithTopology() [4/4]

template<class MemorySpace>

template<class RankViewType, class IdViewType>

auto Cabana::CommunicationPlan< MemorySpace, Mpi >::createWithTopology	(	Import	,
		const RankViewType &	element_import_ranks,
		const IdViewType &	element_import_ids,
		const std::vector< int > &	neighbor_ranks )

inline

Neighbor and import rank creator. Use this when you already know which ranks neighbor each other (i.e. every rank already knows who they will be sending and receiving from) as it will be more efficient. In this case you already know the topology of the point-to-point communication but not how much data to send to and receive from the neighbors.

Parameters

element_import_ranks	The source rank in the target decomposition of each remotely owned element in element_import_ids. This import rank may be any one of the listed neighbor ranks which can include the calling rank. The input is expected to be a Kokkos view in the same memory space as the communication plan.
element_import_ids	The local IDs of remotely owned elements that are to be imported. These are local IDs on the remote rank. element_import_ids is mapped such that element_import_ids(i) lives on remote rank element_import_ranks(i).
neighbor_ranks	List of ranks this rank will send to and receive from. This list can include the calling rank. This is effectively a description of the topology of the point-to-point communication plan. Only the unique elements in this list are used.

Returns: A tuple of Kokkos views, where: Element 1: The location of each export element in the send buffer for its given neighbor. Element 2: The remote ranks this rank will export to Element 3: The local IDs this rank will export Elements 2 and 3 are mapped in the same way as element_import_ranks and element_import_ids

Note: Calling this function completely updates the state of this object and invalidates the previous state.; Unlike creating from exports, an import rank of -1 is not supported.

The documentation for this class was generated from the following file:

core/src/impl/Cabana_CommunicationPlan_Mpi.hpp

Public Member Functions

Protected Member Functions

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ CommunicationPlan()

Member Function Documentation

◆ createWithoutTopology() [1/4]

◆ createWithoutTopology() [2/4]

◆ createWithoutTopology() [3/4]

◆ createWithoutTopology() [4/4]

◆ createWithTopology() [1/4]

◆ createWithTopology() [2/4]

◆ createWithTopology() [3/4]

◆ createWithTopology() [4/4]