Tpetra::Distributor: Split out execution of comm pattern into separate class
Created by: mhoemmen
Tpetra::Distributor currently combines setting up a communication pattern, with executing that pattern. Methods for executing the communication pattern are templated on Packet type.
We might like to have different Distributor back-ends, in order to support communication protocols other than MPI 2-sided. (For example, we could have an MPI 1-sided implementation, or a PGAS implementation, or a Kokkos-wrapping-PGAS implementation.) This implies a base class with subclasses for the different implementations.
In order to make execution of a communication plan happen through virtual methods, we would need to template the class on Packet, not the methods. (Templated methods can't be virtual.) However, the setup code does not depend on the Packet type. We would not want to build all that setup code redundantly for all Packet types (there are a lot of them!).
This suggests splitting the setup code into a separate class from the execution code.