GitLab

Created by: DrBooom

@trilinos/tpetra

The noncontiguous Tpetra::Map constructor has a reduceAll at line 465 of Tpetra_Map_def.hpp:

 if (numGlobalElements != GSTI) {
      numGlobalElements_ = numGlobalElements; // Use the user's value.
    } else { // The user wants us to compute the sum.
      reduceAll<int, GST> (*comm, REDUCE_SUM, as<GST> (numLocalElements),
                           outArg (numGlobalElements_));
    }

And one at line 616:

  GO minMaxOutput[3];
      minMaxOutput[0] = 0;
      minMaxOutput[1] = 0;
      minMaxOutput[2] = 0;
      reduceAll<int, GO> (*comm, REDUCE_MAX, 3, minMaxInput, minMaxOutput);
      minAllGID_ = -minMaxOutput[0];
      maxAllGID_ = minMaxOutput[1];
      const GO globalDist = minMaxOutput[2];

Mark thinks that these could be fused into a single call. Since this Map constructor is one of the most expensive and frequent calls in the R4-5 scaling, any reduction in all-reduce calls will help.