Tpetra: Add option to allocate "device" memory only when in use?
Created by: mhoemmen
@trilinos/tpetra Some Tpetra classes implement "DualView" semantics, where they maintain mirrored allocations in both "host" and "device" memory, and let users sync data between the two. Here, "host" memory implies "large and slower," and "device" memory implies "small and faster" (for some meaning of "fast"). Currently, Tpetra classes that implement DualView semantics allocate on both host and device. This won't support the use case where users want to treat device memory as a temporary workspace.
To fix this, we would need an option for device memory to stay unallocated if not in use. Sync to device memory would allocate device memory, and sync to host memory would deallocate device memory.
Note that Kokkos::DualView would need to store both of its Views by pointer (std::shared_ptr?) rather than by value, if it wants to implement these semantics. (DualView itself has view semantics. If you copy the struct, and one copy allocates a View, then the other copy won't get it, unless the Views are held by pointer, rather than by value.)