Update interpolation to support hicsparse backend #275
Conversation
|
Thank you @l90lpa for finalising the endeavour! One of the nasty surprises one might have at the moment is that single-precision fields get interpolated with the openmp backend rather than hicsparse because it is not possible with hicsparse to have mixed precision for sparse matrix and dense vectors. Probably we should fail, as also the haloExchange(on_device) etc. is not going to be correct with current implementation. Things are just not going to be in the state as expected, or executed where expected. We could try to support this in the future by supporting multiple value arrays (one in double and one in float) in the SparseMatrixStorage and keeping them in sync, or make another single-precision SparseMatrixStorage. I think we should increasingly assume that we work with multiple memory spaces and that users should be aware of this more. It should then be up to the user to make sure that the fields are up to date in the correct memory space after the interpolation is called in order to avoid data transfers when staying on device for future computations. if (tgt.hostNeedsUpdate()) {
tgt.updateHost();
}similarly to how haloExchange are required before stencil operations. I will look a bit more at the tests later. |
This PR modifies interpolation code to enable support for the
hicsparselinalg backend when using matrix based interpolation methods. It should be considered as a first draft as I think there are a number of areas that one might want to change. Here are some of my thoughts and modifications contained in this PR:Backendobject inlinalg/sparse.syncmethod because this can introduce unnecessary data movements.
on_devicehalo exchange.