Open
Description
🚀 The feature, motivation and pitch
https://github.com/YagaoDirac/Pytorch-extension-from-YagaoDirac-v2/blob/main/v2%20with%20basic%20test.py
I implemented this 2 weeks ago. It's probably a better implementation of the Linear layer. It speeds up the training while let people stack much more such layers directly without any trick.
Alternatives
In the code I implemented 3 different types of similar purpose. Each of them are tested and can work individually( if my tests are not too wrong).
Additional context
Also, if you decide to add this to pytorch, remember to rename it.
Notice the "untested workflow" in the file. It's probably a better way, and can be integreted into the layer itself.
More info in the file.
cc @albanD @mruberry @jbschlosser @walterddr @mikaylagawarecki