Poster: Fast On-Device Adaptation with Approximate Forward Training

Enabling real-time machine learning (ML) model adaptation to previously unseen, but highly specific contexts and environments can vastly extend the capability of mobile and ubiquitous AI systems. Cloud-aided approaches often fall short of meeting the time constraints without assuming pre-acquisition of data. Other existing approaches targeting efficient training on mobile devices focus on the generally complex context-agnostic tasks where achieving the performance of DNNs without proper backpropagation-based training is unlikely. In this work, we introduce a novel approximate forward training scheme to leverage the relationship that the updates to the parameters of a specific linear (and convolutional) layer in each training step are the linear combinations of outputs from the previous layer. Our preliminary results demonstrate the feasibility of this approach on mobile platforms.