Lifelong Unsupervised Learning

This chapter focuses on lifelong unsupervised learning. Much of the work in this area is done in the contexts of topic modeling and information extraction. These two areas are well suited to lifelong machine learning (LML). In the case of topic modeling, topics learned in the past in related domains can obviously be used to guide the modeling in the new or current domain [Chen and Liu, 2014a,b, Wang et al., 2016]. The knowledge base (KB) (Section 1.3) stores the past topics. Note that in this chapter, we use the terms domain and task interchangeably as in the existing research; each task is from a different domain. In terms of information extraction (IE), LML is also natural because the goal of IE is to extract and accumulate as much useful information or knowledge as possible. The extraction process is thus by nature continuous and cumulative. The extracted information earlier can be used to help extract more information later with higher quality [Liu et al., 2016]. These all match the objectives of LML. In the IE case, the knowledge base stores the extracted information and some other forms of useful knowledge. Even with the current primitive LML techniques, these tasks can already produce significantly better results regardless of whether the data is large or small. When the data size is small, these techniques are even more advantageous. For example, when the data is small, traditional topic models produce very poor results, but lifelong topic models can still generate very good topics. Ideally, as the knowledge base expands, more extractions will be made and few errors will incur. This is similar to our human learning. As we become more and more knowledgeable, it is easier for us to learn more and also make fewer mistakes. In the following sections, we discuss the current representative techniques of lifelong unsupervised learning which are geared toward achieving these properties.

Authors and Affiliations

1. Google, Inc., USA

   Zhiyuan Chen

2. University of Illinois, Chicago, USA

   Bing Liu

Reprints and permissions

© 2017 Springer Nature Switzerland AG

Policies and ethics