Documents which contain accounts of historical events are quite common. Biographies or descriptions of entity histories like histories of places or organizations are examples of such timeline documents. Their content is explicitly or implicitly associated with timestamps indicating occurrence time of described past events. The collections of such timeline documents can be quite large and can pose challenge for readers trying to make sense of them. We then introduce a novel research task, Comparative Timeline Summarization (CTS), as an effective strategy to discover important similarities and differences in large collections of timeline documents for providing contrastive type of knowledge. We propose a novel summarization framework which relies on a dynamic affinity-preserving mutually reinforced random walk for the CTS task and evaluate it on diverse Wikipedia categories and New York Time news collections. The ROUGE evaluations demonstrate the superior performance of our method on summarizing contrastive and diverse themes over competitive baselines.