Program Summary
The Communicating with Computers (CwC) program aims to enable symmetric communication between people and computers in which machines are not merely receivers of instructions but collaborators, able to harness a full range of natural modes including language, gesture and facial or other expressions. For the purposes of the CwC program, communication is understood to be the sharing of complex ideas in collaborative contexts. Complex ideas are assumed to be built from a relatively small set of elementary ideas, and language is thought to specify such complex ideas—but not completely, because language is ambiguous and depends in part on context, which can augment language and improve the specification of complex ideas. Thus, the CwC program will focus on developing technology for assembling complex ideas from elementary ones given language and context.
Specific technologies that CwC will develop include: A corpus or library of elementary ideas; algorithms for assembling complex ideas from elementary ones given language and context; and algorithms for figuring out what to do or say during communication.
The CwC program is organized around three use cases of increasing difficulty:
- Blocks World In this use case, humans and machines must communicate to build structures with toy blocks. The human or the machine will be given an assignment – a particular structure to build – and will have to communicate with the other to get the job done.
- Biocuration This use case involves communication about the biological sciences literature between human biocurators, who read the literature and compile machine-readable records of the contents of papers, and machine biocurators such as those under development in DARPA's Big Mechanism program.
- Collaborative Composition This use case will explore the process by which humans and machines might collaborate toward the assembly of a creative product—in this case, contributing sentences to create stories.
If successful, CwC could advance a number of application areas, most notably robotics and semi-autonomous systems. For example, CwC could allow operators to describe missions and give direction, before and during operations, using natural language. Conversely, when CwC-enabled robots or semi-autonomous systems encounter unexpected situations that require additional inputs from operators they would be capable of requesting assistance in natural language. Such natural language-based interactions would be far more efficient and flexible than programming or the rigidly preconfigured interfaces currently in use.