TextToHBM: A Generalised Approach to Learning Models of Human Behaviour for Activity Recognition from Textual Instructions is a project funded by the DFG. It started in September 2016 and will continue for three years. Kristina Yordanova is the principal investigator in the project and within its context she investigates methods and develops tools for the automatic learning of behaviour models from textual instructions.
Computational models for activity recognition aim at recognising the user actions and goals based on precondition-effect rules. One problem such approaches face, is how to obtain the model structure. To reduce the need of domain experts or sensor data during the model building, methods for learning models of human behaviour from textual data have been investigated. Existing approaches, however, make various simplifying assumptions during the learning process. This renders the model inapplicable for activity recognition problems. To address this problem, this project aims at developing a generalised methodology for learning the model structure from textual instructions. The methodology shall combine existing and novel methods for model learning.
- A methodology for extracting the action semantics from text shall be developed. The methodology shall address the challenge of identifying causal relations between elements in texts with short and simple sentence structure.
- To ensure the model generalisation and the incorporation of context information, methods for ontology learning shall be investigated. A core challenge here is the ontology extension based on causal, spatial, and functional properties of the entities in the problem domain.
- To learn the model semantics, methods for language grounding shall be investigated. The semantics shall be represented in terms of precondition-effect rules. The resulting methodology shall address the issues associated with learning these rules and translating them into an appropriate for activity recognition format. It shall also address the problem of learning an optimal model through reinforcement learning methods.
- To evaluate the methodology, the learned models shall be applied to various activity recognition tasks and their performance compared to that of handcrafted models.
If successful, the methodology will reduce the time and resources needed for developing computational models of human behaviour for activity recognition. Furthermore, it will contribute to the better understanding of the general problem of structured behaviour representation and the learning of complex behaviour models.
