ONGOING PROJECTS

  • Commonsense Reasoning about Space and Motion
  • Embodied Perception Grounding and Interaction
  • Multimodal Visuo-Locomotive Behaviour Interpretation
  • Visuo-Auditory Perception (focus on Moving Image Studies)
  • Linguistic Summarisation of Visual Imagery
  • Semantic Q/A with Video and Point-Clouds

(some part of this page under maintenance).

Commonsense Reasoning about Space and Motion

Focus is on developing general, declarative methods for comonsense knowledge representation and reasoning about space and motion:

  • theoretical characterisation of domain-independent  spatio-temporal dynamics (and fundamental aspects of modelling dynamic spatial systems)
  • support for linguistically rooted path and (spatio-temporal) motion predicates encompassing space, time, space-time, as first-class entities 
  • implementation within frameworks such as constraint logic programming, answer set programming, inductive logic porgramming
  • online and offline support for question-answering

vision-eye-architecture.jpeg

keywords 

deep semantics, declarative reasoning, visual computing, visual abduction, spatial learning, narrative, CLP / ILP / ASP / ASPMT(QS)

Embodied Perception Grounding & Interpretation

Focus is on general mechanisms for grouning of everyday activitiy specific interactions between people and the environment.

  • relational grounding of everyday activity
  • human-interaction interpretation
  • learning visuo-spatial representations
  • semantic question-answering with interaction datasets

embodied-1.jpeg

keywords

lorem ipsum, ipsum lorem, ipsum lorem, ipsum.

Multimodal Visuo-Locomotive Behaviour Interpretation

Focus is on the study and computational analysis of high-level perceptual and task-specific decision making capabilities in a visuo-locomotive context, e.g., navigation and wayfinding in large-scale builtup environments.

  • spatial memory and orientation
  • semantic analysis of eye-tracking
  • egocentric vision analysis
  • sensorimotor analysis

Large-scale buildings such as hospitals and airports are of special interest.

MModal-Visuolocomotive.jpeg

keywords

lorem ipsum, ipsum lorem, ipsum lorem, ipsum.

Visuo-Auditory Perception

Focus is on the large-scale study of high-level visuo-auditory event perception (for narrative film, immersive media). We have produced a range of digital (narrative) media developed specifically as a test-bed for experiments in visuo-auditory perception research focussing on:

  • change blindness, and continuity peerception 
  • sound based localisation of events & objects
  • impact of visual cues on people's attention

The work links with computational methods for computing the "Shape of the Moving Image", which involves -amongst other things- automated methods for the analysis of the cinematographic aspects (e.g., cuts, camera movements, shot types) of movie scenes.

main-slider.002.jpeg

keywords

digital (narrative) media, moving image studies, event perception, computational analysis of cinematography, AI for cognitive film studies / media studies

Linguistic Summarisation of Visual Imagery

Focus is on high-level, assistance-focussed summarisation of lar-scale visuo-spatial imagery using natural language. We seek to demonstrate the expressive capability of of our internal spatio-linguistic predicates. Analytical natural language based externalisations of visual imagery (image, video, eye-tracking data) are central from the viewpoint of human interaction, evidence-based qualitative analysis, and sensemaking.

Please note that language generation (NLG) per se is not the area of focus; we interface state of the art language generation techniques with our internal, declarative representations of space and motion.

drive-scene-wacv16.jpeg

keywords

lorem ipsum, ipsum lorem, ipsum lorem, ipsum.

Semantic Q/A with Video and Point-Clouds

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, 

  • relational grounding of everyday activity
  • human-interaction interpretation
  • learning visuo-spatial representations
  • semantic question-answering with interaction datasets

pointcloud-4.jpeg

keywords

lorem ipsum, ipsum lorem, ipsum lorem, ipsum.