A novel deep-learning-based computational method using object recognition to quantify visual context in naturalistic, multimodal stimuli demonstrates that a concept's perceived abstractness or concreteness dynamically depends on its visual context.
Digest

When we learn and use language, we deal with two main types of concepts. Concrete concepts, which refer to things we directly experience (like a chair, running or the colour blue), and abstract concepts, which refer to ideas that we are unable to sense directly (like truth, democracy or love).

Most studies have looked at how people process these concepts in isolation, such as by reading single words on a screen. This revealed that the human brain processes each concept differently, with concrete concepts typically activating brain regions involved in sensory and motor experiences, and abstract concepts activating regions involved in emotion and complex thinking.

However, the experiments conducted in these studies do not represent real life situations, where humans often encounter and process both concepts simultaneously. For instance, at the same time as processing language, someone may also be seeing, hearing, and experiencing other things in their environment.

Kewenig et al. wanted to understand whether the brain processes abstract and concrete concepts differently depending on what a person may be visualizing at the same time. To achieve this, they used a technique known as functional MRI to record which regions of the brain are activated as participants watched different movies.

The team found that when abstract concepts (such as love) appeared with related visual information (such as people kissing), the brain processed them more like concrete concepts, engaging sensory and motor regions. Conversely, when concrete concepts (like a chair) appeared without related visual information, the brain processed them more like abstract concepts, engaging regions involved in complex thinking. This suggests that the way the human brain processes meaning is very dynamic and constantly adapting to available contextual information.

These findings could help improve artificial intelligence systems that process language and visual information together, making them better at understanding context-dependent meaning. They might also benefit people with language disorders by informing the development of more effective therapies that consider how context affects understanding. However, more research is needed to confirm these findings and develop practical applications, particularly studies testing whether similar brain patterns occur in other natural situations.