Adventures in Cognitive Science 7: Language, Simulations and their Role in Cognition

Source: pixbay.com

In this post I am going to present the language and situated simulation (LASS) theory. Traditionally, it was often assumed that there is one single type of representation (e.g. amodal symbols, statistical representations, embodiment or linguistic context-vectors) that underlies knowledge. The LASS theory tries to combine two types of representations, it suggests that knowledge is represented in a linguistic form in the brain's language system and as situated simulations in the brain's modal system. This theory is somehow related to the action-sentence compatibility effect (ACE), I presented in my last post, and emphasizes the role of embodiment even more. This post will again be based on a chapter in a book, namely “Language and simulation in conceptual processing” written by Lawrence Barsalou, Ava Santos, Kyle Simmons and Christine Wilson, which was a part of the book “Symbols and Embodiment: Debates on meaning and cognition”, edited by Manuel de Vega, Arthur Glenberg and Arthur Graesser.

The LASS Theory of Conceptual Processing

The research on knowledge usually focuses on categories of things and how these concepts are represented in the cognitive system. Instead of representing knowledge as a holistic image, humans use an attention system, to focus on the different components of an experience and form concepts that represents knowledge about them. Meaning that instead of “storing” every detail of the whole picture, humans break down the picture into its most relevant parts (categories), those categories are used for the development of concepts, that represent these categories. Representing and processing such concepts relies heavily on language and situated simulation. The authors focus on linguistic forms, units or patterns of language, and not on amodal symbols (which were used in traditional theories of knowledge representation).
The authors investigate the following four aspects of the LASS framework:

1. linguistic processing
2. situated simulation
3. mixtures of interactions of language and situated simulation
4. the statistical underpinning of language and situated simulation

Linguistic Processing

It is assumed that the linguistic system gets immediately activated when a word is perceived in order to categorise the linguistic form. The simulation system also gets immediately activated when a word is perceived, but the peak of its activity is after the peak from the activity of the linguistic system. Once the word is recognised, associated linguistic forms are generated as inferences and as pointers to associated concepts. Once the associated linguistic forms are generated, they are used for a rather superficial processing strategy. If this superficial processing is not enough, then more information about a concept is retrieved. Evidence for this proposal comes from work on lexical processing: whether a word is actually a word or not can be done by only looking on the linguistic form, no further processing steps are required. Experiments which were mostly focused on the phonetic aspects of words (the “sound” of a word) show that there is relative little activation of meaning, compared with experiments that focus more on semantic aspects (the “meaning” of words).

Situated Simulation

As written above, as soon as the linguistic system gets activated, it also actives associated simulations, not only for the perceived linguistic form, but also for associated linguistic forms. Activated linguistic forms are pointers to simulations that are potentially useful for representing the meaning of the perceived word. These simulations tend to be situated, so preparing the individual for situated actions (think back of the ACE). The authors of the chapter assume, that these simulations are activated automatically and quickly (within 200 milliseconds of perceiving the word), but still they are not the main influence on conscious cognition (until the effects of the simulations take place, this is still the linguistic system). The authors also assume, that situated simulations represent in-depth information about concepts (which might be the reason why it takes longer for this system until it influences conscious cognition). Also, basic symbol processing operations like predication (usage of concepts in logical operations), conceptual combination (combining of concepts) and recursion (concepts referring to themselves) result from operations on simulations.

Mixtures of Interactions of Language and Situated Simulation

A mixture of both systems are assumed to underlie a variety of tasks: when superficial linguistic processing is enough for handling a task, the processing relies mostly on the linguistic system, if it is not enough, the simulation system has to be consulted to deliver enough information about the required concepts. The two systems are not independent of each other, but they interact with each other. Complex linguistic interactions come from the interplay of both systems: e.g. when a speaker says “I am looking forward to the concert this evening”, a simulation that represents his expected enjoyment while listening to the concert. This simulation gets translated into linguistic forms, which are again translated into utterances. Simulations are also assumed to play a big role in more abstract processes like decision-making, planning and problem solving.

Statistical Underpinnings of Language and Situated Simulation

Both systems are strongly influenced by the statistical structure of their respective domains: in the simulation system, simulators capture the statistical frequencies of properties and their relations in experience, while in the linguistic system the frequency of words, associations between them and their relations to syntax are encoded statistically. The authors already pointed out the role of statistical representation earlier, by also highlighting this aspect, they kind of try to bring it into LASS.

Evidence for the LASS Theory

The much older dual-coding theory from Allen Paivio, which has many similarities to the LASS theory, such as assuming two basic systems, but also some differences, e.g.the dual-coding theory says, that deep conceptual processing happens in both systems, not only in the simulation system. A lot of evidence for cognition relying on two systems has been found, since the dual-coding theory was proposed in 1971, which can also be used to support the LASS theory.
Apart from this evidence, the authors also looked at some more recent studies that supported the LASS theory:

Word Association

In this experiment the participants were given a word and they had to come up with associative words, as soon as a participant stopped generating words, the trial was over. Those generated words were written down and later on the order in which they were uttered was analysed. According to the LASS theory there will by three types of responses:

1. linguistically related response: if the response was linguistically related to the given word, it would fall under this category, e.g. the response “hive” to the word “bee”, since “bee-hive” is a commonly used compound phrase. According to the LASS theory those responses are more likely to come from the linguistic system and they should come earlier than the others.
2. taxonomically related response: if the response did not fall under the first category, but was a taxonomic response, e.g. “animal” for the word “dog”. It is less clear, where they originate from, according to the LASS theory, but it is thought that they also come from the linguistic system.
3. object-situation response: all responses that did not fall into either the linguistic or taxonomically related categories, e.g. “sunshine” for the word “golf”. According to the LASS theory those responses are more likely to come the simulation system and should occur later.

The results from the experiment supported the LASS theory, because linguistically-related responses were produced earlier than object-situation responses. Taxonomic responses occurred, as expected, slower than linguistically-related responses and faster than object-situation responses.

Property Generation

Similar to the experiment above, the participants received a word and had to name properties of this word, e.g. one question was: “What characteristics are typically true of dogs?” The participants had 15 seconds time to name properties. According to the LASS theory it was predicted, that participants should come up with more object-situation responses, since they have more time and linguistically related responses should again come up before the other categories of responses. The experiment showed that both assumptions were true.
A variation of this experiment were the brain-activity of the participants was measured in a fMRI-scanner was also done. The findings of the fMR experiment supported the findings from the other experiment, it also showed that at first those brain regions which are responsible for language processing are active and later on associative brain areas get activated (areas of the brain which are “supporting”, meaning that they connect and process information coming from other brain areas), which suggests that the simulation is run in this brain areas (similar results have been found by another neurological study from Marc Jeannerod, that was also looking at simulations in the brain, this study also found that an associative area, the parietal cortex, was highly active while the brain was running simulations).

Property Verification

In another experiment participants were first shown an object (e.g. “horse”) and a sequence of properties (e.g. “mane”). The participants had to decide whether a property was part of the first word, the times to verify whether a property is part of the object and the accuracy of doing so. For testing the linguistic system the participants got either word pairs with true object-properties (like “taxi” — “meter”) or false object-property pairs (like “pliers” — “river”). According to the LASS theory, these kinds of pairs should trigger a response from the linguistic system. The simulation theory was tested in a similar manner, but notice that associated word pairs like “banana” — “monkey”, are false object-properties, since “monkey” is not a property of “banana”. According to the LASS theory, the participants would take longer for these kinds of words longer to classify, because more in-depth knowledge, coming from a simulation has to be gathered, in order to make this decision.
The results of this experiment support the claims made by the LASS theory: for false and unrelated examples the participants were using their linguistic system, for false but related examples, they were using their simulation system. A variant of this experiment was also done with fMRI scanners and it again supported the earlier findings.

Abstract Concepts

Findings that support the dual-coding theory suggest, that memory tends to be better for concrete concepts that for abstract concepts. The reason for this is that concrete concepts are represented by both systems, while abstract concepts are only represented by the linguistic system. This view got a lot of support and also studies using brain-imaging are supporting it. The authors point out that this view has one fundamental logical problem: just a language alone can not represent a concept, because it is not grounded in experience (the authors are referring to Searle's Chinese room argument) and therefore suggesting that simulations play a central role when processing abstract concepts. They also found evidence for this conclusion, also in fMRI studies. The reason for why the earlier studies showed different results was because of their study design: they only found evidence for linguistic processing of abstract concepts, because they used tasks that encouraged superficial linguistic processing.
In a different study, in order to not only encourage the superficial linguistic processing, participants were shown a word for an abstract concept for 5 seconds and then they had to verify it for a series of pictures, e.g. if the concept “convenience” would fit for a picture of a politician speaking to a crowd. And the study showed, that related brain areas (e.g. in the case of the politician speaking to a crowd, the ones that are also active for social interactions) got activated, indicating that a simulation was run. Also, the linguistic system was not more active for abstract concepts than for concrete concepts.

Conclusion

The LASS theory suggest, that when it comes to representing knowledge two types of representation systems are involved: the linguistic system and the situated simulation system. Knowledge about concepts comes from an interplay of those two systems, where the linguistic system is giving more quick and shallow responses, which are enough for quite some occasions, while the simulation system is doing more in-depth, but slower responses. The LASS theory is similar to the dual-coding theory, but they do differ, since the dual-coding theory suggests, that deep conceptual processing happens in both systems.