User:Hpsych3139/sandbox

Perceptual Simulation

Perceptual simulation is the idea that the mind can imitate what our senses perceived from an environmental stimulus. Once that stimulus is no longer present the brain can still construct this object in the mind. What the brain has constructed internally seems very similar to what was perceived by the senses externally. This is a major topic in the field of cognitive science. The Idea that the sensory areas of the brain can mimic what the sensory organs can perceive when the stimulus is no longer present was first suggested by David Hume back in the 18th century. Many cognitive scientists today continue to study the idea of perceptual simulation and its importance to cognitive science.

History

David Hume was one of the first people to initiate the idea of perceptual simulation. His idea was that when a stimulus is presented to us we first perceive it with our sensory organs; our eyes, ears, mouth, nose and touch. The sensory organs then pass this information onto the sensory areas of the brain that process and store the information of the perception. When the stimulus is no longer present, the perception still remains in the brain where it can be called upon to produce a simulation identical to the stimulus that was actually perceived.

Another key figure in perceptual simulation was Alexander Bain . He agreed with Hume about the idea that the brain can revive the senses that we perceived from a stimulus. He also proposed that it would only make logical sense that the mimicked perception occupied the same parts of the brain as the original perception did. He believed this perception could not be revived in any other area of the brain as he states in his book “it is only in this way that its identity can be preserved, a feeling differently embodied would be a different feeling”. If the perception was simulated by different areas of the brain then it would not be the same perception. He also proposed the idea that all thoughts and ideas must originate from a perceptual experience. Thoughts and ideas are not just spontaneously developed in the brain, they must be developed by simulating things that were actually perceived in the physical world. Any knowledge you have must be a combination of things you have once perceived in the world.

figure 1

figure 2

With the cognitive revolution in the mid twentieth century many cognitive scientists moved away from this idea of perceptual simulation. Many started to believe that the representations of perceptual stimuli in the brain were non-perceptual. John B. Watson was one of the scientists to reject the idea of perceptual simulation. He did not believe that thinking of an object involved actually perceiving it in our mind, for example, he did not believe that thinking of a dog would lead us to construct a visual image of a dog in our mind. Instead he believed that thinking of an object consisted of subtle movements of the vocal apparatus, simply thinking of saying the word dog instead of constructing a visual image of one. His idea was further elaborated by Zenon Pylyshyn who believed the brain constructed what we perceive in the world by mental descriptions, words that would relate to the object. For example, when thinking of a dog barking, perceptual simulation theorist would say that the auditory areas of the brain would simulate actual barking in the mind verses Pylyshyn would say the mind would simple produce the word barking. The newer theorist differed from Hume and Bain on their relation to perception. Hume and Bain believed that the mental representation would mimic the perceptual representation (see figure 1).Watson and Pylyshyn held a non-perceptual view that mental representations were made up of amodal symbols (see figure 2). Amodal symbols are stored in different areas of the brain then the areas used to perceive the stimulus. When the stimulus gets perceived and transferred to these new areas it gets changed into a different symbol, like a word.

However more current research shows that perceptual simulation is evident and that it plays an important role in human cognition.

Perceptual Simulation

What is it?

First let’s start with Perception. In perception an external stimulus is delivered to the sensory organ. This causes activation of the sensory areas of the brain where information about the stimulus is processed and then stored. Simulation is the idea of imitating something. Perceptual simulation then is the idea of the brain imitating what was perceived by the senses in an earlier experience. For example when an individual sees an apple, later when the apple is no longer present in their environment, their mind can imitate the appearance of the apple. This can be done for any of the five senses: vision, hearing, tasting, touching and smelling. When an individual thinks of an object in their head like “dog” the brain is imitating their perceptual experiences of a dog. The brain may imitate the visual aspect of the dog, like the shape and colour or it may imitate one of the other senses like the auditory aspects of a dog like barking or panting. The brain may imitate any of the five senses when an individual thinks of an object in their head.

How Does the Brain Simulate Perception?

It starts with perceptual symbols. Perceptual symbols are records in the brain that represent the perception. When we perceive a stimulus, parts of the brain are activated and a specific neuronal circuit is activated, this circuit gets recorded so that these perceptual symbols can be called upon later for simulation. There are four core properties to perceptual symbols

properties of perceptual symbols

Property	Function	Example
Schematic Perceptual symbols	A perceptual symbol is not the record of the entire brain state that underlies a perception. Instead it is only a small subset that represents a coherent aspect of the state. Perceptual symbols only represent what you attend to during perception	When an individual is perceiving a dog they may not have attended to the shape of its ears or the length of its tail even though they would have been part of the individuals perception. Since they were not attended to the shape of the ears and the length of the tail would not get stored in the perceptual symbol of dog
Once a Perceptual symbol is stored, it is not concrete	Additional perceptual experiences can lead to altering connections of the original pattern of the perceptual symbol	When an individual perceives a dog, the perceptual symbol of the colour of the dog is stored, for example, brown. However, when they experience a new dog with a different colour, for example, black, the connections made for "dog" get changed to adapt to the new information. Now their perceptual simulation of dog includes both brown and black
perceptual symbols are componential, not holistic	The brain can represent parts of a perception instead of all the components of a perceptual system at once	The brain can represent parts of a dog like its shape and size while leaving out parts like its colour or movement
Perceptual symbols are multi-modal	Perceptual symbols operate on all sensory areas of the brain, vision, hearing, smelling, tasting, touching	when the brain is perceiving a dog, not only are the perceptual symbols encoding the visual aspects, but they also encode the sound of barking, the feeling of fur and so on

Following perceptual symbols is simulators. Simulators are a combination of perceptual symbols in long term memory. Related symbols become organized into a simulator so that it can construct a simulation in the absence of a stimulus. For example, when an individual is looking at a house they perceive the front of it, then the side of it, then the back of it, and then the inside of the house. If all these perceptions remained independent of each other then the brain could never simulate the whole object “house”. However the brain combines these perceptions together in a simulator so that a simulation of the entire object can be produced.

Next part of perceptual simulation are the frames. Since individuals experience more than one house they must have something in the brain that organizes all the experiences of houses into one category of houses. This is the idea of frames. A frame contains information from all experiences with a specific object. A frame is never experienced wholly; an individual would never experience all the houses they have experienced at once when thinking of "house". Instead parts of the frames information gets activated to construct a simulation. With these frames a simulator can produce potentially an infinite set of simulations because it can take part of one experience and combine it with a part of another experience. For example, when an individual constructs a simulation of a dog in their mind they can take the colour of one dog they have perceived and combine that with the shape of a different dog they have perceived.

Lastly the simulation. The simulation is what the simulator produces by taking perceptual symbols from the frame of the object the individual is trying to produce.

Support for Perceptual Simulation

Support that human cognition uses perceptual simulation

Support for the idea that we imitate perception in our brain as oppose to just having amodal symbols such as lists of features can be seen in the study on Mental Rotation Task . In this study subjects are given pairs of 3 dimensional objects that would be the same shape or different shape. One of the 3 dimensional objects in the pair would be rotated a certain degree. The subjects were told to determine if the shapes were identical or not. The researchers hypothesized that if the subjects were using perceptual simulation and mentally rotating the objects in their mind then it should take longer to answer if they are the same shape or not the more rotated the second image was from the first. This is exactly what they found. They concluded that the subject must have been doing mental rotation, simulating the shape in their mind and then rotating it to determine if they were the same. The researchers also suggested that degree of rotation would not affect the time it takes if they used some kind of amodal featured list. Since they found a time effect based on the degree of rotation they concluded that the subjects used perceptual simulation as opposed to amodal lists.

More support of perceptual simulation can be found in patients with unilateral neglect syndrome. This is an inability to perceive stimuli in one half of the visual field. In this study patients were asked to imagine standing on a familiar street standing one direction. When asked what they remember they could only name things from one side of the street, for example the right side. When asked to stand the other direction (opposite side becomes the right side) they could only remember what was on the other side of the street. This is evidence showing that their mind takes on a similar form as perception.

Support for perceptual simulation is also found in studies on language. One study found that subjects are faster at reading a sentence if they were primed with a previous sentence that shared the same sensory modality. For example if primed with the sentence “cookies are yummy” vs “music is loud” they will be faster at reading “sugar is sweet” if they were primed with “cookies are yummy” because they both use the sensory modality “taste”. If reading the sentence "cookies are yummy" did not involve sensory areas of the brain then this priming effect would not occur because amodal systems would store information about "cookies are yummy" and "music is loud" in the same non-perceptual areas of the brain so both sentences would prime "sugar is sweet" equally. However, since "cookies are yummy" elicits activation in taste sensory areas of the brain, the brain will be already activated for "sugar is sweet" because this sentence also elicits activation of taste sensory areas.

Support that perceptual simulation uses similar brain areas as perception

Hume and Bain had suggested that perceptual simulation involves the same brain areas as perception does. With Imaging studies and studies on brain damage patients there is evidence that similar areas are used for both.

In one study researchers took MRIs of individuals during visual recall, which would be considered perceptual simulation using the visual senses. They found activation in the primary visual cortex during visual recall which was also activated during visual perception.

Another way to see if similar areas are used for perceptual simulation and perception is to look at patients with brain damage in an area involved in perception to see if there are deficits also in perceptual simulation. In one study done on the auditory sense they found that damage to the right temporal lobe, an area involved in music, lead to deficits in both the perceptual and simulation task. This indicates that the right temporal lobe is involved in both perception and simulation.

One criticism to the view that perceptual simulation is an exact replica of what we perceive is that not all the same brain areas are used in the simulation that are used in the perception, so the simulation is not identical. There is evidence that some areas of brain damage produce deficits in perception but not simulation and evidence that some areas of brain damage produce deficits in simulation but not perception.

Influence in Cognitive Science

Perceptual simulations major influence in cognition is how humans develop, understand and represent concepts. We develop our understanding of concepts and categorizations of concepts by simulating previous experiences with these concepts in our mind to see if the new experience relates to it. Concepts are a very important part of human cognition because our conceptual systems are what helps us get along in the world so an individual knows what is dangerous versus what is safe. Perceptual simulation helps develop these conceptual systems and represent them. Once simulators are established they put into action a basic conceptual system that represents categorizes of things in the world and produces categorical inferences. For example an individual has a category for “snake” based on the different features; they slither, they have no arms or legs, they make a hissing sound, they bite, they are dangerous. Based on this knowledge, when the individual comes across something that is slithering and hissing, they can simulate in your mind a snake and realize this new object they are encountering must be a snake and therefore is probably dangerous. Once a simulator becomes established for a category, we can use it to categorize new stimulus when we encounter them . If the simulator for a category can produce a satisfactory simulation of a perceived stimulus then it belongs to the category. If it cannot produce a simulation, the new stimulus does not belong to that category so the brain must try a different simulator or create a new one.

Perceptual simulation can also lead to productivity of concepts. The human mind can produce an infinite number of concepts that go far beyond those experienced. Productivity is the ability to construct an unlimited number of complex representations from a finite number of symbols by combining the symbols. For example one can simulate a dog never encountered before such as a pink dog. They can do this because perceptual symbols for colours become organized together so simulations of a dog can cycle through the symbols within the field of colours to try out various combinations.

Limitations to Perceptual Simulation

Limitation	Meaning	Reason
Simulations are always partial and sketchy	a simulation is never a complete representation of the perception. leaves information out	use selective attention. Not all information in the perceptual field gets attended to and processed to be stored for later simulation.
Simulations are likely to be biased	the stored perceptual information goes beyond what is presented. for example, see a series of dots in a circle formation, the brain stores it as a circle not a bunch of dots	biased towards storing information in the most simplest way so it is easier to process
simulator is not just a collection of what we perceive from the environment	a simulator is not just what comes from the external stimulus, internal factors such as genetics influence our simulation of perception	genetics play a role in how we interpret things. we might interpret something differently than someone else perceiving the samething so the external stimulus gets stored differently. For example, someone who has a fear of dogs versus someone who does not. When both are perceiving the same dog, their mental representations of that dog will be different because one finds it frightening and the other does not
perceptual simulation is not good at abstract concepts	difficult to use the senses to simulate concepts that are unobservable, unable to be perceived with your senses eg. thought, what does thought look like, feel like? sound like?	harder to perceptually simulate these concepts because people don't really perceive them with their senses.