SIMON BARON-COHEN’S STUDY ON THE THEORY OF MIND AND AUTISTIC ADULTS (2001)
In 1997, Baron-Cohen and his colleagues conducted a study using the Eyes test. After reviewing their results, they decided that a “revised” version of the test needed to be made so that the study could be improved and conducted again.
The problems with the original test are listed below.
- Answering the question involved a forced choice option between only two responses. The range of scores was too narrow to produce accurate results.
- The test scores were too narrow to differentiate between people with actual autism and “lesser variant” or “broader phenotype” groups (e.g. first relative of the autistic person).
- Ceiling effects are likely to occur due to the narrow score range; this means individual differences are difficult to detect.
- The emotions described in the test were of two types: Basic and Complex. Basic emotions (e.g. happy and sad) were too easy and made ceiling effects more likely to occur.
- Some items were linked to gaze direction and perception (e.g. ignoring and noticing). This factor was considered to be a clue that made the test easier to complete for some people.
- There were more female faces than male faces, so a possible bias could have occurred.
- The two response options were always semantic opposites of each other (e.g. sympathetic and unsympathetic), which was making the test far too easy. It’s like asking: is this black or white?
- Since the test involved linking pictures and words, there may have been issues with participants not correctly understanding the meanings of certain words.
To fix the first three limitations, the following steps were taken:
- The number of items (photographs) was increased from 25 to 36.
- The number of forced-choice response items was increased from 2 to 4.
To fix the fourth and fifth limitations, the following steps were taken:
- The test items only involved Complex mental states; Basic states were eliminated.
- All items linked to gaze perception were removed from the test.
To fix the last three limitations, the following steps were taken:
- An equal number of female faces and male faces were used.
- The correct answer was accompanied by three similar words, as opposed to opposites.
- A glossary was given to participants; they could consult it at any time during the study.
- To test a group of adults with Asperger’s or HF autism with the revised Eyes test.
- To test a sample of normal adults with the revised Eyes test and investigate if an inverse correlation between scores on the AQ and Eyes test could be found.
- To test if previous sex differences could be replicated with the revised Eyes test.
- Adults with AS/HFA will score significantly lower on mental state judgements on the revised Eyes test. However, they will not be impaired on the control task of gender judgements.
- Adults with AS/HFA will score significantly higher in the AQ test.
- Normal adult females will score higher than males on the revised Eyes test.
- Scores on the AQ and Eyes test will be inversely correlated.
The participants were sorted into four groups.
Group 1 – A group of 15 males diagnosed with AS or HF-A, who were recruited through support groups and the U.K. National Autistic magazine. They were from different educational levels and socioeconomic classes.
Group 2 – A large group of 122 “normal” adults from adult community/educational classes in Exeter and public libraries in Cambridge. They came from a variety of educational and professional backgrounds.
Group 3 – A group of 103 “normal” male (53) and female (50) undergraduate students at Cambridge University. As this university has relatively high eligibility criteria, it can be assumed that members of this group have high IQ.
Group 4 – A group of 14 randomly selected individuals who were similar to Group 1 in their age and IQ.
Two main tests were used in this study: the Autism Quotient (AQ) and the revised Eyes test.
- Autism Quotient test: This is a self-report questionnaire that measures the degree to which a normal adult of normal IQ possesses traits that are linked to autism. It is scored from 0 to 50.
- Eyes test: This test measures a person’s “theory of mind”. It consists of 35 items, which are pictures of eyes expressing different feelings and emotions. A person chooses a word that best describes the expression from four pre-written options (multiple-choice format). In this study, there was also a glossary of words at the end of this test.
All participants were tested with the revised Eyes test in a quiet room at Cambridge or in Exeter. The test was administered to them inidividually.
Group 1 also completed a gender recognition task while answering the Eyes test; they had to judge the gender of the person whose eyes were being shown. Normal adults were not given this task because previous studies showed that they always achieved ceiling results for this test so, to save time and effort, the researchers only gave the gender recognition task to Group 1.
Group 1, 3 and 4 also completed the AQ test.
All participants were also asked to read the glossary (found at the end of the Eyes test). They could use this glossary during the test to find meanings of any difficult or unfamiliar words.
The following results were collected from the study:
- There was no difference of glossary-usage between groups (no one checked more than two words).
- Group 1 performed significantly worse than the other groups on the Eyes test.
- Group 1 scored 33 or above (out of 36) on the gender recognition control task.
- Group 2, 3 and 4’s performance on the Eyes test was not significantly different from each other.
- Females scored higher than males on the Eyes test.
- Group 1 scored higher on the AQ test than Group 3 and 4.
- In Group 3, males scored higher on the AQ than females.
- There was no correlation between Eyes test and IQ.
- There was no correlation between AQ and IQ.
- In Group 3, the Eyes test was inversely correlated with the social skills and communication categories.
Type of research method
This was a quasi (natural) experiment with self-report.
Type of data collected
Quantitative data (numerical scores on the test).
The factor of autism.
- Collection of quantitative data: Quantitative data (i.e. scores on tests) was collected in this study. This data is all numerical, which means it can be analysed easily while comparisons can also be made (such as the comparison of group scores). It also means that the study’s results are objective, which makes them free from personal bias.
- Replicability and reliability: Because the psychometric tests all have a fixed format with close-ended questions, they can be taken again and again. This makes the study replicable, so other researchers can confirm the consistency of the findings. As the study’s procedure and tasks are replicable, the results are more likely to be reliable.
- Control: The researchers controlled variables like age, sex and IQ. This means that they could be more sure that it was only the factor of Autism that was affecting the scores, as opposed to a factor like age confounding the results.
- Validity: Psychometric tests do not always test what they claim to test. For example, was the Eyes test measuring theory of mind or was it just measuring the participants’ capability of completing an Eyes test? How do we know for sure that theory of mind was tested alone? However, in the original paper, the researchers do attempt to justify the validity of the test.
- Ecological validity and mundane realism: The stimuli were just static images of eyes. In real-life social situations, we interpret emotions of real people who are not stuck in one expression; the situation is much more different. This lowers the ecological validity of the study and creates an issue of mundane realism with regards to the task. Also, the lab setting for some participants also lacks ecological validity.
- Informed consent: This was obtained from all participants.
- Confidentiality: Confidentiality was respected.
- Emotional or physical harm: There is no evidence of the study or tests causing any type of harm.
- The right to withdraw: Participants could withdraw from the study.
- Debriefing: There was no mention of debriefing.
Reference: Baron-Cohen, S., Wheelwright, S., Hill, J., Raste, Y. and Plumb, I. (2001). The ‘Reading the Mind in the Eyes’ Test Revised Version: A Study with Normal Adults, and Adults with Asperger Syndrome or High-functioning Autism. Journal of Child Psychology and Psychiatry. 42(2): 241-251.