A Comparative Analysis of Three Quantitative Research Studies

Quantitative Research Studies Analysis

Introduction

Quantitative analysis is useful for its objectivity and clarity, but abstract concepts inevitably must be used in the interpretation of the results. Qualitative methods may not be as objective, as they rely on constructivist and interpretive research paradigms (i.e. inductive rather than deductive reasoning in analysis), but quantitative methods become just as conceptual when trying to make sense of the research findings. The three research studies discussed in this paper provide good examples to illustrate this point.

Dawes and Bishop, Kidd and Hogben, and Torppa, Tolvanen, Poikkcus, Ekiund, Lerkkanen, Leskinen, and Lyytinen all use quantitative analysis and the normative research paradigm. I chose them, because they all adhere to measurable and quantifiable data in a way that is consistent with the "real" experience that is valued in positivist research. None of the studies uses a qualitative design, but a discussion of the research findings and their implications reveals the fact that abstract concepts will always come into play in one form or another. All three research studies focus on the ability to process and interpret sensory information, and this is a phenomenon that will always necessarily remain somewhat abstract.

In a certain sense, any research study intended to fill in gaps in understanding will involve abstract concepts. The difference between qualitative work and the quantitative work discussed here is that abstract concepts are included within data analysis during qualitative research, whereas quantitative research reduces abstract concepts to a level that can be codified into numbers. In the sections that follow, quantitative methods are shown to produce measurable, objective results, but abstract thinking and speculation must be used during the researchers' interpretation of data.

Dawes and Bishop (2008):

Maturation of Sensory Processing in Children

Dawes and Bishop (2008) examined the process of maturation undergone by auditory and visual temporal processing abilities in children. This is an example of a situation in which researchers may be interested in understanding the processes of sensory and cognitive development over the course of time. For this reason, it is useful to establish "normal" levels of sensory functioning for various groups organized according to age and other factors. One example of a normative research study intended to explain auditory and visual development is Dawes and Bishop (2008).

Their work is focused on sensory development and the development of physical structures of the peripheral auditory system, but this inquiry is complicated by the fact that other, non-sensory factors are also at work to determine responsiveness to stimuli. A lack of attention, a lack of motivation, or an inability to use strategic listening can all interfere with test results, and with young children repeated trials are often necessary - which compromises the validity of results. Therefore, special care needs to be taken in order to promote accuracy in studies intended to build understanding of physical development in this regard.

Thus, Dawes and Bishop used normative research to examine auditory and visual processing abilities among adults and children. Three aims of this study, as explained by Dawes and Bishop, include the following:

- The researchers sought to characterize the development of audio and visual processing tasks, especially those associated with the study of reading and language disorders. However, one non-temporal auditory task and one temporal visual task were also used. All of these were used to asses within group and within subject variability.

- A relationship has been hypothesized to exist between auditory and visual processing. In this regard, the researchers were also interested in testing the hypothesis that common mechanisms are used to process both auditory and visual information; in order to do this, it was necessary to determine if the development of sensitivity to auditory stimuli is related to the development of sensitivity to visual stimuli.

- The study was designed to test whether or not children may have their language and literacy development impacted by impairments to auditory processing. It has been suggested that impaired auditory development can be associated with impaired ability to a deficit in speech processing, and that a link can therefore exist between auditory perceptual impairment and language/reading issues. Therefore, the researchers sought to identify any possible relationships between perceptual tasks to a speech based clinical measure of auditory processing (the SCAN-C) and a measure of communication skills (The Children's Communication Checklist (Bishop, 2003).

Several visual and auditory temporal processing tests were used, including the speech-based clinical measure of auditory processing, the SCAN-C, and also to a measure of communication skills, the Children's Communication Checklist (CCC-2; Bishop, 2003). The procedure involved assessing participants and comparing results with the norm for their respective age ranges.

The researchers note that a significant amount of variance was not accounted for by auditory and visual procedural skills, and they suggest that this could suggest differences in auditory processing from one individual to the next. They found that with detection of FM stimuli did improve over the age range of children studied (6 to 10 years) but that IRN detection did not improve. This led the researchers to draw tentative conclusions about pitch processing development.

Strengths and Weaknesses

This normative research study used quantitative analysis of scores on measures of auditory and visual processing. In the sense that the performances of participants were compared against norms that may not be truly representative of the total population, this normative study is subject to error. Depending on the accuracy of the norms against which the numbers in the present study are compared, the results may be misleading. Another limitation of this study is its small sample size. Only 18 adults and 98 children were involved in the study, and only 21 children provided test-retest data.

Another questionable aspect of the study is represented by the implications of its finding that perceptual task performance generally increases with age. With the exception of temporal processing and coherent form perception, perceptual task performance improves with age, but this does not necessarily suggest that sensory processing is improving with age. The researchers note that the improved performance may be related to other factors, such as capacity for communication, concentration, and strategic listening.

In this case, quantitative methods were used for the purpose of providing hard data, but accuracy is always lost when abstract concepts - such as sensory processing ability - are codified into numbers. This is especially true in the present example, because certain factors such as concentration and level of interest are difficult to quantify. However, the researchers did attempt to mitigate such problems by using controls such as the Children's Communication Checklist (Bishop, 2003). To the greatest extent possible, the influence of other major factors was taken into account.

Kidd and Hogben (2007):

The Saltation Illusion and Dyslexia

Kidd and Hogben (2007) conducted a research study based on the notion that auditory temporal processing ATP abnormalities may be associated with the disorder known as dyslexia. One of the most prominent sources of adversity associated with dyslexia is phonological awareness, the ability to extract, reflect on, and manipulate the sounds of language (Kidd and Hogben, 2007). For this reason, it has been hypothesized that auditory sensory processing ability can play a role in dyslexia. Again, normative research is used.

The quantitative analysis used by Kidd & Hogben was intended to answer the following questions:

- Do dyslexic readers perform poorly on auditory tasks compared with competent readers?

- Is auditory task performance related to reading and phonological processing skills? (p. 983)

Kidd and Hogben cite research evidence that suggests significant differences in the processing of auditory information when the performances of dyslexic individuals are compared to those of non-dyslexic individuals. However, they also cite some research studies that show that discrepancies may be caused by the influence of dyslexic individuals who perform extremely poorly and other studies in which no difference is observable at all. Thus, they question the validity of the results supporting the auditory temporal processing (ATP) hypothesis for explaining dyslexia.

As one of many research studies designed to identify any auditory factors contributing to dyslexia, this work focused on a particular aspect of auditory processing. The method used here to assess temporal processing of auditory information among individuals with dyslexia involves the Auditory Saltation illusion. This phenomenon is an illusion of motion perceived by most individuals when exposed to multiple auditory stimuli in different locations. People ordinarily perceive one stimulus to "jump" from one location to another. The basis for using this phenomenon to assess ATP among dyslexic individuals is that the experience of the auditory saltation illusion relies on the, "capacity to derive a veridical percept from rapidly presented stimuli" (p. 984) and that the limits of ATP may therefore be reflected by the temporal boundary between the percepts of auditory saltation and veridical localization.

Research studies involving this method, like other research studies searching for auditory attributes of dyslexia, have produced conflicting results, so Kidd and Hogben sought to revisit these findings by investigating the question of whether or not poor saltation task performance characterizes dyslexia. Participants included 19 adults with dyslexia and 20 competently reading adults. Two trials of a 2-alternative forced-choice saltation task were used together with standard measures of reading and phonological processing in order to determine whether dyslexia is "characterized by poor saltation task performance and whether saltation thresholds are related to reading and phonological processing" (982).

Research instruments included the Matrices subtest of the Kaufman Brief Intelligence Test (Kaufman & Kaufman, 1990), which was used to establish whether or not participants were of acceptable levels of nonverbal intelligence for this study, and the Phonemic Decoding Efficiency subtest of the Test of Word Reading Efficiency (TOWRE; Torgesen,Wagner & Rashotte, 1999), which was used to establish the presence of dyslexia in the experimental group. Among the participants with dyslexia, poorer saltation thresholds were observed. However there was considerable overlap among threshold distributions overlapped considerably, and thresholds alone were insufficient for predicting group membership.

Strengths and Weaknesses

In this quantitative work, previously establishes norms were used as a reference point by which to assess the performance of a saltation task - which requires answering a question to determine the limits at which one is affected by the Auditory Saltation Illusion. The saltation task may be useful as a measure of auditory temporal processing, but it may not be useful for measuring capabilities that are significantly correlated with dyslexia. If a relationship is observed to exist between saltation task performance and dyslexia, it is not necessarily a causal relationship.

Another limitation of this study comes from the fact that a two-alternative forced choice process was used. With only two alternatives from which to choose, the potential for inaccuracy is increased by a lack of options. Although the two-alternative method is useful for its clarity, it also inevitably reduces and simplifies the information being measured. They write: "We chose to use this task primarily because it is objective but also because our previous studies had shown that this task is more reliable and more efficient when used in individuals who are largely inexperienced with psychophysics (Kidd & Hogben, p. 992).

The researchers attempted to compensate for the reductionist nature of qualtitative work by using careful discernment in their interpretation and discussion of the results. They note, for example, that negative relationships were observed between log saltation thresholds across groups and measures of reading and phonological processing but that these relationships were not significant.

While quantitative methods are generally free of speculation, they are still subject to inexplicable inconsistency in results. That is the case in this example. The researchers find themselves struggling to interpret the data and make sense of inconsistency - and this is where abstract reasoning and conjecture end up playing a role in quantitative work. In qualitative work, the researcher is free to speculate as part of data analysis, but in quantitative work speculation takes place while interpreting the numbers.

Torrpa et al (2007):

Factors Affecting Word Recognition and Reading Comprehension

Torppa, Tolvanen, Poikkcus , Ekiund, Lerkkanen, Leskinen, and Lyytinen (2007) conducted a study was based on Jyvaskyia Longitudinal Study of Dyslexia (JLD), a study of 100 children in Finland who are genetically predisposed to dyslexia and 100 children in a control group. The researchers were interested in learning whether or not word recognition and reading comprehension could be indicative of heterogonous developmental paths. Although recognition of words and reading comprehension are strongly related, they are not exactly the same, and they may develop at different rates. The researchers also examined the relationship between early literacy skills and future reading ability; in order to do this, they devised five categories of reading development in the follow-up study: poor readers, slow decoders, poor comprehenders, average readers, and good readers.

The focus of the study was to test the claim that identifiable subtypes exist within the total population of students with reading difficulties and to give special attention to the development of word recognition capability compared to reading comprehension ability. The researchers acknowledge that word recognition and reading comprehension are strongly related, but they cite studies that suggest subtle differences. The researchers also examined the results of JLD study participants with regard to language and literacy development, reading experience, and the familial dyslexia risk.

Word recognition and reading comprehension were assessed in comparison with one another and also in relationship to early reading experience and skills, including receptive language, expressive language, morphological knowledge, phonological awareness, letter knowledge, memory, rapid serial naming, and intelligence,

Assessments were administered to 1803 students, and of these 193 were student in the JLD study. The assessments were given twice during the first grade year, when the children were seven years old, and twice the following year. After excluding some students who for one reason or another did not participate in both studies, the results of 1750 students were analyzed in comparison with that of the students in the JLD study. The researchers found that, of the children studied, those with familial risk for dyslexia were overrepresented in the "slow decoder" subtype.

One weakness of this study was that it is based on categories derived based on a mixture modeling procedure that may have been subject to error. Various factors could be at work in any process used to assess the skill with which students decode words. The findings of the study are all based on the relationships of various factors to categories into which students were assigned based on their performances on various assessments. As with any standardized test, factors such as the students' skill at test taking strategy, level of fatigue, and level of interest on the day of the test all may have come into play. This research method uses abstract terms (i.e. "slow decoder," "poor comprehender") as a basis for quantitative analysis.

Torrpa et al attempted to mitigate the disadvantages of this method by using a population that is quite large. The students with known genetic risk for dyslexia numbered fewer than one hundred, but they were compared with a large population of almost two thousand other students. Furthermore, multiple assessments were given; word recognition and reading comprehension were assessed four times during the first grade school year.

Ethical Considerations

One ethical issue that is relevant in any research study involving children is that they are not old enough to give their legal consent about the process. They will be affected by their participation, either positively or negatively, so care should be taken to ensure sensitive and confidential treatment of all information obtained. For example, one example of a negative consequence of the study conducted by Torppa et al might be that a student in the group at risk for dyslexia may suffer an injury to the self-esteem when her or his parents explain the reason for their involvement with the study - especially in cases where the child is classified as a slow decoder, slow comprehender, or slow reader. In fact, these labels seem unnecessarily blunt and derogatory; perhaps some effort should have been made to establish subtypes in a more tactful way.

Summary

Despite the different focuses of these research studies, all three studies involved the use of established norms. Dawes and Bishop (2008) examined the process of maturation undergone by auditory and visual temporal processing abilities in children, and their norms were established by previous research data. Kidd and Hogben (2007) were interested in determining whether dyslexic readers perform poorly on auditory tasks compared with competent readers and whether auditory task performance related to reading and phonological processing skills (p. 983), and the norms for this study were based on the performance of "competent readers. Torppa et al (2007) were interested in learning whether or not word recognition and reading comprehension could be indicative of heterogonous developmental paths and also in examining the relationship between early literacy skills and future reading ability, and in this study the researchers established norms by deriving them from the data that they were analyzing. In every case, the establishing of norms creates the possibility of error.

Furthermore, as stated in the introduction to this paper, quantitative analysis produces results that still must be interpreted in order to be useful. Even though quantitative, positivist work is generally free of speculation, a certain amount of speculation must take place as scholars make use of the data yielded by them. Dawes and Bishop (2008), Kidd and Hogben (2007), and Torppa et al (2007) have all adhered to research paradigms that are not interpretive, and for this reason a process of interpretation must take place for those who read the studies and reflect on the results.

References:

Dawes & Bishop. (2008). Maturation of visual and auditory temporal processing in school-aged children. Journal of Speech, Language, and Hearing Research (51): 1002-1015.

Kidd & Hogben. Does the auditory saltation stimulus distinguish dyslexic from competently reading adults? Journal of Speech, Language,and Hearing Research. (50): 982-998.

Torppa, Tolvanen, Poikkcus , Ekiund, Lerkkanen, Leskinen , & Lyytinen. (2007). Reading development subtypes and their early characteristics. The International Dyslexia Association.