Education and WorkSIRS.

School Improvement Research Series
(SIRS)

Research You Can Use

Close-Up #11

Teaching Thinking Skills

Kathleen Cotton

Perhaps most importantly in today's information age, thinking skills are viewed as crucial for educated persons to cope with a rapidly changing world. Many educators believe that specific knowledge will not be as important to tomorrow's workers and citizens as the ability to learn and make sense of new information. --D. Gough, 1991

INTRODUCTION

Throughout history, philosophers, politicians, educators and many others have been concerned with the art and science of astute thinking. Some identify the spirit of inquiry and dialogue that characterized the golden age of ancient Greece as the beginning of this interest. Others point to the Age of Enlightenment, with its emphasis on rationality and progress (Presseisen 1986, p. 6).

In the twentieth century, the ability to engage in careful, reflective thought has been viewed in various ways: as a fundamental characteristic of an educated person, as a requirement for responsible citizenship in a democratic society, and, more recently, as an employability skill for an increasingly wide range of jobs.

Deborah Gough's words quoted at the beginning of this report typify the current viewpoint in education about the importance of teaching today's students to think critically and creatively. Virtually all writers on this subject discuss thinking skills in connection with the two related phenomena of modern technology and fast-paced change. Robinson, for example, states in her 1987 practicum report:

Teaching children to become effective thinkers is increasingly recognized as an immediate goal of education....If students are to function successfully in a highly technical society, then they must be equipped with lifelong learning and thinking skills necessary to acquire and process information in an ever-changing world (p. 16).

Beyth-Marom, et al. (1987) underscore this point, characterizing thinking skills as means to making good choices:

Thinking skills are necessary tools in a society characterized by rapid change, many alternatives of actions, and numerous individual and collective choices and decisions (p. 216).

The societal factors that create a need for welldeveloped thinking skills are only part of the story, however. Another reason that educators, employers, and others call for more and better thinking skills instruction in schools is that American young people, in general, do not exhibit an impressive level of skill in critical or creative thinking. The following observation from Norris's 1985 review is typical:

Critical thinking ability is not widespread. Most students do not score well on tests that measure ability to recognize assumptions, evaluate arguments, and appraise inferences (p. 44).

Likewise, Robinson notes that:

While the importance of cognitive development has become widespread, students' performance on measures of higher-order thinking ability has displayed a critical need for students to develop the skills and attitudes of effective thinking (p. 13).

There is yet another major force behind the call for improved thinking skills instruction. Educators are now generally agreed that it is in fact possible to increase students' creative and critical thinking capacities through instruction and practice. Ristow (1988) notes that, in the past, these capacities have often been regarded as:

a fluke of nature, a genetic
predisposition....qualities [that] are either possessed or not possessed by their owner and that education can do very little to develop these qualities (p. 44).

Ristow then goes on to say:

However, a great deal of the research currently being reported indicates that the direct teaching of creative skills can produce better, more creative thinkers.

Presseisen makes this point even more forcefully, asserting that:

The most basic premise in the current thinking skills movement is the notion that students CAN learn to think better if schools concentrate on teaching them HOW to do so (p. 17).

DEFINITIONS

Thinking skills. Critical thinking. Creative thinking. Higher-order thinking. Those who take an interest in this field of study soon realize that they cannot go tossing off these terms in a casual manner, since there are no universal agreements as to their precise meanings.

CRITICAL THINKING, for example, has been variously defined as:

Beyer goes on to say that "critical thinking has two important dimensions. It is both a frame of mind and a number of specific mental operations" (p. 271). Norris (1985) agrees, stating that:

Having a critical spirit is as important as thinking critically. The critical spirit requires one to think critically about all aspects of life, to think critically about one's own thinking, and to act on the basis of what one has considered when using critical thinking skills (p. 44).

Lists of alternative definitions could also be generated for other terminology commonly used in the thinking skills literature. In an attempt to come to terms with these definitional differences, Alvino, in his 1990 "Glossary of Thinking-Skills Terms," offers a set of definitions which are widely--though not universally--accepted by theorists and program developers. For purposes of the present report, these definitions are applicable. They include:

THE THINKING SKILLS RESEARCH

This summary is based on a review of 56 documents. Thirty-three of these are reports of research studies or reviews and are cited, with annotations, in the Key References section of the bibliography. Twenty-three are descriptive, theoretical, or guidelines documents or are concerned with research in areas other than the effectiveness of programs and practices. These reports are itemized in the General References.

Of the 33 key documents, 22 are research studies or evaluations, and 11 are reviews or syntheses of research. Subjects of these investigations include: general (or unspecified) student populations - 12 reports, elementary students - 9, secondary students - 9, and both secondary and postsecondary students - 3. The research involved regular, gifted, EMR, and Chapter 1 student populations; a representative range of racial/ethnic groups; and a balance of urban, suburban, and rural settings. Only three of the reports deal with student populations outside the United States. Five of the reports have teachers as well as students as their subjects.

The effects of many individual practices and whole programs were investigated. Many reports looked at the effects of instruction in various clusters of higherorder thinking skills, including analysis, synthesis, and evaluation, together with the related skills and subskills of making predictions, making inferences, self-questioning and other metacognitive functions, formulating hypotheses, drawing conclusions, elaborating, solving problems, making decisions, identifying assumptions, determining bias, recognizing logical inconsistencies, and others.

Other reports looked at specific instructional practices, such as tutoring, using thinking skills software programs, and using advance organizers. Five were concerned with the effects of training teachers to conduct thinking skills instruction. The full thinking skills programs investigated by the research are discussed in the section on findings.

Outcome areas were likewise numerous, including student achievement as measured by assessments in the areas of reading comprehension, mathematics, general science, biology, physics, chemistry, art, social studies, and geography. Other outcome areas studied include SAT scores, commercial and locally developed higher-order thinking skills test scores, IQ test scores, and behavioral outcomes such as engaged time/level of participation. Research studies addressing effects on student attitudes or self-concepts were insufficient to allow for any general conclusions.

RESEARCH FINDINGS

EFFECTS ON STUDENT OUTCOMES

THINKING SKILLS INSTRUCTION ENHANCES ACADEMIC ACHIEVEMENT. A broad, general finding from the research base is that nearly all of the thinking skills programs and practices investigated were found to make a positive difference in the achievement levels of participating students. Studies which looked at achievement over time found that thinking skills instruction accelerated the learning gains of participants, and those with true or quasi-experimental designs generally found that experimental students outperformed controls to a significant degree. Reports with such findings include: Barba and Merchant 1990; Bass and Perkins 1984; Bransford, et al. 1986; Crump, Schlichter, and Palk 1988; Freseman 1990; Haller, Child, and Walberg 1988; Hansler 1985, Horton and Ryba 1986; Hudgins and Edelman 1986; Kagan 1988; Marshall 1987; Matthews 1989; MCREL 1985; Nickerson 1984; Pearson 1982; Pogrow 1988; Ristow 1988; Riding and Powell 1985, 1987; Robinson 1987; Sadowski 1984-85; Snapp and Glover 1990; Sternberg and Bhana 1986; Tenenbaum 1986; Whimbey 1985; Wong 1985; and Worsham and Austin 1983.

RESEARCH SUPPORTS INSTRUCTION IN MANY SPECIFIC SKILLS AND TECHNIQUES. Gains on learning and intelligence measures were noted in response to providing instruction in a variety of specific techniques, including:

VARIOUS INSTRUCTIONAL APPROACHES ENHANCE THINKING SKILLS. In addition to instruction in specific mental operations, research also supports the use of several teaching practices as effective in fostering the development of thinking skills, including:

These practices are also associated with increases in student engaged time/level of participation (Cotton 1988; MCREL 1985; Freseman 1990).

COMPUTER-ASSISTED INSTRUCTION HELPS TO DEVELOP THINKING SKILLS. Although the approach taken differed across the various kinds of instructional software studied, all of the CAI programs designed to improve students' thinking skills were effective. The programs focused on skill building in areas such as verbal analogies, logical reasoning, and inductive/deductive thinking. Supportive research includes Bass and Perkins (1984); Horton and Ryba (1986); Riding and Powell (1985, 1987); and Sadowski (1984-85). The computer-oriented HOTS Program originally developed for Chapter 1 elementary students also shows positive results; however, developer Stanley Pogrow (1988) notes that the heart of the program is the teacher-student interaction called for by HOTS activities.

RESEARCH SUPPORTS THE USE OF SEVERAL SPECIFIC THINKING SKILLS PROGRAMS. The research consulted in preparation for this report is not all-inclusive, and no doubt there are studies and evaluations supporting the effectiveness of programs other than those identified here. The following programs are cited here because they are widely known and used, are representative of the kinds of thinking skills programs in current use in schools, and have been studied by researchers. Programs found to be effective include:

TRAINING TEACHERS TO TEACH THINKING SKILLS LEADS TO STUDENT ACHIEVEMENT GAINS. Developers and researchers of most of the effective programs cited above claim that teacher training is a key factor in the programs' success. The majority of these programs have a strong teacher training component, and developers consider this training to be as important as the program content in bringing about the learning gains noted. In addition to the key role of staff development in the programs cited by reviewers Sternberg and Bhana (1986) and Baum (1990), a positive relationship between teacher training and student achievement was also identified in studies conducted by Crump, Schlichter, and Palk (1988); Hudgins and Edelman (1986); MCREL (1985); and Robinson (1987).

PROGRAMS, STRATEGIES, AND TRAINING ARE IMPORTANT, BUT... In drawing conclusions about the effectiveness of particular thinking skills instructional strategies, whole programs, or staff development approaches, several researchers also offer a caveat to those who might make curriculum decisions based on this information. Essentially, they say, yes, these programs, practices and training activities CAN BE effective, but their effectiveness is partially dependent on factors other than the methodologies themselves. In a typical expression of reservation, Sternberg and Bhana, at the conclusion of their 1986 review of several thinking skills programs, write:

...the success of a given program depends on a large number of implementation-specific factors, such as quality of teaching, administrative support, appropriateness of the program for the student population, and the extent to which the program is implemented in the intended manner" (p. 67).

Sternberg and Bhana's observation about the match between program and student population also serves to remind us of another truism: just as there is no one certifiably "best" approach to teaching many other things, there is no one best way to teaching thinking skills. At the end of a study comparing different approaches to teaching critical thinking, Bass and Perkins write, "Like so much educational research, our final results were not supportive of just one instructional technique" (p. 96).

THE CONTROVERSIES IN THINKING SKILLS INSTRUCTION

Is it better to teach thinking skills to students via infused programs or separate curricula? Is it better to teach these skills directly or to create situations whereby students learn them inferentially through being placed in circumstances which call for them to apply these skills? How much classroom time is required in order for thinking skills instruction to be effective, i.e., for students to master higher-order skills and be able to transfer them to other learning contexts? Is successful thinking skills instruction partly a matter of establishing a certain classroom climate, one that is open and conducive to "thinking for oneself"?

Differences of opinion--sometimes profound ones--have been expressed by theorists, developers, and classroom teachers in response to these questions. What does the research say?

INFUSED VERSUS SEPARATE PROGRAMS. Of the demonstrably effective programs itemized above, about half are of the infused variety, and the other half are taught separately from the regular curriculum. In addition, while several documents in the thinking skills literature (e.g., Bransford, et al. 1984; Baum 1990; and Gough 1991) offer support for infusion of thinking skills activities into subjects in the regular curriculum, others (Freseman 1990; Matthews 1989; Pogrow 1988; and Baum 1990) provide support for separate thinking skills instruction. The strong support that exists for both approaches (in the research, not to mention in the views of warring experts) indicates that either approach can be effective. Freseman represents what is perhaps a means of reconciling these differences when he writes, at the conclusion of his 1990 study:

...thinking skills need to be taught directly before they are applied to the content areas....[I] considered the concept of teaching thinking skills directly to be of value especially when there followed immediate application to the content area...(p. 48).

In a similar vein, Bransford (1986) says:

"Blind" instruction [in which students are not helped to focus on general processes or strategies nor to understand how new concepts and strategies can function as tools for problem solving] does not usually lead to transfer to new tasks....as the instruction focuses on helping students become problem solvers who learn to recognize and monitor their approaches to particular tasks, transfer is more likely to occur (p. 69-70).

DIRECT VERSUS INFERENTIAL LEARNING. Approaches such as inquiry development and the techniques used in the HOTS program involve guiding students through the process of figuring out what strategies to apply and where those strategies can lead them. Some researchers and developers (e.g., Hansler 1985; Orr and Klein 1991; Pogrow 1988) offer evidence that this approach enables students to learn thinking skills, rather than merely learning ABOUT them. HOTS Program developer Stanley Pogrow calls the process "controlled floundering"-- "floundering" because students must feel their way (along a line of reasoning, for example), but "controlled" because teachers stay with them and assist them to work through the steps of their tasks.

Others favor direct instruction in the steps of whatever thinking process the teacher wants the students to learn. Teachers using this approach typically demonstrate the process using events and ideas which are familiar to the students and then applying the same generic process to unfamiliar material, usually new content from the school curriculum. Proponents claim that many students, particularly those whose out-of-school lives have offered little exposure to higher-order thinking, cannot be expected to develop these skills inferentially and must be taught them directly. The efficacy of direct instruction in a variety of thinking skills is demonstrated in the work of Freseman (1990); Herrnstein, et al. (1986); Pearson (1982); and Wong (1985), among others.

Again, it would appear that either approach can be effective, and a blend of the two may well be most effective. Pearson, for example, favors both direct instruction and guided practice:

...I think the justification exists for placing more emphasis on direct explicit teaching, interactive discussions, substantive feedback, and control and self-monitoring strategies (p. 26).

TIME REQUIREMENTS FOR THINKING SKILLS INSTRUCTION. This topic is not so much the subject of controversy as of uncertainty; even the experts seem uncertain as to how much time should be devoted to thinking skills activities in order for students to learn those skills well. Of course, time requirements will be different for different students, and experience shows that some students become adept thinkers with no explicit instruction at all.

The research can only address the time question obliquely, since most researchers don't design studies in which different groups of students are exposed to different amounts of instruction. What the research does show is that those commercial or locally developed programs which have made substantial differences in students' academic performance are quite timeintensive. Instrumental Enrichment requires three to five hours of instruction per week over approximately two years. Philosophy for Children, a K-12 curriculum, calls for three 40-minute periods weekly. Odyssey is made up of 100 45-minute lessons. Programs such as HOTS, which are designed especially for at-risk students who have limited experience in understanding and applying higher-order strategies, require even more time. Pogrow (1987) says:

It takes an extensive amount of time to produce results--at least 35 minutes a day, four days a week, for several months, for true thinking skills development to occur (p. 12).

Given these kinds of time demands, conducting meaningful thinking skills instruction clearly requires a high level of staff commitment and administrative support.

CLASSROOM CLIMATE. Research shows that positive classroom climates characterized by high expectations, teacher warmth and encouragement, pleasant physical surroundings, and so on, enhance all kinds of learning. In the thinking skills literature, there is an especially strong emphasis on the importance of climate. Orr and Klein (1991) go so far as to say that:

Teachers and administrators should systematically evaluate the general culture of their classrooms and schools and should estimate how this culture affects their ability to promote critical reasoning habits among students
(p. 131).

The point made by these writers and many others is that moving beyond one's mental habits and experimenting with new ways of looking at things--the very stuff of thinking skills instruction--involve risk. In order for students to be willing to participate in such activities, they:

...need to feel free to explore and express opinions, to examine alternative positions on controversial topics, and to justify beliefs about what is true and good, while participating in an orderly classroom discourse (Jerry Thacker, as quoted in Gough 1991, p. 5).

Here again, research can provide illumination only indirectly; however, it is the case that the validated programs in the research base include both teacher training components and classroom activities which emphasize establishing open, stimulating, supportive climates.

How might this be accomplished? Thacker lists twelve recommended teacher behaviors, all of which will be familiar to good teachers, for fostering a climate conducive to the development of thinking skills:

SUMMARY

Findings emerging from the thinking skills research reviewed in preparation for this report include:

In both school settings and in the world outside of school, it is crucial for people to have "skills in questioning, analyzing, comparing, contrasting, and evaluating so that [they] will not become addicted to being told what to think and do" (Freseman 1990, p. 26). Putting into practice the findings from the thinking skills research can help schools to teach these skills and students to gain and use them.

KEY REFERENCES

Barba, R. H., and Merchant, L. J. "The Effects of Embedding Generative Cognitive Strategies in Science Software." JOURNAL OF COMPUTERS IN MATHEMATICS AND SCIENCE TEACHING 10/1 (1990): 59-65.

Examines the effects of science software which incorporates generative cognitive strategies such as recall, integration, organization, elaboration, and visualization. Experimental students outperformed controls on tests of both higher-order cognitive skills and science content knowledge.

Bass, G. M., Jr., and Perkins, H. W. "Teaching Critical Thinking Skills with CAI." ELECTRONIC LEARNING 14/2 (1984): 32, 34, 96.

Investigates the relative effects of CAI and conventional instruction in enhancing the critical thinking skills of seventh grade students. Of the four kinds of skills taught and tested, students performed better after CAI instruction in two and better after conventional instruction in the other two.

Baum, R. "Finishing Touches--10 Top Programs." LEARNING 18/6 (1990): 51-55.

Identifies ten thinking skills programs that have proven effective in increasing students' cognitive performance. Programs are organized by whether they involve infusing thinking skills into the established curriculum or provide a separate thinking skills course. All programs include teacher training.

Bransford, J. D.; Burns, M. S.; Delclos, V. R.; and Vye, N. J. "Teaching Thinking: Evaluating Evaluations and Broadening the Data Base." EDUCATIONAL LEADERSHIP 44/2 (1986): 68-70.

As a follow-up to Sternberg and Bhana's article in the same issue (see below), these authors cite data indicating that approaches which involve teaching from a metacognitive or problem-solving perspective enhance skill transfer in reading comprehension, mathematics, and writing, thus producing gains in student achievement.

Cotton, K. CLASSROOM QUESTIONING: CLOSE-UP NO. 5. Portland, OR: Northwest Regional Educational Laboratory, May 1888.

Synthesizes findings from 37 research reports on the relationship between teacher's classroom questioning behavior and a variety of student outcomes. Found that, when teachers ask higher cognitive questions, conduct redirection/probing/reinforcement, and/or increase wait time, the cognitive sophistication of student responses increases.

Crump, W. D.; Schlichter, C. L.; and Palk, B. E. "Teaching HOTS in the Middle and High School: A District-Level Initiative in Developing Higher Order Thinking Skills." ROEPER REVIEW 10/4 (1988): 205-211.

Presents results of an evaluation of the effectiveness of training nearly all teachers and administrators in an Alabama school district in the Talents Unlimited model for teaching higher-order thinking skills. Teacher self-reports were positive, and the performance gains of middle and high school students on thinking skills assessments indicated that the program was successful.

Eriksson, G. I. "Choice and Perception of Control: The Effect of a Thinking Skills Program on the Locus of Control, Self-Concept and Creativity of Gifted Students." GIFTED EDUCATION INTERNATIONAL 6 (1990): 135-142.

Compares the effects of two thinking skills programs on the affective variables of locus of control and self-concept, and on the creativity measures of originality, fluency, and flexibility. Thinking skills instruction had a significant, beneficial effect on locus of control and creativity, but no significant effect on self-concept.

Farha, J. L., and Milbrandt, M. S. "BEYOND LOLLIPOP TREES": TEACHING THINKING SKILLS THROUGH ART. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA, March-April 1989. (ED 307 968)

Seeks to determine the effect of an innovative art education program on the art content learning and general achievement of fourth, fifth, and sixth graders. Experimental students outperformed controls on learning of art content; effects on achievement were inconclusive.

Freseman, R. D. IMPROVING HIGHER ORDER THINKING OF MIDDLE SCHOOL GEOGRAPHY STUDENTS BY TEACHING SKILLS DIRECTLY. Fort Lauderdale, FL: Nova University, 1990. (ED 320 842)

Examines the outcomes produced by a geography curriculum which included direct instruction in visualizing, prioritizing, summarizing, making inferences through drawing analogies, and problem solving using divergent thinking. Effects on the achievement and self-concepts of participating seventh graders were generally positive, though below predicted levels in some areas.

Haller, E. P.; Child, D. A.; and Walberg, H. J. "Can Comprehension Be Taught? A Quantitative Synthesis of ‘Metacognitive' Studies." EDUCATIONAL RESEARCHER 17/9 (1988): 5-8.

Synthesizes findings from 20 studies on the effects of teaching students the metacognitive skills of awareness, monitoring, and regulating of their own understanding of material read. Findings indicate a substantial effect of such instruction on students' reading comprehension.

Hansler, D. D. STUDIES ON THE EFFECTIVENESS OF THE COGNITION ENHANCEMENT TECHNIQUE FOR TEACHING THINKING SKILLS, 1985. (ED 266 432)

Describes research studies demonstrating the effectiveness of cognition enhancement activities (also called inquiry training) in developing elementary and secondary students' skills in constructing and testing hypotheses.

Herrnstein, R. J.; Nickerson, R. S.; de Sanchez, M.; and Swets, J. A. "Teaching Thinking Skills." AMERICAN PSYCHOLOGIST 41/11 (1986): 1279-1289.

Investigates the effects of a year-long thinking skills course on the general mental abilities and specific cognitive skills of Venezuelan seventh graders. Experimental students significantly outperformed controls on both general and special measures.

Horton, J., and Ryba, K. "Assessing Learning with Logo: A Pilot Study." THE COMPUTING TEACHER 14/1 (1986): 24-28.

Compares the performance on cognitive tasks by New Zealand junior high school students who had used the Assessing Learning with Logo program with the skills of those who had received no thinking skills instruction. Logo-using students outperformed controls in five of six areas.

Hudgins, B., and Edelman, S. "Teaching Critical Thinking Skills to Fourth and Fifth Graders Through Teacher-Led Small-Group Discussions." JOURNAL OF EDUCATIONAL RESEARCH 79/6 (1986): 333-342.

Investigates the effects on both teacher and student behavior of a teacher training program in teaching critical thinking skills to small groups. Students exhibited increased critical thinking behavior in some areas after their teachers received training.

Kagan, D. M. "Evaluating a Language Arts Program Designed to Teach Higher Level Thinking Skills." READING IMPROVEMENT 25/1 (1988): 29-33.

Reports on the effects of a language arts program with a strong focus on developing high-order thinking skills in sixth graders. Lessons were developed using portions of two commercial instructional packages. Participating students made significant gains, as measured by four different divergent thinking skills instruments.

Marshall, J. D. "The Effects of Writing on Students' Understanding of Literary Texts." RESEARCH IN THE TEACHING OF ENGLISH 21/1 (1987): 30-63.

Examines the effects of three kinds of writing assignments on the later comprehension and analysis of literary texts, as measured by a test calling for a written interpretation of a literary work. Students whose earlier assignments had called for more lengthy and in-depth writing exhibited most sophisticated reasoning and higher quality written products than those who had previously done only brief, restricted writing or no writing at all.

Matthews, D. B. "The Effect of a Thinking-Skills Program on the Cognitive Abilities of Middle School Students." CLEARING HOUSE 62/5 (1989): 202-204.

Reports findings of a study of the effects produced by a nondomain-specific thinking skills program upon the Cognitive Abilities Test scores of eighth and ninth graders. Post-test students outperformed pre-test students on all subtests except one.

Mid-Continent Regional Educational Laboratory. REPORT OF THINKING SKILL INSTRUCTIONAL ACTIVITIES. Denver, CO: Mid-Continent Regional Educational Laboratory, Inc., August 30, 1985. (ED 273 970)

Examines the effects of training teachers in how to foster in their students 18 higher-order thinking subskills in the three areas of learning-to-learn skills, content thinking skills, and basic reasoning skills. Assessments indicated that students of participating teachers improved in all areas addressed by the teacher training program.

Nickerson, R. S. RESEARCH ON THE TRAINING OF HIGHER COGNITIVE LEARNING AND THINKING SKILLS. Final Report. Report No. 5560. Cambridge, MA: Bolt, Beranek and Newman, Inc., 1984. (ED 248 455)

Summarizes reports in five categories--programs for teaching thinking skills, theoretical and conceptual reports, literature reviews and syntheses, reports of empirical research, and other related reports.

Pearson, P. D. A CONTEXT FOR INSTRUCTIONAL RESEARCH ON READING COMPREHENSION. Champaign, IL: University of Illinois at Urbana-Champaign/Cambridge, MA: Bolt, Beranek and Newman, Inc., 1982. (ED 215 307)

Reviews research which demonstrates that skills such as drawing inferences, making predictions, and monitoring one's own understanding of written material can be explicitly taught. Observes that direct teaching of these skills is often overlooked and should be included in reading instruction.

Pogrow, S. "HOTS: A Thinking Skills Program for AtRisk Students." PRINCIPAL 67/4 (1988): 19-24.

Describes the HOTS (Higher Order Thinking Skills) program, elaborates on its appropriateness for the learning needs of at-risk students, and provides results from several user sites demonstrating the program's effectiveness.

Riding, R. J., and Powell, S. D. "The Effect on Reasoning, Reading and Number Performance of Computerpresented Critical Thinking Activities in Five-year-old Children." EDUCATIONAL PSYCHOLOGY 7/1 (1987): 55-65.

Compares the reading, mathematics, and reasoning test performance of children participating in 13 weeks of computer-presented problem-solving activities with the performance of children who did not participate. Treatment subjects outperformed controls on reasoning tasks; other results were mixed.

Riding, R. J., and Powell, S. D. "The Improvement of Thinking Skills in Young Children Using Computer Activities: A Replication and Extension." EDUCATIONAL PSYCHOLOGY 6/2 (1986): 179-183.

Offers results of a replication study of the effects of computer-presented thinking skills activities on the reasoning test performance of four-year-olds. As in the original study, treatment students outperformed controls, but to an even greater degree.

Ristow, R. S. "The Teaching of Thinking Skills: Does It Improve Creativity?" GIFTED CHILD TODAY 11/2 (1988): 44-46.

Determines the effects of locally developed creative thinking program on the creative thinking test performance of third graders. Program participants outperformed controls on two of the three areas assessed--flexibility and originality--but not on the third--fluency.

Robinson, I. S. A PROGRAM TO INCORPORATE HIGH-ORDER THINKING SKILLS INTO TEACHING AND LEARNING FOR GRADES K-3. Fort Lauderdale, FL: Nova University, 1987. (ED 284 689)

Studies the effect of a series of teacher inservice sessions and the use of the Junior Great Books Program on (1) teachers' skills in and attitudes toward providing thinking skills activities to third graders, and (2) the performance of students on tasks at different levels of Bloom's THE TAXONOMY OF EDUCATIONAL OBJECTIVES. Significant improvement of both teachers' and students' skill levels resulted.

Sadowski, B. R. "Research Notes: Critical Thinking and CAI." JOURNAL OF COMPUTERS IN MATHEMATICS AND SCIENCE TEACHING 4/2 (1984-85): 12-13.

Reviews three studies on the effects of using CAI programs to improve students' critical thinking skills. Results were generally positive among the research surveyed.

Snapp, J. C., and Glover, J. A. "Advance Organizers and Study Questions." JOURNAL OF EDUCATIONAL RESEARCH 83/5 (1990): 266-271.

Seeks to determine the effect on the quality of students' answers to study questions produced by exposing students to advance organizers prior to reading the study material. Experimentals outperformed controls in responding to both lower- and higher-order questions.

Sternberg, R. G., and Bhana, K. "Synthesis of Research on the Effectiveness of Intellectual Skills Programs: Snake-Oil Remedies or Miracle Cures?" EDUCATIONAL LEADERSHIP 44/2 (1986): 60-67.

Reviews a large number of evaluations of five of the most widely used thinking skills programs--Instrumental Enrichment, Philosophy for Children, SOI (Structure-ofIntellect), Problem Solving and Comprehension, and
Odyssey. Finds most of the evaluations sufficiently flawed to call into question the positive claims made for the programs evaluated.

Tenenbaum, G. "The Effect of Quality of Instruction on Higher and Lower Mental Processes and on the Prediction of Summative Achievement." JOURNAL OF EDUCATIONAL RESEARCH 80/2 (1986): 105-114.

Investigates the relative effects of (1) enhanced cues, participation, reinforcement, and feedbackcorrectives, (2) mastery learning, and (3) conventional instruction on the science and math achievement of elementary and secondary students. The cues, etc. condition was most effective and the most dramatic differences were noted in the higher mental processes.

Tierney, R. J.; Soter, A.; O'Flahavan, J. F.; and McGinley, W. "The effects of reading and writing upon thinking critically." READING RESEARCH QUARTERLY 24/2 (1989): 134-173.

Seeks to determine the learning conditions under which college students are most likely to think critically. Those who read an editorial passage about the specified topic before writing a letter to the editor exhibited more critical thinking than students in other learning conditions.

Whimbey, A. "Test Results From Teaching Thinking." In A. L. Costa (ed.). DEVELOPING MINDS: A RESOURCE BOOK FOR TEACHING THINKING. Alexandria, VA: Association for Supervision and Curriculum Development, 1985, 269- 271.

Reviews studies of the effects thinking skills instruction on test performance and on the transfer of cognitive skills to new and different situations. Found that thinking skills instruction both improves academic performance and enables students to become better problem solvers in other situations, both in and outside of school.

Wong, B. Y. L. "Self-Questioning Instructional Research: A Review." REVIEW OF EDUCATIONAL RESEARCH 55/2 (1985): 227-268.

Reviews 27 studies on the effects of instruction in self-questioning on the prose processing skills of students in the elementary grades through college. Found such instruction effective when it is direct and explicit, training is sufficient, and adequate time to generate questions is provided.

Worsham, A. W., and Austin, G. R. "Effects of Teaching Thinking Skills on SAT Scores." EDUCATIONAL LEADERSHIP 41/3 (1983): 50-51.

Investigates the effect of THINK, a thinking skills program, on the SAT verbal scores of low-performing high school seniors in Baltimore. Participants outperformed controls to a "highly significant" degree on all three SAT verbal measures: vocabulary, reading comprehension, and total score.
GENERAL REFERENCES

Alvino, J. "A Glossary of Thinking-Skills Terms." LEARNING 18/6 (1990): 50.

Defines major terms used by thinking skills theorists and program developers. Reminds readers that there are some definitional differences among experts in the field.

Anderson, R. N.; Greene, M. L.; and Loewen, P. S. "Relationships Among Teachers' and Students' Thinking Skills, Sense of Efficacy, and Student Achievement." THE ALBERTA JOURNAL OF EDUCATIONAL RESEARCH 24/2 (1988): 148-165.

Examines relationships among teachers' and students' sense of self-efficacy, reasoning abilities, and student achievement. Twenty-four teachers and their third and sixth grade students participated. Several types of relationships were noted, but causal links were difficult to determine.

Beyer, B. K. "Critical Thinking: What Is It?" SOCIAL EDUCATION 49/4 (1985): 270-276.

Addresses the confusion experienced by educators and others as to what constitutes critical thinking. Offers a brief history of views of critical thinking, information about what critical thinking is NOT, and a current definition agreed upon by many specialists in the field.

Beyth-Marom, R.; Novik, R.; and Sloan, M. "Enhancing Children's Thinking Skills: An Instructional Model for Decision-Making Under Certainty." INSTRUCTIONAL SCIENCE 16/3 (1987): 215-231.

Presents a rationale for providing students instruction in thinking skills and offers a model to be used in teaching students the particular skill of decision-making under certainty, i.e., when full information about each choice is available.

Costa, A. L. "How Can We Recognize Improved Student Thinking?" In A. L. Costa (ed.). DEVELOPING MINDS: A RESOURCE BOOK FOR TEACHING THINKING. Alexandria, VA: Association for Supervision and Curriculum Development, 1985, 288-290.

Identifies student behavioral indicators that teachers can observe and record to determine whether instruction in thinking skills is having a beneficial effect on students' intellectual development. The author claims that attention to these indicators often provides more useful information than do typical standardized tests.

Gallo, D. "Educating for Empathy, Reason and Imagination." THE JOURNAL OF CREATIVE BEHAVIOR 23/2 (1989): 98-115.

Argues that the attributes which characterize empathy correlate with those of effective critical thinking and imagination. Thus, empathetic "role-taking" can sharpen one's imaginative and critical thinking capacities.

Gore, K. "Diskovery: Computers and Thinking Skills: The HOTS Program." LANGUAGE ARTS 68/2 (1991): 153- 158.

Describes the HOTS (Higher Order Thinking Skills) program in relation to the learning needs of at-risk students, and argues that the close match between learning needs and program provisions has made possible the considerable success of the HOTS curriculum.

Gough, D. THINKING ABOUT THINKING. Alexandria, VA: National Association of Elementary School Principals, 1991. (ED 327 980)

Summarizes five study reports concerning the nature of higher-order thinking skills and the most effective methods for teaching them. While focusing on different aspects of the topic, the authors of these reports are in agreement that thinking skills should be integrated across the curriculum rather than taught in isolation.

Heiman, M., and Slomianko, J., eds. THINKING SKILLS INSTRUCTION: CONCEPTS AND TECHNIQUES. Washington, DC: National Education Association, 1987. (ED 306 559)

Presents a series of 35 essays on the importance of teaching thinking skills and on instructional strategies for developing these skills in students. The essays address thinking skills instruction within and across disciplines.

Hunter, Brother E. "Focus on Critical Thinking Skills Across the Curriculum." NASSP BULLETIN 75/532 (1991): 72-76.

Argues that, to be prepared for the future, students need to have questioning minds and develop the skills of critical thinking. Claims that instruction and practice in thinking skills should be incorporated into all secondary-level academic courses.

Kober, N. "What Critical Thinking Approach is Best?" THE SCHOOL ADMINISTRATOR 48/1 (1991): 14-17.

Offers transcripts of brief interviews with three experts in the field of teaching thinking skills: Robert Swart, co-director of the Center for Teaching Thinking in Andover, MA; E.D. Hirsch, author of CULTURAL LITERACY and other educational books; and Matthew Lipman, author/developer of the PHILOSOPHY FOR CHILDREN curriculum.

Norris, S. P. "Synthesis of Research on Critical Thinking." EDUCATIONAL LEADERSHIP 42/8 (1985): 40-45.

Reviews research on the elements of critical thinking; the nature, merits, and flaws of various critical thinking tests; and frequently encountered errors in reasoning. Argues that critical thinking must be coupled with content knowledge and that better assessments are needed to determine accurately the effectiveness of critical thinking instruction.

Onosko, J. J. "Comparing Teachers' Instruction to Promote Students' Thinking." JOURNAL OF CURRICULUM STUDIES 22/5 (1990): 443-461.

Compares the classroom practices, instructional materials, and assignments of teachers who place a great deal of importance on students' development of higher-order thinking skills with those of teachers who place less value on such skill development.

Orr, J. B., and Klein, M. F. "Instruction in Critical Thinking as a Form of Character Education." JOURNAL OF CURRICULUM AND SUPERVISION 6/2 (1991): 130-144.

Argues that the development of critical thinking capabilities results less from being taught disembodied "skill bits" than from being "initiated into school communities that grant importance to critical reasoning." Claims that, ideally, instruction in critical thinking should be aimed at fostering the character trait of a "critical spirit."

Orton, R. E., and Lawrenz, F. "A Survey and Analysis of Factors Related to the Teaching of Critical Thinking in Junior High Mathematics Classrooms." JOURNAL OF RESEARCH AND DEVELOPMENT IN EDUCATION 23/3 (199): 145- 155.

Presents the results of a survey of the classroom practices of 146 junior high mathematics teachers to determine the incidence of activities which tend to foster critical thinking skills and those which tend to impede development of these skills. Outcomes were mixed; implications for teacher education are cited.

Paul, R. W. "Bloom's Taxonomy and Critical Thinking Instruction." EDUCATIONAL LEADERSHIP 42/8 (1985): 36- 39.

Acknowledges the extensive influence of Bloom's THE TAXONOMY OF EDUCATIONAL OBJECTIVES on educators' thinking and planning. Provides an analysis and critique of the Taxonomy, calling attention to its limitations for developing a critical thinking curriculum.

Paul, R. W. "Critical Thinking Research: A Response to Stephen Norris." EDUCATIONAL LEADERSHIP 42/8 (1985): 46.

Identifies research sources beyond those identified by Norris' article (see above) that can be helpful in understanding critical thinking and designing instructional programs. Also argues that educators should concentrate on the kinds of thinking and thinking problems that are most common in our everyday lives and address the many obstacles to rational thought.

Pogrow, S. "The HOTS Program: The Role of Computers in Developing Thinking Skills." TECHTRENDS 32/2 (1987): 10-13.

Describes ways the HOTS (Higher Order Thinking Skills) program approach differs from other programs using computers to enhance thinking skills. Emphasizes the need for instructional software to "force" the average and below average student to make inferences, rather than simply teaching them about inferences.

Pogrow, S. "Teaching Thinking to At-Risk Elementary Students." EDUCATIONAL LEADERSHIP 45/7 (1988): 79-85.

Describes the HOTS (Higher Order Thinking Skills) program, provides an overview of users' successful experiences with the program, and cites a series of conclusions about the learning needs of at-risk students and ways to meet those needs.

Presseisen, B. Z. CRITICAL THINKING AND THINKING SKILLS: STATE OF THE ART DEFINITIONS AND PRACTICE IN PUBLIC SCHOOLS. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA, April 1986. (ED 268 536)

Focuses on past and present definitions of and approaches advocated for teaching critical thinking skills, including overviews of the writings of key theorists. Discusses the current interest in improving students' critical thinking skills and instructional approaches for doing so. Makes recommendations for future research.

Renner, J. W. CURRICULA WHICH PROMOTE REASONING. Paper presented at the United States-Japan Seminar on Science Education, Honolulu, HI, September 14-20, 1986. (ED 278 565)

Cites findings indicating that ninth grade science students achieve more content understanding, retain more information, and exhibit higher IQ measurements when taught using an approach called the "learning cycle" than when traditionally taught.

Schmidt, P. "A Concern About...Restructuring Learning to Improve the Teaching of Thinking and Reasoning for All Students." CONCERNS 22 (1991): 24-31.

Discusses issues related to higher-order learning, including definitions, relevance in contemporary life, principles to guide educators as they restructure their approaches to include greater focus on higher-order learning, teacher training needs, and recommended steps for state education agencies.

Strahan, D. B. "A Research-Based Approach to Effective Middle Grades Instruction." CLEARING HOUSE 60/4 (1986): 149-155.

Discusses research on the development of reasoning skills in adolescents in relation to the general effective teaching research. Describes and gives examples of a "guided thinking" approach to instruction which is based on findings from these two bodies of research.

Tingle, J. B., and Good, R. "Effects of Cooperative Grouping on Stoichiometric Problem Solving in High School Chemistry." JOURNAL OF RESEARCH IN SCIENCE TEACHING 27/7 (1990): 671-683.

Compares the chemistry problem-solving performance of students working individually with those in cooperative groups whose members were organized heterogeneously according to proportional reasoning ability. No statistically significant differences were noted.

This publication is based on work sponsored wholly, or in part, by the Office of Educational Research and Improvement (OERI), U.S. Department of Education, under Contract Number RP91002001. The content of this publication does not necessarily reflect the views of OERI, the Department, or any other agency of the U.S. Government.

November 1991

Northwest Regional Educational Laboratory
101 SW Main, Suite 500, Portland, OR 97204
Telephone (503) 275-9500
School Improvement Program: School Improvement Research Series
Last Update: 9/7/97 - Contact Webmaster