Key research findings about questioning skills to promote higher order thinking

A study in the 1980s revealed that over 90% of questions posed by teachers prompted factual recall only.

Dains, D. (1986). Are teachers asking the right questions? Education 1, (4) 368–374.

Another study of that period revealed a figure of 75-80%.

Dillon, J. T. (1988). Questioning and teaching: A manual of practice. New York: Teachers College Press.

When teachers increase the time allowance for students to think before they reply to questions, responses become longer and reach a higher cognitive level. Similar benefits occur when teachers also wait a few seconds after students have answered, plus other students become more participative.

Rowe, M. B. (1974). Wait-time and rewards in instructional variables, their influence on language, logic, and fate control: Part one—Wait time. Journal of Research in Science Teaching, 11, 81-94.

Rowe, M. B. (1986). Wait-time: Slowing down may be a way of speeding up! Journal of Teacher Education, 37(1), 43-48.

Teachers have a tendency to echo student answers; when they do not do so other students are more attentive to their peers’ responses.

 Craig, J. & Cairo, L. (2005). Assessing the relationship between Questioning and Understanding to Improve Learning and Thinking (QUILT) and student achievement in mathematics: A pilot study. Appalachia Educational Laboratory (AEL).

During discussions, thought-provoking statements are viable alternatives to questions in order to stimulate student participation.

Wilen, W.W. (1991). Questioning skills, for teachers. National Education Association of the United States.

Individual, pair or small group interviews conducted by teachers reveal students’ awareness and understanding and provide an opportunity for teachers to notice and develop their questioning competency.

Moyer, P. & Milewicz, E. (2002). Learning to question: Categories of questioning used by preservice teachers during diagnostic mathematics interviews. Journal of Mathematics Teacher Education, 5, 293–315.

I have attached a lesson observation form based upon the above research findings:

Questioning skills observation form

Concept checking – a valuable technique for any educator

In this entry, I will demonstrate the art of concept checking, i.e. using questions to elicit whether students have understood a difficult concept, or are able to distinguish commonly confused concepts.

Concept checking is a tried and tested technique in teaching grammar and vocabulary. Personally, I learnt the technique when I was training to become a TESOL practitioner. There is a good description of concept checking for language teaching at http://www.teachingenglish.org.uk/article/checking-understanding

However, in this entry I would like to show how concept checking can be used for other subject teaching. In every subject there will be terminology that is hard to grasp or is easily confused with a related concept. For instance, how about “Internet” and “World Wide Web”? In everyday conversation they are used interchangeably but they are not the same.

My example concepts are ‘plagiarism’ and ‘copyright infringement’. These two concepts are often confused or conflated no matter how clearly they are presented. On more than one occasion I have been tasked to help university students and lecturers comprehend the difference between them.

Step 1: Present the concept(s) as clearly as possible

“Plagiarism occurs when someone tries to pass off another person’s work or ideas as their own without acknowledgement.” http://edc.polyu.edu.hk/PSP/teacher.htm

“Copyright infringement occurs when someone copies, publishes or distributes a piece of writing, music, picture or other work of authorship without permission.” http://en.wiktionary.org/wiki/copyright

Step 2: Pose concept checking questions

What kind of offence is plagiarism – moral or legal? (moral)

What kind of offence is copyright infringement – moral or legal? (legal)

Who carries out punishment for plagiarism? (higher education institutions / the academic community)

Who carries out punishment for copyright infringement? (courts of law)

How can plagiarism be avoided? (acknowledge sources)

How can copyright infringement be avoided? (seek permission or pay)

Is there any flexibility in plagiarism rules? (No)

Is there any flexibility in copyright law? (Yes)

Reasons for concept checking

  • It intercepts and corrects misconceptions at an early stage, prior to application of knowledge and assessment.
  • It informs decisions whether to re-teach the concepts in another way or continue with, e.g., additional input.
  • It is a tool for formative assessment.
  • It can improve learning.
  • It is a diagnostic tool for differentiation by readiness.
  • It is a tool in inquiry-based learning and other inductive approaches.
  • It is a model of effective study skills.

Loop input: A valuable teaching or training strategy

First off, I want to thank Tessa Woodward for the idea of loop input. She introduced the concept in a 1986 article in The Teacher Trainer journal and articulated it again in her 1991 book, Models and Metaphors in Language Teacher Training: Loop Input and Other Strategies.

Since reading about this training strategy, I have experimented and found it to have the distinct advantage of making the content of professional development workshops highly memorable.

So what is it? In short, with loop input the message of the training and its means of delivery coincide. This is best understood through examples, and in this post I would like to present a couple that I have developed. This has been done before, for example in this entry on John Hughes excellent blog elteachertrainer. However, the additional contribution I would like to make is to provide examples beyond the language teaching profession. Below are two examples of loop input for trainers of all disciplines.

Example 1: An opening activity to introduce the topic of thinking skills

I was tasked by a secondary school to provide a staff development session on the topic of developing students’ thinking skills. When I was writing the materials for this session, I decided that I needed a dynamic, interactive opening activity that would also serve to introduce the topic. My goals for this activity were mainly to engage the participants, but following its completion I wanted to be able to provoke initial reflections on different ways of thinking. This would then lead into more detailed consideration of the ways of thinking that are needed for success in different school subjects, e.g. history or mathematics.

As I was searching for inspiration, I recalled an entertaining language activity in Jill Hadfield’s Intermediate Communication Games that I had used many times in ESOL lessons and had always proved a winner. It is called Detective Work and is a card game designed originally to practise reporting past events. Students work in small groups, turn up one card at a time from a pile, and discuss the clues to the murder that are on the cards. In the process, they should use several verb forms.

I adapted Detective Work in two ways. Firstly, I changed the context of the murder to the school in which I was leading the professional development session. Seeking approval beforehand, I made one of the vice-principals the victim and one of the teachers the perpetrator. This was the cause of some hilarity in the session. Secondly, after the task was completed and the groups had all solved the murder mystery, in plenary I posed the question, “How did you solve the crime?”. This led to a discussion of the distinctions between deductive and inductive reasoning. Having just directly experienced deductive reasoning themselves, teachers appeared not to confuse it with inductive reasoning, as could easily happen. Moreover, sometime later teachers from this school remarked upon that task to me. Their recall was partly due to how much they had enjoyed playing the role of detective and competing with other groups to solve the murder. My hope is that they also recalled the message of the activity.

Example 2: A complete development session on the topic of learner autonomy / self-directed learning

My overall goal for this whole-day staff development session with 70+ teachers at a secondary school was to help teachers grasp the importance of scaffolding the process by which students become more independent. I also hoped that the outcomes of this session would dovetail with earlier professional development at this school on the topic of differentiating instruction.

So, instead of leading a conventional training session, which typically would include input on research findings from me followed by discussion work on how to apply those findings in the school’s distinct learning environments, I opted to give the participants more freedom of choice.

At the outset, I helped teachers to synthesize a plausible working definition of “learner autonomy” from several that had been sourced from the literature. Then, I provided eight possible learning objectives for the session and invited teachers to select two or three that were most relevant to their individual needs. They also selected the sequence in which they would try activities designed to bring them closer to their chosen learning goals. These activities had been designed as self-access materials with accompanying instructions. The teachers were aware of a prescribed, overall time limit and managed their time accordingly.

At the end of the time limit, teachers came together and reflected on whether they had chosen learning objectives wisely, what they had actually learned, and whether they had managed their learning appropriately. Participation in this process led participants naturally to the apparently paradoxical conclusion that independent learning still needs to be guided by teachers, at least until students’ metacognitive awareness has developed sufficiently.

Conclusion

I have found loop input to be a useful addition to my training strategies repertoire. It is not always appropriate, but sometimes combining the message and the process is potent and memorable.

Hadfield, J. (1990). Intermediate communication games. Nelson.

Woodward, T. (1986). Loop input – a process idea. The Teacher Trainer, 1:6-7. Pilgrims.

Woodward, T. (1991). Models and metaphors in language teacher training: Loop input and other strategies. Cambridge University Press.

Problem-Based Learning: Scaffolding in problem crafting and at the problem identification stage

My focus in this 2005 investigation was the appropriacy of learner support (scaffolding) incorporated by Problem-Based Learning (PBL) facilitators in their design of problem materials and offered real-time during Stage 2: Problem Identification of the PBL process devised by and utilized at Temasek Polytechnic (TP) in Singapore. For more information, please see TP’s webpages on PBL at http://www.tp.edu.sg/home/pbl.htm

I collected and examined PBL materials from various Subjects/Courses at Temasek Polytechnic for indications of scaffolding, and interviewed facilitators concerning their beliefs about the quality & quantity of learner support that should be purposefully incorporated into the design of the problem materials and/or offered during Stage 2 itself. I strove to understand how the need to assist certain learners in their comprehension of problem scenarios can be balanced with the generally recognized desirability of authenticity in PBL problem crafting.

My conclusion was that, although PBL is a form of self-directed learning, scaffolding remains appropriate before and at Stage 2 in the interest of inclusiveness.

Contextual information

TP Diploma Courses are sub-divided into Subjects sub-divided into Topics.

The TP PBL Process:

Stage 1:  Group setting

Stage 2:  Problem identification

Stage 3:  Idea generation

Stage 4:  Learning issues

Stage 5:  Self-directed Learning

Stage 6:  Synthesis and Application

Stage 7:  Reflection and Feedback

Introduction

I was prompted to examine scaffolding in PBL problem materials and at the problem identification stage by two articles, and by a request to develop an academic staff development workshop on the topic of advanced PBL problem design.

The first article, by Puntambekar and Hübscher (2005) was not specifically related to PBL, yet did focus on ‘complex learning environments’ such as project-based and design-based classrooms. It raised concerns about a perceived current emphasis on the tools of scaffolding rather than the scaffolding process. The authors claim that “…although the new curricula and software tools now described as scaffolds have provided us with novel techniques to support student learning, the important features of scaffolding such as ongoing diagnosis, calibrated support, and fading are being neglected.”

Hence, I decided to investigate whether such concerns were warranted in the context of PBL in Temasek Polytechnic. In order to make the research task more manageable and to concentrate my thinking on PBL problem design issues, I chose to focus only on the crafting of PBL problems and the facilitation of Stage 2 – problem identification. The latter was included for consideration because I consider that the design of the problem and the facilitation of Stage 2 are inextricably linked.

Another significant issue was raised by Greening (1998) who highlighted the “implications of PBL modes for students with a non-English background and from a cultural perspective”, and supplied evidence of the value of scaffolding in this area. About 10% of TP students are non-Singaporean, therefore the researcher considered it relevant for his secondary focus to be the inclusiveness of PBL problem design for international students and for any student with less well-developed English language proficiency.

The notion of scaffolding

What is scaffolding? In its original sense, it “…consists essentially of the adult ‘controlling’ those elements of the task that are initially beyond the learner’s capacity, thus permitting him to concentrate upon and complete only those elements that are within his range of competence” (Wood, Bruner, and Ross, 1976). As the learner makes progress in gaining mastery of manageable elements, the adult or teacher gradually restores control of the more challenging elements to the learner. The ultimate goal, of course, is independent overall proficiency.

As an everyday example, an adult holds onto a bicycle seat to take control of a child’s balance while the child becomes proficient in keeping her feet on the pedals, holding the handlebars, steering, etc. After some practice, the adult decides to place a hand on the seat and is prepared to grip tightly only if the child loses her balance. Support is reduced and eventually withdrawn. “A good scaffolder looks for the point where a student can go it alone, and allows the individual to proceed on his or her own initiative.” (Hogan, 1997)

Six types of support that can be provided by an adult or expert were identified by Wood, Bruner and Ross:

  • Getting the learner interested
  • Simplifying the task
  • Providing direction
  • Highlighting crucial features of the task
  • Managing frustration
  • Modeling the task

The idea of scaffolding has been connected to the idea of making available a space for growth that matches a learner’s Zone of Proximal Development (ZPD), as defined by Vygotsky (1978). To continually match a learner’s ZPD, an expert follows a process of:

  • Ongoing diagnosis
  • Calibrated support
  • Fading

The scaffolding metaphor was conceived with one-to-one teaching in mind. It is not immediately obvious how to transfer scaffolding to a classroom situation where the facilitator is outnumbered by the learners, or how scaffolding may be integrated into the learning materials that are used in a class of students.

Scaffolding in PBL problems

If scaffolding in the sense described above were incorporated into the design of PBL problem scenarios, what form might it take?

To digress slightly, let us first consider what ‘elements’ may need to be scaffolded in PBL. They include domain-specific knowledge & skills, and process skills such as time management, interpersonal skills, communication, critical thinking, etc. Besides these, there are other enabling elements that are essential for success in PBL but which may not be mentioned in syllabus documents.

For example, since PBL problems are frequently presented as quasi-authentic written statements, language proficiencies such as the following are vital:

  • general/academic vocabulary range sufficient to gain understanding of the problem statement
  • reading sub-skills, e.g., an ability to guess meaning from cotext
  • linguistic versatility and agility sufficient to paraphrase and summarise the main (factual) points of a problem statement
  • dictionary skills to research unknown, opaque lexical items
  • awareness of grammatical structures employed in factual statements and opinions

Here is an example of the scaffolding of assumed language elements in problem design:

After pre-assessment of learners’ reading skills and vocabulary range, a facilitator opts to ‘control’ challenging vocabulary in the problem statement in one of the following ways:

  1. by pre-teaching the challenging vocabulary
  2. by grading the text, for instance by using everyday vocabulary rather than technical terms  
  3. by presenting the problem statement online, with challenging vocabulary hyperlinked to a glossary
  4. by being willing to respond to vocabulary questions

Taking control of the vocabulary element leaves learners free to focus on traditional PBL elements such as discriminating between fact and opinion in the text. Then, if there were a second problem on the same topic, learners could go through the same problem identification stage with reduced lexical support, the degree and nature of which is decided by the facilitator in the learning context based on ongoing diagnosis. For instance, fewer lexical items in the problem statement could be hyperlinked to the glossary.

In the above example, the facilitator intervenes to remove a potential barrier to problem identification through informed calibration of the language content of a problem statement. In addition, by choosing technique 3 above instead of technique 2, the facilitator is able to maintain the authenticity of language used in the problem statement. This is important for them to enter the discourse community of their chosen profession. It also supports learners during self-directed learning because they may be able to use relevant terminology as search items.

As a second example, consider a problem statement in which there is an exophoric reference, i.e. the significance of the reference is not explicit from the text itself, but is obvious to those in a particular situation or culture. For instance, a problem crafter makes reference to consumer behaviour as ‘kiasu’. Singaporeans understand the implications instantly but this is not the case for many international students who have recently arrived in the country. In this situation, the provision of cultural notes could support international students in their comprehension. This technique to provide support can be reduced and withdrawn as the international students become more familiar with Singapore culture, but they need such support in the short-term to give them an equal chance of succeeding in meeting the learning outcomes associated with PBL.

Scaffolding in PBL problem design at TP

Are the above examples of the scaffolding of problem statements characteristic of scaffolding in PBL problem design at TP?

The sample problem statements (and supporting materials) that I scrutinized showed evidence of various forms of cognitive and affective learner support:

  • a ‘hook’ to engage learners
  • some means to activate learners’ schemata
  • sufficient contextualisation
  • relevance to future careers
  • steadily increasing complexity of problems over time and with multiple exposures to the PBL process
  • division of very large problems into smaller, more manageable problems
  • logical sequencing of a series of connected problems
  • multimodal and/or multisensory presentation of information, e.g. memos, live interviews with clients, statistics, etc.

There was faithful application of common principles for effective problem design distilled from the work of Savin-Baden and Howell-Major (2004), Dolmans and Snellen-Balendong (1997), and Barrows (1994) and recommended to facilitators by TP academic staff developers.

Problems should:

–      require the learning of new core knowledge

–      align with learning outcomes of the programme of study

–      adapt to learner’s prior knowledge

–      be presented in a context that is relevant and authentic* to the current or future profession of the learner

–      stimulate learners to elaborate through cues in the problem

–      encourage integration of knowledge

–      stimulate self-directed learning by encouraging generation of learning issues and research

–      encourage discussion and exploration in the subject matter

It was difficult for me to judge the authenticity of the contexts, but I accepted the assurances of the facilitators of these problems. As a linguist I was able to see that there was some substitution of language in problem statements; a layperson’s vocabulary was being used when in real life there would be terminology specific to the professions.

Learning support was purposefully incorporated into PBL problem design but could not be said to adhere to the original notion of scaffolding because of a lack of dynamism and adaptability in the learning materials.

Scaffolding at Stage 2 of the TP PBL Process

PBL facilitators at TP reported scaffolding at Stage 2 through selective and discerning use of:

–      questioning strategies

–      paraphrasing and probing strategies

–      summarizing to refocus

In their training, TP PBL facilitators are made conscious of the need to “Model, support, observe & fade” (Barrows, 1988). It can be argued that scaffolding during problem identification can compensate for the static nature of learner support in problem statements.

One might also contest that learners scaffold for each other during Stage 2 because they work in collaborative groups and each have different strengths. Perhaps, for example, in a PBL group there is a learner who has the necessary linguistic ability or cultural insight that the others lack. This learner can scaffold for the others. However, Puntambekar and Hübscher (2005) provide evidence for their opinion that learners are unlikely to be applying principles of instructional scaffolding.

Discussion

If learners struggle with comprehension of the problem statement because of English as a Second Language (ESL) or cultural issues, facilitators have the option either to incorporate scaffolding into PBL problem design or to scaffold comprehension during Stage 2.

I suggest that it is inefficient and distracting to deal with ESL and cultural issues at Stage 2 when learners really need to focus on the challenge of identifying facts, and the facilitator needs to focus on the scaffolding of that skill.

Moreover, it has become a practical option to incorporate scaffolding into problem design because of the emergence of educational technologies that can reduce the burden on PBL problem crafters in terms of the authoring of dynamic, adaptive problem materials, and diagnostic tests.

Scaffolding in problem design may also be more realisable in a PBL setting where students experience the PBL process frequently. There will then be opportunities, to use Bruner’s terminology, for multiple ‘routines’ in the same ‘format’.

Conclusion

TP PBL facilitators have designed problems that are true to the principles of PBL problem design sourced from seminal works. There is learner support in the problems that were analysed, but it is not characteristic of the original notion of scaffolding. However, TP PBL facilitators do report scaffolding during Stage 2 of the TP PBL process.

To remove barriers to meeting PBL learning outcomes and for more inclusive learning, scaffolding of the assumed elements of language proficiency and cultural awareness is vital, and can be built into the design of problems. Incorporating scaffolding at this stage has the added advantage of making feasible more authentic language use in problem statements which in turn can support learners in their self-directed learning.

Questions for teachers

Do you facilitate Problem-Based Learning or Enquiry-Based Learning? How do you support learners through the process? Is this scaffolding dynamic? Please provide examples from your experience. Thanks!

References

Barrows, H. S. (1988). The Tutorial Process. Springfield, Illinois: Southern IllinoisUniversitySchool of Medicine.

Barrows, H. S. (1994). Practice-based Learning. Problem-based learning applied to medical education.Illinois: Southern IllinoisUniversitySchool of Medicine.

Dolmans, D.H.J.M. & Snellen-Balendong, H. (1997). Seven Principles of Effective Case Design for a Problem-based Curriculum. Medical Teacher, Sep97, Vol. 19, Issue 3.

Hogan, K. (1997) Introduction. In: Hogan, K. and Pressley, M. (eds.) Scaffolding Student Learning: Instructional Approaches & Issues, Cambridge, Massachusetts: Brookline Books, p. 2.

Puntambekar, S. and Hübscher, R. (2005). Tools for Scaffolding Students in a Complex Learning Environment: What Have We Gained and What Have We Missed? Educational Psychologist, 40(1), 1 – 12.

Savin-Baden, M. and Howell Major, C. (2004). Foundations of Problem-based Learning. Maidenhead, Berks: Open University Press.

Vygotsky, L. S. (1978) Mind in society: The development of higher psychological processes. HarvardUniversity Press.

Wood, D.J., Bruner, J.S., & Ross, G. (1976) The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17, 89-100.