Fall 2009 CONFCHEM	Excellence in Education with CCLI: Notes from Recent Awardees An on-line conference, September and October 2009
	Abstracts	Papers	Instructions	Discussion Archive

 

"An Analysis of Discourse in Peer-Led Team Learning (PLTL)"

Regina F. Frey (gfrey@wustl.edu), Pat L. Brown, R. Keith Sawyer, Washington University, Department of Chemistry, The Teaching Center, and Department of Education, One Brookings Drive, Saint Louis, MO 63130

Abstract

Peer-Led Team Learning (PLTL) is a structured method for helping students engage actively in collaborative conversations. The method originated in undergraduate chemistry courses, but is now used in math and in other science classes as well. Previous studies have shown that PLTL results in improved learning in undergraduate chemistry. However, researchers have not studied the group mechanisms and discourse processes that lead to this improved outcome. This study is the first to explore those mechanisms and processes. We observed videotapes of PLTL sessions and analyzed the actions and interactions of peer leaders and of students. We found that peer leaders used two distinct interactional styles, which we call instructional and facilitative. The effects of these two interactional styles on the students’ dialogue, participation, and knowledge building will be presented. 

Introduction

Introduced in the mid-1990s, peer-led team learning (PLTL) is currently being used in many types of institutions, especially in general chemistry.^1-3 PLTL introduces students to effective group study by supplementing the lecture with formalized study groups that enforce active learning. The PLTL model consists of study groups containing 6-8 students that meet weekly and are facilitated by a student (peer) leader who has previously performed well in the course and who has close interaction with the instructor of the class while being a peer leader. The students solve problems selected by an instructor of the course using a distinct instructional strategy, such as “round robin,” “scribe,” “pairs,” and “small groups.” There are many variations on these basic instructional strategies, and the peer leaders are encouraged to creatively modify the strategies to increase participation and collaboration.

Our earlier evaluation study showed that the Washington University PLTL program is effective at improving students’ academic performance by, on average, one-half of a letter grade; this improvement is seen at other institutions as well.^1,2 Our earlier evaluation study also contained an attitudinal survey about the program, and the results of this survey revealed positive attitudes by the participants toward PLTL as a vehicle for learning and toward the study of chemistry in general.

However, the group mechanisms and discourse processes that lead to the improved performance have not been studied. The objective of this project is to better understand the discourse processes that occur between the peer leaders and students in PLTL groups. Using the findings from this project, we are enhancing our peer-leader training and improving the design of our PLTL sessions, and eventually plan to improve the construction of our PLTL problem sets.

The project consists of three inter-related studies. In the first study, we focused on two specific research questions: (1) What interactional strategies do peer leaders use during the PLTL sessions?  Are some of these strategies more or less effective at encouraging student discourse? (2) How do students talk among themselves in the PLTL sessions? The goal in the second study was to describe and characterize student knowledge building that occurs during PLTL sessions. The major research questions we addressed in this second study are: (1) What peer-leader actions facilitate student collaborative discourse? (2) How are students’ contributions responsive to those of other students? (3) What types of collaborative discourse practices are used by students working in small groups that lead to building knowledge of chemistry content? The goal of the third study in this project, which we are just beginning, is to describe the characteristics of PLTL problems that help lead to increased student discourse and increased conceptual knowledge building. In this paper, we will describe the findings from the first two studies in the project.

PLTL program at Washington University in St. Louis

Structure of the PLTL program at Washington University

The general chemistry lecture course at Washington University is a two-semester series enrolling 600 - 680 students each semester. Unlike the national program, students self-select into our PLTL program. However, once a student has joined a PLTL group, attendance at each session is mandatory. More details about our implementation of the PLTL model is described in reference 2. Currently approximately 65% of the students in general chemistry participate in PLTL chemistry program each semester.

Methods

Participants in the project

In fall 2006, during the 14-week semester, three PLTL sessions of each of 15 returning peer leaders were videotaped. The three recorded sessions were the 4^th, 7^th, and 9^th sessions, and approximately 60 hours of video data were recorded. For the project, all verbal interactions in 6 PLTL groups for two parts of session 1 were analyzed from these recordings. We used a purposeful sampling approach to identify cases and parts of problems that were “information-rich.^4,5 The 6 groups were chosen based on observation of all the video data. Based on our experiences working with peer leaders, we purposefully selected: (1) groups that represented a range of peer leader styles (instructional or facilitative); and (2) two parts of session 1 that represented typical PLTL activities (content review, problem solving, and concept discussion). The problem solving and concept discussion were from a two-part problem concerning the de Broglie matter wave. The content review and the concept discussion were large-group discussions and the problem solving was small groups of 2-4 students.

To analyze the discourse in the first study, we watched the videos for the 6 selected peer leaders and searched for re-occurring patterns connected to our research questions. First, we identified, labeled, and time-stamped the amount of peer-leader talk, student talk, individual tasks, and off-task behaviors that occurred during the selected PLTL activities. Second, all participants in the 6 selected groups, and the three PLTL activities (content review, problem solving, and concept discussion) were transcribed. Each transcription was divided into segments of talk, in which a segment of talk represented an individual’s contribution to the discussion. An individual’s talk could consist of multiple utterances depending on how many ideas were included in one segment of talk. Each utterance was assigned a code. The coding scheme involved two levels of coding: categories and codes. The categories we developed were consistent with the constant comparative method of qualitative data analysis.⁶ Categories are general types of speech observed and include 4 broad areas: (1) explanation, (2) content question, (3) facilitation, and (4) problem solving. Codes refer to specific types of identifiable speech within these 4 categories. For example, closed question and open question are both codes within the category “content question.” 

To calculate the reliability of the coding manual, two trained coders were used; one was an author and the other was a co-instructor of the general chemistry series. The second coder was blind to the motivation and hypotheses of the study. In training, the second coder was provided with the coding manual and was trained to apply the proper variable to each turn of the training transcripts. When disagreements arose, the two coders discussed differences and either a rule for coding was decided upon or a revision was made to the coding manual. Cohen’s Kappa is an inter-rater reliability measure for qualitative studies.^7,8 Bakeman and Gottman⁷ characterized a Cohen’s Kappa of greater than 0.75 as excellent. On the training transcripts, our Cohen’s Kappa was 0.81; thus, meeting the criteria for excellent inter-rater reliability. The data from the “training” transcripts were not used in the final analysis. After the coders attained reliability on the training transcripts, they coded all of the data for the content review and the selected de Broglie problem. The Cohen’s Kappa for the final stage of coding was 0.90.

In the second study concerning the knowledge building in PLTL sessions, we examined the discourse in detail in two PLTL groups as they both solved the selected de Broglie problem. One of the groups was led by a peer leader with a more facilitative style, and the other by a peer leader with an approximately equivalent combination of facilitative and instructive styles. Using the coding manual from our first discourse study, the videotapes of the two groups were transcribed verbatim. In addition, the written transcripts were annotated with relevant gestures to include what students were doing when they were not talking. The same two coders were used and the Cohen’s Kappa was 0.91 for the final coding, which meets the criteria for excellent inter-rater reliability.

Results

The analysis from the first study on the discourse of the six selected PLTL groups addresses: (a) types of peer leader and student talk, (b) time spent on PLTL activities, (c) student interactions, and (d) participation across typical PLTL activities (content review, problem solving, and concept discussions).

Instructional verses Facilitative Discourse

Peer-leader and student statements were coded to determine differences among leader-student and student-student discourse.  Table 1 lists peer-leader discourse and student discourse during the content review, the problem solving, and the concept discussion.

Table 1. Number of utterances and percentage in each category during PLTL activities

¹Discourse coded as: closed and open questions and explanations
²Discourse coded as: managerial/structure, course issues, feedback, refocusing, and restating

As shown in Table 1, there are differences in peer-leader talk during the PLTL group session. We separated these differences into instructional talk, where the peer leader directs the conversation and instructs the students, and facilitative talk, where the peer leader facilitates conversation among the students. In instructional talk, leaders use content questions and explanation statements; in facilitative talk, leaders use talk coded as managerial/structure, course issues, feedback, refocusing, and restating. As seen in Table 1, most of the peer leaders used a combination of instructional and facilitative discourse during the session. During the content review, the instructional discourse ranged from 12% to 64% percent of the total peer-leader talk and facilitative discourse ranged from 35% to 88%; during the small-group problem solving, the instructional discourse ranged from 0% to 55% and facilitative discourse ranged from 45% to 100%; during the concept discussion, the instructional discourse ranged from 6% to 67% and the facilitative discourse ranged from 33% to 94%. Across the different PLTL activities (content review, problem solving, and concept discussion), Gillian used primarily facilitative talk: during the content review, she had 15 statements (or 88% of her talk); during the problem solving, although she made only six statements, all were facilitative talk; during the concept discussion, 17 of Gillian’s 18 statements (or 94% of her talk) were facilitative discourse. The rest of the leaders used more equivalent percentages of instructional and facilitative discourse during the content review and selected problem.

In addition to investigating peer-leader discourse, we found that student discourse varied during the content review and the two different parts of the de Broglie problem. First, as seen in Table 1, during the content review in Tamal’s, Lance’s, and Matt’s groups, the student discourse was predominately problem solving (ranged from 62% to100%). Conversely, the most frequent type of student discourse observed in Gillian’s and Rachael’s groups was facilitation and occurred 46% and 41%, respectively. However, Gillian’s students made twice as many facilitation comments as Rachael’s students (40 comments versus 20 comments, respectively).  Second, during problem solving, all groups of students utilized the three types of student talk (content question, problem solving, and facilitation) to solve the problem. For all groups, the student talk was distributed among content questions (15% to 30%), problem-solving talk (28% to 49%) and facilitation (24% to 44%). Third, during the concept discussion, we noted differences in the number of statements students made during the concept discussion.  For example, Gillian’s, Rachael’s, and William’s student’s made 43 to 65 statements versus Lance’s, Tamal’s, and Matt’s students who made only 5-9 statements. In Gillian’s, Rachael’s, and William’s groups, students’ comments were distributed among content question (6% to 21%), problem solving (21% to 32%), and facilitation (44% to 51%).  

Time

In Table 2, the percentage of time in each section that consisted of peer-leader discourse, student discourse, individual tasks, and off-task behaviors was tabulated for the 3 PLTL activities (content review, problem solving, and concept discussion).

Table 2. Percentage of Time of Peer-leader and Student Discourse, Individual Tasks, and Off-task during PLTL Activities

Note. S=Students; PL=Peer leader; IT=Individual task; OT=Off-task

As shown in Table 2, the time spent on the content review ranged from 175.8 to 388 seconds; the time spent on problem solving ranged from 368.2 to 655.8 seconds; and the time spent for the concept discussion ranged from 270.1 to 531.2 seconds. First, in most of the groups and across the PLTL activities, off-task time ranged from 0-25% of the time. However, in one group (Lance’s), the amount of off-task time was significant; in the content-review section, more than 2 minutes of the 5 minutes and 30 seconds (41% of the total time) was off task, and more than 3 minutes of Lance’s 8 minute concept discussion (or 42% of the total time) was off task. Second, in most of the groups and across the different PLTL activities, students spent from 0 to 36% of the total time working individually. However, in Matt’s problem-solving activity which lasted over 10 minutes, students spent more than 5 minutes (or 47% of the total time) working individually on tasks; and during the concept discussion, Matt’s students spent 2 minutes and 30 seconds (or 55%) of the 4 minute and 30 second total time working individually.

Third, we noticed differences among the groups in the percentage of time students talked during different PLTL activities. As shown in Table 2, across the different PLTL activities (content review, problem solving, and concept discussion), the total student-talk time in Gillian’s group ranged from 62% to 85% of the total time. In the rest of the PLTL groups, the amount of student-talk time was variable across the different PLTL activities and ranged from 12% to 70% of the total time. For example, although Tamal’s students talked for 70% of the total time during the concept discussion, they only talked 44% of the time during the content review. 

Fourth, we noticed differences in the amount of peer-leader talk time across the 3 PLTL activities. As seen in Table 2, across the PLTL activities (content review, problem solving, concept discussion), peer-leader talk time was infrequent during problem solving (peer-leader talk time ranged from 7% to 14% of the problem-solving time). Gillian’s talk time was consistently less than the other groups. During the review, Gillian talked for 10% of the time; during problem solving, she talked for 8% of the time; and during the concept discussion she talked for 13% of the time. In the rest of the PLTL groups (Lance, Tamal, Rachael, Matt, and William), the amount of peer-leader talk time accounted for a greater percentage. During the content review, their talk time ranged from 14% to 61%; during the concept discussion their talk time ranged from 13% to 27%.

Chains of Interaction

Chains of interaction begin with a peer-leader action, followed by student(s) responses, and end when the peer leader re-enters the conversation. We considered one or two student-student interactions short chains of interactions, while four or more student-student interactions were considered long chains of interaction. Table 3 displays the number and percentages for the 6 PLTL groups.

Table 3. Comparison of Chains of Interaction for 6 PLTL Groups During PLTL Activities

As shown in Table 3, there were PLTL groups that had mostly short chains (chains that consisted of 1-2 interactions) and groups that consistently had long chains (chains that consisted of 4 or more student-student interactions). Most of the peer leaders had short chains of interactions during large-group discussions (content review and concept discussion); that is, short chains ranged from 50% to 100% of the total interactions. Short chains of interaction were less apparent during the problem-solving activity ranging from 33% to 72% of the total interactions. In addition, during problem solving where students worked in groups of 2-4 students, all of the groups had some longer (4+) chains of interactions (22% to 67% of total interactions). Across the different PLTL activities (content review, problem solving, concept discussion), Gillian consistently had chains of interactions that included 4 or more students. During the content review, she had 5 long chains (or 31% of the total interactions); during the problem solving, she had 3 long chains (or 50% of the total interactions); during the concept discussion, she had 7 long chains (or 39% of the total interactions). Other peer leaders had long chains of interaction, but the pattern was not consistent across all three PLTL activities. For example, Rachael had 5 long chains of interactions (or 31% of the total interactions) during the content review, but only had 3 long chains of interaction (or 21% of the total interactions) during the concept discussion. In addition, during Rachael’s concept discussion, seven (or 50%) of the chains of interactions included only 1 student.

Participation

We investigated student participation during the content review and concept discussion.   The problem-solving portion of the problem was not considered in this analysis because all students participated and had approximately equal contributions due to working in a group of only 2 to 4 students. 

Table 4. Student Participation

¹S=students were indistinguishable from each other due to overlapping speech or camera angle. 

Table 4 illustrates that there are differences in student participation and individual student contributions during the large-group discussions among the 6 PLTL groups, with 40% to 100% of students contributing to the discourse. Across the content review and concept discussion, Gillian had 100% participation and approximately equivalent contributions from individual students. During the content review, individual contributions ranged from 6% to 21%; during the concept discussion, individual contributions ranged from 3% to 17%. Although some of the other peer leaders had high participation (80% to 100%), students’ individual contributions were unequally distributed. For example during the content review, Tamal had 100% participation, but the individual contributions ranged from 7% to 41%. In addition, in Tamal’s group, F3 dominated the conversation by providing 41% of the total contributions made by all of the students. We also noted that during the concept discussion, only half of Matt’s and William’s students contributed to the conversation. The rest of the participants had unequal student participation during the content review (0% to 58% of the individual contributions) and concept discussion (0% to 50% of the individual contributions). 

Knowledge-building Discourse

To study which type of discourse led to knowledge building, we analyzed the discourse that occurred during Part I of the de Broglie problem for two PLTL groups (Gillian and Matt), where the students worked in small groups. Matt's group spent approximately 2 minutes more time on parts I and II than Gillian's group (1046 seconds and 916 seconds, respectively), but the two groups differed dramatically in the ways that they used that time (see Figure 1).  

Figure 1. Time of peer leader and student discourse, individual tasks, and off task during Parts I and II of the de Broglie problem.

First, students in Gillian’s group spent more than twice as much time talking as did Matt’s students (675 seconds versus 320.8 seconds), and Matt’s students’ spent almost ten times as much time as Gillian’s students working individually (454.9 seconds versus 46 seconds). In addition, Matt’s students worked in silence during their individual time, while Gillian’s students were silent for only 15 seconds during their individual task time.

From Table 1, Gillian primarily used facilitative discourse (92%), and from Figure 1, most of the time spent solving the problem was her students discussing. In addition, collectively, Gillian’s excerpts show that students had extended discourse episodes and exhibited collective knowledge building. For example, Gillian’s students’ discourse indicated that they were thinking about the salient features of the problem, and students’ comments were more often made in coordination with each other, rather than independent of each other. The transcripts show that Gillian’s students acknowledged, built upon, and elaborated on each other’s ideas when discussing the problem. Additionally, students’ explanations went beyond algebraic manipulations and began to address the underlying concepts. A critical component of effective knowledge building is that it supports and facilitates student collaboration as students engage in explaining, clarifying, and debating their ideas^9,10 Last, Gillian’s students used managerial/structure statements and refocusing statements that were directed at collaboration and learning processes. Hence, in Gillian’s group, students were active participants in making sure everyone understood the process necessary to solve the problem and all students made intellectual contributions necessary to solve the problem.

In contrast, Matt’s contributions provided limited opportunities for students to discuss the underlying concepts. Matt provided students with an equivalent combination of facilitative and instructional discourse, 33% statements and 66% statements, respectively (see Table 1). In addition, students spent more than 50% of the time working the problem individually. Neither the peer leader nor the students made a strong effort to address the issue that the group spent a substantial time working individually in silence. 

Matt’s students exhibited shorter discourse episodes, frequently engaged in individual tasks, and mostly provided each other with algebraic manipulations that did not deal with the underlying concepts. The excerpts show that Matt’s students asked questions that were often task-oriented and used to prepare the small groups for individual tasks. The lack of explicit focus on the important features of the problem and underlying concepts led students to a false sense of competence. For example, during one episode, although all three students discussed and agreed upon the equations to use, they all performed different calculations. Analysis of student discourse revealed that students had not all used the same equation. Research suggests that asking closed questions and providing non-elaborated help involves less cognitive restructuring or clarifying on the part of help-givers and does not enable help-receivers to correct their misconceptions or lack of understanding.¹¹ In addition, in Matt’s group, not only was the main focus on getting the correct answer, but the participatory structure was unequal.

In summary, Matt’s group was ineffective at promoting group knowledge-building discourse and students focused instead on the individually attempting to understand the content. Neither the students nor the peer leader encouraged in-depth conversations of the underlying concepts associated with the problem. In addition, the transcripts demonstrate that students in Matt’s group showed little evidence of building upon, debating, and elaborating upon each other’s ideas. 

Conclusion

There is a great deal of research evidence that students who participate in PLTL acquire higher levels of chemistry understanding than students who learn individually and alone. This project begins to examine exactly how the discourse in PLTL groups contributes to improved chemistry understanding. From our analysis, it appears that not all peer-group experiences are equivalent in promoting chemistry understanding.

First, peer leaders who used equal amounts of facilitative and instructional discourse had shorter chains of interactions and unequal participation. Conversely, the use of a high percentage of facilitative discourse was related to increased chains of student-to-student interactions and more equal student participation. Second, we observed that when a peer leader used primarily facilitative discourse, students displayed extended discussions that went beyond applying equations in a rote manner and began to develop an understanding of the concepts. Conversely, when a peer leader used more equal combinations of discourse coded as instructive and facilitative, students spent considerable time working individually and their discourse focused on the algebraic steps necessary to solve the problem. Third, we observed that students with a highly facilitative peer leader engaged in intellectual conversations where they asked each other questions, provided procedural and conceptual explanations, and “checked” each other’s understanding of the problem. In addition, these students continued this discourse even when working on individual tasks. Hence, even though at times these students had slightly incorrect conceptions, as a result of their discussions they collectively developed a better understanding of the problem. In contrast, students with an instructive peer leader rarely included any reference to, or explanation of the equations or underlying concepts. The conversations in this group were mostly superficial where students provided each other with equations and non-elaborate explanations, and there was little evidence that the group jointly developed a more in-depth understanding of the content from their discourse.

In summary, these findings suggest that peer leader’s styles are quite variable, and that if all peer leaders used the primarily facilitative style of Gillian, a collaborative group culture would emerge which would encourage students to engage in deep knowledge building.

Acknowledgements

The research in this paper has been funded by NSF CCLI grant award #0633202. The authors thank all of the peer leaders for their efforts in making the PLTL program successful at Washington University in St. Louis.

References

1. Gafney, L., & Varma-Nelson, P. (2008). Peer-Led team learning: Evaluation, dissemination, and institutionalization of a college level initiative. Dordrecht, the Netherlands: Springer.
2. Hockings, S. C., DeAngelis, K. J., & Frey, R. F. (2008). Peer-led team learning in general chemistry: Implementation and evaluation. Journal of Chemical Education, 85(7), 990-996.
3. Tien, L. T., Roth, V., & Kampmeier, J. A. (2002). Implementation of a peer-led team learning instructional approach in an undergraduate organic chemistry course. Journal of Research in Science Teaching, 39(7), 606-632.
4. Beals, D. E., & Tabors, P. O. (1995). Arboretum, bureaucratic, and carbohydrates: Preschoolers’ exposure to rare vocabulary at home. First Language, 15, 57-76.
5. Patton, M. Q. (2002). Qualitative research and evaluation (3rd ed.). Thousand Oaks, CA: Sage Publications.
6. Klaassen, C. W. J. M., & Lijnse, P. L. (1996). Interpreting students’ and teachers’ discourse in science classes: An underestimated problem? Journal of Research in Science Teaching, 33(2), 115-134.
7. Bakeman, R., & Gottman, J. (1986). Observing interaction: An introduction to sequential analysis. New York: Cambridge University Press.
8. Lunn, M. (1998). Applying simple k-sample tests to conditional probabilities for competition risks in a clinical trial. Biometrics, 54, 1662-1672.
9. Hiebert, J., Carpenter, T. P., Fennema, E., Fuson, K., Human, P., Murray, H., et al. (1996). Problem solving as a basis for reform in curriculum and instruction: The case of mathematics. Educational Researcher, 25(4), 12-21.
10. Yackel, E., Cobb, P., & Wood, T. (1991). Small-group interactions as a source of learning opporuntities in second-grade mathematics. Journal of Research in Mathematics Education, 22, 390-408.
11. Webb, N. M. (1995). Testing a theoretical model of student interaction and learning in small groups. In R. Hertz-Lazarowitz & N. Miller (Eds.), Interaction in small groups: the theoretical anatomy of group learning. Cambridge: Cambridge University Press.