School Composition, Disruptive Classroom Behaviour and Student Results: A Study of Mechanisms of Peer Effects

Introduction

In Sweden, there has been some debate whether there is a need to worry that increasing school segregation in compulsory schools (Holmlund et al., 2014; Bäckström, 2015; Skolverket, 2018) may lead to increased negative peer effects for some pupils, whilst leading to positive peer effects for others, thereby increasing differences in performance between pupils and schools. The Swedish National Agency for Education (Skolverket) has hypothesized that increased peer effects, caused by increased segregation, could explain some of the Swedish decline in results in international tests such as PISA and TIMSS (Skolverket, 2009). Such a claim has been challenged both by researchers (Gilljam & Persson, 2010) and in the political debate (Heller-Sahlgren, 2015). One argument, also recognised by Skolverket (2009), is that there is a lack of Swedish research into the field of peer effects.

The latter might be true to some extent, but I argue that such an argument disregards extensive educational research into “compositional” or “contextual” effects in education (Dreeben & Barr, 1988), which reports evidence on the effects of class composition on student’s results (Rutter & Maughan, 2002). I argue that such effects should be viewed as synonymous to peer effects (Wilkinson, 2002), since the mechanisms that create these effects are a result of the dynamics of the teaching process and the interaction between students, the class and the teacher (Wilkinson, Parr, Fung, Hattie & Townsend, 2002; Thrupp, Lauder & Robinson, 2002; Weinert, Schrader & Helmke, 1989). The relationship between class composition (peers), the teaching process and students’ results was the central theme of Swedish educational research during the 1960’s and 1970’s (Dahllöf, 1967; Lundgren, 1972; Marklund, 1985), which to some extent contradicts the claim that there is a lack of Swedish research into peer effects.

Some researchers have argued that one important mechanism creating peer effects is disruptive classroom behaviour (Lazear, 2001; Blatchford, Edmonds & Martin, 2003; Lavy, Paserman & Schlosser, 2011). In my forthcoming research, I will test whether Ulf P. Lundgren’s specification of the frame factor theory can be used to explain peer effects in education, where disruptive classroom behaviour is one of the factors creating peer effects. In this study, I examine the possible relationship between disruptive classroom behaviour, school composition and students’ outcomes in the Swedish setting, following previous studies by Lavy and Schlosser (2007) and Lavy, Paserman and Schlosser (2011). This study sets the stage for a planned future revisit to the frame factor theory.

Previous research

In the international literature, there is evidence for several different hypotheses about peer effects. For instance, Schindler Rangvid (2003) reports evidence for a positive linear peer effect using Danish PISA data; attending a school with a higher share of high-performing peers increases the individual’s results. In her analysis Schindler Rangvid also finds evidence for a non-linear peer effect since the positive effects of high-achieving peers is greater for low-achieving peers than for high-achieving. The size of the effects diminishes the further up in the achievement distribution she looks.

Similar results have been reported in several other studies (Hoxby, 2000; Hanushek, Kain, Markman & Rivkin, 2003; Sirin, 2005; Ammermueller & Pischke, 2009; Burke & Sass, 2013; Brunello & Rocco, 2013), including Sweden (Szulkin & Jonsson, 2007; Sund, 2009). The effect sizes and type of hypothesis supported empirically (linear/non-linear-, positive/negative peer effect) differs between the studies mainly because of study design, methods used and hypotheses made. There are also other differences. For instance, some studies indicate that the more detailed the independent variables measured, the greater the effect sizes are (Ewijk & Sleegers, 2010; Sirin, 2005; Gustafsson & Yang Hansen, 2018). Other studies indicate that the more controls performed for school, class or teacher fixed effects, the smaller the effect sizes are (Burke & Sass, 2013; Boucher, Bramoullé, Djebbari & Fortin, 2014), or even that they may be non-existent (Vigdor & Nechyba, 2007).

A commonality between the studies mentioned is that all of them are studies in economics or educational economy. The question of peer effects though, is in no way neglected in educational research. The question at hand – how different classmates influence each other – has a long history in the educational sciences, with many researchers pointing to the Coleman report as a starting point (Coleman et al., 1966; Barr & Dreeben, 1977; Rutter & Maughan, 2002). The educational research though, is rarely labelled as research into “peer effects”. It is more commonly referred to as research into “compositional” or “contextual” effects (Dreeben & Barr, 1988).

The educational research has reported similar results to those referred to above (Barr & Dreeben, 1977; Beckerman & Good, 1981; Weinert, Schrader & Helmke, 1989; Bourke, 1986), but with some differences. One important difference that needs to be addressed is which theoretical framework was used to interpret the results and how this may have influenced possible conclusions concerning mechanisms that generate peer effects (see for example Bourke, 1986; Parkerson, Lomax, Schiller & Walberg, 1984).

In the economic literature, it is common to distinguish exogenous factors from endogenous factors (Epple & Romano, 2011). Much of the economic research referred to above uses exogenous factors as independent variables in the analysis. These factors are measured with different types of registered data of the pupils’ socio-economic status (SES), including parents’ educational background and different measures of their immigration background. Hereby the studies de facto say very little about possible casual mechanisms creating the peer effects observed. Why does a larger share of male classmates or a larger share of classmates with highly educated parents influence an individual students’ results?

The other way to study possible peer effects is to use endogenous factors as independent variables in the analysis (Yeung & Nguyen-Hoang, 2016). These differ in the sense that they, firstly, are often viewed as being dependent of the exogenous factors mentioned above,¹ and secondly that they are measures of what individuals do, how they act in the classroom, how they behave and how they perform. The first are often measured with surveys or observations, the second – performance – with aptitude tests or registered data on grades. In comparison to exogenous factors, the casual effect of these types of factors is more intuitive. If some classmates disrupt instruction or make a lot of noise, it will likely affect the other students in the class. An example of research dealing with these types of causal mechanisms, with a described causal relationship between exogenous and endogenous factors, and students’ results, is found in Bourke (1986).

In this study, I will focus on the possible effects of disruptive classroom behaviour on students’ results, as well as explore the possible relationship between such exogenous and endogenous factors. It is likely that these factors are correlated to some extent. Exploring and specifying these relationships will contribute to future research in the field.

In Nordic educational research, there are examples of studies of disruptive classroom behaviour (Duesund & Oedegaard, 2018; Bru, 2006). There are also studies of the impact of disruptive classroom behaviour on student’s results, reporting a negative effect of disruptive classroom behaviour (Kristoffersen, Krægpøth, Skyt Nielsen & Simonsen, 2015; Smith & Reimer, 2018). In Sweden, there is a lack of such studies.

In the international literature on peer effects, there are also studies which report on the negative effects of disruptive behaviour. One such example is Lavy and Schlosser (2007) who study Israeli primary- and upper secondary schools. They find evidence that a larger share of girls in a class leads to positive peer effects for all students in the class due to its negative correlation with disruptive behaviour in the classroom instruction.² The more girls there are in the class, the less disruptive behaviour (Bertrand & Pan, 2013). The effects of a larger share of girls is equal between boys and girls; they all benefit from going to school with a larger share of girls. The results also indicate that a larger share of girls is correlated with less violence in the classroom, improved pupil-to-pupil and pupil-to-teacher relations, enhanced school experiences and decreased teachers’ fatigue. The study doesn’t show whether girls and boys change their behaviour with respect to a larger or smaller share of boys, thereby the effect seems to arise from class composition (Lavy, Paserman & Schlosser, 2011; Lazear, 2001; Cheema & Kitsantas, 2014).

The Frame Factor Theory

Many of the studies referred to thus far, share a focus on the immediate peer effect on an individual students’ results, not how possible peer effects arise from the interaction between class composition, pupil-pupil interactions, and teacher-pupil interactions. In contrast, this study employs Ulf P. Lundgren’s specification of the frame factor theory, which focuses on the interaction between the group (the class) and the teaching process (Haertel, Walberg & Weinstein, 1983).

The discussion of the mechanisms behind peer effects strikes at the heart of the frame factor theory as put forward by Lundgren (1972). The centre of attention is the instruction time needed for students to learn the curricula unit at hand, a concept first introduced by John Carroll in his model of school learning in 1963 (Caroll, 1963; 1989; Haertel, Walberg & Weinstein, 1983), and further developed by Urban Dahllöf into early versions of the frame factor model (Dahllöf, 1967, 1999).

According to Lundgren’s frame factor theory, three main frame factors steer and limit the teaching process. These are (i) the curriculum (the goals and content), (ii) the time available for instruction and (iii) the class composition, more precisely what amount of time the class needs to achieve the goals of the curriculum. If (iii) is greater than (ii), then it will have a negative effect on the outcome (students’ results). According to Lundgren, this also means that in classes with a larger share of low-performing students, the teacher must spend more instruction time on elementary units in the curricula, at the expense of more advanced units.

In his thesis, Lundgren (1972) tests this theory empirically, finding evidence to support it. Lundgren’s frame factor theory does not (at least explicitly) focus on the possible effects of peers’ behaviour. He does mention the correlations in his data between students’ academic abilities and the occurrence of disruptive behaviour, and how this might steal time for instruction, but the theory focusses on academic skills and the time students need to learn the curricula unit at hand.

Other theories, such as Benjamin Bloom’s theory on mastery learning, which also originates from Carroll’s model of school learning, put greater emphasis on how time at hand for instruction is spent, in relation to the time different students need to learn the curricula unit at hand. In this research the concept of “time-on-task” is significant; the more time students spend on task, the more they will learn. In this perspective, disruptive classroom behaviour will lead to time-off-task, thereby leading to poorer results for the students (Bloom, 1974; Arlin, 1979; Karweit, 1984; Karweit & Slavin, 1982; Carroll, 1989).

From the late 1970’s until the present day, major strides have been made in developing the frame factor theory. Early expansions of the theory were made by Lundgren himself, with the intention of creating a broader social theory (Broady, 1999; Englund, 1990), but the contributions from others have been immense. Development of the theory has been characterized by a continuous broadening of the theory (with social and historical perspectives) and with a growing number of frame factors in the analysis (Rapp, Segolsson & Kroksmark, 2017; Persson, 2015; Öberg, 2019).

In the present study, I set the stage for a future revisit to Lundgren’s original frame factor theory (1972) as a way of explaining the mechanisms that create some of the peer effects. Since disruptive classroom behaviour steals time from instruction, I believe that this aspect must be included in a frame factor analysis of the mechanisms of peer effects. Here, I therefore explore whether disruptive classroom behaviour affects students’ results.

The research questions

The aim of this study is to examine the possible relationship between disruptive classroom behaviour and students’ outcomes on the school level in Sweden. Hence, I will investigate whether similar results as reported by Lavy & Schlosser (2007) and Lavy, Paserman & Schlosser (2011) can be found in Swedish lower secondary schools.

Since analysis of these outcomes is also determined by a school’s pupil composition (measured as SES, sex and immigration background), the study will also explore the relationship between these exogenous factors and the endogenous factor of disruptive classroom behaviour.

My research questions for the study are:

Method

Data Set and Sample

To conduct the study, I set up a joint dataset with data from the Swedish National Agency for Education (Skolverket) SALSA-database and data from the Swedish Schools Inspectorate (Skolinspektionen) School Survey. The dataset comprises variables on school composition, disruptive classroom behaviour and school results. All data is accessible through each authority’s webpage.

Each term the Swedish Schools Inspectorate conducts a survey amongst pupils, teachers and parents as part of their supervision and quality assurance of Swedish schools. Over a period of four terms (two years, 2015 and 2016), they surveyed all Swedish lower secondary schools (one fourth of the schools each term). The pupils surveyed were in year nine and on average fifteen years old. The pupil surveys had an average response rate of 80% and the teacher surveys 69%. In total 81 816 pupils and 33 629 teachers in 1704 schools answered the surveys (see table 2).

The data from the pupils and teachers’ surveys have been merged with data from the Swedish National Agency for Education SALSA-database, comprising variables of the pupil composition at the school level (year 9) and their results. The data sources are matched by the unique school codes that Sweden employs for statistical use. The survey data from the fall and spring term were merged with the corresponding grades from the end of that school year, as illustrated in figure 1.

The variables used in the analysis are categorized and presented in table 1. The way in which they were measured is described under “Measures”. Table 3 presents descriptive statistics for the dataset.

VARIABLE	CATEGORY	SOURCE
Schools’ grade point average (GPA)	Dependent	Skolverket
Parents’ average educational background	Independent (exogenous)	Skolverket
Share of newly arrived immigrants at school	Independent (exogenous)	Skolverket
Share of boys at school	Independent (exogenous)	Skolverket
TEACHER INDEX of classroom environment	Independent (endogenous)	Skolinspektionen
PUPIL INDEX of classroom environment*	Independent (endogenous)	Skolinspektionen

*Note that this variable is not used in the final models, the reason for this is described in the “Missing cases” section

Results

Mediation Analysis

Since the M2 results indicate that some of the original effects of schools’ composition in M1 are mediated through the occurrence of disruptive classroom behaviour (TEACHER INDEX), mediation tests were conducted. In figure 2, the mediation analysis regarding the effects of the share of boys is illustrated in a path diagram. I table 8, the results from each separate mediation analysis is reported. All mediation analyses were tested against the conditions set up by Baron and Kenny (1986), and they were all met. Sobel’s test for significant mediation is reported in table 8.

As the results indicate, some of the original effects from the school compositional variables (SES-variables) are in fact mediated through disruptive classroom behaviour, thereby indicating that there seems to be theoretical relevance carried by the classroom environment in explaining the occurrence of different peer effects.

	DIRECT EFFECT	INDIRECT EFFECT	TOTAL EFFECT	SOBEL’S TEST (P)
Share of boys	-0.156	-0.,071	-0.227	<0.001
Parents’ educational background	0.680	0.068	0.748	<0.001
Share of N.A immigrants	-0.489	-0.096	-0.585	<0.001