Frontline Learning Research Vol.9 No.2 (2021) 179 - 185
ISSN 2295-3159
1University of Antwerp, Belgium
2University of Oslo, Norway
This discussion addresses the methodological approaches used in the special issue on student transitions and diversity. We provide an overview of how the different papers contribute to methodological development in the field, in particular by their use of advanced multi-factor analyses and accounting for diversity in student transitions at several levels. The discussion touches upon challenges concerning (a) the distinction of analytical levels, (b) different conceptualisations and operationalisations of diversity and (c) the types of data collection methodologies. We conclude by discussing future steps to widen the methodological lens in transition research.
Keywords: transition; higher education; diversity; discussion; methodological challenges
In their introduction, the guest editors of the special issue “From Micro to Macro: Widening the Investigation of Diversity in the transition to Higher Education” describe a twofold aim: (a) to widen the perspective on research concerning the role of diversity in the transition process and (b) to contribute to further methodological developments in the field (De Clercq, Jansen, Brahm and Bosse, 2021). The guest editors acknowledge the need for a multidimensional approach (cf. Coertjens, Brahm, Trautwein & Lindblom-Ylänne, 2017; Richardson, Abraham & Bond, 2012) and claim that the special issue will provide research that employs a methodology that goes beyond single-factor analyses and uses cutting-edge data analysis to investigate different levels of diversity in the transition to higher education. We will argue that the special issue indeed succeeded in presenting a coherent set of papers that use cutting-edge data analyses. Next, we will discuss how the issue has contributed to the efforts of endorsing a multidimensional approach in research on the transition to higher education, but that this approach has not yet reached its full potential. Finally, we will present some methodological challenges that remain open for future research.
Student transition and diversity are complex subjects. Educational researchers that take on the challenge of studying these topics need a well-developed conceptual understanding of the phenomena in focus. The range of conceptualizations spans widely, from seeing diversity and transitions as characteristics of individual learners, to sociological approaches that employ these concepts to understand whole student populations and their changing compositions over time (e.g. Rendon, 1994; Guri-Rosenblit, Šebková, & Teichler, 2007). No matter which conceptual approaches are chosen, it is important to select appropriate methodologies that can shed light on the many factors deemed relevant for understanding student transitions and diversity.
The authors in this special issue have gone to great lengths to develop conceptual frameworks that account for the complexity of the phenomena under investigation and to select appropriate methodologies. While most of the employed methods might not be novel in themselves, they may be considered innovative in the field of transition research, which has traditionally been characterized by single factor analyses. Several of the included studies use advanced and state-of-the-art multi-level analyses on survey data. Van der Zanden et al. (2021) study the relationship between students’ perceptions of the learning environment in secondary school and their subsequent academic achievement and adjustment during their first year at university. The study’s three-wave longitudinal design allows the researchers to analyse changes within individuals, differences between students and interaction effects between them. Similarly, Willems et al. (2021) conduct SEM analyses on a longitudinal dataset to explore to what extent students’ psychosocial variables at the end of secondary education can predict their academic achievement and adjustment during the first year at university. It is novel here that the authors also investigate whether the predictive power might differ depending on whether students are enrolled in professional or academic programs. Jenert and Brahm (2021) draw on a particularly rich longitudinal dataset. In addition to survey data on students’ self-efficacy, anxiety and motivation at three points during their first study year, the authors collected longitudinal data from four to five interviews of 14 students throughout the year. The methodological novelty lies in the combination of latent profile analysis to identify different student profiles and the triangulation with the qualitative interview data. In a similar vein, Bohndick et al. (2021) conduct a latent profile analysis on a cross-sectional survey collected at the end of the first study year to identify different profiles of first year students’ perceptions of the study requirements. In addition, the authors show through SEM that the differences in perception can be largely explained by individual factors such as self-efficacy and volition. Also drawing on student survey data, De Clercq et al. (2021) use multilevel analyses to investigate to what extent variation in first year students’ achievement can be explained by multifactorial models that include individual background and psychosocial variables as well as factors at the study program level. The novelty of this approach lies in the combination of both contextual and individual factors in the same models. Compared to the relatively homogenous approaches used by the other authors, Messina Dahlberg et al. (2021) choose a very different methodological approach to studying student transitions and diversity. They employ ethnographic methods to analyse the links between the wider institutional and policy context and the personal narratives of migrant students beginning higher education. Especially in the context of this special issue, this contribution provides a useful complementary perspective on the phenomena in focus. Lastly, Balloo and Winstone’s (2021) contribution has an unusual but interesting format. They demonstrate how university administrators can use institutional data to investigate the reasons behind the varying rates of success during first year transition. While the analytical approaches presented in the article are not novel in themselves, it can be argued that disseminating methodological knowledge among administrators is an equally important contribution to the practical goal of reducing the first-year achievement gap between different student groups.
In the previous section, we highlighted how the papers in the special issue succeeded in presenting research that generates new insights by using cutting-edge data analysis to investigate different levels of diversity in the transition to higher education. In the next paragraph, we will discuss where the papers in the special issue could have done more to endorse a multidimensional approach in research on the transition to higher education. We will highlight some limitations of the papers with regard to (a) connecting two or more levels in transition research, (b) different conceptualisations and operationalisations of diversity and (c) the range of data collection methodologies used.
In the introductory chapter of this special issue, the guest editors write that “the common aim of the studies composing the special issue is to connect either two or more levels (micro/meso/macro) in transition research or different types of diversity by using a range of methodologies” (De Clercq et al., 2021, p. 3). While the papers clearly do not focus on just one level (micro, meso or macro), the attempt of connecting more than two levels has not been realised in any of the papers in the special issue and the distinction between different levels sometimes feels artificial. For instance, in Van der Zanden et al. (2021), both variables at the meso level (teacher practices in secondary education) and the main variables at the micro level (social and emotional adjustment) were measured by means of self-report questionnaires. While multilevel analyses do take the multi-structuredness of the data into account, it could be questioned whether asking students how they perceived the teaching context is a valid operationalization of the (meso) teaching context, or rather just another variable (i.e. student perception) at the micro level. De Clercq et al. (2021) tried to overcome this limitation by considering both self-reported measures and objective characteristics of the teaching context.
The difficulty in separating micro- from meso-level variables without acknowledging their interdependence can be further illustrated by the research of Willems et al. (2021). The authors frame diversity at the meso level as ‘programme diversity’, pointing to differences especially between academic and professional disciplines with regard to their aims, conditions and teaching methods. These differences are assumed to mediate the explanatory value of secondary students’ academic motivation, self-efficacy, learning strategies and subsequent academic adjustment during the first year at university. The authors note that “it remains unclear whether the unveiled differences in predictive power of the abovementioned determinants emerge from differences between the HE programmes (contextual), or differences between students within the two systems (individual)” (p. 44). Indeed, it appears questionable whether these two aspects can and should be separated in this way; the students who choose professional or academic study programs are themselves central elements who influence the programs just as much as the programs influence the individual characteristics of the students. Messina Dahlberg et al. (2021) acknowledge this close interdependence by defining diversity “in terms of an intricate process in which the agency of human beings does not exclusively reside in the single individual, but is always situated, i.e. related to contexts, from specific micro-moments, to macro-structures, of which policies are an important part” (p. 146).
Another example of the problematic distinction between the micro, meso and macro levels can be found in the paper by Jenert and Brahm (2021). They study how subgroups of students with different personal characteristics perceive and interact with their academic and social study environment. The meso level here seems to be seen as the design of study environments and their different academic and social characteristics. The social characteristics of a study environment are, amongst others, understood as the “quality of students’ social relationships” (Jenert & Brahm, 2021, p. 51). This raises the question whether it makes sense to distinguish the individual characteristics of students from their social study environment, as that environment is made up of the students themselves and shaped by how they interact with each other. The authors themselves state that “students’ personal characteristics when entering higher education influence how they experience the study environment” (p. 50), which supports our doubt whether student perceptions are an appropriate methodological basis for providing valid operationalisations of the teaching context (meso level). On the other hand, the paper by Bohndick et al. (2021) could support using students’ perceptions of environmental requirements as a valid measure for assessing the meso level. The authors argue that “as the perceived requirements result from both the characteristics of the students and the institutional context, they allow to investigate the transition at the interface of the micro and the meso level of HE” (p. 78). Messina Dahlberg et al. (2021) address this challenge explicitly and chose an ethnographic approach to give “further fuel to the debate about the notion of analytical levels and on the kinds of (dis)advantages that it may bring, in relation to an alternative perspective according to which there are always macro aspects in the micro and vice versa” (p. 150). We argue that combining (mainly retrospective) self-report measures with other (more concurrent) measures (e.g., log data, observations or document-analyses) could help to measure different levels in a single study (see also Gijbels & Loyens, 2020). We will elaborate on this benefit in our discussion.
Diversity cannot be thought of as a standalone phenomenon that exists for its own sake. We can speak of diversity only in relation to something, such as the diversity of the student population. The variables used to operationalize diversity differ between the studies in the special issue and sometimes even within the same study. For example, Jenert and Brahm (2021) operationalize diversity in the student population by measuring context-dependent variables that change over time (e.g., anxiety and self-efficacy). At the same time, the selection criteria for the interview participants reflect an operationalization of diversity based on unchangeable variables such as academic family background, nationality and gender. This illustrates the conceptual and methodological challenges related to the complex notion of diversity. Another example is the study by Bohndick et al. (2021) that uses a multi-layered conception of diversity that considers the students’ individual, performance-related differences and the social differences related to their background. At the same time, the authors frame diversity as the organizational differences of the types of institutions and the students’ participation in different study programs. While the first conception refers to the diversity of the student population, the latter conception refers to diversity of a different kind of population – namely, all higher education institutions and study programs. In Bohndick et al.’s paper, the lines between these two conceptualizations become blurry, raising the question whether it makes sense to treat these two types of diversity at the same level or whether this is like comparing apples and oranges. It is important for future research to develop more precise and differentiated terminology and to avoid conflating different empirical phenomena under a single label (see e.g. Gale & Parker's, 2014 conceptual paper on a typology of student transition as an example of how diversity could be conceptually further unpacked).
The aim of the special issue is to better understand diversity in order to provide a fair chance of success for all students, independent of their diverse backgrounds and experience. In line with this goal, diversity is used primarily as a neutral scientific concept to explain differences in first-year achievement and experience. We would have welcomed more discussions of the normative notions linked to the term diversity and the consequences of higher education institutions embracing diversity as a desirable state of the student population. For example, should study programs and initiation weeks be adapted to first-year students’ individual characteristics in order to increase their chances of success? Or is it problematic to categorise students and treat them differently according to their backgrounds, even if well-intentioned? While these questions cannot be easily answered, it would be valuable to know more about the different authors’ reflections on these issues, based on their empirical insights.
While we already acknowledged that the papers in the special issue use state-of-the-art data analysis, it is fair to say that their data collection methods are much more traditional and could have made better use of mixed or multiple data sources. With the exception of the more ethnographic approach used by Messina Dahlberg et al. (2021) and the use of administrative records by De Clercq et al. (2021), all of the papers use self-report instruments as the primary data source. For instance, in Van der Zanden et al. (2021), students are asked how they perceive their school teachers’ support. The authors acknowledge that the perception of teacher practices is dependent on myriads of unique interactions with different teachers during school time. This raises a methodological question of validity concerning what students actually report when answering questions about their teachers’ support. Do students think of particular teachers, with whom they had positive or negative experiences, or do they think of the average experience they had during school? If students completed the questionnaires during class, would they be tempted to think of the particular teacher present in the room? Without further inquiry, these questions are impossible to answer, illustrating the limitations of self-report measures. The authors themselves suggest that triangulation with other data sources such as teacher interviews or classroom observations would improve the validity of the measures of the teachers’ support practices. However, the papers in the special issue are heavily based on self-report instruments, and data triangulation seems to be generally underdeveloped.
While several authors in the special issue highlight the importance of understanding transition as a process rather than as a state (Tett et al., 2017), we found it surprising that none of the papers explored the use of process measures. While several authors used longitudinal data sets, it could be questioned whether surveys – even if conducted at several points in time – are the best methodological approach to capture transition processes, especially when they are not accompanied by other sources of data. Surveys are always only snapshots, fixed in time and space. De Clercq et al. (2021) acknowledge that “the transition to HE cannot be fully understood through the achievement process and through cross-sectional designs” (p. 114). However, they do not propose other types of data (e.g., trace data, log data, sensor data,…) but rather suggest solving the issue by implementing longitudinal designs that still depend on self-report surveys. This argument is in line with their assumption that “achievement can be conceived as an index of a successful transition” (p. 97). We question this assumption and strongly advocate for incorporating other types of data that provide more insights into the temporal changes that constitute students’ transitional processes. While the guest editors referred to other methods (e.g., social network analyses) in their introduction, the aim of presenting a widened analytical lens remains unfulfilled in most of the papers in the special issue.
Looking back to the overall aim and challenges that the guest editors established in their introduction, we can conclude that they succeeded in producing a coherent special issue with a clear focus on multiple levels in the transition to higher education and on diversity. Clear strengths are that all of the papers go beyond considering individual, micro-level characteristics and use more than a single-factor analysis of student transition. At the same time, some methodological challenges should be addressed in future research.
We suggest combining the analytical rigor demonstrated in the special issue with more multi-method designs that collect data from multiple sources, including the use of process measures (see for example Järvelä & Bannert, 2021). We are well aware that this is no easy task; combining measures that operate at different levels of granularity (e.g., a general self-perception measure about how students study in general versus trace data that assesses students’ activities every second) will create new methodological and theoretical challenges. We are nevertheless convinced that this step is worth taking in order to advance this area of study. However, as researchers we should be aware that we can only go one step at the time. From this perspective, the special issue paved a solid way for future researchers to take the next step.
This collaboration has been supported through an EARLI mentoring grant.
Coertjens, L., Brahm, T., Trautwein, C., & Lindblom-Ylänne, S. (2017).
Students’ transition into higher education from an international
perspective. Higher Education, 73 (3), 357–369.
Gale, T. & Parker, S. (2014). Navigating change: A typology of student
transition in higher education. Studies in Higher Education, 39 (5), 734–-753.
Gijbels, D., & Loyens, S. (2020). Commentary: Measuring strategic
processing in concert: Reflections and future directions. In D.L. Dinsmore,
L.K. Fryer & M.M. Parkinson (Eds.), Handbook of Strategies and Strategic Processing (pp.332-341). New
York: Routledge.
Guri-Rosenblit, S., Šebková, H. & Teichler, U. (2007). Massification
and diversity of Higher education systems: Interplay of complex dimensions. Higher Education Policy, 20, 373–389.
Järvelä, S., & Bannert, M. (2021). Temporal and adaptive processes of
regulated learning - What can multimodal data tell? Learning and Instruction, 72, 101268.
Rendon, L. I. (1994). Validating culturally diverse students: Toward a new
model of learning and student development. Innovative Higher Education, 19, 33–51.
Richardson, M., Abraham, C., & Bond, R. (2012). Psychological
correlates of university students’ academic performance: A systematic
review and meta-analysis. Psychological Bulletin, 138,
353–387.
Tett, L., Cree, V. E., & Christie, H. (2017). From further to higher
education: Transition as an on-going process. Higher Education, 73(3), 371–406.
Balloo, K., & Winstone, N. E. (2021). A primer on gathering and
analysing multi-level quantitative evidence for differential student
outcomes in higher education. Frontline Learning Research, 9(2), 121–144. https://doi.org/10.14786/flr.v9i2.675
Bohndick C., Bosse, E., Jänsch V. K., & Barnat, M. (2021). How
different diversity factors affect the perception of first-year
requirements in higher education. Frontline Learning Research, 9(2), 78–95. https://doi.org/10.14786/flr.v9i2.667
De Clercq, M., Galand, B., Hospel, V., & Frenay, M. (2021). Bridging
contextual and individual factors
of academic achievement: A multi-level analysis of diversity in the
transition to higher education. Frontline Learning Research, 9(2), 96–120. https://doi.org/10.14786/flr.v9i2.671
De Clercq, M., Jansen, E., Brahm, T., & Bosse, E. (2021). From micro to
macro: Widening the
investigation of diversity in the transition to higher education. Frontline Learning Research, 9(2), 1–8.
https://doi.org/10.14786/flr.v9i2.783
Jenert, T., & Brahm, T. (2021). The interplay of personal and
contextual diversity during the first year
at higher education: Combining a quantitative and a qualitative approach. Frontline Learning Research, 9(2), 50–77.
https://doi.org/10.14786/flr.v9i2.669
Messina Dahlberg, G., Vigmo, S., & Surian, A. (2021). Widening
participation? (Re)searching
institutional pathways in higher education for migrant students - The cases
of Sweden and Italy. Frontline Learning Research, 9(2),
145–169. https://doi.org/10.14786/flr.v9i2.655
Van der Zanden, P. J. A. C., Denessen, E., Cillessen, A. H. N., &
Meijer, P. C. (2021). Relationships
between teacher practices in secondary education and first-year students’
adjustment and academic achievement. Frontline Learning Research, 9(2), 9–27. https://doi.org/10.14786/flr.v9i2.665
Willems, J., van Daal, T., van Petegem, P., Coertjens, L., & Donche, V.
(2021). Predicting freshmen’s
academic adjustment and subsequent achievement: Differences between
academic and professional higher education contexts. Frontline Learning Research, 9(2), 28–49.
https://doi.org/10.14786/flr.v9i2.647