CIES scholars have documented the rise of international large-scale assessments (ILSAs) over the past two decades. These assessments include global projects such as the Programme for International Student Assessment (PISA) and Trends in International Mathematics and Science Study (TIMSS), as well as regional projects such as the Latin American Laboratory for Assessment of the Quality of Education (LLECE) and the Southern and Eastern Africa Consortium for Monitoring Educational Quality (SACMEQ). Research about the implications of ILSAs to policymaking and the secondary analysis of data produced by these assessments is very common in academic journals and at the CIES Annual Meeting. However, scholars have mostly overlooked the organizational structure of ILSAs. This project seeks to begin addressing this gap.
The idea for this project emerged when Stephanie took Dr. Pizmony-Levy’s course titled ITSF 5035 Social Analysis of International Assessments. The course provides students with a comprehensive curriculum on ILSAs including: (a) the methodology behind international assessments; (b) history of international assessments and conditions that facilitated their emergence and diffusion; and (c) critical analysis of the ways in which stakeholders – scholars, media, and policy makers – use data from international assessments. After working for the Australian Council for Educational Research (ACER), which is involved in different ILSAs (e.g., PISA and ICCS), Stephanie was interested in exploring which other actors are driving this organizational field.
In order to describe the organizational structure of ILSAs, we used descriptive statistics alongside social network analysis. Specifically, we collected data on the individuals and institutions involved in developing ILSAs for the following organizations: OECD, IEA, LLECE, CONFEMEN, SACMEQ, EQAP and NEQMAP. For each organization, we coded publicly available reports (a total of 49) related to the assessment and published by the organization itself. For example, we coded information from the acknowledgements sections of reports. This information allowed for the creation of datasets and matrices that form the basis of our analysis.
What did we find? Across all documents, we identified 872 individuals and 273 institutions that are involved in ILSAs. These institutions are based in 103 countries. Although ILSAs are presented as a “global project,” a large number of organizations involved in this work are situated in the Global North. The United States, for example, has the largest number of institutions represented in the network (30 institutions, 11%), followed by Germany (13 institutions, 4.8%), and the Netherlands (13 institutions, 4.8%). Figure 1 illustrates a two-mode network of institutions (circles) and the ILSA documents to which they are connected (squares). Documents are colored according to the organization commissioning the assessment. A considerable number of institutions are involved in both IEA and OECD assessments. The regional organizations are more distant from the center of the network.
What is next? Our initial findings are promising. We are now extending the analysis to explain variation across countries.
Oren Pizmony-Levy is an Assistant Professor in the International and Comparative Education Program.
Stephanie Templeton is an alumnus of the International Comparative Education Program. She is currently a Junior Research and Evaluation Associate at School-to-School International.