Abstract

This article proposes an indicator that describes each Brazilian municipality’s learning level of its elementary school students, as well as learning inequalities among groups of students defined by socioeconomic status, race, and sex. This indicator is necessary because learning inequalities are growing and are not observed by the country’s main educational indicator, the Basic Education Development Index (IDEB). This study initially conceptualizes justice in education, positioning itself in the debate on theories of justice. Then, it presents the methodological aspects of the indicator and the results obtained. The main finding is that the subjects and school grades in which the highest number of municipalities increased their learning levels are also those in which there was a strong increase in inequalities.

right to education; educational assessment; justice in education; education inequalities; IDEB

Resumo

Este artigo propõe um indicador que descreve, para cada município brasileiro, o nível de aprendizagem de seus estudantes do ensino fundamental e as desigualdades de aprendizagem entre grupos de estudantes definidos por nível socioeconômico, raça e sexo. Esse indicador é necessário porque as desigualdades de aprendizagem são crescentes e, contudo, não são observadas pelo principal indicador educacional do país, o Índice de Desenvolvimento da Educação Básica (IDEB). O estudo inicialmente conceitualiza justiça em educação, situando-se no debate sobre teorias de justiça distributiva. Em seguida, apresenta os aspectos metodológicos do indicador e os resultados obtidos. A principal constatação é que as disciplinas e séries escolares nas quais se verificou o maior número de municípios aumentando o seu nível de aprendizagem são também aquelas em que se verificou forte aumento das desigualdades.

direito à educação; avaliação educacional; justiça em educação; desigualdades educacionais; IDEB

Résumé

Cet article propose em indicateur qui décrit, pour chaque municipalité brésilienne, son niveau et ses inégalités d’apprentissage. Cet indicateur est nécessaire car les inégalités d’apprentissage augmentent et ne sont pourtant pas observées par l’indicateur éducatif principal du pays, l’Indice de Développement de l’Éducation de Base (IDEB). L’étude onstatelize initialement la justice em éducation et se situe dans le débat sur les théories de la justice distributive. Ensuite, on onstate les aspects méthodologiques de l’indicateur et les résultats obtenus. La principale constatation est que les matières et les séries scolaires où l’on a onstat le plus grand nombre de municipalités augmentant leur niveau d’apprentissage sont également celles où l’on a onstate une forte augmentation des inégalités.

droit à l’éducation; évaluation éducative; justice em éducation; inégalités éducatives; IDEB

Resumen

Este artículo propone un indicador que describe, para cada municipio brasilero, el nivel y las desigualdades de aprendizaje y sus diferencias. Ese indicador es necesario porque las desigualdades de aprendizaje son crecientes y, no obstante, no son observadas por el principal indicador educativo del país, el Índice de Desarrollo de la Educación Básica (IDEB). El estudio inicialmente conceptualiza la justicia en educación, situándose en el debate sobre teorías de justicia distributiva. Después, presenta los aspectos metodológicos del indicador y los resultados obtenidos. La principal constatación es que las disciplinas y grados escolares en que se verificó un mayor número de municipios con aumento de su nivel de aprendizaje son también en los que se verificó un fuerte aumento de las desigualdades.

derecho a la educación; evaluación educativa; justicia en educación; desigualdades educativas; IDEB

Introduction

Brazilian education systems have traditionally been marked by inequality and have played a prominent role in perpetuating the country’s deep social disparities. During the process of the country’s redemocratization in the 1980s, universal access to enrollment in the first year of school was still not guaranteed in rural and/or poorer areas of the country (Fletcher, Ribeiro, 1987). Furthermore, Brazilian schools generally had high repetition rates, consequently limiting the education of many while strongly favoring those who continued studying until completing secondary education (Klein, Ribeiro, 1988). Since then, the country’s education systems have undergone reforms in light of the reorganization of the national political space, reformulation of legal frameworks, renewal of the scientific field, and alignment of the country’s education policies with international discussions on the topic.

Once access to enrollment in Elementary Education was universalized in the 1990s, two central objectives were assumed by education policies to make them more effective in ensuring the right to education: breaking with the pedagogy of repetition and improving learning outcomes. Educational policies in the 1990s and 2000s relied on outcome indicators and goal-setting systems that contributed to the institutionalization of normative discourses, interpreting these challenges in their own way and outlining objectives to be pursued. The most important indicator was the Basic Education Development Index (IDEB), created in 2007, with its historical series starting in 2005. Explicitly, the IDEB took on these two objectives of its time and built a normative discourse interpreting them as the pursuit of simultaneous increases in average approval rates and average learning scores, as measured by the Basic Education Assessment System (SAEB) tests¹ 1 . The SAEB is a set of large-scale external assessments, of which the IDEB is a part. Please see https://www.gov.br/inep/pt-br/areas-de-atuacao/avaliacao-e-exames-educacionais/saeb for more information in Portuguese. (Fernandes, 2007Fernandes, Reynaldo. (2007), "Índice de Desenvolvimento da Educação Básica (Ideb)". Textos para Discussão, n. 16. Brasília: Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. 26 p.).

The policies implemented during this period expanded educational opportunities and produced improvements in three educational outcomes that express the right to education: access to enrollment, retention, and learning. However, it is also true that inequalities were restructured during this period and began to assume new patterns. Regarding learning outcomes, two problems have been more frequently pointed out: they are low compared to other countries (UNESCO, n.d.; Franco, 2002Franco, Creso. (2002), "Educação das elites no Brasil: a Bélgica não existe". Trabalho e Sociedade, v. 2, n. 4, pp. 13-15.), and there are significant and growing inequalities (Alves, Soares, Xavier, 2016; Alves, Ferrão, 2019).

As existing indicators do not address inequalities, they are insufficient to describe pressing challenges in our educational reality, to frame inequalities as a social issue, or to guide actions promoting justice in education. In the early 2020s, the educational debate began to recognize the need for a revision of indicators, incorporating inequalities as a problem to be addressed.² 2 . See the 2021 webinar series “Fundeb and Educational Inequalities” available on the UNESCO Brazil YouTube channel: https://www.youtube.com/user/unescoPortuguese.

In 2021, the National Congress approved, through a constitutional amendment, a new version of the Basic Education Maintenance and Development Fund and Appreciation of Professionals (FUNDEB), providing for the distribution of part of the funds supplemented by the Union (2.5% of the fund’s resources) based on attendance and learning indicators and considering the reduction of socioeconomic and racial inequalities, something that the IDEB cannot achieve. In 2022, the IDEB reached the end of the cycle envisaged in its goal-setting system.

In this context, we present the theoretical and methodological assumptions and results of the second edition of the Inequality and Learning Indicator (IDeA). The first edition of the indicator was proposed by Soares, Ernica, and Rodrigues (2019) and published on an internet portal.³ 3 . See https://portalidea.org.br/. The objective of this indicator is to describe the level of student learning and the patterns of inequality among groups of students defined by socioeconomic status, race, and sex for each Brazilian municipality.

In the next section, we will present the conception of justice in education on which IDeA is based. In the following section, we will outline the general statistical construction of the indicator. Then, we will highlight general patterns of the Brazilian educational reality revealed by the indicator. Finally, in a brief conclusion, we will revisit some central points of the historical justification and the indicator’s theoretical construction.

Defining justice in education

Although the proposed indicator focuses on the analysis of learning, it must be understood within a larger conceptual context – that of justice in education. This context that guides us is built in relation to a debate that has been ongoing since at least the mid-1950s. There is a wide range of concepts and ways to measure justice in education, which is part of the dispute between normative ideals present in societies. This dispute takes place at the intersection of the political and scientific fields, each with its own rules and modus operandi. For this reason, the proposed concepts and measures express the conflicts within each of these fields, as well as the tensions between them.

Inequalities in reports from international organizations and in State policies

Measures of inequality are common in reports produced by international organisms and, yet, are less common in State policies. They are common in reports such as the Organisation for Economic Co-operation and Development’s (OECD) “Education at a Glance,” the United Nations Educational, Scientific and Cultural Organization’s (UNESCO) World Inequality Database on Education,⁴ 4 . See https://www.education-inequalities.org/ . and Les chiffres clés de l’éducation dans l’Union Européenne (The Key Numbers in Education within the European Union) by the European Education and Culture Executive Agency (EACEA – Eurydice).

When examining recent State policies, polyphony prevails. UNESCO, in its 2018 “Handbook on Measuring Equity in Education,” mapped equity measures in seventy-five national education plans (Alcott et al., 2018). It analyzed the plans for countries in Africa, Asia, Latin America, Oceania, and – to a lesser extent – North America and Europe. The authors conclude that the most frequent measures of inequality lie in school enrollment, addressing gender disparities. Measures of completion rates are rarer. In the United States, the policy in place since 2015, defined by the Every Students Succeed Act (ESSA), stipulates the country’s measures of inequalities. ESSA monitors learning through students’ proportions across four levels: below basic, basic, proficient, and advanced. These proportions are calculated for different jurisdictions: school administrative dependency, school districts, and states. Each jurisdiction presents legally required data for groups defined by sex, race/ethnicity, and parent’s education level.⁵ 5 . See https://www2.ed.gov/rschstat/landing.jhtml?src=pn and https://www.nationsreportcard.gov/ However, ESSA lacks an indicator that synthesizes this information, which faults at not being able to establish a desirable relationship between results demonstrating the right to education and hindering the formation of a discourse guiding public policies.

The 2020 report “Equity in School Education in Europe: Structures, policies, and student performance” (European Commission/EASEA/Eurydice, 2020) mapped out the European framework. The report analyzes the characteristics of education systems, State policies, and educational outcomes of forty-two education systems in thirty-seven countries. It also examined the official documents of these forty-two systems. Almost all mention concepts related to equity, although few provide explicit definitions of it. Among those that explicitly refer to equity, its definition and the subjects to take into account vary significantly.

Assumptions for a definition

Our definition of justice in education was conceived for a democratic society in which the State holds power to regulate the education system and is directly responsible for, if not all, a significant part of the educational provision. It also refers to education systems where there are no formal barriers to the schooling of any social group.

In this definition, we seek to encompass both the ideal of distributive justice in education and the ideal of recognizing diversities. On one hand, it aims to establish a standard for the fair distribution of outcomes that materialize the right to education. Understanding inequality as a relationship (Bobbio, 1993Bobbio, Norberto. (1993), Igualdad y libertad. Barcelona: Ed. Paidós Ibérica.), the definition of a fair distribution of educational outcomes will be constructed through a relational approach (Carter, Reardon, 2014), comparing results achieved by groups of students.

On the other hand, this definition aims to be compatible with a society in which historical-cultural diversity and moral and political values are not only legitimate but also the recognition of these diversities is a value that precedes individual prerogatives. This second requirement concerns, above all, the nature of the common culture that schools should transmit. As we will argue further, it should allow different individuals to exercise citizenship, engage in fair competition for positions and goods in society, and pursue their own diverse ends. Thus, the fair distribution of education, including a certain common culture, is a condition for the equitable recognition of diversity.

The subjects referred to in our definition of distributive justice in education constitute the school-age population of a specific territory, corresponding to a certain level of government responsible for education systems (e.g., municipality, state, federal). The agents on whom the definition focuses and seeks accountability are the State and the institutions responsible for serving the population. Therefore, our definition of educational justice is related to educational institutions – the State and education systems – and not to individuals – families, students, and teachers – and their practices.⁶ 6 . The distinction between principles of justice directed towards institutions and individuals is present in Rawls (2006 [1971]: 62 onwards) and is revisited by Allen (2016). Building upon this distinction, we focus on the State, which is responsible for regulation and most educational provision, although Article 205 of the 1988 Brazilian Federal Constitution mentions the family as being co-responsible for education.

We assume that justice in education aims toward fair outcomes, which is how the right materializes itself.⁷ 7 . The objective here is not to discuss the necessary conditions to produce these results. However, in discussing the outcomes, we intend to emphasize that resources and learning conditions should be provided with the aim of achieving these results. We focus on three outcomes. The first is universal access to school enrollment at the appropriate age; the second is school attendance, in which we distinguish two outcomes: the pace of the trajectory and the completion of educational stages. Thus, all enrolled individuals must be enrolled at the appropriate age and must progress in their studies with a regular trajectory between school grades and complete mandatory educational stages. The third is the acquisition of knowledge that should be developed during the school years, comprising a common culture and being necessary for the development of individuals’ potential and the exercise of citizenship.⁸ 8 . Like Satz (2007), our definition of citizenship traces back to the classic work of T. H. Marshall (1977), who conceptualized it as the civil, political, and socioeconomic conditions necessary for someone to be a full member of society. The order of these outcomes is important because access to enrollment precedes attendance, which precedes learning. In other words, when analyzing learning, it is necessary to consider whether access to school and attendance have been properly ensured. Otherwise – for example, if learning is observed in isolation – injustices that occurred earlier may be disregarded.

Principles of justice

Once the outcomes are defined, it is necessary to determine how they should be distributed among individuals within just education. We agree with various authors who argue that these outcomes cannot be addressed by a single principle of justice. Instead, a concept of justice in education requires different principles suited to each outcome (Cameron, Daga, Outhred, 2018; Brighouse, 2010Brighouse, Harry. (2010), "Educational equality and school reform". In: G. Haydon (ed), Educational Equality. London: Continuum International Publishing Group.; Satz, 2007Satz, Debra. (2007), "Equality, adequacy, and education for citizenship". Ethics, v. 117, n. 4, pp. 623-648.; Reich, 2013Reich, Bob. (2013), "Equality, adequacy and K-12 education". In: Allen, Danielle; Reich, Bob (eds.), Education, Justice, and Democracy. Chicago and London: The University of Chicago Press.). For this reason, we will discuss the principles of justice suitable for our purposes to then synthesize our conceptualization of justice in education.

The first principle of justice to consider is equality of outcomes among individuals. It does not accept any variation in outcomes between individuals and, therefore, is limited in its application to certain outcomes – notably, access to enrollment and completion of compulsory education. Thus, the primary condition for an education system to be just is that all school-aged individuals in the given territory have, throughout their compulsory education, access to enrollment adequate for their age and the school years they have already completed.⁹ 9 . Goals 1 and 2 of the 2014 Brazilian National Education Plan (PNE 2014) express this ideal by stipulating, respectively, that “all children aged 4 and 5 should be enrolled in Early Childhood Education” and that “the entire population aged 6 to 14 should be enrolled in the 9-year Elementary Education” (Brazil, 2014). Furthermore, once enrollment is ensured for all individuals, they must all complete the education that is compulsory by law.¹⁰ 10 . Target 4.1 of UNESCO’s Sustainable Development Goals expresses this ideal by stating that all girls and boys should complete primary and secondary education (UNESCO, 2015). This is also the direction that guides the goals of the Brazilian PNE, although its targets foresee partial achievements of this objective: 95% completion rate in Primary Education at the recommended age, universal enrollment of the population aged fifteen to seventeen and a progressive increase in net enrollment rates in Secondary Education (Brazil, 2014).

However, equality of outcomes cannot be applied to other outcomes: under certain precise conditions, some variation in outcomes between individuals is not only inevitable but also acceptable. These cases include the pace of trajectories and learning. After all, it is not reasonable to suppose and assume as a normative ideal that an entire birth cohort, once enrolled in the initial grade at the appropriate age, will progress at the same pace, without variation between individuals, until compulsory schooling is complete.

It is also not reasonable to stipulate as a normative ideal that learning outcomes be equal between individuals, aiming for all students in the same grade to have equal learning outcomes by having identical levels of proficiency. After all, if it were possible to produce a situation of equal proficiency among all individuals within the population, this situation would require sacrificing the development of individuals who could achieve levels of rare excellence. This situation would be undesirable, as it would not only violate individuals’ right to develop their potential but also because society as a whole may possibly benefit from the excellence of some.

Many indicators address the issue of variation in outcomes between individuals by using averages. However, they do not always explicitly state the normative and methodological assumptions made in doing so. As a hypothesis, averages assume that higher results compensate for lower ones. Thus, they synthesize the given population into an abstract subject: the average student. This average student is produced by erasing all the characteristics that distinguish and define individuals and social groups. This assumption is not appropriate because, first and foremost, education is a right of individuals who cannot be abstracted in favor of a nonexistent subject: the average student.

As Waltenberg (2006) points out, behind averages is a utilitarian principle of justice according to which what matters is maximizing the sum of the results observed in the population – which can be summarized in the result of the average individual – and not the distribution of these results among observable individuals and social groups. Mutatis mutandis, analyzing the just distribution of education through averages, is like analyzing the just distribution of wealth by Gross Domestic Product (GDP) per capita.¹¹ 11 . The PNE’s goal seven (Brazil, 2014) adopts averages to set learning goals by targeting values for the national IDEB average. By inducing the pursuit of maximizing the sum of results in the population without providing any information about the pattern of distributions of the values they summarize, averages accept any variation in results, including the most unjust: large inequalities between individuals or high rates of individuals with low results. Moreover, averages can grow alongside inequalities.¹² 12 . See Alves, Soares and Xavier (2016).

These limits of averages are often bypassed by comparing averages of groups of students defined by social characteristics such as socioeconomic level, race, and sex.¹³ 13 . This is the solution used in many works by multilateral organizations and academics, such as the OECD’s Education at a Glance report or UNESCO’s World Inequality Database. Although this solution allows for a certain analysis of inequalities between groups, it transfers the limits of the measure to within the groups. Since an average student represents each group, what matters is the sum of the group’s results and not the distribution of results among individuals within the group. However, some distributions of results may be synthesized by acceptable averages but contain unacceptable situations – for example, many students with significantly low results.

To address variations between individuals, there are alternatives to averages, which, as Waltenberg (2006) shows, stem from other theories of justice. In one way or another, their recent formulations are influenced by the work of John Rawls (1971-2006),¹⁴ 14 . See Kymlicka (2003) for a systematic overview of this debate. who renewed the debate on distributive justice and paved the way for justice theories seeking to establish a distinction between unacceptable variations in results because they characterized unjust inequalities and variations that can be accepted by institutions. It is within this debate opened by Rawls that we intend to frame our proposal.

In the field of education, two justice principles alternative to averages – sometimes treated as exclusive and oftentimes reconcilable – vie for defining which variation in outcomes between individuals can be accepted: equality among groups and the universalization of an adequate basic level (Satz, 2007Satz, Debra. (2007), "Equality, adequacy, and education for citizenship". Ethics, v. 117, n. 4, pp. 623-648.; Reich, 2013Reich, Bob. (2013), "Equality, adequacy and K-12 education". In: Allen, Danielle; Reich, Bob (eds.), Education, Justice, and Democracy. Chicago and London: The University of Chicago Press.). They were developed first in the American debate and then explored in various other contexts, albeit not always explicitly.¹⁵ 15 . Meunier (2005) takes stock of the formation of these notions in the Anglo-Saxon debate and how they have been received in nine European countries. Both principles were developed at the intersection of the political and scientific fields, and both emerged primarily in the political field.

The principle of equality among groups, sometimes referred to as equity, emerged greatly in the American field of education after the Brown v. Board of Education 1954 case, which determined that education should be provided on equal terms (Reich, 1989). In addition to the Brown case, the principle of equality became central in the political debate with the civil rights movement and the measures passed and implemented during Lyndon B. Johnson’s administration, such as the Civil Rights Act of 1964 and the legislation on social welfare called War on Poverty, also in 1964, of which the 1965 Elementary and Secondary Education Act was a part.

The construction of equality as a central value in the political field had consequences in the scientific field, of which we highlight two: it brought the issue of inequalities in education already addressed to the center of sociological debate (Walters, 2007Walters, Pamela B. (2007), Betwixt and between discipline and profession: A history of sociology of education. History of Sociology in America. (Ed. C. Calhoun): University of Chicago Press: Chicago.) and contributed to the revival of the social justice debate. In 1966, the famous “Equality of Educational Opportunity” report, organized by James Coleman – commissioned by the federal government in compliance with a provision of the Civil Rights Act – was published (Walters, 2007Walters, Pamela B. (2007), Betwixt and between discipline and profession: A history of sociology of education. History of Sociology in America. (Ed. C. Calhoun): University of Chicago Press: Chicago.; Brooke, Soares, 2008). In 1971, John Rawls published his Theory of Justice. Although its centrality has been disputed, the agenda derived from the Brown case continues to feed works on inequality and segregation to this day (Reardon, Owen, 2014).

The second principle has three different names – adequacy, sufficiency, and minimum standard – which, despite their nuances, express the same idea: setting a certain floor of results to be made universal. It appeared in the public debate later, especially in the 1980s, also because of political events. In 1983, during the Ronald Reagan administration, the document A Nation at Risk brought about a turning point in the debate on education, which, instead of prioritizing equality, began to emphasize raising the learning level and to pay special attention to two groups: students not learning the minimum considered acceptable and those achieving high levels of proficiency. In 1989, the Kentucky Supreme Court ruled that education should be provided at an adequate level, and from the 1990s onwards, other state supreme courts made decisions based on the same thesis. These events shifted the political agenda from the principle of equality to the principle of the minimum adequate level and had as a corollary the 2001 No Child Left Behind program during the George W. Bush administration (Reich, 1989). This shift in the political debate had an effect both on the research agenda in the sociology of education, removing the subject of inequalities from the heart of the debate (Walters, 2007Walters, Pamela B. (2007), Betwixt and between discipline and profession: A history of sociology of education. History of Sociology in America. (Ed. C. Calhoun): University of Chicago Press: Chicago.), and on theories of justice, reinforcing the central role of the adequate minimum level principle (Reich, 1989).

The principle of an adequate minimum level stipulates a floor that must be universal for all individuals and, consequently, only accepts variations above this level. It is a universalist principle of justice and is not organized according to the defining characteristics of social groups (Fleurbaey, 1995Fleurbaey, Marc. (1995), "Equal opportunity or equal social outcome?" Economics and Philosophy, 11, pp. 25-55.). It includes a concern not covered by averages: attention to those in the most disadvantaged positions.¹⁶ 16 . This is a principle used in measures of socioeconomic poverty and is also present in goal 5 of the Brazilian National Education Plan (Brazil, 2014), which stipulates that all children should be literate by the third year of Elementary Education. Therefore, it has the virtue of avoiding deprivation in absolute terms, which is a strong argument in its favor.

However, it does not say anything about the variation in results above the minimum floor, accepting any variation from then on, including clearly unjust ones. For example, it accepts that the proportion of individuals with high levels of proficiency is concentrated in certain social groups. Satz (2007)Satz, Debra. (2007), "Equality, adequacy, and education for citizenship". Ethics, v. 117, n. 4, pp. 623-648., who advocates an egalitarian conception of adequacy, and Reich (2013)Reich, Bob. (2013), "Equality, adequacy and K-12 education". In: Allen, Danielle; Reich, Bob (eds.), Education, Justice, and Democracy. Chicago and London: The University of Chicago Press., who advocates placing the principle of equality at the core of the debate, agree that the strongest objection to the principle of an adequate minimum standard comes from the fact that it disregards the fact that education is a positional good. In other words, the extrinsic value of education - its convertibility into positions in the social space and into symbolic and material goods - is relative and dependent on others not achieving such results. In short, the principle of an adequate minimum standard does not prevent relative deprivation. This is a strong argument against it.

Therefore, the principle of basic adequacy is a justice principle that makes it acceptable for the greatest social benefits associated with school education to be controlled by specific social groups that will achieve higher results. In summary, it ignores the reproduction of privileges for groups in a more advantageous position in the school space, which is why it is an insufficient justice principle for our purposes. For its virtue to be mobilized – combating the absolute deprivation of the most disadvantaged – it must be associated with the principle of equality or equity.

Equality and inequality are relationships, and, as such, addressing them requires answering two questions: Inequality of what? Inequality between whom? Consequently, the treatment of inequality and equality requires justice principles constructed based on the characteristics of individuals (Fleurbaey, 1995Fleurbaey, Marc. (1995), "Equal opportunity or equal social outcome?" Economics and Philosophy, 11, pp. 25-55.).

John E. Roemer, in Equality of Opportunity (1998), proposes that inequalities in results between population groups defined by attributes that explain these results and for which individuals cannot be held responsible should be considered unjust – and therefore unacceptable. Roemer’s definition accepts differences between individuals within groups but does not accept differences between groups. He refers to such groups as types:

The Roemerian principle of equality of opportunity is the subject of criticism. There is a recurring question of its applicability to basic school education because there is no acceptable basis for holding children and adolescents responsible for their school results, given that they are not fully formed moral subjects (Cameron, Daga, Outhred, 2018; Fleurbaey, 1995Fleurbaey, Marc. (1995), "Equal opportunity or equal social outcome?" Economics and Philosophy, 11, pp. 25-55.; Reich, 2013Reich, Bob. (2013), "Equality, adequacy and K-12 education". In: Allen, Danielle; Reich, Bob (eds.), Education, Justice, and Democracy. Chicago and London: The University of Chicago Press.; Satz, 2007Satz, Debra. (2007), "Equality, adequacy, and education for citizenship". Ethics, v. 117, n. 4, pp. 623-648.). A second criticism argues that this principle alone does not address the pattern of result distributions, which may occur at unacceptably low levels (Reich, 2013Reich, Bob. (2013), "Equality, adequacy and K-12 education". In: Allen, Danielle; Reich, Bob (eds.), Education, Justice, and Democracy. Chicago and London: The University of Chicago Press.) and/or may include unacceptably low results for those worst-off (Brighouse, 2010Brighouse, Harry. (2010), "Educational equality and school reform". In: G. Haydon (ed), Educational Equality. London: Continuum International Publishing Group.; Fleurbaey, 1995Fleurbaey, Marc. (1995), "Equal opportunity or equal social outcome?" Economics and Philosophy, 11, pp. 25-55.).

In the literature, there are proposals to address the first criticism regarding undue accountability. UNESCO (Cameron, Daga, Outhred, 2018) preserves the Roemerian measure under the name of impartiality. It seeks to restrict the measure’s objective to group comparison, observing situations of injustice among them and removing the debate about individual accountability. UNESCO also combines it with other principles. Fleurbaey (1995)Fleurbaey, Marc. (1995), "Equal opportunity or equal social outcome?" Economics and Philosophy, 11, pp. 25-55., on the other hand, does not adopt the Roemerian measure and proposes an alternative solution: society and the State should be held responsible only for some results that should be made universal, shifting the responsibility for the remaining variation to the family.

There are also proposals in the literature to address the second criticism regarding the pattern of result distribution. Fundamentally, these proposals suggest aligning the distribution of observed results with distributions assumed as normative standards. Fleurbaey (1995)Fleurbaey, Marc. (1995), "Equal opportunity or equal social outcome?" Economics and Philosophy, 11, pp. 25-55., although not specifically addressing education, proposes replacing equality between groups with universal minimum levels, transferring responsibility for the remaining variation to individuals and families. Satz (2007)Satz, Debra. (2007), "Equality, adequacy, and education for citizenship". Ethics, v. 117, n. 4, pp. 623-648. also advocates for the principle of universal minimum levels, which she calls adequacy, but acknowledges that education is a positional good, hence advocating for equality between groups in achieving higher results.¹⁷ 17 . “Adequacy views must look to not only the bottom of the distribution but also to the top of the distribution. Children of all walks of life must have a fair chance at access to elite universities and the career opportunities that depend on such access. The more that education is positional, the more that adequacy will converge with vertical equality of oportunity views” (Satz, 2007:643-644). Brighouse (2010)Brighouse, Harry. (2010), "Educational equality and school reform". In: G. Haydon (ed), Educational Equality. London: Continuum International Publishing Group. and Reich (2013)Reich, Bob. (2013), "Equality, adequacy and K-12 education". In: Allen, Danielle; Reich, Bob (eds.), Education, Justice, and Democracy. Chicago and London: The University of Chicago Press. also advocate for equality between groups, formulated as equal proportions of individuals from the lowest to the highest results, and argue for universalizing a minimum level.

The proposed approach to address acceptable result variations in the text partially incorporates Roemer’s definition. It argues for equality between social groups defined in terms of Roemerian types, calling it equity to distinguish it more precisely from equality between individuals.

However, the text recognizes the relevance of criticisms of the Roemerian model, agreeing that equality between groups is not a sufficient principle and that variations in results among individuals within groups need to be addressed. The text proposes the following solution: advocating that the distribution of results among individuals should adhere to a certain normative pattern called a level. On the one hand, the level should ensure the universalization of adequate minimums, called sufficiency. On the other hand, it should also encourage the concentration of students with higher results, hereby called excellence.

Regarding the issue of undue accountability, inspired by Rawls (1971/2006) and Allen (2016)Allen, Danielle. (2016), "Two concepts of education". In: Education and equality. Chicago and London: The University of Chicago Press., we emphasize that its definition of justice in education seeks to impact institutions, not individuals and their practices. Thus, the combination of the equity principle with the level principle aims only to identify the unjust inequalities for which the State and institutions responsible for educational provision should be held accountable and act upon. Like UNESCO (Cameron, Daga, Outhred, 2018), the text seeks to exclude from its approach the issue of accountability for remaining individual variations within groups. Inspired by Fleurbaey (1995)Fleurbaey, Marc. (1995), "Equal opportunity or equal social outcome?" Economics and Philosophy, 11, pp. 25-55. and Allen (2016)Allen, Danielle. (2016), "Two concepts of education". In: Education and equality. Chicago and London: The University of Chicago Press., we argue that these variations should shift toward individuals – families, students, and teachers – and their practices, proposing another debate.

In summary, we characterize a given situation as equitable or inequitable by comparing the distribution of results among groups of students formed by social characteristics strongly associated with the variation in these results, namely socioeconomic status, race, and sex. For example, if a difference is identified in the distributions of trajectories of elementary school students and/or the distribution of proficiency in Mathematics between self-declared white and black students of a particular grade, this signifies that racial identification is associated with these inequalities among these groups and, therefore, they are unjust and should be addressed by the State and institutions responsible for providing education.

We need to establish another conceptual precision for the approach to learning. From the vast curriculum debate, the text retains that there is a set of knowledge that, in contemporary societies, should constitute a common culture (Williams, 2015Williams, Raymond. (2015), "A cultura é algo comum". In: ____ Recursos da Esperança. São Paulo: Ed. Unesp.; Bourdieu, 2019a; 2019b). These pieces of knowledge have both intrinsic value, as they allow the development of attributes assumed as values in themselves, and extrinsic value, as they are expressed in social power relations that have consequences on individuals’ possibilities to occupy positions associated with rights and duties and appropriate wealth produced by social cooperation (Satz, 2007Satz, Debra. (2007), "Equality, adequacy, and education for citizenship". Ethics, v. 117, n. 4, pp. 623-648.; Reich, 2013Reich, Bob. (2013), "Equality, adequacy and K-12 education". In: Allen, Danielle; Reich, Bob (eds.), Education, Justice, and Democracy. Chicago and London: The University of Chicago Press.; Young, 2016Young, Michael F. D. (2016), "Por que o conhecimento é importante para as escolas do século XXI?". Cadernos de Pesquisa [online]. v. 46, n. 159, pp. 18-37. Disponível em: < https://doi.org/10.1590/198053143533>.
https://doi.org/10.1590/198053143533>... ).

This knowledge includes, for example, certain social uses of written language and mathematical knowledge. At the same time, it is true that these pieces of knowledge are far from exhausting the curriculum and even the common culture to be transmitted; they are central. Claiming that they form a common culture in no way means that they form equal people. On the contrary, due to their intrinsic values, these forms of knowledge allow forms of subjectivization and the development of ways of thinking, sensitivity, and practices that can be oriented in various directions. Due to their extrinsic values, they are essential for people to exercise their rights and duties and appropriate the fruits of social cooperation. This common culture is thus part of citizenship rights and is one of the conditions for diversity to manifest and be recognized on fair terms.

This knowledge can be – at least in part – objectified in standardized test results, which, in turn, can be measured and standardized in proficiency scales. These measures should be socially and pedagogically relevant. They can also be used to construct quantitative indicators that, in turn, express conceptions of justice in education.

Justice in education

Having presented the results and principles of justice, we can summarize the definition of justice in education with which we are working. In view of the precedence structure of the results that characterize justice in education, the institutions responsible for ensuring the right to schooling for the population of a given territory must guarantee the following achievements.

Firstly, universal access to school enrollment that corresponds to the age and school years already completed by the individual, from the starting grade to the end of compulsory school education. Secondly, they must ensure that the completion of compulsory basic education is universalized, accepting a very limited variation only in the pace of the trajectories, but in such a way as to ensure both equality between groups (equity) and a level defined by both the universalization of a minimum pace (sufficiency) and the search for the highest possible proportion of students with the maximum pace (excellence).

Thirdly, as far as learning is concerned, a just education must ensure both equality between the proficiency distributions of individuals from all social groups (equity) and a level defined by both the universalization of a minimum learning level (sufficiency) and the highest possible proportion of students with higher learning (excellence) across all grades.

Inequality and Learning Indicator (IDeA)¹⁸ 18 . Due to limited space, it is not possible to present all the technical aspects of the indicator’s construction in detail. A specific article is being prepared for this purpose. You can download the database from the IDeA Portal (https://portalidea.org.br/) and consult the results in greater detail.

Various indicators can express the definition of justice that we propose in education. These indicators will serve as ways to verify how this definition materializes. If the observed results do not meet the standard stipulated by the concept, they may be considered unjust. Prioritizing the problems and outcomes to be achieved, as well as setting goals must be based on the analysis of concrete situations; this is the realm of policy.

Brazil already has an extensive data system that permits constructing indicators compatible with this definition. The best indicator to evaluate access to education is the net enrollment rate for different age groups, calculated at different levels of government. At the municipal level, this can be done with data from the Demographic Census, conducted every ten years, or by population projections coupled with results from the School Census. For states and some metropolitan regions, this rate can be calculated annually using data from the National Household Sample Survey (PNAD). Based on this information, we know that access issues are limited to Early Childhood Education, which is not yet universal, despite substantial progress in recent years. We also know that there is a significant dropout problem that begins in the second segment of Elementary Education and becomes more acute in High School.¹⁹ 19 . Net enrollment rates in the country in 2018 were as follows: 81.4% of children aged four and five were enrolled in schools or daycare centers; 98.4% of people aged six to fourteen were enrolled or had already completed elementary school; 55.3% of young people aged fifteen to seventeen were enrolled in high school or had already completed basic education (see http://simec.mec.gov.br/pde/graficopne.php. Consulted on May 29, 2019).

It is possible to analyze school retention by observing birth cohorts’ school flow and completion rates over the stages of Basic Education. Demographic Census, PNAD, and School Census data allow us to build these cohorts and follow them longitudinally while identifying each student enrolled in a school at any time, for each calendar year, whether they are enrolled and, if so, their grade level. Thus, it is possible to calculate completion rates and the pace of trajectories for each cohort. It is possible, within each cohort, to measure completion rates and trajectory paces for different social groups.²⁰ 20 . Soares, Alves and Fonseca (2021) analyzed the inequalities of trajectories with an approach compatible with the one we have presented.

In this article, however, we will focus on the construction of an indicator to monitor learning, the IDeA. With the same methodology, this indicator addresses two dimensions: the level and the equity/inequality of learning.

To characterize the learning level, IDeA uses, on the one hand, the distance between the distribution of proficiency in learning a subject observed in a particular municipality’s set of students in the analyzed grade and a distribution of learning assumed as a desirable reference at the current moment in the country.

The calculation of this reference distribution was defined by Soares and Delgado (2016). The methodological procedure is analogous to the one used in defining the goals of the IDEB.²¹ 21 . See Fernandes (2007) on the construction of IDEB. Initially, the change that should occur in the distribution of learning for the set of Brazilian students in the Program for International Student Assessment (PISA) was calculated to achieve performance equivalent to a typical OECD country.²² 22 . The countries considered by Soares and Delgado (2016) were: Australia, Austria, Belgium, Canada, Korea, Denmark, Finland, France, Germany, Iceland, Ireland, Italy, Japan, Norway, New Zealand, Portugal, Spain, Sweden, Switzerland and the United States. Then, this change was applied to the distribution of learning for the set of Brazilian students measured by the Prova Brasil.²³ 23 . This procedure does not assume that learning in PISA and Prova Brasil are equal or equivalent. It only assumes that the size of the gap of all Brazilian students in PISA is an adequate measure of the improvement that the distribution of Brazilian students’ learning should have in Prova Brasil. The resulting distribution was considered a benchmark and, therefore, an appropriate goal for this moment in the country’s history.

There are two caveats about the reference distribution. The first is that the competencies measured in Prova Brasil and PISA are not the same, and therefore, the reference distribution for Prova Brasil should not be interpreted in terms of PISA. The second is that other reference distributions are possible, and the one chosen is justified because it focuses on the international context and meets the two requirements for measuring learning levels: sufficiency and excellence.

For each grade and year, the continuous distribution may be discretized according to the learning bands on Prova Brasil’s scale, which leads us to the shares below.

To characterize and measure situations of equity or inequality, IDeA calculates the distance between the distribution of learning within groups in the municipality, composed of individuals with certain characteristics correlated to proficiency: socioeconomic level, race, and sex. Thus, for each grade and subject, distances are calculated between the distribution of proficiencies of individuals with lower socioeconomic levels and those with higher socioeconomic levels, between the share of proficiencies of self-declared black individuals and self-declared white individuals, and between the share of proficiencies between girls and boys.

Defining the specific way to calculate these distances in municipalities was possible after overcoming three challenges outlined below. The first challenge was the choice of a measure suitable for our purposes since, as we will argue, commonly used indicators for measuring income inequality are not appropriate for them. The second challenge was constructing a methodology that allows for the calculation of inequalities in situations where there are few students in the groups that will make up the measures. The third challenge was to define interpretative ranges for the values obtained.

Gini coefficient, Theil index, and Kulbach-Leibler divergence

The widespread use of income inequality measures in the public debate suggests that they can be used to measure inequalities in learning. However, they are not suitable for analyzing justice in education as we define it. For this reason, before presenting the measure that is compatible with our definition -- the Kulbach-Leibler divergence --, we must argue why we do not use two commonly remembered ones: the Gini coefficient and the Theil index.

Both the Gini coefficient and the Theil index can be understood as measures that compare statistical distributions and summarize the distance between a given situation and a distribution in which all individuals have equal incomes. These measures implicitly make three assumptions: firstly, that the given distribution should be compared with a distribution in which there is equality of results between individuals; secondly, that there is a total and fixed amount of income between individuals; thirdly, that there can be a transfer of quantities of income from one individual to another, decreasing the income of those who concentrate more to increase that of those who concentrate less, in order to produce situations that show less inequality.

However, these assumptions cannot be taken for the reality of education. Firstly, because, as previously argued, equality of results between individuals is not an adequate principle, and certain variations in results can be accepted. Secondly, in the field of education, it doesn’t make sense to assume a total and fixed amount of knowledge to be shared. Thirdly, when one person shares their knowledge with another, they don’t become less knowledgeable than they were before -- they don’t become “poorer” in knowledge.

Therefore, in order to measure learning inequalities, it is necessary to construct indicators that simultaneously accept three assumptions of the reality of education: a) the distribution taken as a normative reference must accept some variation in proficiency; b) knowledge must be assumed to be variable and virtually infinite; and c) the transfer of knowledge must not imply a decrease in the knowledge of the person passing it on.

Neither the Gini coefficient nor the Theil index meet these three requirements at the same time. The Gini coefficient compares a given income distribution with a reference distribution in which all individuals have the same value without allowing the reference distribution to be altered to accept any difference between individual scores.

The Theil index summarizes the distance between a given distribution and a state of equality represented by a discrete uniform distribution in the domain of people, in which all people get the same proportion. The Theil index, used to measure income distribution, is calculated in the realm of people; in other words, it deals with the income allocated to each person.

To address learning distribution, IDeA adopts the measure of distance between distributions implicit in Theil’s index with two modifications. Firstly, we changed the domain of the information analyzed: we addressed proficiencies instead of people – in other words, we dealt with the proportion of people in learning scores. This change in the domain has an important effect on the indicator values, as it means that it is not necessary to assume a finite amount of knowledge to be distributed, which is the case when considering the domain of people. By comparing distributions of people rather than learning scores, one can assume that people can increase their scores without others having to reduce their learning.²⁴ 24 . Transposed to the learning analysis, the Gini and Theil would deal with the distribution of proficiency scores in the domain of individuals. As these indicators assume a fixed quantity of goods to be distributed, for a decrease in learning inequality, there would have to be a reduction in the concentration of people in the highest proficiency values. In the case of income, this can be done through redistributive policies defined in taxation and public spending, for example. Yet, this is not suitable for analyzing education. The democratization of literacy, for example, has never required these elites to know less than they did before school expansion policies. Secondly, we take as a reference a situation in which the scores have different proportions of people. In this way, we ensure that all three assumptions for a measure of learning inequality are met simultaneously.

The IDeA algorithm can be described as a generalization of the Theil index for continuous distributions. Two distributions in the domain of learning scores are compared using the Kullbach-Leibler divergence. The measure used in the IDeA is built by calculating the ratio between the densities in the two distributions at each point in the distribution domain. For technical reasons, the measure works with the logarithm of this ratio, weighted by the density of students in the distribution assumed as the reference. This measure can be interpreted as the change that must occur in the given distribution for it to become the distribution taken as a reference.

This measure was used for both learning levels and inequalities. To analyze the learning level in a given subject and grade, we calculated the distance between the empirical distribution of the proficiencies of all the students in a municipality and the distribution taken as a reference.²⁵ 25 . If a municipality has a higher concentration of students at higher proficiency levels than defined in the benchmark, KL will indicate this by assigning it positive values. In present-day Brazil, there are few municipalities in this situation and there are no cases of positive KL in the two subjects and school years analyzed. The reference distribution has historical validity: in the event of a significant improvement in the country’s education, another reference distribution will need to be constructed. To analyze learning inequalities in a given subject and grade, we calculated the distance between the distribution of proficiencies of the socially disadvantaged group and the distribution of proficiencies of the socially privileged group.

The IDeA was calculated for Brazilian municipalities and is restricted to describing the learning of individuals enrolled in the last year of each elementary school segment.²⁶ 26 . For more detailed information on the methodological aspects of the IDeA, see Soares, Castilho and Delgado (2018). For this reason, it is an indicator restricted to the school population taking the test and does not provide information on those who have left the education system or who have not completed the segment despite being old enough to do so.

Prova Brasil data from 2007 to 2017 was used to compile the samples needed for our calculations and enable building historical series. This data was grouped into four sets of three editions each: 2007-09-11, 2009-11-13, 2011-13-15, and 2013-15-17. From now on, they will be referred to by the last year of the triad. With the information from these tests, we calculated the learning level in Portuguese Language and Mathematics in fifth and ninth grade. For each subject and grade, we also calculated the learning inequalities between lower and higher socioeconomic groups, black and white students, and girls and boys.

Student Imputation

The quality of the measures that compose the IDeA depends on the number of students in the municipalities. Students who took the test inferred the proficiency distributions used to calculate distances – both to generate learning level measures and inequality measures. When a municipality’s number of students is small, the estimates become unreliable, as the information they produce does not accurately represent the proficiency distribution of all students. This situation is common in municipalities with small populations, and it becomes even more frequent when comparing inequalities among groups within municipalities.

Most Brazilian municipalities exhibit a small population size. According to the 2021 population estimate from the Brazilian Institute of Geography and Statistics (IBGE), encompassing a total of 5,570 municipalities, the lowest 25% of these had populations of up to 5,540 inhabitants, comprising merely 2.3% of the national population. Half of the municipalities had populations up to 11,732, constituting 7.7% of the overall population, while 75% had populations up to 25,765, accounting for 19.1% of the total population. The nation’s population is concentrated significantly in a limited number of municipalities, with 80.9% residing in the top quarter of the most populous ones. Furthermore, half of the population resides in a mere 3.6% of the municipalities with over 154,600 inhabitants, and 33% is concentrated in the top 1% of municipalities (56) with populations exceeding 430,000 inhabitants.

This challenge can be addressed through the application of statistical models. Despite the limited information in many municipalities, sufficient data exists at the municipal level to estimate proficiency distributions for each. This applies to both the overall student population – essential for calculating learning levels – and specific subsets of students defined by socioeconomic status (NSE), race, and sex – essential for calculating inequality. By using the parametric bootstrap technique, we initially fitted models to estimate proficiency distribution parameters. Subsequently, values were generated based on these models, reflecting what would be observed if the student population of the municipality were larger while still abiding by the municipality’s unique characteristics.²⁷ 27 . This problem is the subject of a specific article, which is currently being prepared. Here we will present the solution in its most general aspects.

The procedure for expanding the samples was as follows: we identified the municipalities that needed their samples expanded and the number of observations to be added to these samples, whether for calculating their learning level or for calculating their inequalities. Simultaneously, we fitted a model for all Brazilian municipalities, allowing us to estimate proficiency distribution parameters from which we could generate information to add to municipal samples. With this information, we generated proficiency values that were integrated into the original municipal samples.

Initially, we set the sample sizes of students below which simulated data would be necessary, even considering the sum of students from the three editions of the test under analysis. For the learning level of the municipality’s overall student population, we set a minimum sample of 100 students. To calculate intramunicipal inequalities, we defined a minimum size of thirty students in each group analyzed.

We used different criteria to determine the amount of simulated data in the groups used for inequality analysis and groups for learning level analysis. For the groups considered for inequality analysis, we always included the simulated number of students necessary for the group to reach at least thirty individuals. This way, they were compared with groups with a minimum of thirty individuals. However, to establish student samples to calculate learning levels, this same procedure was inadequate. If we were to include the simulated number of students until the municipal sample reached 100 students, in the case of small population municipalities, the samples would be predominantly formed by simulated data. Such an outcome could distort the municipality’s characteristics and yield imprecise measures.

Thus, to ensure sufficient samples and minimize these risks, we proceeded as follows. In each triad of tests and in each grade analyzed, we looked at the distribution of the number of students in municipalities with fewer than 100 students. Then, for each distribution, we calculated the difference between the first quartile and the minimum value. The value thus obtained was assumed to be the maximum number of students to be included in the samples, which ensured that all municipalities had at least the sample size of the first quartile. However, the number of students actually included in each municipality varied according to the number needed to reach 100. Whenever the difference to 100 was less than the maximum number to be included, the quantity actually included was less than the maximum possible number of inclusions.²⁸ 28 . For example, for the triad completed in 2015, the smallest sample of students in fifth grade was fifteen and the first quartile was forty-seven students, so the maximum number of students to be studied was thirty-two. The municipality of Castanheiros (in the state of Rondônia), which had thirty-six students, received thirty-two simulated students, resulting in a sample of sixty-eight students. On the other hand, Carmolândia (in the state of Tocantins), which had ninety-one students, received only nine students from the simulation.

In the second stage, we utilized the available set of municipal data to adjust a multilevel regression model for estimating the parameters of the proficiency distributions from which we would generate values to add to the municipal samples. We opted for a two-level hierarchical regression model: municipality and student. As explanatory variables, we used the year of the test, sex of the student, race of the student, NSE of the student, and average NSE of the student’s school. The model estimated effects, valid for the whole country, which measure the impact of each of these variables on proficiency. In addition, for each municipality, the model estimated the specific effect that the variables NSE, race, and sex have on student proficiency.

With this procedure, it was possible to calculate expected student proficiency values for all Brazilian municipalities, for each edition of the test, for the fifth and ninth grades, and for the two subjects assessed, according to the explanatory variables. It was also possible to estimate the variability around these values, which was obtained from the standard deviation of the random effect associated with the students.

We constructed a set of proficiency distributions with these parameters. Since proficiency distributions vary between schools based on their NSE, we chose to generate distributions by school. In all schools, eight distributions were built for student types defined by all possible combinations of values for three variables: NSE (1 and 5), race (black and white), and sex (girls and boys).

These distributions were used to expand municipal samples. To generate values used for learning level calculation, we initially calculated the proportion of enrollments among schools in each municipality and, in each school, the proportions of students by NSE, race, and sex. Considering these proportions, independent draws were made for NSE, race, and sex to select one of the eight student types. Once the type was defined, the proficiency to be integrated into the municipal sample was determined by drawing within the proficiency distribution of the previously drawn type. The number of draws per school was defined by the number of students needed in the municipality, respecting the proportions of students from that municipality among the schools.

To obtain the values that would make up the samples to calculate the inequalities, we proceeded as follows: in each school, we initially determined the defining characteristic of the group whose sample we wanted to expand (e.g., NSE 5). Two independent draws were then made for the other two characteristics (e.g., race and sex), which defined the selection of one of the four possible types (e.g., NS5, black, and girl). Once this was done, the proficiency to be included in the municipal sample was defined by drawing lots within the proficiency distribution. The number of draws in schools was defined by the required number of students to comprise the sample of the specific group in the municipality, adhering to the distribution of proportions of students in that group across schools in the municipality.

Interpretative categories of the measure

It is then possible to calculate the IDeA after the minimum samples are ensured for all municipalities. It is important to note that, in the case of municipalities with originally small student populations, the calculated values are dependent on the simulation process and are subject to greater random fluctuations.²⁹ 29 . When reporting the results by municipality, we inform readers when the figures were calculated using a simulated sample.

In theory, the Kulbach-Leibler (KL) divergence can take unlimited positive or negative values. Empirically, however, most of its values will be expressed as negative numbers – except for gender inequalities. Therefore, the smaller the values, the greater the distance between the distributions. These values were grouped into ranges that can be translated into Prova Brasil terms to make interpretation easier.

For the interpretation of the learning level, the KL scale was divided into five ranges: low, low-medium, medium, medium-high, and high. The values were defined based on the triad of tests completed in 2017 and then applied across all analyzed time frames.

These values were defined based on the interpretation ranges of the IDEB constructed by Soares and Xavier (2013)Soares, José Francisco; Xavier, Flávia P. (2013), Pressupostos educacionais e estatísticos do Ideb. Educação e Sociedade, v. 34, n.124, pp. 903-923.. This choice makes sense because both the IDEB and the IDeA are grounded in Prova Brasil, and therefore – although the values of each of these indicators may vary across municipalities – the interpretative ranges of the measures should be similar.

The interpretative bands for inequalities follow a pattern for SES (socioeconomic status) and race and another for sex. For SES and race inequalities, we defined three main interpretative bands: inequality, equity, and atypical situations. Equity corresponds to situations where we assume that proficiency distributions are equivalent. Inequality is the range in which we assume that proficiency distributions are different and favor students with higher SES or who are white. By clustering, this range was further subdivided into three ranges based on the size of the distances between distributions: inequality, high inequality, and extreme inequality. Atypical situations are the band where we assume that distributions are different and favor students with lower SES or who are black, contrary to the literature. While this band merits attention, interpreting it requires caution. Few municipalities fall into this category; they have a small population, underwent sample expansion through imputation, and the observed distances closely resemble the equity situation.

For inequalities by sex, there are no atypical situations, so there are only two main bands: equity and inequality. Inequalities, either in favor of boys or girls, were divided by clustering into three levels: low inequality, inequality, and high inequality. The levels of inequality by sex are different because learning inequalities by sex are smaller than those by NSE or race; furthermore, there is both a well-defined group with values close to equity, called low inequality, and there is no well-defined cluster that can be characterized as extreme inequality.

In conclusion, the defined values were:

Results

This section will present two sets of results obtained by the IDeA.³⁰ 30 . The results for each municipality are available on the IDeA Portal, where there are a number of resources for visualizing the data. Descriptive papers are being prepared which will present more detailed results, supported by maps and graphs. Due to space limitations, this article will limit itself to the tables that present the more general patterns of the results. The first identifies patterns in the relationship between learning levels and inequalities. The second identifies the characteristics of municipal learning distributions that are most strongly associated with variation in the position of municipalities between the learning level and inequality bands.

Regarding the relationship between the learning level and inequalities by NSE and race, as observed in Tables 4 to 7 , we identified a general pattern applicable to both disciplines. When analyzing the data synchronously within a given set of exams, a general increase in the learning level was shown to have a correlation with a rise in the proportion of municipalities experiencing inequality. Thus, in the case of municipalities with higher levels of learning, those with equity in terms of NSE or race are rare, while those with inequalities are more frequent. Conversely, in the case of municipalities with lower learning levels, those with equity in terms of NSE or race are more frequent, and those with higher levels of inequality are rarer.

This pattern is consistent over time and has been verified in all analyzed datasets. When analyzing historical series, the main change observed is that the pattern intensified in situations with a greater increase in the learning level and remained consistent in situations with greater stagnation in the level of learning.

Two pairs of tables below highlight this conclusion. Both present data for the first set, concluded in 2011, and the last set, concluded in 2017. The first pair, presented in Tables 4 and 5, corresponds to the situation where the country experienced the greatest improvement in its learning level: Portuguese Language in the fifth year. Note how the increase in municipalities at higher levels of learning leads to a decrease in cases of equity and an increase in cases of inequality, both by NSE and race, deepening the general pattern.

Two pairs of tables below show this conclusion. Both present data for the first triad, ending in 2011, and the last, ending in 2017. The first pair, shown in Tables 4 and 5, shows where the country experienced the most improvement in its learning levels: Portuguese Language in fifth grade. Note how the increase in municipalities at the highest learning levels leads to a decrease in cases of equity and an increase in cases of inequality both by NSE and by race, thus deepening the general pattern.

The second pair of tables, numbers 6 and 7, show where the country’s learning levels have stagnated the most: ninth-grade Mathematics. Note how the pattern of inequalities by NSE and race remains constant over time.

The relationship between learning levels and sex inequalities follows a different pattern than for NSE and race. Firstly, inequalities by sex do not show a correlation with the learning level in any of the school subjects. Secondly, the pattern of inequalities varies considerably according to the subject analyzed: there are many more municipalities showing equity in Mathematics than in Portuguese.

When one looks at the evolution over time of this data, shown in Tables 8 and 9, one notices the persistent patterns of inequalities, which remain relatively constant both when there is a greater increase in the learning level and when the level is more stagnant. The pairs of data below show these results.

The second set of results concerns the characteristics of the proficiency distributions of students in municipalities that are most strongly associated with the variation in the municipalities’ positions within the ranges of learning levels and inequalities.

Firstly, we argue that the decrease in the number of students at the lowest levels of proficiency, below the basic level – which could characterize sufficiency – is important for them to move from the low to the medium learning level in the IDeA. However, for municipalities to have a high learning level in the IDeA, it is necessary that they ensure a high proportion of students at the adequate/advanced level (approaching the ideal of excellence), in addition to a small proportion of students with low learning (approaching the ideal of sufficiency).

Tables 10 and 11 below show this conclusion. They show the median proportion of students in the municipalities in proficiency bands that discretize Prova Brasil scale into three: below basic, basic, adequate/advanced (Soares, 2009Soares, José Francisco. (2009), "Índice de desenvolvimento da educação de São Paulo Idesp: bases metodológicas". São Paulo em Perspectiva, v. 23, n. 1, pp. 29-41.). In turn, the municipalities are grouped according to their learning level on the IDeA. We chose to present the results for Portuguese Language in the fifth grade because that is where there has been the greatest improvement in the learning level over the years.

The synchronic analysis of results, considering each triad of tests, shows a gradual reduction in the share of students in the below basic proficiency range and an increase in their share in the adequate/advanced range as the learning level seen in the IDeA increases. However, this variation between IDeA learning levels does not occur at the same intensity: as the IDeA learning level increases, the share of students with higher proficiencies increases more intensely than the reduction in students with lower proficiencies. The diachronic analysis of results reveals what happened to this pattern in the case of a general increase in the municipalities’ learning levels. What we see is that the observed medians remain stable, with a significant variation, which confirms the conclusion: in more recent years, we see an increase in the median share of students with higher proficiency at the highest levels of the IDeA.

As we saw earlier, the group of municipalities with the highest learning level in IDeA is predominantly composed of unequal municipalities. Therefore, when we observe the learning distributions of municipalities among levels of inequality, we conclude that the greatest inequalities in IDeA are associated with a double advantage for privileged groups: students from these groups have a lower chance of achieving lower learning levels (approaching the ideal of sufficiency) and a higher chance of reaching higher learning levels (approaching the ideal of excellence).

To analyze the pattern of proficiency distributions in the case of inequalities, we calculated the ratio between the proportions, for each proficiency level in Prova Brasil, of students from each social group. We always indicated the socially advantaged group (NSE 5; white students; boys) in the numerator and the socially disadvantaged group (NSE 1; black students; girls) in the denominator. All calculations were made for all municipalities. We will show the data for fifth grade Portuguese Language by NSE, as it is the situation with the greatest increase in the level of learning and inequalities.

Tables 12 and 13 show the median ratios between the share of students in the municipalities in Prova Brasil proficiency bands for NSE for each inequality band in the IDeA. When we look at the municipalities from the equity group to the group with the greatest inequalities, we see a twofold increase in the advantages of the socially privileged groups.

Conclusion: this and that

Education indicators are grounded in an interpretation of the state of education they seek to impact. They serve as tools to scrutinize this reality and construct a normative perspective that seeks to provoke transformation. The identification of a pressing social concern – the urgency of addressing education inequalities and monitoring learning levels – led to the formulation of the IDeA indicator.

The analysis of the reality of education in Brazil over the ten-year period examined by IDeA reveals a consistent pattern: Brazilian municipalities are generally more equitable at lower learning levels. When they demonstrate higher learning levels, they tend to show more inequality. Exceptions to this pattern are rare. Furthermore, IDeA shows that over time, in situations where learning levels see an increase, this pattern intensifies—meaning that as the learning level rises over the years, the share of equitable municipalities decreases while the share of unequal municipalities increases. Conversely, when learning levels are stagnant over time, the pattern remains consistent.

When we analyze the distributions of student learning in the municipalities, we note that the municipalities with the highest learning levels also have a lower share of students in the below-basic level and a higher share of students in the advanced level of Prova Brasil; also considering that the most significant difference between municipalities, when they reach the highest learning levels, is the increase in the share of students in the adequate/advanced level.

In turn, when we analyze the learning distributions of the groups of students in the municipalities taken to measure inequalities, we note that greater inequality is associated with both: i) the lower probability of the favored group developing lower learning (approaching the ideal of sufficiency); ii) their higher chance of achieving higher learning (approaching the ideal of excellence).

Considering these results, we argue that attention to socially disadvantaged students who have very low learning levels has enormous potential to improve overall outcomes, both in terms of increasing the learning level and reducing inequalities. Indeed, monitoring the lower tail of the proficiency distribution curve is fundamental, as it complies with the principle of sufficiency.

However, both the specialized literature and the empirical data we present support the idea that monitoring lower levels of learning is not enough to characterize a state of justice in education. Education is a positional good, and the social benefits associated with it, which individuals benefit from, are largely defined by the difference that individuals have in access to the highest levels of learning about other individuals. What the empirical data shows are that not only is an increase in the level of learning associated with an increase in inequality, but this increase is also associated with an increase in the educational advantage that socially advantaged groups have over others, which translates into both a lower probability of obtaining low levels and a higher probability of access to higher learning levels.

In Brazil, better learning opportunities translate into increased inequalities because the groups with the most assets take more advantage of them, increasing their position over the others. As we have seen, this pattern has been reinforced over the years. In our proposal, an increase in the level of learning must go hand in hand with a decrease in inequalities across the entire distribution of proficiencies. If not, we stress that the country will lose sight of very important elements in the reproduction of education inequalities. More importantly, it will run the risk of fueling mechanisms that reproduce inequalities by not stimulating policies that counteract the secular tendency of benefiting privileged groups, which are associated with education advantages.

Notes

Data availability

Publication Dates

History