Efficiency of Expenditure on Education and Learning by Brazilian States: A Study with Data Envelopment Analysis

This work aims to demonstrate the efficiency degree achieved by the Brazilian states in the allocation of public resources for education. We used multivariate statistical analysis, with Data Envelopment Analysis and Multiple Regression, with data from 2001 to 2011. The results show that some states achieved good efficiencies such as Minas Gerais and Rio Grande do Sul. These states succeeded in conciliating reasonable expenses with good scores in education assessments and tests. Maranhão and Pará had unsatisfactory scores in teaching, but also lower budgets of public expenditure. Thus, focusing on Brazilian Public Education, the results show the efficiency degree in public spending and education outcomes, using several variables and time periods.


Introduction
The current development level of democratic principles and citizenship requires a review of how information on the activities of the public sector are disclosed to citizens. Citizen participation will shortly rise to a level where the mere control and checks for compliance to the laws will no longer be sufficient to meet the information needs.
In this context, the search for information on performance becomes a reality for which the public sector must be prepared. There is a gap in the disclosure of information on public management and its effective performance appraisal. Links are necessary to enable conclusions on the management of public resources and human and social development. And the development of new scientific methods can help improve the process of disclosing evidence-based results and of social control.
This study proposes a form of measuring and evaluating the results obtained with the allocation of public funds to education. The general objective of this work is to assess the degree of efficiency achieved by the Brazilian states in allocating public funds to education.
Nowadays, governments around the world are emphasizing the importance of education. In addition, the expansion of societies is partially explained by the accumulation of human capital, mainly provided by education. Increase the efficiency level of education spending has become a goal in government programs in several countries. At the same time, education systems are subject to measures intended to contain expenditure and improving educational outcomes (Giménez, Prior & Thieme, 2007).
On the same side, there is a lack in the academic literature of studies in management and accounting, focusing on education efficiency and public expendi-ture. There are also some gaps on the methods applied to identify and emphasize efficiency on public expenditure on education. Governments do not know if the money they are spending on education is having good results. And the academie could contribute to find new ways to evaluate the public expenditures on education and its results.
Data Envelopment Analysis (DEA) is a technique that could help to evaluate efficiency of public expenditure on education, because the technique is based on linear programming designed to measure the performance of Decision Making Units (DMU). Thus, DEA aims to know the Decision Making Units (DMU), which are able to maximize the production of outputs with a certain level of inputs. The method also allows minimizing the supply of inputs, keeping the quantity of outputs constant. The results can be compared to an efficiency frontier formed by the best practices of their peers. Gupta and Verhoeven (2001) assessed the efficiency of public spending on education and health in a sample of 37 African countries in the period of 1984 to 1995, compared with each other and, then, compared with Asian and Western countries using the FDH method, which is a little different from the DEA method. Afonso and Aubyn (2005) conducted a comparative study on the efficiency of health and education sectors based on a sample of member countries of the Organization for Economic Co-operation Development (OECD) and applied the DEA ad FDH methods.

Previous studies on the performance of education expenditures
The Agasisti's study (2011) also used the DEA method and the OECD's datasets. The study focused on the analysis of efficiency of higher education systems in European countries. Agasisti (2011) used as inputs the public spending (as a percentage of GDP), the rate of admission of students to higher education institutions, and the student-teacher ratio and, as outputs, the percentage of population aged 25 to 34 years with a higher education degree, undergraduate enrollment rates, employment rates of the population between 25 and 64 years old per educational level, and enrollment rate of foreign students.
The studies carried out by Hauner (2008) aimed to find explanations for efficiency differences in the public sector regarding public spending on health, education and social welfare by the Russian subnational governments. By employing the DEA method, the author points out that if the less efficient regions reached the most efficient standards, the outputs could be achieved with about 50 to 70% of the real public spending.
The research conducted by Zoghbi, Mattos, Rocha and Arvate (2011) aimed to evaluate the efficiency of public expenditure on primary education by municipalities in the state of São Paulo in 2005. The spending per student in primary education was assumed as an input variable. The output variables were: Age-Grade Distortion rate, Pass Rate, Brazil Proficiency Index, and the IDEB index, an indicator that is a linear combination of all these indicators. Their approach also presents a difference in relation to the others: a section where the municipalities were grouped by Administrative Regions, another by population size, another by GDP size, and a last one in which they were grouped according to the political party of the municipal manager.
Other scientific works (Agasisti & Johnes, 2009;Agasisti, 2011) were directed to the analysis of the efficiency of universities, from a cross-country perspective. The vast majority of these works were performed at universities in Europe. There is a lack of literature in South America in this field.
And Data Envelopment Analysis was developed by Charnes, Cooper and Rhodes (1978), based on the ideas from Farrell (1957), in the seminal work «Measuring the efficiency of decision making units». The authors applied linear programming to estimate an empirical production technology frontier for the first time. And Rhodes, and his doctoral thesis, applied Data Envelopment Analysis to evaluate education.

Data Envelopment Analysis in studies on education expenditures
Data Envelopment Analysis (DEA) is a technique based on linear programming designed to measure the performance of decision-making units (Senra, Nanci, Soares de Mello & Angulo-Meza, 2007). The concept of efficiency usually considered in DEA is the best form of converting inputs to products or outputs. Such outputs, or results, are related to the operational scale and management capacity of a decision-making unit (DMU), taking into consideration its production frontier. Thus, the efficiency frontier is empirically estimated based on the analyzed DMUs (Joro & Korhonen, 2015). Agasisti (2014) summarizes the technical efficiency measured by DEA as the capacity of a DMU in producing outputs, considering the existing inputs constraints.
In addition to the existing convexity, the DEA model usually assigns weights freely to each input or output so as to maximize the DMUs' productivity (Agasisti, 2011). The weights used are converted into a single virtual input or virtual output. The reason of both these items result in the efficiency associated to the DMU. The result of the virtual input or output determines the DMU's relative efficiency. The technique Leonardo FLach, Luísa Karam de mattos, anderson renan WiLL y Lucas Facciuto roscheL used is an attempt to find the best virtual unit for each real unit (Aristovnik & Obadic, 2014).
By adapting the DEA model proposed by Cuellar (2014) to the Brazilian regions analyzed in this study, we could consider the example of analysis of four regions (i = A, B, C, D), using one input x i to deliver two outputs y 1, i and y 2, i . Table 1 illustrates the example.
Assuming that the country regions have the same amount of inputs ( , region A is more specialized in (or is more focused on) in the production of output 1, while region B is it in the production of output 2 (y 1, A > y 2, A ; y 1, B < y 2, B ). Region C produces a mix of outputs, but it does not produce as much as the specialized regions (y 1, B < y 1, C < y 1, A ; y 2, A < y 2, C < y 2, B ). Finally, region D also has a mix of outputs similar to region C, but the performance of region D is lower (y 1, D < y 1, C ; y 2, D < y 2, C ) (Cuellar, 2014).
In the case of region D, we assume that it wants to produce outputs in the same level as of y 1, D and y 2, D . So, the aim is to determine the potential outputs (εy 1, D and εy 2, D ), where ε is the expansion factor) that region D can reach. It can achieve such efficiency if it adopts efficient peers' practices. In this regard, the main goal is to maximize the expansion factor of region Dε. To do this, it is necessary to consider the split of its inputs in its own practices and to copy the behavior of the other three regions to produce the outputs. This can be represented by the input constraint l 1 x A + l 2 x B + l 3 x C + l 4 x D ≤ x D , where l 1 , l 2 , l 3 , l 4 ∈ [0.1] are the intensities (weights) in copying the behavior of other regions. And all together they must be equal to or less than the input available for region D (x D ). On the other hand, the output constraints show that the weighted sum of the output must be equal to or greater than the potential output. In this case, the input allocation must deliver l 1 y 1, A + l 2 y 1, B + l 3 y 1, C + l 4 y 1, D of output 1 and l 1 y 2, A + l 2 y 2, B + l 3 y 2, C + l 4 y 2, D of output 2, which can be expanded at factor ε. Finally, the l 1 + l 2 + l 3 + l 4 = 1 constraint is imposed to allow the frontier convexity, taking into consideration the variable returns to scale (Cuellar, 2014).
The linear programming problem to find the optimum expansion factor for region D and its corresponding weights can be described as follows: Subjected to: 2) l 1 y 1, A + l 2 y 1, B + l 3 y 1, C + l 4 y 1, D ≥ εy 1, D (2.3) l 1 y 2, A + l 2 y 2, B + l 3 y 2, C + l 4 y 2, D ≥ εy 2, D (2.4) l 1 + l 2 + l 3 + l 4 = 1 (2.5) l 1 , l 2 , l 3 , l 4 ≥ 0 (2.6) The expansion factor ε measures the distance between the D's production and the efficiency frontier, which is defined by the linear combination that envelops the efficient countries. If ε* > 1 means that the country in question is within the frontier (i.e. is it inefficient), while ε* = 1 means that the country is on the frontier (i.e. is efficient). The l i value reflects the weights used in the programming to calculate the location of the inefficient region (Cuellar, 2014).
The linear programming problem, according to equations (2.7) to (2.11), is output-oriented. This means that inputs are fixed to accomplish potential outputs. When it is input-oriented, outputs would be fixed to evidence potential inputs, as described in the following linear programming problem: Notice that factor φ was included in the inputs, while the outputs are fixed. This means that the linear programing seeks for the factor φ, which allows to reduce the inputs to a certain level of outputs. This model is used to determine the wasted resources to achieve certain levels of output (Cuellar, 2014).
The efficiency frontier is empirically estimated based on the analyzed DMUs (Joro & Korhonen, 2015). Agasisti (2014) summarizes the technical efficiency measured by DEA as the capacity of a DMU in producing outputs, considering the existing inputs constraints.

Dataset and method of analysis
The present study is conducted with data of budget execution and the indicators' results for 2001 to 2011. For comparison purposes, analyses were carried out including, in addition to the data of budget execution, the values relating to the states' Gross Domestic Product. The 11-year period of analysis is justified by the fact that it allows to extend the time span required for the investment returns and avoids making shortterm comparisons.
In the first part of the analysis, we searched for budget execution data on expenditure reports per function. The reports were obtained from the website of the National Treasury Secretariat linked to the Ministry of Finance of Brazil. In a first analysis, data on the Education budget were used as expenditure per capita, dividing the amounts spent in education by the total population of the states; then, as a second form of analysis, by the number of students enrolled in the secondary school.
Data on the population of each state were obtained on the IBGE's website, which carries out census every ten years: The second part of data collection was focused on data relating to results indicators.
When obtaining the education expenditures index per capita (either per inhabitant or per student attending the secondary school) and the results indicators, the efficiency of allocation of public spending would be determined for the established period using the method of data analysis.
To conduct this survey, the DEA method is used to make comparisons on the efficiency of production units, called DMUs, which employ similar processes to transform multiple inputs into multiple outputs. So, by allowing that various perspectives on the understanding of inputs and outputs are grouped into a single indicator, this technique appraises the efficiency of operational units, which allows a perception of the relative organizational performance (Macedo, Casa Nova & Almeida, 2009).
Concerning the model orientation, the option is for the outputs-oriented model. Macedo, Casa Nova and Almeida (2009) understand that the choice for the inputs-oriented model should be made whenever outputs are not controllable by managers. In the rationale developed in this study, the outputs achieved can be improved by an adequate application of resources in education (input variable), although they are not an exclusive effect of this variable. Furthermore, the dynamics of the public resources management, whose objective is not the profits made every year, but the simultaneous application of the resources obtained, offers a direction to the understanding that inputs should not be reduced, but better results should be achieved from the resources used.

Variables and research stages
The form of resources management in Brazil, which is shared by different levels of the public administration of our federalist structure, also has limitations regarding the educational areas that can be assessed, since the states are responsible for the public provision of access to secondary and primary education, but do not have exclusivity regarding the latter.
Based on these considerations, the present study aims to expand the analyses to four-year results (2005, 2007, 2009 and 2011) using the input and output variables, as described in Table 2.
In this way, different aspects of analysis are considered to allow for more robust conclusions on the performance of the DMUs studied and to reduce the effects of a unilateral view of the subject. The «payback» time of expenditures and their conversion into results is also reduced when the calculations for each different period considered are made.
The IPCA (Broad Consumer Price Index) was used for monetary correction. For adjustment of the financial amounts as a result of monetary correction of the same fiscal year, the average adjustment factor, as expressed by Eq. 1, is used, as follows: Eq. 1: After the monetary adjustment, the adjusted values are proportionally adjusted to the size of the population in each state and the number of students attending the secondary school, and thus the indicator of per capita spending in education and culture, the GDP per capita, and the spending per secondary school student made by the Brazilian states are obtained.
On the IBGE website, one can find the population data in each of the censuses made. As in the period of this study the count of the population was only carried out in 2010, and the previous one in 2000, it was possible to estimate the population in the other periods of time using the following formula: Eq. 2: Growth factor = (POP2010/POP2000) 1/10 The monetary correction indices are applied to the data relating to budget expenditures, and the per capita annual spending with education and culture of each Brazilian state is obtained from the data on the population and population growth factor.
In order to mitigate the effects of choosing just one period for the comparison of expenditures and results, the efficiency indicators for each year were calculated based on the SAEB's education assessment, using the Note: Mean spending value is an estimate based on the total spending in education and culture divided by the state population. Calculation of the mean value is made considering the expenditures of the last 5, 4, 3, 2 and 1 years.
Output Grades scored by first-year primary education students according to the National System for Evaluation of Primary Education (SAEB.) Grades scored by students at the final years of the primary education according to SAEB. Grades scored by secondary students according to SAEB.
Premises: results imputed by both exclusive and shared obligations in education provision.
2 Input Mean spending value in education and culture per student enrolled in the secondary school Note: the mean spending value is an estimate based on the total spending in education divided by the number of students enrolled in the secondary school. Calculation of the mean value is made considering the expenditures of the last 3, 2 and 1 year.
Output Grades scored by the students of secondary school according to SAEB.
Premises: results imputed only by exclusive obligations in education provision.
3 Input Mean per capita expenditure in education and culture. Mean GDP per capita.
Note: the mean expenditure is an estimate based on the total expenditure on education divided by the state population. Calculation of the mean value considers the expenditures of the last 5, 4, 3, 2 and 1 year.
Output Grades scored by students of the first years of primary school according to SAEB Grades scored by students of the final years of secondary school according to SAEB. Grades scored by secondary school students according to SAEB.
Premises: results imputed by both exclusive and shared obligations in education provision; influence of the wealth level on the results attained by the states.

Input
Mean expenditure in education and culture per student of secondary school. Mean GDP per capita.
Note: the mean expenditure is an estimate based on the total amount spent in education divided by the number of students enrolled at the secondary school. Calculation of the mean value considers the last 3, 2 and 1year.
Output Grades scored by the secondary school students according to SAEB.
Premises: results imputed by exclusive and shared obligations in education provision; influence of the wealth level on the results attained by the states. For the analyses that are limited to the states' expenditures and results for the secondary education, the total spending in education was considered, given that there are not sufficient and reliable data for all Brazilian states about the expenditures on the sub-function «Secondary school», and this total is divided by the total number of students enrolled in the secondary school in each year. So, the efficiency indicator was calculated using as the input variable the average spending per student of the secondary school in the last three years, the average spending in the last two years, and the spending in the year of the education assessment.  At the bottom area of Fig. 1

Primary education and secondary education, with gdp per capita
This stage of analysis resumes the premise that the priority of the states comprises the primary and secondary education. It uses as an input variable the GDP per capita of each state, as a way to represent the economic activity and wealth produced, which may have effects on the outputs. Table 3 shows the efficiency indicators of the model, carried out using the SIAD software. In this model, the outputs are made up of the results of three educational levels assessed by INEP. As inputs, the spending per capita as well as the GDP per capita are considered, and the mean values are calculated considering the period of one to five years prior to the year when the education assessment was carried out.
According to Cavalcante and Macedo (2011), the increase in the number of variables can make that more DMUs are located on the efficiency frontier.
Notice that in this model there are always more than seven states with maximum efficiency, which increases the number of components in the first quartile (blank on the Table) and diminishes the number of components of the two central quartiles.
In addition to the states already cited as efficient according to the model without the GNP per capita variable (Minas Gerais, Rio Grande do Sul and Maranhão), Ceará and Piauí always scored 1.00 too and were between the most efficient DMUs. Maranhão, Ceará and Piauí, in the introductory section of analysis, stood out because of the lower availability of funds for education, combined with the fact that they are among the lowest GNP per capita in the country. Ceará has a greater amount of resources spent in education, when compared to the other two states, but is also among the states with the lowest GDP per capita.
Amazonas, Mato Grosso and Rio de Janeiro were permanently among the least efficient states. Amapá and Goiás were among the lowest indicators of efficiency in three of four segments. Table 4 describes the statistical results, applied to the most efficient units, obtained when using as input variables the annual expenditures per capita in education and the GDP per capita, in the period from one to five years prior to the respective education assessments, correlated with the results of the assessments for the first grades of primary school, last grades of primary school and secondary school.
Unlike the regression analysis applied to the first model, where it was not possible to infer conclusions, the regression analysis applied to the efficient DMUs group for this model, which considers the GNP per capita as one of the input variables, showed results of higher correlation when the grades attributed in the assessments of the secondary school (higher R-squared and lower F-value and P-value for the variables) were considered as output variables.   Table 3 Model 3   The results are consistent with the fact that this educational level is an exclusive obligation of the states, while the other educational levels studied here are the states' responsibility but in conjunction with the municipalities.

Secondary education, exclusively, and gdp per capita
The last stage of the analysis considers, just as in the second stage, only the exclusive obligation of the states in education: the secondary school. To the approach presented in the second stage, the states' wealth variable is added, including the GDP per capita to the inputs. Table 5 shows the efficiency indicators as calculated for the fourth model. As input variables it was used the estimate (restriction on the financial statements of part of the states) of the average spending per student of the secondary school, obtained by dividing the total expenditure in education by the number of students enrolled in the secondary school in each year, and the average GDP per capita of the same period in which the information on education expenditures (one to three years) were appropriated. The output variables comprise the results of secondary education, according to assessments conducted by INEP.
In the assessment of the results of the secondary school only and with the addition of the wealth variable, only two states attained maximum efficiency in all segments of time considered: Ceará and Rio Grande do Sul. The states of Minas Gerais, Maranhão and Piauí also are at a top position, particularly in the upper quartile, as occurred in the previous model.
Likewise, Amazonas, Mato Grosso and Rio de Janeiro were always among the least efficient states. Goiás, again, was one of the least efficient states in three or four time periods (2005, 2007 and 2009). Table 6 presents the statistical results applied to the most efficient DMUs group, obtained when using as input variables the annual average expenditures in education per student of the secondary school and the GDP per capita of the period from one to three years prior to the respective education assessments, correlated to the results of the assessments of the secondary school. According to what was observed, only the state of Rio Grande do Sul was always in the upper quartile in all years and models of analysis, although it was not always on the efficiency frontier (efficiency = 1.00).
Minas Gerais also stood out and was outside the upper quartile only when the expenditures per student of second school and the GDP per capita and the results of the education assessments in 2009 were compared, considering the period of three years. These two states are at the top position primarily because they had excellent scores in INEP's assessments. Except for the second school assessment of Minas Gerais in 2009 and the first grades of primary school in Rio Grande do Sul in 2011 (both at the 6th position in the ranking), these states were always among the five best states in the country as to all educational levels and periods of time studied.
Other two states that were among the best ones, although less often, were Maranhão and Pará. However, their efficiency is due more to low expenditure, because the scores given by INEP assessments were never been expressive and their results among the poorest scores is constant.
Still regarding the best efficiency indices, the tables also converge in some major points in a given period: The state of Pernambuco increased expenditures in education in the most recent periods, but has not yet succeeded in improving their results in education assessments accordingly. It was among the most efficient in 2005 due to low spending: until 2006 it was the state with the lower per capita spending in education.
Distrito Federal always scored good results in the education assessments. However, it is also one of the states with the highest expenditures per capita and the highest GDP per capita. The fact that it was among the most efficient in 2007 only was due to the fact that it achieved the best grades in secondary school in this year, combined with a «lower cost», due to the fact that between 2003 and 2007, the fiscal period considered for the 2007 financial statement, the Distrito Federal had lower per capita expenditures than three or four other states.
As for Mato Grosso do Sul and Santa Catarina, both states have similar characteristics: neither high nor low expenditures in education, combined with good education assessments (predominantly among the top five and never below the ten states with the best scores in the assessments). In 2011, both states indicated an optimum combination in their diverse educational levels, especially in secondary school, which place them in an outstanding position in efficiency in the last segment of time under analysis.
There was no state that was always in the lower quartile among the least efficient DMUs. The states that were predominantly less efficient are: Amapá, Amazonas, Rio de Janeiro and Sergipe. Rio de Janeiro, in economic terms, can be compared to Rio Grande do Sul, since it has one of the largest GDP per capita in the country and reasonable expenditures in education. Sergipe in turn is not different considering the regional comparison: expenditures in education are higher than the neighboring states, but the scores attained in education assessments are not consistent with the budget increase, and for this reason it appears among the less efficient states.
Amapá was always among the five states with the highest expenditures in education per capita, among the five ones with the worst performances in the primary school and reasonable results for the secondary school: nothing that could detract it from the concept of inefficiency, considering the amount of funds allocated.
The neighboring state, Amazonas, did not have such high expenditures and showed some evolution in the education assessments, especially in secondary school, where it was up from the 26th position in 2005 to the 16th position in 2011; nevertheless, its results are still low when compared to the budget spending allocated to education.
Future studies could apply this model and this method to different municipalities in Brazil, in order to analyse public expenditure in education.