Efficiency Estimates and Rankings Employing Data Envelopment and Stochastic Frontier Analyses: Evaluating the Management of U.S. Public Colleges

: This paper estimates and compares operating efficiencies of publicly owned associate degree granting colleges in the United States using data envelopment analysis (DEA) and stochastic frontier analysis (SFA). Comparisons are based on panel data for 698 colleges over four academic years, 2005-09. Included are both constant and variable returns to scale DEA estimates along with half and truncated normal inefficiency SFA estimates. The values 0.56 vs. 0.45 represent the largest mean DEA-SFA efficiency differential. DEA results indicate that 13% of colleges are fully (100%) efficient while SFA puts that percentage at only 1.7%. Comparisons of rankings based on efficiency performance generated a weak 0.65 correlation. Encouragingly, despite the financial turmoil initiated by the global crisis, the findings indicate that colleges have managed large efficiency gains over the four-year period. By 2008-09, DEA estimated efficiency increased to approximately 60%. Given continuing reductions in higher education public funding and increasing interest in public management reforms, the results should be of both managerial and public policy interest.


Introduction
This paper provides efficiency estimates and rankings for publicly owned and managed colleges in the U.S. The analysis is based on panel data observations of 698 associate degree granting colleges operating over the four academic years, 2005-09, and includes, therefore, the potential efficiency impacts imposed by the global financial crisis.To assess the robustness of estimates, operating efficiencies are evaluated and compared using both data envelopment analysis (DEA) and stochastic frontier analysis (SFA).In order to further the evaluations, estimation is performed under both constant and variable returns to scale DEA models and both half-normal and truncated normal SFA models.DEA as a nonparametric approach and SFA as a parametric approach represent the two standard techniques for estimating technical efficiencies.Both have been applied to a wide variety of industries in evaluating and ranking the managerial and operating performance of forprofit firms, non-profit organizations, and government agencies.That has brought interest in using both approaches so as to produce some confidence in the range of possible efficiencies.DEA-SFA efficiency comparisons have been applied across several industries.However, the public sector and U.S. higher education in particular has escaped rigorous evaluation.Thus, the present paper's empirical focus on DEA-SFA estimation of publicly owned college efficiencies offers several opportunities.First, the increasing interest in public management reforms has created greater managerial pressures to improve operating efficiencies.Yet, very little empirical evidence has been offered in the evaluation of college operating efficiencies.Secondly, the focus on public colleges can provide potential insights into the possible efficiency effects arising from the financial crisis induced budget cuts.Those managerial responses should be of interest since it is unlikely for the pre-crisis public funding levels to return anytime in the near future.Third, to date, U.S. higher education is not among the studies subject to DEA and SFA comparative evaluations and testing.Lastly, within the U.S. system of higher education, the two-year public college sector has witnessed the largest enrollment growth over the last decade and comprises more than sixty percent of the public sector higher education institutions.Studies of higher education production outcomes have been concentrated on research universities, largely neglecting the two-year sector of public higher education.The paper proceeds with the next section providing an overview of the literature.That is followed by an outline of the DEA and SFA methodology, and then a section explaining the data.The final two sections of the paper present the empirical results and concluding remarks.

Literature Review
Throughout the literature, it is well recognized that DEA is attributed to the seminal work of Charnes, et al. (1978) while SFA is jointly due to Aigner, et al. (1977) and Meeusen and van der Broech (1977).Contributions in further developing the approaches since those beginnings are numerous and well documented elsewhere.That need not be repeated here.Rather, methodological advancements along with empirical applications and implementation issues are provided by Cooper, et al. (2007) and Cook and Zhu (2008) for DEA and by Coelli, et al, (2005) and Kumbhakar and Lovell (2003) for SFA.These works are further supported by some 4000 published DEA research papers (Emrouznejad, et al., 2008).That volume of literature cannot be reviewed here.Instead, the following presents an overview of the empirical literature pertaining to DEA and SFA comparative efficiency estimates.That subset of the literature appears to consist of eight studies only one of which is an application of both DEA and SFA to higher education.The studies are wide in variety and investigate the operating efficiencies of Hawaiian swine farms (Sharma et al., 1997), Dutch dairy farms (Reinhard, et al., 1999), Bangladesh farms (Wadud and White, 2000), United Kingdom hospitals (Jacobs, 2001), English Channel fisheries (Tingley, et al., 2005), Canadian universities (McMillan and Chan, 2006), Greek dairy farms (Theodoridis and Psychoudakis, 2008), and Indian leather companies (Bhandari and Maiti, 2011).These studies rely on the basic idea that efficiency is based on firms producing the maximum output for a given set of inputs.A corresponding production frontier exists.Efficiency scores range from zero to one with the latter referring to efficient firms resting on the frontier while inefficient firms lie below the frontier with scores below the value of one.In the Hawaiian swine farm study by Sharma, et al. (1997), mean efficiency estimates range from 0.64 under DEA estimation to 0.75 under SFA estimation.In the evaluation of Dutch dairy farms, Reinhard, et al., (1999) develop models of both technical and environmental efficiencies and find a mean efficiency range of 0.44 to 0.89.For Bangladesh farms, Wadud and White (2000) find that mean efficiencies vary between 0.79 using DEA and 0.86 using SFA.For UK hospitals, a host of different model specifications employed by Jacobs (2001) generated mean efficiencies ranging from 0.65 under DEA to 0.88 under SFA.
The Tingley, et al. (2005) investigation of three different fishing fleets results in DEA vs. SFA efficiency estimates ranging from 0.56 to 0.65, 0.63 to 0.76, and 0.61 to 0.79.McMillan and Chan (2006) evaluated the operating efficiencies of 45 Canadian universities.In using different variables to define four DEA and four SFA models, the efficiency estimates range from an average of 0.91 to 0.98 under the DEA versions and from 0.89 to 0.95 under the SFA versions.The study by Theodoridis and Psychoudakis (2008) reports Greek dairy farm efficiencies on the order of 0.63 and 0.68 using DEA and 0.81 using SFA.A single comparative evaluation was not possible for the Bhandari and Maiti (2011) study of Indian leather companies.They present multiple specifications by year over seven years.Using an average of their 2002-03 results, there appears to be a 0.55 efficiency arising from DEA estimation and a 0.83 average efficiency derived from SFA estimation.In summary, the average estimated efficiencies range from a low of 0.44 under DEA estimation to a high of 0.98 using a SFA model.The minimum efficiency difference was found to be 0.07 while the maximum difference was more than six times greater at 0.45.The studies by Reinhard, et al., (1999), Wadud and White (2000), and McMillan and Chan (2006) indicate that DEA relative to SFA technical efficiencies are somewhat greater.The remaining five studies find greater efficiency scores in using SFA as opposed to DEA.However, the results come from eight different industries housed in seven different countries.In addition, each study uses a different set of variables, employs different times, and performs the analysis under different model specifications.Thus, it is best to use the results to conclude that DEA relative to SFA approaches can affect the estimation and evaluation of firm is operating efficiencies but the extent to which differences can arise appears to be industry dependent.However, none of the studies provide U.S. based efficiency insights.In addition, the single higher education study is based on efficiency estimates for a single academic year dating back to 1992-93.None of the studies include a period spanning beyond the financial crisis.Hence, there should be additional DEA-SFA efficiency estimates and comparisons that comprehensively include a U.S. experience, a public sector experience, and both a pre and post financial crisis operating experience.To that end, the present paper turns to an investigation of publicly owned postsecondary educational institutions in the U.S. operating during the 2005-09 academic years.

Efficiency Models
Data Envelopment Analysis (DEA): DEA presents a non-parametric technique for estimating a production frontier based on output and input observations for a group of decision-making units or DMUs (e.g., see Cook and Zhu, 2008).The resulting frontier is comprised of efficiently operating DMUs and is said to envelop the other DMUs that rest below the frontier and in relative terms are operating inefficiently.In the present context, colleges serve as decision-making units.Each college is producing an output, y, using a set of k=1, K inputs.Using standard notation, the formal output-oriented DEA model for the it college can be stated as follows: Where j=1, n colleges, the s and e are output and input slacks (both being ≥0), and i  measures the increase in output potential for each college, i.e., the estimated DEA production potential compared to the actual production.Hence, i  ≥1.The weights,  , on outputs and inputs give rise to variable returns to scale (VRS) in production and are due to Banker, et al. (1984).In this case, the underlying technology of production can be one of increasing, decreasing, or constant returns to scale.The more restrictive constant returns to scale (CRS) model originally developed by Charnes, et al. (1978) eliminates the last equation.The technical efficiency with which each college operates is based on its actual production accomplishment relative to its estimated production level for the frontier, i.e., Technical efficiency, therefore, varies in the range 0≤TE≤1 with the value of one representing an efficient college operating on the frontier.Technical efficiencies estimated under the CRS model will be less than or equal to the technical efficiencies coming from the more flexible VRS model.It is customary to estimate both efficiencies and use the results to compute scale efficiency (SE) as the ratio of CRS to VRS efficiencies.That custom will be followed in the empirical analysis of this paper.If SE=1, then a college is operating at the most efficient scale or optimal size.A SE less than one would be due to decreasing returns to scale(over production) or increasing returns to scale (under production).

Stochastic Frontier Analysis (SFA):
Continuing with the same notation established above, the stochastic production frontier model can be expressed as (e.g., see Coelli, et al, 2005) exp( ) Where x is a vector of inputs along with the parameters,  , to be subsequently estimated.The v's are the usual random errors (or stochastic noise) that is independently and identically distributed as N(0, They represent random shocks that are beyond the control of individual colleges in affecting the production of educational output.The u's are non-negative random variables also assumed to be independently and identically distributed and truncated at zero,N( 2 , u  ).They represent the potential technical inefficiency of college operations due to, for example, input characteristics and managerial decision-making.Many alternative distributional assumptions exist for the inefficiency term, including half-normal, exponential, and gamma.For evaluation purposes, the more widely used truncated normal and half normal will be employed in the empirical analysis to follow in this paper.From estimates of the stochastic production frontier comes the predicted technical inefficiency of individual colleges as determined by the estimated production with inefficiency present relative to the estimated production with inefficiency removed.Thus, technical inefficiency is exp( ) / exp( ) exp( ) In addition, varies in the range 0≤TE≤1.If inefficiency is absent, then TE=1 and the college is technically efficient.Unlike DEA, implementation of SFA requires the specification of a particular functional form and then proceeds with estimation via maximum likelihood.The Cobb-Douglas and translog are the most common specifications.Although the latter offers greater flexibility, it is also more demanding for estimation purposes and in preliminary tests using the current data, the likelihoods failed to converge.Therefore, the SFA model is estimated with the Cobb-Douglas form With the returns to scale being determined empirically by the sum of the input coefficients.The variance parameters (Coelli, et al, 2005) of the model are used to determine test the significance of including the inefficiency term in estimation of college production or proceeding with the ordinary least squares specification.Each of the above modeling approaches has advantages and disadvantages.These have been extensively addressed in the literature.In general, a major advantage of the DEA approach is that it does not require specification of a functional form for the production technology.SFA implies a search for a best-fit functional form.However, that deterministic quality of DEA also acts as a disadvantage relative to the SFA approach in that SFA allows direct statistical hypothesis testing related to the production technology.On another matter, DEA is useful in that the technical efficiency estimates are based on comparisons of individual units to a set of peers.A disadvantage is that DEA estimates tend to be very sensitive to outliers.SFA estimates are based on averages and, therefore, are less sensitive to such data variability.In addition, DEA ignores measurement error while SFA contains both a random noise component and an inefficiency component.
Data: For this study, data is obtained from the U.S. Department of Education, Integrated Postsecondary Education Data System (IPEDS) and include 698 publicly owned and operated two-year colleges accredited to offer associate degrees over the four academic years, 2005-2009.The academic years represent the most current and consistently measured set of data available at the time of the study.Since public colleges are government funded through appropriations based on student credit hour enrollments, the IPEDS output measure is the total twelve-month credit hour production.This is in keeping with earlier work pertaining to U.S. public higher education (Sav, 2004(Sav, , 2012) ) and accounts for credit hour enrollments over all semester terms, intersessions, and summer offerings.The two-year colleges being evaluated are considered to produce a single education output, i.e., they do not engage in graduate education or educational production beyond the associate degree.Moreover, unlike comprehensive four-year degree granting colleges and research universities, research output is not considered in the present paper as an institutional output.On the input side, the number of faculty employed is used as a labor input.In addition, the average faculty salary is included as a wage variable in an attempt to control for inter-institutional differences in productivity.Two capital input measures are used: the value of buildings and the value of equipment.Pell grant funding per student enrolled is included as an input to account for the government funding impact on credit hour production.These grants are low-income loans and comprise an increasing proportion of student recipients across many postsecondary campuses but a larger proportion at two-year colleges relative to four-year comprehensive and research type universities.Data was not available to capture other professional and labor inputs related to academic and student services support.However, IPEDS does report the institutional expenditures for both support functions.For proxies to other labor inputs, annual academic support and student support expenditures are converted to expenditures per student.
Table 1 provides a summary of the variables to be used in the DEA and SFA estimates of college operating efficiencies.On average, colleges produced nearly 140 thousand credit hours while employing 121 faculty at a salary of $55 thousand.Low income grant funding exceeded the internal college expenditures per student on both academic and student service support.Table 1 also shows the academic year percentage changes for each variable.On the output side, the percentage increases in credit hours clearly accompanies the financial crisis driven recession.The increase in government provided grants helped stimulate those increases.Not surprisingly, the increases in faculty employment lagged well below the production increases.In addition, the higher education budget cuts are evident in the 2008-09 decreases in college expenditures on both academic support and student services.

Results
Since the SFA efficiencies are parametrically dependent, the maximum likelihood estimates of the Cobb-Douglas production function are first presented.These appear in Table 2 for both the half normal and truncated models.So that it possible to obtain efficiency estimates by academic year and make comparisons to DEA estimates, the time varying SFA technical efficiency specification of Battese and Coelli (1992) is used in the Table 2 estimates.In addition, the truncated SFA model includes the Battese and Coelli "Inefficiency Trend".Here, the statistical significance of the Trend, confirms that college operating inefficiencies vary over time.Its positive coefficient indicates that inefficiency is decreasing, i.e., colleges are becoming more efficient.
The statistical significance of gamma indicates that the stochastic specification with the inefficiency measure included is preferred over that of ordinary least squares.In both SFA models, the log likelihood ratio and all the parameter estimates are statistically significant at the 1% and better level of significance.As expected, the coefficients indicate that faculty input, along with increased faculty productivity, enhance credit hour production.That also occurs with increases in capital inputs and government subsidies via increases in low-income grants.Somewhat at odds with expectations are the negative production effects occurring with respect to increases in academic and student support services.In that these expenditures are serving as proxies for non-faculty employment, their negative effects suggest that there may exist some degree of over staffing in terms of college administrative and support personnel.However, data that are more refined related to administrative and support staffing would have to be made available to better explore that possibility.It is now possible to turn to the efficiency estimates obtained from the DEA models and make comparisons to those that are derived from the SFA estimates.Table 3 provides the efficiency estimates.Mean efficiencies and other descriptive statistics of the efficiency distributions are shown in upper part of Table 3.The lower part of Table 3 presents the mean efficiencies for each academic year.Overall, the DEA mean results indicate that colleges are operating considerably below their optimal capacity.The SFA efficiency estimates confirm that result: both the half-normal and truncated normal SFA efficiencies are less than the variable returns DEA efficiency estimate.However, for the four technical efficiency estimates, the distributions are positively skewed and, therefore, there are some highly efficient colleges.The DEA variable returns to scale efficiency is greater than either SFA efficiency estimate.The difference is statistically significant at the one percent level and better (the t-value is 12.9).In fact, the differences between any evaluation of the DEA and SFA means are statistically significant at the one percent and better level of significance (the smallest t-value for any paired combination was found to be 7.4).The largest DEA vs. SFA difference in mean efficiency is approximately 0.117 or 11.7% efficiency.That is a smaller differential than that found in previous studies: the Reinhard, et al. (1999) Dutch farming differential was 0.45 and the U.K. hospital efficiency difference was 0.23.It is, however, a larger differential than the 0.09 DEA-SFA efficiency difference that McMillan and Chan (2006) found to exist among Canadian universities.That study was based on a 45 Canadian universities compared to the current sample of 698 U.S. colleges.Thus, differences in results might be attributed to differences in institutional heterogeneity that is built into the samples.It is expected that there exists greater homogeneity among the 45 Canadian universities relative to the 698 U.S. colleges.As a result, one would anticipate more efficiency variability among the present sample of 698 colleges compared to 45 Canadian universities sampled by McMillan and Chan.It is also important to note that the McMillan and Chan (2006) efficiency estimates range from 0.89 to 0.98 and, therefore, far exceed the current range of efficiency estimates.Yet, the McMillan and Chan (2006) efficiencies were the highest among the eight DEA-SFA published studies.Six of those eight studies generated efficiency estimates about 60%, 50%, and as low as 44%, thereby indicating that the present set of colleges are not unusually low in operating efficiency.
Examining the dynamics of college operating efficiencies, it is comforting to find that college efficiency improvements have occurred with each academic year.Those estimated improvements are present in all four DEA and SFA models.However, the mean technical efficiencies might be disappointingly low in the eyes of some public managers.Even with the annual efficiency improvements, under the CRS model and the halfnormal SFA model, the efficiencies rise to just above the 50% mark by the 2008-09 academic year.
Comparing across models, the truncated SFA and CRS-DEA efficiencies are lower than their counterparts are.The SFA differences can only be attributed to the different distribution assumptions related to the inefficiency term, while the DEA differences are due to the scale efficiencies contained in the CRS vs. VRS models.Those scale efficiencies (SE) as reported in  In a final analysis, rankings were created based on individual college efficiency scores.The rank correlations are presented in Table 5.The highest correlation occurs between rankings based on the alternative SFA models, thereby indicating that relative college performance is somewhat invariant with respect to the two distributional assumptions for the inefficiency term.However, it must be recalled that the absolute performance is affected; that was previously apparent in the results presented in Table 3.The weaker correlation existing between the two DEA models is, of course, due to the underlying returns to scale differences present in the CRS compared to the VRS model.Comparing across techniques, both SFA model rankings show a higher correlation with the CRS ranking (approximately 0.9) than with the VRS ranking (approximately 0.65).That seems somewhat at odds with the ability to statistically reject constant returns to scale under both SFA empirical results and the DEA findings that increasing returns prevailed at some 87% of the colleges.To a large extent, this can be due to the skewness of the efficiency distributions that are also embedded in the rankings.That is, the substantially more skewed CRS relative to VRS efficiency scores are more closely matched by both SFA distributions and, as a result, produce a higher correlation in ranking the individual college performances.

Conclusion
This paper provided DEA and SFA estimates of operating efficiencies for 698 publicly owned and operated two-year colleges accredited to offer associate degrees in the U.S. Efficiencies were estimated using four academic years, 2005-09.When evaluated at the overall mean efficiencies, college efficiencies are estimated to be between 44.7% and 56.4%.However, under all modeling results, colleges were found to operate at higher efficiency levels with each passing academic year.In fact, DEA results indicated that operating efficiency increased to an estimated 60.7%.Over the four-year period, there was no empirical evidence to suggest that the financial crisis, recession, and accompanied higher education budget cuts had a noticeable impact on college operating efficiencies.That is to the credit of those managing college resources and production.Comparing DEA to SFA estimates, the results show greater efficiencies when employing DEA models relative to SFA models.The mean efficiency difference is largest when employing a variable returns to scale DEA model compared to a SFA model with a truncated inefficiency distribution.The estimated efficiency difference is approximately 0.12 and arises from a DEA efficiency score of 0.564 vs. a SFA score of 0.447.The mean efficiency score under DEA constant returns to scale was estimated to be 0.474 and under a half-normal SFA model, it was 0.514.Upon examining the distribution of college efficiency scores, the estimates indicate a more skewed SFA compared to DEA distribution.All distributions were positively skewed, but using the variable returns DEA and truncated SFA distributions as benchmarks, a significantly larger proportion of colleges are beyond 90% efficiency under DEA relative to SFA; 13% vs. 1.7% of the 698 colleges, respectively.The findings offer caution in using a single DEA or SFA modeling approach for purposes of ranking individual colleges according to their efficiency performances.That caution seems warranted either based on the finding of a fairly weak correlation of approximately 0.65 between performance rankings derived from variable returns DEA and the half-normal or truncated normal SFA college rankings.Interestingly, that correlation is identical to that reported by McMillan and Chan (2006) in their study of Canadian universities using 1992 data.However, ranking comparisons are subject to a host of additional tests beyond the scope of the current inquiry.For example, Martin and Roman (2006) present and apply six different methodologies for testing the DEA determined performance rankings of Spanish airports.A fruitful avenue for future research might involve an extension of those methodologies to more rigorous validations of comparisons between DEA and SFA efficiency rankings.Based on the present findings, it is recommended that additional research be conducted along those lines but also to investigate and provide a better understanding of the underlying determinants of institutional efficiency.While environmental factors have been integrated into SFA models and multi-stage techniques have been developed for DEA models, little has been accomplished in the way of rigorously offering to decision makers the managerial and institutional ingredients that create efficiency.In addition, there appears to be a lack of comparative efficiency evaluations between for-profit and non-profit institutions.With the increasing entry of for-profit colleges and universities and the global higher education competitiveness created by on-line technology, the analyses presented in the current paper combined with these recommendations should be incorporated into future research agendas.

Table 3 : Efficiency Estimates Under DEA and SFA Specifications DEA SFA
college efficiencies range from 0.474 to 0.564 depending upon the constant or variable returns to scale specification.That suggests that with given resources, colleges can, on average, increase credit hour production by approximately 53% to 44%.Of course, the constant returns to scale efficiencies are lower than variable returns to scale due to the presence of scale efficiencies.Thus, the scale efficiencies are presented separately: i.e., scale efficiency is determined the ratio of CRS to VRS efficiencies.
Table3remain below the value of one and, therefore, indicate colleges are on average scale inefficient.In a separate DEA analysis, however, it was found that 87% of the colleges operate under increasing returns to scale.The increasing returns to scale apparent in both SFA estimates lend support to those findings.Yet, the DEA scale inefficiencies indicate that most colleges are producing below optimal size.That suggests that the associate degree granting colleges could take on more credit hour production while realizing decreasing average costs.The results for the mean 2005-09 efficiencies mirror the annual efficiency differences with the more flexible VRS model showing the larger 0.564 efficiency under DEA estimation and the more general truncated model showing the smaller 0.447 efficiency under SFA estimation.That 11.7 percentage point efficiency difference is only somewhat narrower at 9.35% when examined at the median efficiencies for the two estimations.The distributions of efficiencies are not substantially different between the CRS and VRS models or between the two SFA results.Comparing across models, the VRS and truncated SFA efficiency estimates show the least amount of skewness with the VRS distribution being relatively flatter.A more thorough distributional picture is provided in the Table4frequencies.The frequencies are for the 2005-09 mean efficiencies as estimated under the VRS and the truncated SFA.As indicated, while 13% or 91 of the colleges are efficient under the VRS estimation, only 1.7% or 12 of the colleges approach that efficiency level when evaluated under the SFA model.Similarly, 73% of colleges exceed the 50% efficiency mark under VRS and only 57% do so under SFA.