by Joseph A. Russo, Director of Student Financial Strategies, Notre Dame

Each year a growing level of national angst follows the publication of the USNWR rankings.  Several other rankings also are published which attempt to measure various institutions, and not always in a positive light.  In the future, as a result of federal legislation (HEOA 2008), there will be yet another ranking by sector referred to as “lists of shame,” which will depict the highest rankings by cost, net cost, and rate of increase in costs of postsecondary institutions.  These attempts to measure various institutional attributes will be published in the context of a growing demand by policymakers, consumers, and the media for more accountability.  The current Administration in Washington speaks of “outcomes” and many state legislatures likewise seek to identify new means to encourage efficient performance in a reduced funding environment.

Moreover, policy shifts by other providers of educational funding for college, such as the Gates Millennium Scholarship Foundation, also focus increasingly on outcomes, not simply access, for postsecondary education investments.

While such a focus is appropriate in tough economic times, some might argue that reasonable expectations for performance should also be considered in evaluating “returns of investments.”  Related questions often follow, including who should go to college, how long should they be supported, and who benefits and who should pay and how.  Indeed, some may even ask how “college” should be defined (Baum and McPherson, “What is College?,” Chronicle of Higher Education, August 8, 2010).

One important measure of institutional performance is the commitment to serving low-income families.  Some attempts to evaluate such commitment use only one instrument of measure: the percentage of an institution’s first year, full-time enrollment of Pell recipients.  Lists of schools by Pell enrollment are published which simplistically suggest that those with the highest percentage of such Pell recipients are the most committed to serving low-income families.  Some observers even suggest that institutions whose Pell enrollments are low should have their eligibility for federal student aid as well as their federal tax exempt status revoked.  As interesting as Pell enrollment rates may be to measure commitment to low-income families, using more than this single criterion might paint a different picture.

Typically not included in the aforementioned evaluations are other pertinent details.  For example, what percentage of Pell recipients is dependent versus self supporting?  What percentage of the institution’s first year Pell recipients returns for the second year?  What percentage graduates?  What percentage graduates on time?  What percentage is “gainfully employed” within a reasonable period of time after graduation, and what percentage is accepted into graduate/professional school?  What is their average level of student debt and what is the student loan default rate among these Pell recipients?  What was their level of satisfaction with their experience at the institution?  Did these students have their full financial need met each year and if so, how?  What percentage of the institution’s scholarship/grant resources was used to support these Pell students versus non-Pell students?

The fairness of using such a narrow assessment is certainly questionable, especially if “commitment” includes such critical outcomes as student success and satisfaction, meaningful careers, manageable levels of student debt, and the acquisition of the most basic skills for having benefited from a quality experience, including the ability to communicate clearly, organize and prioritize, adapt to change, perform critical analysis, and be responsible for one’s decisions and be civil and tolerant.

Using only one measure instead of a more comprehensive evaluation would similarly be unfair if other single measures were so used: for example, an institution might annually be able to demonstrate that 13% of its graduates enter full-time public service careers and be able to make a convincing case that this is a clear demonstration of its strong commitment to serve the needs of low-income families.  To suggest that this indicator alone is how an institutional commitment is demonstrated is neither fair nor reasonable, as each institution has its own mission, curriculum, history, resources, etc.  The focus of a certificate program intended to train dog groomers or truck drivers is not the same as a state teachers’ college or that of a research university.  Of course, using the measure of percentage of graduates entering public service as the only indicator of commitment to low-income families would be similarly simplistic.

Or consider the institution where 80 percent of its undergraduates regularly participate in at least one community service volunteer program before they graduate, an effort that not only includes the school’s local community but programs across the country and even around the world; where activities occur not only during school periods but literally 365 days a year; where the coordination of all of this public service requires a full-time staff of two dozen employees and a separate major facility on campus.

On the other hand, an institution with 13% of its graduates entering public service or with a pervasive campus-wide community service program may enroll only 9% Pell recipients.  Considering either measure by itself alone misses all sorts of essential variables.  Many more examples using only one criterion would be equally unfair and questionable.

American higher education is rightly considered the very best in the modern world.  Part of its strength is the diversity of its institutions: history, location, selectivity, mission, source of support, endowment, curriculum, cost, etc. all vary and vary considerably.  While such diversity presents challenges not faced by other countries, and contains flaws and constraints which may call for improved policies and standards of accountability, respect for diversity of mission should always remain a solid principle for evaluating overall effectiveness and commitment to serving society…both in good economic times and bad.  Each institution has a contribution to make.

As we all prepare to carry out the provisions of the federally mandated “Net Price Calculator,” which is intended to provide prospective consumers the ability to compare and rank institutions by “net cost,” there is a potential for much confusion and even abuse.  A faulty definition of terms may create one of the most disruptive new dynamics in American higher education since the USNWR rankings first appeared.  Astute consumers would do well to ask institutional representatives many other questions about the value of the investment they are considering.  These include the many retention and success indicators of performance such as those outlined above for all of the students an institution enrolls, including its Pell recipients.

The opinions expressed in this article are those solely of the author and not those of the Educational Policy Institute.