Do We Make the Grade?

Trusteeship
November/December
2006
Number: 
6
Volume: 
14
By 
Peter Ewell
Trusteeship

As board members, we are accustomed to looking after the financial health and fiscal integrity of our institutions. Virtually every board meeting we attend features reviews of budgets, revenue projections, capital needs, or the approval of specific expenditures. But at many institutions the board only rarely gets to look directly at the heart of the academic enterprise: the quality of teaching and learning.

There are many reasons why this is beginning to change. And change it should. As in the far-more-visible world of elementary and secondary education, the products of our colleges and universities are under growing public scrutiny because of the overwhelming importance of developing the nation’s “educational capital” in a globally competitive world. At the same time, competition within higher education for more and better students means that attention to academic quality must be paramount if the institutions for which we are responsible are to prosper.

As in other realms of institutional operations, it is up to the faculty and administration to uphold and improve academic quality. But it is up to the board to understand it and to see that it gets done. Ensuring academic quality is a fiduciary responsibility; it is as much part of our role as board members as ensuring that the institution has sufficient resources and is spending them wisely.

Most boards have academic affairs committees that are responsible for broad oversight of the institution’s academic functions, including programs, curricula, teaching, research, and faculty affairs. These committees have a specific responsibility for ensuring that academic quality assurance and improvement mechanisms are in place. They can be expected, for example, to look carefully at the evidence about student learning or the results of academic program review and engage in interpretive dialogue with senior academic administrators and faculty committee chairs to determine potential implications and what improvements can be made. Occasionally, such dialogue will result in a recommendation to the full board about a potential strategic direction with respect to academic programming or a needed investment. For small boards that have no discrete academic affairs committee, the full board will have to judiciously assume these responsibilities.

Defining the Territory

One way to organize the processes that make up academic quality assurance and explain how they fit together is to visualize the institution’s teaching and learning functions as they would work in a typical business enterprise. Given this perspective, questions such as the following will be familiar to most boards of directors in other settings:

How good is our product? For colleges and universities, the principal product is student learning, and the quality of learning outcomes should be a central concern. And just as in a manufacturing enterprise, quality needs to be examined from at least two perspectives: the ultimate quality of the product on completion (what a student knows and can do upon graduation) and the “value added” by the “production process” of instruction (how much more a student knows and can do upon graduation than he or she did upon entering the institution). Determining these is the business of assessment.

How good are we at producing our product? Like every other “production process,” college- level instruction entails a certain amount of “waste.” Not all who enter our institutions as freshmen will complete their programs, and many of those who do will “stop out” for some period of time or will otherwise finish late. Patterns of student flow into and through our institutions are important to monitor because they affect both costs and outcomes. As a result, every institution should know something about student retention and completion rates.

Are our customers satisfied? Like all businesses, colleges and universities have a range of stakeholders, and the perceptions they maintain about the institution are important to monitor because they will strongly affect whether the stakeholders continue to relate to and invest in us. Among the key stakeholders from whom we must seek such information are students, potential students and their parents, employers, civic opinion leaders, and members of the public in the regions we serve. Periodically examining stakeholder perceptions and opinions can help tailor our product and anticipate emerging needs.

Do we have the right “mix” of products? Like businesses that typically offer diverse “product mixes,” colleges and universities offer degrees in many fields and provide instruction at multiple levels. Regardless of the levels of outcomes achieved and the efficiency of the production process, therefore, institutions should periodically take stock of their “product portfolios” to determine whether they are offering the right things at the right levels in the light of graduate quality and the marketplace. Stock-taking of this kind is typically a part of academic program review.

Do we make the grade? Finally, businesses that want to stay competitive must obtain external certifications or ratings of their quality. For products, this may take the form of becoming certified by Underwriter’s Laboratory or obtaining the Good Housekeeping Seal of Approval. Companies themselves can pursue international quality certifications such as ISO9001 or seek the Malcolm Baldrige National Quality Award. For colleges and universities, this basic quality certification is institutional accreditation, and it is critical for institutions if they are to remain in good standing.

Analogies between the academy and the world of business, of course, can be overdrawn. But characterizing some of the basic processes of academic management and quality assurance in business language emphasizes the fact that academic affairs should not be considered alien territory by members of the board who do not happen to be academics. Familiar principles of wise management and judicious oversight should inform these functions just as they do any other.

Defining the Board’s Role

The basic questions posed here are not only colloquial but are intentionally pitched at a level appropriate for board involvement. The role of the board in academic quality assurance, as in any other area, needs to be defined in terms of explicit boundary conditions that guard against overstepping the line between necessary fiduciary responsibility for the institution and directly managing its operations. This is always a delicate balance, and it is more so in academic affairs, where issues tend to be more complex and lines of authority less clear-cut than in more “businesslike” areas of institutional functioning such as finance, personnel management, communications, and fund-raising.

Issues are more complex in part because faculty from different disciplines may have different values with respect to what is most important in the curriculum and what constitutes “high quality” performance. There is a lot more ambiguity in these matters than in whether the fiscal bottom line is black or red. Lines of authority are less clear-cut because academic matters typically are managed by consensus, and individual faculty and departments are legitimately accorded a great deal of autonomy in defining what they do with respect to instruction and in carrying it out. Both of these factors mean that coming to closure on academic matters may take a good deal of time.

On the one hand, this means that boards need to display patience in allowing the process of academic deliberation to run its course. On the other, board members need to continue to press for answers and avoid the temptation to stop asking questions just because the process seems stalled.

For boards that have an academic affairs committee, these dialogues become deeper, and the need to maintain appropriate balance becomes even more important. Academic affairs committees, for example, will be expected to examine evidence of academic quality in greater detail than the full board and discuss emerging implications with academic leaders and senior faculty members. These discussions may well raise questions about curricular change, the need for greater attention to faculty development, potentially significant investments in instructional technology, or program inventory.

Because of their unique position of being able to see such evidence from the point of view of the institution as a whole, rather than from the perspective of particular departments or schools, committee members may be able to shed a different interpretive light on some of this evidence and should not hesitate to do so. But as for the board as a whole, committee members should be mindful that their role is about strategic direction, not the details of how things should be done. One way to begin to achieve clarity with respect to the board’s proper role in these matters is to consider the following principles:

Running the curriculum is the faculty’s responsibility; the board’s role is to remind them of that responsibility. Principles of shared governance at any institution mean that primary responsibility for the academic program is vested in its faculty.This means that faculty first must define the learning outcomes students are expected to achieve in each academic program and for the institution as a whole and then design and deliver a curriculum consistent with these objectives. This is a collective responsibility. Although it is normal for there to be a good deal of variance with respect to how each faculty member teaches toward established learning objectives, it is both appropriate and important for the board to insist that all faculty have such objectives, that instructors are conscious about designing learning activities consistent with these objectives, and that they are collecting evidence that these objectives are being attained.

What is important for the board is to remind the faculty that these expectations are an integral part of their academic responsibility. It is beyond the board’s appropriate role to question or dictate the content of these intended outcomes or the particular instructional designs and approaches used to achieve them.

Keep focused on strategic issues. Strategic issues are “mission-critical”—that is, they are issues that if left unattended will threaten the institution’s ability to fulfill its purposes. Balanced budgets are mission-critical, which is why boards pay so much attention to achieving them. All of the “basic business questions” noted earlier are mission-critical in this sense. For example, student retention is mission- critical because it affects both tuition revenue and the reputation the institution needs to attract new students. Similarly, program mix is mission-critical because it both defines the student markets the institution tries to tap and because it delineates the boundaries of the type of institution it aspires to be.

Above all, maintaining the quality of student-learning outcomes is mission-critical because it validates the claims the institut ion i s making about i t s graduates—claims that if unsubstantiated will affect graduates’ employment and postgraduate opportunities and ultimately the institution’s ability to attract new students.

In this regard, strategic questions reflect how well the institution is doing its basic job of graduating students who are competent and well prepared. They should not address the specific changes needed in response to negative assessment results. Similarly, strategic questions reflect the new opportunities for institutional markets or programs that may be revealed in the course of a program review. They are not about designing the content or instructional approaches that will characterize new programs.

Expect and demand a culture of evidence. In the past, colleges and universities viewed academic quality as an intangible—impossible to measure and in the eye of the beholder. This traditional view rested largely on institutional resources and reputation as a proxy for academic quality. The current premise of academic quality assurance is entirely the opposite—that it is possible to assemble meaningful and generalizable evidence of academic quality and to act on it to improve teaching and learning. But because many of these processes are new and unfamiliar, there remains a tendency at many institutions to make assertions about student learning or program quality based largely on anecdote.

Boards should not let that happen. Conversations about academic quality in any of its dimensions should be based on evidence. Boards should insist on this, and administrators and faculty members should come to expect it. Whenever claims about quality, effectiveness, or improvement are made, boards should always ask, “How do we know that?” If evidence-based answers are not offered, the follow-up question should be “What would it take to find out?”

Under many circumstances, adequate evidence may not be available for legitimate reasons. Methodologies for gathering appropriate evidence in some areas may be unavailable, inapplicable to the institution’s circumstances, or simply too expensive to pursue cost effectively. But the board nevertheless has a responsibility to ask the questions.

Recognize that evidence about academic quality raises issues but rarely gives final answers. Much of the evidence generated by academic quality reviews ends up being presented in numeric form and, as a result, has an air of precision that suggests “final answers.” In fact, on most occasions these data will represent not the end but the beginning of a conversation. This will be the case particularly in the deliberations of the academic affairs committee, where evidence about academic quality should be regularly presented and thoroughly discussed. When presented with such data, committee members should not just take them at face value but should ask administrators what they think the data mean and what action implications grow out of the findings.

In looking at information like this, moreover, committee members should be aware that such statistics mean little without an interpretive context. One way of providing context is to establish a point of comparison. For example, committee members might ask how a given statistic about academic performance compares with the same information from the previous year or to similar figures nationally or for a set of peer institutions. Another way to provide context is by breaking down the statistic further to examine how it looks for different campus populations. For example, questions might be raised about how men performed in comparison with women, how particular academic programs fared in comparison with one another, or how students receiving institutional aid performed.

Further, because academic quality is complex and elusive, no single piece of evidence tells the whole story. Committee members instead should ask administrators to provide evidence drawn from multiple sources and to engage the campus in holistic conversation about the “big picture” that emerges from a presented body of evidence.

Make reviewing evidence of academic quality and improvement a regular and expected board-level activity. It is easy in the context of pressing board business such as approving budgets, looking at construction progress reports, and handling legal matters to put off looking at academics. Yet teaching and learning constitute every institution’s main business, and academic quality should be of paramount concern to the board. Because the faculty and administration bear most of this responsibility, the board’s overall level of engagement in this area need not be time consuming—but it does need to be systematic. This implies that regular occasions to consider information about academic quality should be built into the board’s annual schedule. This can be done in many ways, including annual reviews by the full board of “dashboard” performance indicators that contain data on academic quality, making discussions of quality an integral part of a strategic planning exercise, or making assessment results a topic for a board retreat where more in-depth conversations can occur.

Reviewing results of the sources of evidence about academic quality should be an explicit responsibility of the academic affairs committee, which should establish a regular schedule for doing so in collaboration with academic leadership. Finally, the full board should be aware of preparations for institutional accreditation— a process that today focuses more heavily on the assessment of student-learning outcomes—and should thoroughly discuss the resulting accreditation report with administrators to determine what the institution should do in response.

The overall message here is that knowledge about the academic condition of the enterprise is as critical for a board as knowledge about the institution’s fiscal condition. As Harvard University’s Derek Bok put it in a 2005 essay in the Chronicle of Higher Education: “The traditional roles of trustees are both to defend and promote the interests of their institutions and to represent the concerns and needs of the public that does much to subsidize and sustain higher education. Examining the methods used to ensure the quality of education is a natural way of discharging the second role.”

Yet it is important to keep the board’s involvement strategic by ensuring (1) that the right kinds of academic quality-assurance processes are in place and (2) by periodically asking questions about how the administration is using the information it collects about the academic effectiveness to improve teaching and learning.

To reiterate, the board has a fiduciary responsibility to ensure that the institution is meeting its obligations. In this regard, a favorable accreditation outcome is as important to an institution as a clean financial audit. The board also must be assured that the administration is effectively managing the institution and is using tools and approaches consistent with known best practices in academic management. Among these practices are learning assessments, monitoring student flow, seeking feedback from students and stakeholders, and regularly reviewing the quality of academic programs.

The board as a whole, or through its academic affairs committee, ensures that these important processes are in place and are functioning effectively by requiring reports from the president and chief academic officer on their results and on the issues of academic quality that the results raise each year. It is as important for the board to know that these mechanisms are in place and that their results are being used as it is for the board to know that the institution is following sound budgeting and accounting practices.

Finally, as in any corporation, the board has ultimate responsibility for the soundness of the institution’s products and the integrity of its operations. At bottom, this means that it must be able to stand behind the competitiveness of the institution’s graduates with respect to their knowledge and skills and the academic integrity of the curriculum that prepared them. This is what we signify when we stand with our faculty and graduates at every commencement convocation. We need to act on this testimony in the boardroom.