Who Audits the Auditor? Assessing the Quality of the Internal Audit Function

By Frank Bossle, Betty McPhilimy, and Michael Somich    //    Volume 23,  Number 3   //    May/June 2015

Internal audit departments can be major contributors to the effectiveness of governance, risk management, and internal control processes, but they need to focus on the areas of highest risk, perform their services effectively, and clearly communicate the results of their work. An external review of the internal audit function can validate the quality of the unit and ensure a clear alignment of expectations among the audit committee of the board, the college or university’s senior leadership, and the internal audit leader. The question of “Who audits the auditor?” characterizes and reinforces the argument that every auditor should embrace having such a review on a regular basis.

A quality assessment review, or QAR, provides an occasion to consider opportunities to improve the internal audit function. The outcome of the review may provide a roadmap for agreed-upon changes to increase the value of that function to all its stakeholders, including the department itself, senior leadership, and the board—especially the audit committee.

Many internal audit departments overwhelm audit committee members with detailed reports, assuming the trustees can weed out the most important information. Internal audit departments usually present the annual audit plan for audit committee approval, but they may not present updates on changes to the plan or actual completion information. It is crucial that internal audit professionals clearly communicate the impact of their activities.

It is good practice for those who serve on a governing board to ask the following questions:

  • Is the internal audit function adding value? Are metrics in place to evaluate those expectations?
  • Have all parties agreed where internal audit might add value and focus its efforts?
  • Are responsibilities for risk, compliance, and internal audit well delineated?
  • Is internal audit proactively involved in change?
  • Has internal audit ever had an external review? If so, how broad was the scope and how engaged was the audit committee with that process?

The board can play a vital role in pushing an internal audit function into having a quality assessment review. An important part of that review is interviewing senior leaders and audit committee members to ask their perceptions of the scope and quality of services.

The chair of the audit committee of the board at Boston University, J. Kenneth Menges Jr., expressed the expectations of his committee on the quality assessment review process: “We viewed it as a potentially helpful measure in assessing the quality and effectiveness of the internal audit function. We looked for assurance that our internal audit resources were focused on areas of highest risk with the training and tools necessary to perform effectively.”

Walter M. (Sonny) Deriso, Emory University chair of the audit and compliance committee, noted, “These types of reviews, when done well, are invaluable to executive leadership and the board because they help ensure the audit function serves as a real partner in advancing the mission of the institution.”

The Importance of Quality Assessment

Internal audit functions evolve over time, and choices are made that impact the range of services that the internal auditors provide. While it is true that audit committees approve the internal audit charter and annual plan, few audit committee members have the experience to suggest different approaches. Similarly, senior institutional leaders may have reservations about suggesting new ideas or areas of emphasis in the annual audit plan. And, for his or her part, the director of the internal audit function, or chief audit executive, may be reluctant to revisit decisions made in the past. The external review can be the impetus to agree on the key areas of focus for the internal audit staff.

An external quality assessment review provides a great opportunity for an internal audit director to benchmark with, and gain insights from, a team of experienced internal auditors from other organizations who can share effective tools and techniques. Benchmarking internal audit activity can also help establish metrics. These metrics will indicate improvement in areas of partial conformance or nonconformance with the International Standards for the Professional Practice of Internal Auditing (Standards) established by the Institute of Internal Auditors (IIA), the recognized authority in the field, with more than 180,000 members serving in 160 countries around the world. Many internal audit functions have not had an external review despite the fact that one is required to be in full compliance with the standards of the profession.

In January 2002, the Standards were updated to require all internal audit activities— regardless of industry, sector, legal or cultural environment, size of organization, profitability, geographic location, language spoken, or size of staff—to develop and maintain a quality assurance and improvement program (QAIP) every five years. An indispensable component of such a program includes having internal and external assessments. In fact, it is mandatory that every internal audit activity undergoes an external quality assessment conducted by a qualified independent team or independent validator once every five years to comply with IIA Standard 1312. This review encompasses an assessment of compliance with the International Professional Practices Framework of the IIA. A quality assessment review evaluates conformance with the IIA’s definition of internal auditing, the Standards, and whether internal auditors apply the IIA code of ethics.

“The process of going through a QAR was a great way to have the department reflect on our current practices and identify ways we might improve them,” commented Richard Cordova, the chief audit executive at the University of Washington. “The external team then provided thoughts on how we might enhance and supplement some of the ideas and validated the good work we had already begun.”

A quality assessment review offers the opportunity to sit back and reflect on various aspects of the internal audit function, leading to a wealth of insights and recommendations. Through interviews with management, senior leaders, and board members, greater awareness of the department and its role emerges that can help it implement change and evolve into a more effective unit. Involving the institution’s leadership in the process ensures greater effectiveness. Indeed, recognizing the difficulty of changing the course and direction of most operational units, one could easily argue that it is hard to make any substantive organizational or structural changes without having had such a review and using the resulting recommendations to effect a new approach.

Methodology for a QAR

The Institute of Internal Auditors (IIA) has produced a set of tools that streamlines the process for gathering evidence and performing a quality assessment. It has standardized the tool format and methodology to promote ease of use while achieving consistency in assessment techniques and documentation of results—no matter where the internal audit function operates across the globe. The QA Manual, published by The IIA Research Foundation, is the principal guide and set of practical tools to assess conformity to the Standards and, equally important, to reveal opportunities for enhancing the effectiveness and value of internal audit activities.

A variety of individuals or firms can perform the external review, whether it is limited to compliance with the Standards or has a broader scope. A common practice in higher education is for the chief audit executive to select a team of reviewers based on his or her awareness of experienced leaders in the field. Because that can raise concerns about objectivity, the chair of the board’s audit committee should be involved in the selection of members of the review team.

Professional service firms can be valuable in leading reviews because they generally have had significant experience in evaluating a wide variety of internal audit departments. In selecting the review team, it is helpful to get different perspectives and experiences. The ideal review team would also include experienced audit leaders who are proficient in the Standards and who lead their own internal audit function—and thus are able to offer valuable benchmarking insights.

In addition, members from a public accounting firm or a consulting firm that specializes in internal audit services can offer distinct perspectives that ensure materiality and significance are the focus of the internal audit unit’s strategic goals. Similarly, people who have been the recipients of internal audits— such as controllers, business administrators, and associate provosts—provide valuable perceptions of the effectiveness of internal audit recommendations and can offer insights for improving auditor/auditee relationships.

There are two approaches for meeting the requirement of the IIA’s Standards for conducting a quality assessment review:

  • Full External Assessment. This method uses a qualified, independent team led by an audit expert. Team members should also be competent professionals who are well-versed in the Standards, assessment methodology, and successful internal audit practices. The assessment team works on site at the headquarters of the internal audit activity, using the Manual to conduct interviews, surveys, benchmarking, and a review of work papers. The assessment team also drafts a report stating the internal audit department’s conformance or nonconformance to the Standards and providing any recommendations for improvement.
  • Self-Assessment with Independent Validation. This method begins with the internal audit function performing a self-assessment of its compliance with the Standards. A competent independent evaluator who is well-versed in quality assessment methodology then comes in to validate that self-assessment. In addition to reviewing the self-assessment and substantiating the work done by the self-assessment team, the evaluator makes an on-site visit, interviews senior management, and issues a separate report. Although this approach requires a significant amount of the internal audit function’s time to perform the self-assessment, it can save on the review costs and be more economical than a full external assessment while still meeting the requirements of the Standards.

It is important that the audit committee chair and chief audit executive agree on the scope of the review. In many cases, the external review also gathers feedback on whether the internal audit function is meeting key stakeholders’ expectations. It is highly probable that an internal audit unit may comply with the IIA Standards but still not provide the right services. In our experience, internal audit departments may gravitate toward auditing some areas, while not focusing on other important ones, perhaps because they have no demonstrated knowledge of them. Federal research compliance, for instance, is very important, but some auditors may have chosen not to audit that area. The external review provides an opportunity to calibrate audit coverage and risk.

Potential Benefits to the Audit Department

In an external review, the review team will evaluate how the auditors’ backgrounds and skills match up with the institutional risk areas. Well-developed internal audit functions embrace customer satisfaction and seek feedback from senior leaders and department managers. Developing a process to solicit and act upon such feedback is a hallmark of effective audit functions.

Internal audit departments also spend a lot of valuable time reporting their audit results. Many times they provide too much data or information that is not logically organized. They may also devote significant time to achieving agreement on audit recommendations. The best internal audit practices have streamlined reporting processes and tailor written reports to the audiences receiving them.

D. Richard Moyer, associate vice president for audit, compliance, and privacy at Stanford University, explained, “Whether I am conducting a QAR or being reviewed during a QAR, I always learn something from my peers that we can apply to improve the services provided to my organization.”

Some examples of the opportunities that may be suggested in the external review include:

  • An annual audit plan. The internal audit department should consider redefining the “audit universe.” Does it place appropriate emphasis on the most important parts of the university’s operations? At times, internal audit may exclude coverage of critical areas, perhaps because the unit does not have the expertise to audit these areas.
  • Information technology audit coverage. In our experience, a well-developed IT audit function is critical to providing valuable audit coverage. IT audit units may provide a wide range of services such as auditing underlying business systems, data center operations, and IT security, as well as participating in new systems development.
  • Organizational alignment. The Standards prevent internal auditors from designing or developing new systems or procedures. Many chief audit executives choose not to be involved in critically reviewing the internal controls being built into new systems; however, we consider this a vital competency in leading internal audit functions. Developing a process for ongoing interaction with senior leaders ensures the internal audit function is aware of the strategic objectives and challenges of the institution and can devise an audit plan that provides them with valuable feedback.
  • Staff development. Internal auditors must have the right skills, experience, certifications, and professional development to provide expert assessment of internal controls. Internal auditors are rarely as experienced as the managers of the areas they audit. Internal auditors have to gain the confidence of those managers by being able to demonstrate they understand the risks and internal controls in those operations.
  • Benchmarking. The ability to provide appropriate coverage is a function of the prioritization of auditable entities and available resources. There are no hard and fast determinants of the right size of an internal audit function, but careful comparison to peer universities may provide valuable insights. Peer chief audit executives who participate in the external review may be especially helpful in making the right determination.
  • Department operations. These fall into three key areas:
  1. Reporting results of testing: This important aspect may consume large portions of available auditor time, and the audit closing process may be delayed by protracted backand- forth discussions between the auditor and appropriate management to achieve agreement upon any recommended changes. Leading departments work with managers throughout the audit project to reach agreement on the current level of controls and the need for any enhancements.
  2. Establishing a quality assurance and improvement program that identifies specific actions employed on an ongoing basis will ensure that an internal audit function operates in a most effective manner.
  3. Establishing performance metrics will more accurately determine productivity and status reporting.

These are just a few of the leading practices that audit committee members, senior managers, and internal auditors need to consider. The external review provides an opportunity for improvement and the agreed-upon change to increase the value of the internal audit function.

Specific Benefits for Senior Administrators and the Board

Confusion on the role assignments noted above leads to uncertainty on the part of the internal audit stakeholders. We have seen the positive results when the audit committee, management, and internal audit have affirmatively agreed on the functions and responsibilities of internal audit. This establishes a roadmap for the chief audit executive to follow to be successful in achieving the desired results.

A wide variety of leading practices for internal audit have to be closely aligned with the expectations. For example, if senior management wants the internal audit to be a training ground for future business leaders, the audit department will need to invest heavily in skills development that is relevant to the broader institution. A clear point of demarcation relates to defining the internal audit role in change; some internal auditors may resist being involved in change for fear of losing independence and objectivity. By contrast, leading internal audit departments actively participate in change initiatives, providing valuable advice on the control structure without impairing independence to later audit the new system or process.

Reflecting on his experience of a recent quality assessment review at Boston University, Martin J. Howard, senior vice president, chief financial officer, and treasurer, explained: “The benchmarking information and the external QAR team insights on best practices were useful in our continuing refinement of the role and placement of the internal audit function at Boston University. It sharpened our focus on our existing enterprise risk management (ERM) process and improved coordination of compliance activities.”

Risk, Compliance, and the Internal Audit

Often, chief audit executives are not working with board leaders in structuring committee agendas, committee assignments, and board agendas. The work of the quality assessment review team raises the question of what role the chief audit executive should play in such important governance decisions.

We suggest that a broad-scope external review helps improve institutions. Not only will it enhance the internal audit function, it will have a beneficial impact on senior management and the audit committee of the board. If an institution’s internal audit function has not evolved from a primary focus on internal controls to include governance, enterprise risk management, and compliance, the college or university may not be getting the full value of the internal audit function.

The institutional roles for ERM, regulatory compliance, and internal audit are often not clearly defined, and may be attributed to internal audit by default. ERM is a critical issue, and senior leaders need to own the process to ensure administrators have identified and are managing or monitoring significant risks. Likewise, regulatory compliance is a specialized field; higher education institutions are exposed to regulation in many different areas, ranging from student aid to human resources to laboratory safety. An emerging leading practice is to closely align ERM and compliance, using similar tools and approaches.

ERM has been an evolving area in higher education over the last 10 years, and the role of internal audit has varied. After the Sarbanes-Oxley legislation became effective in the early 2000s, the best practice in public companies was to have the audit committee be responsible for the risk-management process. How the process was to be implemented was undefined. It opened up the question of the role of internal audit in the design, implementation, and execution of the ERM process.

Broad-scope quality assessment reviews ask the questions of how developed the risk-management process is in the institution and what role internal audit should play in its development or execution. There are many choices, and it is essential that the audit committee participates in the decisions.

With respect to institutional compliance programs, many models have evolved, but it is rare that the definition of the institutional program, documenting how it meets the Federal Sentencing Guidelines, exists. This should be of significant concern to the audit committee or board. Best practices in audit committee governance place responsibility for oversight of the design and operation of the compliance program with it. The QAR scope often looks at how the institution is designing and managing its compliance program.

This leads to discussion of the role of internal audit in meeting its compliance requirements. The external team raising questions about the program and role of internal audit often causes the institution to focus on an issue it had not previously addressed. Examples include issues such as determining the enterprise-wide impact of Title IX regulations, management of programs involving minors on the campus, ensuring that compliance units coordinate efforts when investigations cross operational areas, and assessing which compliance risks rise to a level requiring the most immediate attention.

Michael J. Mandl, executive vice president of business and administration at Emory University, supports a broad role for internal audit on his campus: “The audit function, when viewed as an institutional partner, can add value across virtually all institutional domains. It is no longer about risk mitigation and constraint—it is about judgment of the appropriate risk to take to best deliver the mission most effectively. This requires not only risk mitigation, but also explicit acceptance of risk. If functioning well, audit can help achieve the right balance of risk acceptance and mitigation while accomplishing the mission.”

A Catalyst for Change

An external quality assessment review is necessary to provide full objectivity. In addition to compliance with the Standards, such a review builds stakeholder confidence by documenting the internal audit commitment to quality and best practices, and the internal auditor mindset of professionalism. An external review also provides evidence to the board, administration, and staff that the internal audit activity is concerned about an institution-wide range of concerns in the internal control, governance, and riskmanagement processes.

Summarizing, Raina Rose Tagle, a partner and the National Higher Education Consulting Leader of the accounting and advisory firm Baker Tilly, notes: “A QAR could and should incorporate a strategic view of the position of internal audit and its activities within an institution. The most effective QAR will lead to a senior leadership and audit committee consensus about the internal audit role in the context of the institutional maturity of the compliance and risk-management programs.”

Reference Publications from the Institute of Internal Auditors

Quality Assessment Manual for the Internal Audit Activity, The IIA Research Foundation, 2013.

20 Questions Directors Should Ask About Internal Audit, 2nd Ed., The Canadian Institute of Chartered Accountants, 2008.

IPPF Practice Guide: Assisting Small Internal Audit Activities in Implementing the Standards, The IIA, 2011.

IPPF: Practice Guide: Measuring Internal Audit Effectiveness and Efficiency, The IIA, 2010.

Available at: http://www.theiaa.org/bookstore

Explore more on this topic:
The owner of this website has made a commitment to accessibility and inclusion, please report any problems that you encounter using the contact form on this website. This site uses the WP ADA Compliance Check plugin to enhance accessibility.