Board Report: 2015-SR-B-018 October 29, 2015
The Office of Inspector General conducted this evaluation to assess the extent to which the Board of Governors of the Federal Reserve System's (Board) model risk management practices in support of its supervisory stress testing efforts are consistent with supervisory guidance on model risk management previously issued by the Board. Specifically, we focused primarily on model validation activities, but we also evaluated broader governance, policies, and controls as warranted.
In 2010, the Dodd-Frank Wall Street Reform and Consumer Protection Act mandated that the Federal Reserve conduct annual stress tests of all bank holding companies (BHCs) with $50 billion or more in total consolidated assets. In late 2010, the Federal Reserve initiated the annual Comprehensive Capital Analysis and Review (CCAR) exercise, which includes quantitative stress tests and a qualitative assessment of the largest BHCs' capital planning practices. CCAR has developed into the cornerstone of the Federal Reserve System's supervisory program for the largest BHCs.
Although the Board oversees supervisory stress testing, it relies on a broad range of Federal Reserve System staff to execute the program.
The use of models in any environment invariably presents model risk—the risk that decisionmaking may be influenced by inaccurate or unreliable models. The Board expects that its supervisory stress testing program will mitigate model risk in a manner consistent with the standards that the Board has outlined in relevant supervisory guidance. A guiding principle for managing model risk is model validation, which refers to the effective and independent challenge of each model's conceptual soundness and control environment.
The Board's model validation function has assessed its validation activities and proactively identified opportunities to improve model validation. In 2014, the model validation function conducted three reviews assessing its performance and that of the broader supervisory stress testing program. As a result of these reviews, the model validation function identified several areas for improvement. Notably, the model validation function found that its staffing approach was not consistent with industry practice, and as a result, it plans to transition to a new staffing approach. While the internal reviews demonstrate a focus on continuous improvement of supervisory stress testing model validation and governance, we believe that the Board can take additional steps to further improve its model risk management practices in support of supervisory stress testing. Specifically, our report outlines findings related to model validation and broader governance practices.
First, we identified certain risks associated with validation staffing and performance management that may not be mitigated by the implementation of a new staffing approach. These risks include insufficient performance feedback to supplemental reviewers, key-personnel dependencies, and inadequate scrutiny of models. Second, we found that although the Board has taken steps to address the risks associated with changes to models that occur late in the supervisory stress testing cycle, some risks remain. Third, we found that the model inventory lacks several components either required or deemed useful by supervisory guidelines. Finally, based on our review of a sample of validation reports, we found that limitations encountered by reviewers during model validation should be made clearly identifiable for management in the validation reports submitted to management.
Our report contains recommendations designed to strengthen supervisory stress testing model validation practices. Management generally agreed with our recommendations and noted that a number of the recommended actions have already been completed or are in the process of being implemented. We intend to conduct future follow-up activities to determine whether the Board's actions are responsive to our recommendations.
