Model risk management meets 3 lines of defense

By Michael Guglielmo, managing director, Darling Consulting Group

Prompted by growing regulatory expectations for more substantial processes and controls surrounding decision support modeling, financial institutions of all sizes continue to expand their model risk management (MRM) practices.

Over the past few months, I’ve had an opportunity to interact with many examiners, financial risk modelers, auditors, chief risk officers, and model risk management executives and managers. One thing has grown clear:

Old school ways of managing model risk don’t cut it anymore.

Model risk becomes bigger challenge

Since the regulatory guidance of 2011, model risk management has evolved to encompass all models within an organization. The need to establish broader coverage prompted institutions of all sizes to develop ways to address model risk in a more holistic fashion.

Most have gone through the process of identifying, cataloging, and prioritizing the various models used throughout the organization based upon their purpose, use, and impact. We’ve seen various approaches, including model classifications, corporate governance policies for multiple classes of models and, in the case of large organizations that can have over a thousand models in inventory, the formation of sizable model risk management departments with teams of quantitative and model governance experts.

While we don’t expect this level of infrastructure to emerge at the community bank level, it is not uncommon for mid-tier banks to have one or more internal model risk management resources dedicated to this management and oversight process.

A three-tiered approach appears to be emerging from the process of addressing model risk. This three-tiered approach distributes roles and responsibilities between the model developer/owner/user: the model risk management group/specialist/organizer: and Audit.

This framework provides the structure and accountability that is needed to address the challenges associated with increased dependence on models to make informed decisions and meet heightened regulatory expectations.

Model manager: First line of defense

While those responsible for the ongoing management and update of a model have traditionally included some form of self-review, a more formalized approach is now expected.

Many institutions have developed or are in the process of developing corporate model risk management policies. They are instituting standards governing model management protocols, including data management, change control, assumption review and approval, and documentation. For numerous model managers, these new corporate standards are necessitating several changes to their normal model update and review processes, and many are finding it challenging to incorporate the needed changes in the midst of their daily routines.

Documentation of the modeling process is the most common challenge we see with existing models in relation to newly developed corporate model risk management policies. Having substantive procedural documentation and keeping it current, while considered important, has often taken a backseat to other priorities of model managers.

However, this is quickly being reprioritized as this documentation represents an important component relied on by model risk management experts and validators. This documentation not only signifies the care and effort taken to properly manage a model, but also provides the benchmark for comparison and validation of expected practices.

We have seen a direct correlation between model documentation quality and strength of the model risk management processes.

Model managers who have invested the time in documenting procedures for model updating, and for validation techniques employed to review data, assumptions, and results tend to have stronger, more reliable processes that are repeatable and instill confidence with strategic decision-makers.

Assumption management and “change control” is another area where model owners are often falling short relative to new expectations. Institutions are now expected to support their assumptions both quantitatively and qualitatively. Requests for our deposit and loan prepayment studies have never been greater as institutions recognize the shortcomings of their existing assumption development and support processes.

In addition, a hierarchical review and approval process related to notable changes made to a model (data, modeling methods, key assumptions, reporting) needs to be instituted. For many organizations, assumptions may be adjusted and other changes made, often without the review and approval of a supervisor or governing body (i.e. the Asset/Liability Committee or ALCO). Today this is no longer acceptable practice.

Moreover, assumption development should involve appropriate resources and internal experts, such as business line managers, and needs to be reviewed and approved by management and, in many cases, the board of directors.

For example, the key assumptions for an institution’s asset/liability management model should be reviewed and approved by ALCO regularly and conceptually approved by the board, at least annually.

Documentation of any notable changes to a model—in effect, a change log—is now expected. This log should include the nature of the change, when it occurred, the expected impact, who performed the change, and who reviewed and approved the change. In the case of key assumption changes, any quantitative and qualitative (expert judgment) support should also be included or referenced.

Model risk management: second line of defense

The model risk management function (the group, individual, or outside resource) represents the second line of defense for model risk management.

This component of MRM is where the “effective challenge” occurs. For larger organizations, centralized oversight of the model risk management process involves the following:

• Establishing corporate governance and control policies and standards.

• Maintaining an inventory of models within the organization.

• Applying a risk-based approach to the validation process that will inform the validation rigor, frequency of review, and the resources required (internal and external).

For smaller organizations, this process is often being managed by an individual, a small team of two or three, or is still considered part of Audit’s domain.

Regardless of bank size, the initial challenges for most organizations have been related to establishing the model inventory: identifying models, distinguishing models from “tools,” correctly setting validation priorities, and matching internal and external resources to the model complexities and validation needs.

In addition, the notion of a “challenger” (benchmark) model has gained prominence, particularly as credit stress-test models have recently emerged. Institutions are being called upon with greater frequency to provide independent control models for liquidity, interest rate risk, capital, and now credit. This additional element, while beneficial, is increasing risk management and compliance costs as experts and technology are required to develop models that either confirm or refute an existing model’s performance.

Audit: third line of defense

With the emergence of the model risk management function, Audit serves as the third line of defense.

Auditors are now tasked with reviewing model risk management practices to affirm that the framework sufficiently provides effective challenge, the internal and external resources used or hired are appropriately matched to the models being reviewed, and processes and procedures are sufficient and properly followed. In addition, Audit is responsible for data integrity (data from core systems).

Moving forward

Regardless of size, examiners are expecting institutions to adopt more substantive model risk management practices. This is a logical extension of regulatory focus on operational risk, greater transparency, and improved accountability.

Adopting a three-tiered approach will provide your organization with the means to effectively manage model risk, particularly as model complexity increases and the need for more sophisticated models grows. This approach will be a blend of art and science, and, in the long run, will result in more reliable models and more confident, proactive decision making.

About the author

With over 25 years in asset liability management, Michael Guglielmo, managing director, at Darling Consulting Group, provides both technical and strategic consulting to a diverse group of financial institutions in the U.S. and abroad. He is also a frequent author and top-rated speaker on a variety of balance sheet management topics. During his tenure at DCG, he has served in various capacities including Director of Financial Analytics. Prior to joining DCG, he managed the asset-liability management and strategic planning process for a regional bank in the northeast.