Your AMS plays a crucial role in helping financial institutions detect and prevent illicit activities. However, to ensure the effectiveness and compliance of these solutions, regular model validations are necessary and prescribed by Guidance on Model Risk Management from the prudential regulators.
During validations, common findings often emerge and highlight opportunities for improvement.
Here, TCA explores some of these findings and outlines potential solutions.
Insufficient Model Governance
Insufficient or informal model governance procedures is one of the most prevalent findings in AML software validations.
Establishing a clear model governance framework is critical to effective model oversight. This framework should outline roles, responsibilities, and accountability measures for all stakeholders involved in developing, implementing, using, and monitoring AMS models.
Solutions
- Develop written policies and procedures on the model’s usage and assign responsibility for model oversight and management.
- Establish formal change control procedures for modifying system settings, review system settings and rules at least annually, and formally document system changes. Also, ensure only appropriate personnel can access the system, and perform user access and role reviews at least annually.
- Perform a model gap analysis to determine what the model can do and what must be supplemented.
- Perform a model risk assessment to identify the risks of relying on the AMS model and track audit findings to be sure they’re properly addressed. Also, establish written procedures for ongoing monitoring of the model.
- Ensure the software provider is subject to annual vendor due diligence reviews.
- Perform an independent system validation periodically and define the frequency of model validations in your policy.
Data Integrity Issues
Another common finding in AMS model validations pertains to data integrity. Poor data quality can lead to inaccurate results and missed suspicious activities. Missing data or inaccurate data flowing from the core system to the software are common causes.
Solutions
Address data quality issues by implementing robust data governance practices.
This includes establishing data quality standards and independently testing data imports. Periodically testing data quality identifies potential system gaps that you can address by correcting the data fed to the software or by manual monitoring if the gap stems from a system limitation.
Also, be sure that users know when new products or services are being implemented and their effect on the software. For example, new transaction codes may require revisions to mapping to ensure that the software captures the information.
Model Performance Degradation
Over time, the performance of AML software models can degrade due to transaction pattern changes, new high risk customer types, new fraud and money laundering techniques (for example crypto currency), regulatory requirements, or system updates.
Because of model drift, you may see decreased effectiveness and more false positives, which undermines the AML Program’s overall efficacy.
Solutions
Compliance officers should identify the above changes during regular updates to the AML-CFT Risk Assessment. Changes to your risk profile may indicate a need for review and change of model thresholds and require the activation of new parameters to ensure the risk changes are addressed by the model. To mitigate performance degradation, it’s essential to conduct regular model monitoring and recalibration.
In addition, establish processes for ongoing monitoring, including tracking key performance indicators (KPIs) and conducting periodic model reviews. By tracking the output of alerts, you can identify the rules producing few or no SARs or those generating very few or no alerts, all of which can indicate that the thresholds need adjusting.
Also consider tracking alerts to cases and alerts to suspicious activity report percentages, since this can help you determine the effectiveness of each rule or scenario. In addition, annual reviews can help you determine whether the model’s rules are working as intended.
It’s also important to stay abreast of persistent and emerging financial crime risks. To address updated regulations or emerging AML threats, adapt alert rules or scenarios or implement new ones.
False Positives and False Negatives
False positives (resulting in a large number of meaningless alerts) and too few alerts (often the result of thresholds being set too high) are common challenges of an AMS.
False positives occur when legitimate transactions are incorrectly flagged as suspicious, leading to unnecessary investigations and compliance costs. This can stem from thresholds being set too low.
Conversely, setting thresholds too high can cause suspicious activities to go undetected, potentially exposing you to regulatory risks and financial losses.
Meaningless alerts can occur when the system has inadequate or outdated rules. Another reason for meaningless alerts is AML software with data quality issues. Inaccurate data can lead to inaccurate alerts or alerts that do not generate at all.
Solutions
To reduce false positives and improve the accuracy of alerts, fine-tune AMS detection rules and thresholds and rely on historical data to identify patterns and adjust system’s rules accordingly.
Also important is customizing parameters and establishing reasonable thresholds relative to your size, complexity, and risk profile.
Essential Protection
AMS model validations are essential for ensuring the effectiveness and compliance of AML/CFT programs. By addressing common findings such as insufficient model governance, data quality issues, model performance, false positives and false negatives, all financial institutions can strengthen their AML capabilities, mitigate money laundering risks, and protect themselves from costly lookbacks, civil money penalties, memoranda of understanding, matters requiring attention and cease-and-desist orders.
Quick Tips
Here are three things to keep in mind as you enhance your AML Software Model Validations.
- The 2011 Supervisory Guidance on Model Risk Management states that “Banks should conduct a periodic review – at least annually but more frequently if warranted.” It goes on to say that “It is generally good practice for banks to ensure that all models undergo the full validation process, at some fixed interval, including updated documentation of all activities.”
- The frequency at which model validations need to be completed is based on the complexity of the institution and if there have been critical changes that affect the system.
- Most examiners recommend a full, independent model validation every 12 to 18 months. Some institutions may wait 24 to 36 months if there haven’t been significant changes to their risk profile or model.
Conclusion
There is no “one size fits all” answer to model validation as each institution’s risk profile is unique. A well-designed AML Program that includes a system to assist with suspicious activity monitoring, risk rating, reporting and recordkeeping, OFAC and 314(a) compliance requires adequate oversight from the Board and Management, well developed policies and procedures, and documentation of risks and system gaps. User access should be compartmentalized based on job function and monitored to ensure that all users are active employees. Strong change management processes should ensure that no one employee can make unauthorized changes to the model and that system parameters and thresholds are supported and documented by thorough analysis.
TCA can help your institution manage its model risk through calibration consulting or independent validation. Call us at 800-934-7347 or send an email to [email protected] to set up an initial consultation to discuss how we can help.
TCA – A Better Way!
