RBI wants to mandate AI kill switch for all banks, human oversight for all decisions

The Reserve Bank of India is planning to bring new rules for banks and financial institutions that use AI. As per the latest draft guidelines, the RBI wants to mandate a kill-switch that would override or stop any AI model being used within a bank's systems.

In Short

• RBI wants AI kill switch for all models
• RBI issues draft guidelines on how AI is used by banks
• All banks must have new risk assessment framework for AI models

At a time when AI tools are becoming more popular, there lies a fear about how these tools are used and the potential risks that may come with it. This is particularly true for AI models being used in banks and other financial institutions that manage sensitive financial data and information. Now, the Reserve Bank of India is planning to issue new guidelines that will make it necessary for all banks to have a kill switch that could turn off any AI model being used in their system.

On Wednesday, the RBI issued new draft guidelines that propose a wide-ranging framework for banks and other regulated entities using AI. If mandated, this would require them to have the ability to instantly override, suspend or deactivate any such system, including through a kill-switch arrangement. That is, a bank must be able to shut down any and all AI systems it uses, essentially with just a flip of a switch.

The RBI said banks must establish override, suspension and deactivation mechanisms to ensure that no AI model operates without the ability to be immediately shut down if it produces harmful or erroneous outputs.

This move comes at a time when Anthropic’s Claude Mythos AI model raised fears of cybersecurity, particularly for financial institutions.

Human oversight, AI risk tiering

But that is not all. The draft guidelines also state that all AI-driven decision-making must remain subject to human oversight. That is, even when AI is supposed to be doing tasks, a human must be overseeing any decisions being made.

The draft framework would apply to all models used by regulated entities, from simple spreadsheet-based calculators to complex frontier AI systems.

The bank will also be held fully responsible for the outcomes of any model it uses, regardless of whether the model was built in-house or acquired from a third-party. RBI has asked banks to carry out proper due diligence before using such models.

The Reserve Bank of India has proposed a risk-based tiering structure under which entities must classify models by risk level and apply proportionate oversight, validation and controls.

If assessed risk exceeds a bank's risk appetite, the guidelines say, the organisation must take swift action, including enhanced controls, restrictions on use, remediation or decommissioning of the model, and submit a report to the board's risk management committee.

The risk tier of a model must be reviewed at least once a year, and high-risk models must receive approval from the Risk Management Committee of the Board before deployment rather than being cleared only by the technology or risk team.

Model Risk Management Framework for all AI models

For the first time, the RBI has placed AI and model governance squarely at the board level. Every regulated entity must have a board-approved Model Risk Management Framework covering all models, whether developed in-house, obtained from vendors or built through a combination of both.

The RBI said, “Considering model usage has expanded significantly and regulated entities are increasingly using models, including those employing artificial intelligence / machine learning, across various business and decision-making processes; weaknesses in their governance, oversight, risk management and controls may expose the regulated entities to financial, operational, compliance, and reputational risks.”

It added, “If not managed effectively, such risks may lead to inaccurate outcomes, flawed decisions, financial losses, operational disruptions, compliance failures and other adverse consequences for entities, consumers and the financial system.”

The central bank also flagged supply chain risk arising from overdependence on a limited number of AI model providers and said banks must actively manage that risk. It further said banks should ensure that deploying AI models does not introduce vulnerabilities into the model.

On customer-facing systems, the RBI said banks must disclose when a customer is interacting with an AI system and must provide the option to switch to a human at any point.

The draft also warned against automation bias, or the risk of employees relying too heavily on AI outputs without applying their own judgement. For generative AI models that interface with customers or external users, the RBI said additional cybersecurity controls should be put in place.

The RBI has invited feedback on the draft guidelines by July 24.

- Ends