Introduction

Artificial intelligence (AI) is the general term used to describe the process of programming computers and machines to think and operate like humans. Machine learning (ML) is a subset of AI that describes computers and programs that may be programmed to operate with minimal human intervention and can in some instances learn and/or update themselves. Various events over the past few years have helped raise the profile of AI/ML and its role in delivering financial services. OpenAI's introduction of the large language model (LLM) ChatGPT in 2022 was a rare moment when an AI/ML technology became directly accessible by the broad public. While momentum around AI/ML had been building over the past decade at least, the ubiquity of competing prominent technologies concentrated attention on all manner of applications, including those in the financial sector.

AI/ML is accelerating a long, ongoing shift in finance from face-to-face interactions conducted in a customer's community to online products and services fueled by advanced algorithms that require little or no human interaction and can occur anywhere. Such applications may have benefits for financial institutions and their clients. Relative to human operation (or models built using earlier technology), AI/ML provision of financial services may be faster and cheaper due to their speed and efficiency, and they may be able to expand service to more individuals due to their ability to analyze alternative data and find latent connections. The financial industry's use of technology is not new, and there are elements to its adoption of AI/ML that resemble past periods. While regulators are used to dealing with these cycles, there is a recognition that the growth of AI/ML is likely to be especially transformative.¹

Adoption of the technology also brings challenges and risks. While AI may eliminate some human bias in financial services, it may also introduce or exacerbate bias. Certain AI-market specific factors—such as using the same data to train models and concentration of technology and technological capacity—may encourage monocultures, or herd-like behavior, that introduce systemic risk. The ability of certain models to learn on their own, free from human oversight, may also allow them to engage in market manipulation, conflicts of interest, or other unlawful activity.

Financial industry use of the technology has attracted policymaker interest. For example, Executive Order 14110 on the "Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence" briefly discusses the Biden Administration's view of AI/ML in finance, among a host of other issues. In addition, the House Committee on Financial Services created a bipartisan AI Working Group to address these issues in January 2024.²

This report discusses the role of AI/ML in financial services broadly. First, it provides a brief overview of some technical terms. Next, it discusses the motivations for the financial industry's use of AI/ML. It then provides an overview of some financial applications of the technology. Finally, it addresses the regulatory framework and considers various policy issues, including any applicable regulations.

Technology Overview in Brief

What Are Artificial Intelligence and Machine Learning?³

AI is the broad label for technologies that give computer systems the ability to learn new concepts or tasks and to reason and draw useful conclusions about the world.⁴ Earlier versions of AI were largely dependent on their human developers that programmed machines with a series of deterministic rules, such as "if … then" statements, alongside troves of subject matter content.⁵ Such technology required specific syntax for questions or commands and produced responses on topics for which it had been programmed. When faced with a situation for which it had not been programmed, it would "freeze."⁶ That dependency would likely disqualify it from the AI in current taxonomy and is largely disappearing. According to some, most applications of AI now are ML, so the terms as used are synonymous.⁷

ML is a subset of AI focused on constructing computer systems that can automatically improve through experiencing and determining certain characteristics inherent in all learning systems, which can include computers, humans, and organizations.⁸ ML attempts to give computers the ability to learn and change without being reprogramed. It uses algorithms—sequenced actions designed to solve specific problems—that improve automatically with various degrees of—and often, little or no—human interaction.⁹ In other words, unlike earlier AI, it is meant to be adaptable. ML systems are intended to determine relationships among variables or recognize patterns in large data sets.¹⁰ Yet ML is not one thing, such as a discrete computer application, nor is it even one technology. Instead, it is a group of systems that is defined in part by the type of information used to train the model (labelled or unlabeled data, for example) and the amount of involvement provided by human trainers (supervised or unsupervised).

Recent Advances in Data and Technology

Technological developments in computing and telecommunications have been a boon to AI/ML, and the financial industry's use of it includes advanced performance computing, which includes hardware, cloud and edge computing, and communication technologies.¹¹ However, the financial industry's adoption of AI/ML depends upon the availability of data, including alternative data not traditionally used in financial decisionmaking, and the ability to process and analyze unstructured data. These technological advances move in tandem with the amassing of large quantities of data (including that which is coming from alternative sources) that require "scalable architecture for efficient storage, manipulation, and analysis."¹²

Recently, graphics processing units—hardware previously used in gaming (and subsequently crypto mining)—has fueled development of deep learning and other advanced forms of AI.¹³ Cloud computing has also increased the amount of data and programs that individuals or companies may store or access, while edge computing allows them to process data at its source in real time.

Some of today's technologies have their conceptual origins decades ago, and they are now being developed.¹⁴ Unsupervised learning's ability to process unlabeled data may identify relationships—for example, in market interactions or among loan applications—where its programmers had not thought to look. Similarly, the ability to process unstructured and alternative data that previously could not be analyzed—such as images and social media posts—has opened it to analysis.

Big Data

Big data refers both to a type of information and how it is used. It refers to large data sets that exhibit specific characteristics such as volume, velocity, variety, and/or variability and requires "scalable architecture" to store, manage, and analyze.¹⁵ The term also generally refers to the data use by managers, companies, and industries to facilitate business decisions.¹⁶

Some advances in ML have been directly attributed to the ability to process more data.¹⁷ The specific big data used in finance depends on the sector or use case. For example, some firms may use big data—including social media—when deciding whether to make loans.¹⁸ In a capital markets and trading environment, big data may include continuous data feeds from various stock exchanges, which can include stock prices and broader market moves and histories of transactions. Data in this environment may also include company-level data that may influence a stock performance, including financial statements and quarterly and annual regulatory filings.

Alternative data refers to information that has not traditionally been used to make financial decisions. In consumer credit underwriting, alterative data refers to non-credit data that has not traditionally been used in determining credit scores, such as rent and utility payments.¹⁹ Banks and credit card and payment companies may also use alternative and unstructured data—such as transaction information and location or GPS data—in their fraud detection and anti-money laundering activities. Asset managers and other capital markets participants may also use GPS and satellite imagery of retailer parking lots or social media mentions that provide insight into a company's performance.²⁰ The availability of different forms of data and the models they use have expanded the ways financial firms may use both.

Recent interest in AI has been marked by a period of increased investment in and development and adoption of AI/ML technologies by various sectors, including the financial industry. According to one bank's research, investment in AI is expected to climb to around $100 billion in the United States in 2025 and nearly $200 billion globally.²¹ Notably, there is likely broad investment from traditional financial institutions—one market research firm suggested retail banks will spend more than $4 billion on AI in 2024—with finance-focused startups attracting investment from venture capital firms.²²

Finance and Artificial Intelligence

According to the Organisation for Economic Co-operation and Development (OECD), AI is "powering digital transformation" that has been rapidly deployed in finance.²³ Adoption of this newer technology is not surprising as finance has often been linked with cutting-edge technology. However, the industry is also known for its use of often obsolete technology and associations with traditional practices that are at odds with new technologies. While in the past, personal banking was rooted in community relationships and conducted in person, now nearly all transactions can be performed online. In capital markets transactions, electronic trading has essentially ended the need for trading to be conducted person to person at the physical location of exchanges.²⁴ Meanwhile, the use of AI/ML (like the related adoption of a broad set of financial technology, or "fintech") over the past few decades also has implications for regulation, which prioritizes industry transparency, explainability, and fairness.²⁵ Finance's adoption of AI/ML thus takes place as technology and its capacity may be evolving more rapidly than societal and regulatory expectations.

Motivation behind the financial sector's adoption of AI is not new. Finance's adoption of AI has been compared to the 13^th-century development of net-present-value calculations, the more recent invention of the automated teller machine, use of expert systems for personal financial planning, and automated trading, all of which employed different technologies.²⁶

In addition to a general gravitation to technology, various characteristics of the financial industry have made it a ripe testing ground for AI/ML. As one commentator expressed: "information processing is the central function of financial markets."²⁷ For example, AI/ML generally requires large quantities of data on which to train its systems. Data is a quintessential feature across the various financial sectors. The industry amasses data from economic indicators, financial markets, individual consumers and businesses, and payments and transactions, among countless other sources. Moreover, while comprised of different subsectors—with diverse functions, services, and products—finance presents quantifiable problems, the solutions for which are often reduced (perhaps overly simplistically) to maximizing returns considering a given risk tolerance while considering some number of other quantifiable variables. As such, the industry is generating large amounts of data and providing the operating environment for AI/ML technologies.

AI/ML may also increase the speed with which financial transactions may take place. For example, high-frequency trading, which is also increasingly automated and programmatic, allows trades to occur in the smallest fractions of a second, speeds at which neither human analysis nor execution are possible. Also, if a financial institution uses AI/ML to determine whether it should make a loan, the speed may be evident in multiple ways. AI/ML-induced speed may allow financial institutions to update their lending models.²⁸ Lenders may also use AI/ML-based lending models, integrated with digital platforms and web-interfaces, to deliver decisions to customers more quickly.

AI/ML Applications in Finance

Financial industry and market participants use AI for a variety of functions. These functions have multiple purposes, including improving efficiency, assisting with decisionmaking, producing analysis and forecasting, increasing profitability, managing risk, and underwriting credit.²⁹ Moreover, they may be used in support functions or front office functions. Back office applications may include post-trade processing, trading profit and loss reconciliations, and data analytics.³⁰ Other applications may include regulatory compliance functions, such as know-your-customer checks and customer identification programs, anti-money laundering and countering the financing of terrorism programs, and anti-fraud. AI may also assist in asset allocation, robo-advising, and trade execution. Many of the actions taken by financial firms are multistep and closely related, and AI may be used for and benefit from different parts of a business or phases of a business process. Moreover, there may be overlap in use cases with different definitions. For example, while asset managers may use AI to assist with asset allocation, certain asset management firms may also employ algorithmic trading. This section provides a brief overview of various financial applications and services that may use AI but is not intended to be exhaustive.

Credit Underwriting

Credit underwriting—assessing the risk of prospective borrowers defaulting on loan repayment—is one of banks' and other financial institutions' primary businesses. Loan-making financial institutions have automated the methods for making loan decisions, including through the use of electronic data.³¹

For example, for consumer loans, such methods generally entail using a mathematical formula (called a scoring model) to determine a consumer's credit score and subsequently whether the firm should make the loan and at what interest rate.³² Various factors—including bill paying history, unpaid debt, outstanding loans, and accounts—go into determining a credit score.³³ Recently, however, some financial companies have considered using AI/ML to augment or replace traditional credit scoring.

Lenders have been using predictive models for decades.³⁴ Traditionally, statistical regression methods used data from the credit reporting bureaus, assigning weights to different variables to help forecast whether an applicant would default on a loan and to determine the likelihood of on-time loan repayment.³⁵ More recently, firms may use ML-based measures based on their ability to analyze large amounts and different types of data—including transaction data—and their ability to discern other important relationships not visible in traditional models.³⁶ According to one study, some ML underwriting models are "more adaptive" and have shown improvements in predictiveness and cost savings relative to traditional models.³⁷

Adoption of AI/ML for credit underwriting may differ among banks and nonbank fintech. Various regulatory requirements may make banks hesitant to use the technology for credit underwriting decisions (see "Policy Issues Regarding the Role of AI/ML in Finance"). Generally, because nonbanks may have different (and in certain ways more permissive) regulatory requirements and supervision than banks do, they may be more willing to use AI/ML technology. Some fintechs claim to have adopted the technology for underwriting, including reportedly for consumer loans, as well as for "buy now, pay later" credit products.³⁸ While banks may be more hesitant to use AI base models for credit underwriting decisions, some have reportedly not shunned the practice altogether. According to various reports, banks may use AI/ML to identify useful variables or relationships and to "explore the potential … to refine" their traditional statistical models used in the credit underwriting process.³⁹ Moreover, some banks—especially large ones and some solely in experimentation phases—reportedly use ML underwriting models for lending.⁴⁰

Chatbots

Chatbots are computer programs that interact (e.g., converse and provide answers) with people online by simulating human conversation through text and voice commands. Banking chatbots provide immediate assistance 24/7, reducing wait times, addressing customer inquiries, providing information on account balances and transaction history, and guiding users through various banking processes. Chatbots used in banking provide $8 billion in cost savings annually, according to one estimate.⁴¹

Regtech

Regtech (regulatory technology) refers to regulated financial institutions' use of technology to meet various regulatory, compliance, and data reporting functions.⁴² Regtech has existed for around a decade.⁴³ Regtech had focused on compliance with regulations related to onboarding and identifying new customers, but more recently it has been described as being instrumental in anti-money laundering, countering the financing of terrorism, fraud prevention, risk management, stress testing, and micro and macroprudential reporting.⁴⁴ For example, banks and other financial institutions use "robotics process automation" to help them comply with reporting requirements and repopulating required data on a regular basis.⁴⁵ In many cases, banks' and other financial institutions' adoption of regtech may involve partnering with third-party providers with expertise in the field.

Monitoring Fraud and Illicit Financial Activity

A key form of regtech aided by AI/ML is detecting, preventing, and reporting unauthorized and illicit financial activities for banks and other financial institutions. The shift to AI/ML solutions is due to its more adaptable approaches and its ability to leverage more types of data. Banks and other financial institutions must detect, prevent, and report on unauthorized and illicit financial activity, and there has been an increase in the number of financial institutions using AI/ML to address the issue.⁴⁶ Banks and other financial institutions can train models on huge volumes of consumer behavior data they generate, allowing the ML models to learn fraud patterns and to then detect fraudulent behavior in practice.⁴⁷ One payment processor, for example, has stated it uses time and location and GPS data to determine whether activity occurring in distant geographies may be fraudulent.⁴⁸ The same company also suggested that ML models can learn and subsequently evaluate certain behaviors, including swiping speed and gestures, when assessing the likelihood of fraud.⁴⁹ Similarly, there may be benefits to using the technology across the "anti-money laundering value chain," including at onboarding and client screening and with particular and immediate benefits coming from transaction monitoring.⁵⁰ In addition, researchers suggest that the technology is also useful in reducing false positives, freeing banks to dedicate resources to actual instances of fraud.⁵¹

Capital Markets Applications

Technology has always played a large role in the financial sector's capital markets activities. AI/ML may be seen as natural extensions of areas such as quantitative finance, which embraced advanced statistical analysis. As such, financial institutions, including the asset management industry and other investment companies, have adopted the technology to identify and exploit investment opportunities, allocate capital, execute trades, and reduce cost, with the latter ultimately allowing them to reach more customers.

Sentiment Analysis

AI/ML technologies help participants distinguish meaningful data points from noise in a practice referred to as signal processing.⁵² AI has become adept at identifying events from which a viable, profitable trading strategy can be generated but where the limited information would have made devising the strategy difficult without the benefit of AI.

Sentiment analysis is the analysis of financial sector news to forecast individual stock or market directions.⁵³ Advances in AI allow firms to analyze a wide variety of data, including unstructured data, to determine a company's popularity—for example, by surveilling social media posts and analyzing traffic through satellite imagery and GPS data. Sentiment analysis may be used across the spectrum of capital markets services where firms or individuals interpret data to allocate capital and determine strategy, including asset management and securities trading.

While sentiment analysis is not new, LLMs and computer capacity may outperform human methods and abilities to evaluate or forecast potential market moves.⁵⁶ The financial press and various researchers have suggested that LLMs may provide an edge in a number of contexts, from deciphering statements from the Federal Reserve to forecasting stock prices.⁵⁷ Some studies have shown that AI-enabled market participants (such as hedge funds) are quick to respond to new data, especially machine-readable disclosures, moving stock prices quickly.⁵⁸ Subsequently, companies have begun to adjust corporate disclosure practices to account for these market practices by learning "how to talk when a machine is listening," with research showing that public companies are adjusting how they speak and the language used in reporting to account for AI used by market participants.⁵⁹

Asset Management

Asset managers are companies that manage individuals' and businesses' capital for a fee, identifying suitable investments based on stated preferences.⁶⁶ The industry has been using AI for "a number of years," according to one report, to generate ideas and for portfolio allocation.⁶⁷ Recent applications employ deep learning models, including neural networks.⁶⁸ Specifically, AI is used to perform various types of analyses to suggest asset allocations (where and how much to invest) that optimize a portfolio (to maximize profit), with evidence suggesting that they may be better at meeting targets than "traditional methods" are.⁶⁹

Robo-advisors are a form of financial advisors under the broader asset management category that use AI to automate investment management. These digital investment advisors may employ various types of AI/ML (including NLP, LLMs, etc.) to develop a profile of an investor including budget, time horizon, and risk tolerance. They came to prominence more than a decade ago as a fintech application in part to introduce investment management to a subset of the population that may not have previously engaged with the service. Generally, these advisors may charge lower fees and require lower minimum balances than personal advisers do.⁷⁰ Additionally, robo-advisor algorithms may be programmed to rebalance a portfolio and perform "tax loss harvesting" and digital document delivery.⁷¹

Some asset managers have also created LLMs and generative AI models to assist with other services. Morgan Stanley, for example, announced in March 2023 that it had partnered with OpenAI (the company that created ChatGPT) to create a tool based on internal content to assist financial advisors in serving clients.⁷² The tool is said to function much like ChatGPT but uses Morgan Stanley's "own expansive range of intellectual capital" to provide answers in "an easily digestible format."⁷³

Trading Applications

Algorithmic Trading and Trade Execution

Algorithmic trading is a subset of quantitative finance that dates to the 1950s that uses "mathematical models, computers, and telecommunications networks to automate the buying and selling of financial securities."⁷⁴ In earlier iterations, algorithmic trading was based on—in technological terms—"deterministic 'rules based' systems," which impose certain constraints.⁷⁵ As discussed earlier (see "Technology Overview in Brief"), these models took their cues directly from humans and lacked the ability to learn. As in other financial industry subfields, technological advances have pushed firms to use newer methods, whose algorithm-enabling trading are considered more robust, adaptable to market conditions, and capable of operating at greater levels of autonomy.⁷⁶ Traders may combine supervised (i.e., learning with the assistance of humans) and unsupervised (i.e., learning without human supervision) learning models, employing a variety of techniques based on how a firm's adoption of AI/ML capacity evolved with technology.⁷⁷ For example, one investment firm may use an LLM such as ChatGPT to develop an investment thesis that is subsequently fed into more traditional "statistical AI" models to back test a hypothesis or determine whether the theory would have been profitable based on previous data.⁷⁸

Various market participants may also use ML for vital process-oriented tasks such as order placement and execution, especially when price impact occurs—the phenomenon in which buyers or sellers of large quantities of securities experience adverse price movements brought about by market reactions to their large buy and sell orders.⁸⁴ As such, traders may employ algorithms and, more recently, neural networks to execute orders in a dynamic way, taking into consideration market forces.⁸⁵

AI-as-a-Service

AI-as-a-service refers to third-party service providers offering AI models to financial firms lacking their own capacity to develop them internally. In the financial services industry, BlackRock's Aladdin resembles such a service. BlackRock describes Aladdin as "end-to-end portfolio management software" that offers trading, operations, and compliance functions on a single platform.⁸⁶ The platform is used by a large number of other financial firms that, together with BlackRock, manage more than $10 trillion in assets.⁸⁷ It uses AI to develop some of its insights, including pricing data or data cleansing, extending those services to its clients. The firm reportedly offers AI/ML-fueled sustainability evaluation and assessment tools and is integrating LLM capabilities to let its clients query Aladdin to help them find information more quickly.⁸⁸ Similarly, core services providers that serve small banks and credit unions, such as Jack Henry and Fiserv, also advertise AI/ML services, including fraud detection and chatbot services.⁸⁹

Policy Issues Regarding the Role of AI/ML in Finance

Not all policy issues associated with AI/ML in finance are new, but concerns have grown as its use becomes more prevalent and as the scope of technology appears capable of exacerbating potential risks. Some policy issues, such as whether AI may introduce or magnify bias, deal with broad issues such as fairness in the provision of financial services. Others, which hinge on the scope of the technology and its potential uniformity, include whether the technology creates or increases systemic risk.

This section provides a brief treatment of the legislative and regulatory framework and selected policy issues: (1) the potential for the technology to introduce or exacerbate bias in the provision of financial services; (2) the lack of "explainability" that stems from increasing model complexity, potentially introducing risk to the financial system; (3) the ability to encourage herd-like behavior, leading to financial stability concerns; (4) data security and privacy issues; (5) the potential to promote market manipulation; (6) the evolving role of big tech's position at the intersection of data, AI/ML, and financial services; and (7) whether, and the extent to which, AI may disrupt financial sector jobs.

Legislative and Regulatory Considerations

Broadly speaking, the legal and regulatory framework applicable to financial institutions and activities are "technology neutral," meaning they do not take into consideration the specific tools or methods used by institutions. For example, lending laws apply to lending whether the lender uses a pencil and paper or a cutting-edge AI-enabled model, and securities laws apply equally to traders on an exchange floor and ultra-fast high-frequency trading rigs. Given this, many policy debates in the area relate to questions about whether existing rules, when applied to new or innovative uses of AI/ML technologies, are adequate and effective, overly restrictive and stifling to beneficial advances, or overly permissive and inadequate protections against the risks presented by the technology.

Policymakers have increasingly focused their attention on AI/ML financial issues, although to date, few legislative or regulatory changes have been specifically directed at AI/ML use in finance. Meanwhile, regulators have communicated their positions and concerns about AI/ML in various areas. For example, in a March 2021 Request for Information (RFI), various federal banking regulators solicited "views on the use of AI in financial services to assist in determining whether any clarifications from the agencies would be helpful for financial institutions' use of AI." The RFI includes a list of existing laws, regulations, guidance, and other regulatory statements relevant for AI and notes that "[s]ome laws and regulations are applicable to any process or tool a financial institution employs, regardless of whether a financial institution utilizes AI or how."⁹⁰ The RFI essentially acknowledged the central issue of assessing the adequacy of existing regulation in the context of an evolving technology, acknowledging that new rules may not be immediately necessary considering that existing ones are still applicable. Yet the acknowledgement appears to be more of a reminder to financial firms of their existing obligations and not the agencies' final stance on policy. Some commentators suggest existing laws should already motivate corporations not to engage in bad behavior and reject additional attempts to regulate as "intrusions" that will "slow American innovation."⁹¹ Alternatively, many firms may be choosing not to use AI because of legal risks, and they may be more likely to employ the technology with clearer regulations.

AI/ML Model Bias

Bias in financial services is the overprovision, limitation, or existence of cost difference in the provision of financial services based on an individual's identity as part of a specific group. While conventional and newer AI/ML may remove one avenue for bias outcomes by eliminating the interpersonal interactions that may lead to discrimination, some believe the technology may introduce or exacerbate biases.⁹⁴ These biases generally fall into two main groups: bias introduced from data and bias introduced by models.⁹⁵ Data may introduce bias in a number of ways, including if the data on which a model is trained includes certain historical biases, which in a financial services context may include discrimination against protected classes.⁹⁶ Models trained on such data may provide credit decisions in ways that perpetuate biases and are illegal.

Model construction and training may also introduce bias. In training periods, for example, certain models may take a longer time to process observations for which there is less information and "learn" how to treat low information interactions. If institutions limit how long they train models, they may adversely affect how such models treat instances with low levels of information in practice.⁹⁷

Regulators have addressed bias in various venues. Federal Reserve Vice Chair for Supervision Michael S. Barr cited a case in which Meta was charged with violating a law that "prohibits discrimination in housing, including housing-related advertising" on the basis of protected classes.⁹⁸ On June 1, 2023, the federal banking agencies requested public comment on a proposed rule that would require companies that use automated valuation models "to adopt policies, practices, procedures, and control systems" that, among other things, ensure the models comply with nondiscrimination laws.⁹⁹ Also, in 2020, the Consumer Financial Protection Bureau issued an RFI on the Equal Credit Opportunity Act (Regulation B/ECOA)—which prohibits discrimination in credit applications because of race, color, religion, national origin, sex, marital status, or age, among others—posing the question of whether it should provide more "regulatory clarity" regarding Regulation B/ECOA to facilitate innovation that would expand credit access without unlawful discrimination.¹⁰⁰

Explainability

Explainability in AI/ML generally refers to the ability of a firm or model designer to understand a model's outputs or how it came to its conclusions. A lack of explainability has significant regulatory ramifications. Regulation B/ECOA requires that when taking adverse actions (such as refusing credit), banks and other financial institutions must provide notice to applicants explaining why they were rejected.¹⁰¹ Such notice should be "specific and indicate the principal reason(s) for the adverse action."¹⁰² Therefore, a firm's use of an overly complex model whose decision and rationale cannot be adequately and specifically explained may directly conflict with Regulation B/ECOA. As use of AI/ML for credit underwriting expands, and as regulatory guidance evolves, determining what is a satisfactory explanation—one that is not too general nor overly technical—may emerge as a key area of negotiation between industry and regulators.

Banks also have risk management requirements, which require they understand the theory and logic of models that manage their risk.¹⁰³ Assuming that AI/ML technologies would be considered models, for the sake of compliance, banks' inability to explain its results may be of supervisory concern.¹⁰⁴

For capital markets participants, such as traders, lack of explainability creates risks for firms insofar as traders may not be able to detect the rationale for a successful or losing strategy.¹⁰⁵ Lack of explainability may also cause compliance issues for market participants if a model engages in illegal practices, such as market manipulation.

There may be ways for developers to mitigate challenges from the lack of explainability and potential bias of certain AI/ML models. For example, researchers have shown that diagnostic tools and testing can address transparency-related challenges and that some such tools can identify what model features may be responsible for an adverse action.¹⁰⁶ In addition, policy and technology experts have recommended—including in response to a 2019 executive order—technical standards that may, among other things, eliminate the accidental or intentional promotion of bias.¹⁰⁷

Data-Related Policy Issues

Use of data by financial firms is subject to data protection and security requirements such as those established under the Gramm-Leach-Bliley Act (GLBA, P.L. 106-102).¹⁰⁸ GLBA requires that financial institutions ensure the privacy and confidentiality of customers and protect against threats and unauthorized access and use of data. GLBA also limits what financial institutions can do with information they have collected, generally requiring that they not disclose information to nonaffiliated third parties unless they notify customers and give them the opportunity to opt out.¹⁰⁹ To comply, banks and financial institutions subject to GLBA typically anonymize or "de-identify" their data before selling it.¹¹⁰

However, concerns about data privacy have increased as models improve and may be able to accurately identify owners of previously anonymized data.¹¹¹ In other words, anonymizing data may no longer be enough to comply with GLBA. Furthermore, because keeping data anonymous is becoming more difficult, even more information may qualify as sensitive.¹¹² As AI/ML models use increasingly more alternative data (see "Big Data"), more of a consumer's social behavior may become subject to commercial surveillance.¹¹³ In other words, as financial institutions seek more alternative types of information to grant financial decisions, individuals may begin to suffer from a greater lack of privacy. While consumers may opt out of information disclosure, this may limit what services they can receive, forcing them to choose between disclosure and collection or going without a service or product.¹¹⁴

Concentration Risk and Systemic Risk Concerns

The market for developing AI/ML may exhibit traditional economic forces seen in other technology sectors, such as high barriers to entry and economies of scale. The high cost to purchase equipment and hire staff may limit to large companies those that are able to develop technology and attract and retain talent.¹¹⁵ Subsequently, companies that have made the investment and attracted market share may be able to offer AI/ML services broadly, profitably, and more easily than new entrants, which would have to begin making the expensive investment and then wrest market share away from the incumbents. Therefore, developing robust models may be economically feasible only for very large companies, while smaller financial institutions may not be able to develop these technologies on their own and may rely on third-party technology providers for the technology.¹¹⁶

The concentration of human capacity and model development at a handful of firms means the number of firms using the same underlying model or data may create or exacerbate systemic risk—financial market risk that poses a threat to financial stability.¹¹⁷ Conditions in the market for AI/ML may create financial system risk in various ways, including through various dynamics that may create herd-like behavior.

In addition, there is a general assumption that the continued growth of data quantity and access contributes to improving AI/ML models. This may improve predictive ability and mitigate potentially harmful attributes such as myopia and human subjectivity that may precipitate asset market bubbles. An alternative theory suggests that current access to data may still not be enough to model all potential outcomes. For example, a model's predictive ability may be good for examples that are represented in the data on which it was trained, but it may be of minimal or no use for situations that do not appear in data (referred to as "overfit").¹¹⁸ "Idiosyncratic" or once-in-a-lifetime events—such as COVID-19, for example—may introduce scenarios that were not represented in training data and for which models cannot account.¹¹⁹ This could cause some AI/ML model performance to "change or deteriorate."¹²⁰

In addition, models created using the same data sets may reach the same or similar insights, informing the collective investment decisions and encouraging similar behavior in large portions of the investment community.¹²¹ Errors, risks, and unrepresentative conditions in the data would be propagated across the system, potentially fostering systemic risks.

Herding Behavior

Herding refers to an investment behavior wherein, for an array of reasons, investors may copy the behavior and strategy of other investors, moving markets as a result and creating risk. Literature suggests that herding has exacerbated previous financial crises and is not a new feature of finance created by AI/ML.¹²² However, commentators (including regulators) suggest that herding is a key risk inherent in use of AI/ML.¹²³ Herding manifests itself through AI if a number of market participants use the same or sufficiently similar algorithms, models, or data sets, creating homogeneity in the market or one-way markets.¹²⁴ This may occur when firms (knowingly or not) use the same third-party provider or an "off the shelf" model employed by others or if an AI/ML vendor's technology encourages the same activity among all clients.¹²⁵ Herding could create illiquidity or price spikes if there are not enough other market participants to take the opposite trade, which may be exacerbated in periods of stress and when financial conditions change.

A related challenge may be that individuals capable of creating or managing systems may be concentrated in a relatively few institutions, which may drive an over-reliance on third-party vendors, comparable to other services, such as cloud computing.¹²⁶ As in other areas with a limited number of providers, concentration of third-party service providers and models may pose risk in the event of technical or other failure of the intermediary.

There are some laws and regulations governing financial institutions' use of third-party service providers, although these are not necessarily aimed at curtailing the risks of herding. For example, depository banks that rely on third-party service providers—particularly those that provide financial AI/ML services—must ensure that these vendors satisfy safety and soundness regulatory requirements under the Bank Services Company Act (P.L. 87-856). According to bank regulators' guidance, a bank's "use of third parties does not diminish its responsibility to meet these requirements to the same extent as if its activities were performed by the banking organization in-house."¹²⁷ Similarly, LabCFTC, the technology-focused division of the Commodity and Futures Trading Commission (CFTC), stipulated in a primer document that CFTC-regulated firms remain responsible for any services performed by third-party service providers of AI.¹²⁸

Market Manipulation

Market manipulation occurs when an individual or firm tries to influence the supply of or demand for a security. It traditionally comes in a variety of forms, including "pump and dump" schemes in which someone engages in a series of securities transactions to make a security seem more actively traded than it is.¹³² ML techniques that aim to improve profit maximization may inadvertently lead to market manipulation. Academic studies indicate that autonomous AI agents may—in an effort to optimize profitability—learn ways to manipulate markets without involving input from a developer.¹³³ Some manipulative tactics that AI/ML may perpetuate include "spoofing" and "pinging," both of which are intended to learn certain information about a market and use it to participants' advantage.¹³⁴ While human traders generally would and should know that these manipulative actions are illegal, AI/ML models may not know, or they may otherwise ignore the prohibition. Additionally, LLMs (and generative AI) may be used to create images or text to fabricate news stories that could move markets in a way that could prove beneficial for certain transactions.¹³⁵ Securities, commodities, and derivatives markets are governed by laws that prohibit market manipulation.¹³⁶ Therefore, manipulation would be considered illegal, regardless of the technology used to perpetrate it.

Conflicts of Interest

AI/ML may create conflicts of interest between firms using the technology and their customers. As with market manipulation, AI agents that are programmed to learn on their own could conceivably maximize benefits to a firm by taking advantage of the firm's own clients. Combined with a lack of explainability and the complexity of large financial operations in general, this could create scenarios in which a conflict of interest is just another inexplicable variable in a large model.

Various securities laws and regulations are intended to protect customers from such conflicts of interest. Regulation Best Interest and the Investment Advisory fiduciary standards, established under the Investment Advisers Act of 1940, for example, oblige advisors to act in their clients' best interest and not put their own interests ahead of their clients' interests.¹³⁷ Additionally, the Securities and Exchange Commission (SEC) proposed a rule that would require firms to identify conflicts of interest that may arise from firms using "predictive data analytics" tools and related technologies and neutralize any such conflicts associated with predictive analytics technologies.¹³⁸ Some market participants have criticized the proposed rule, claiming the SEC's definition of the technology is too broad.¹³⁹

Supervisory Technology

Financial market regulators and supervisors may also use AI/ML-driven technology to oversee companies in their jurisdiction. According to the Financial Stability Board, various drivers are precipitating the development and use of supervisory technology (i.e., "suptech"), including enhanced efficiency (the ability to demand, store, and analyze large quantities of digital data); real-time data analysis; and "pro-active" and "forward-looking" supervision and surveillance.¹⁴⁰ Supervisors may use NLP to manage and synthesize unstructured data and for sentiment and network analysis, among other uses.¹⁴¹

In testimony before the House Committee on Financial Services Task Force on Artificial Intelligence, an official from the Office of the Comptroller of the Currency addressed the agency's initiative of upgrading "core supervision systems … evaluating and exploring use of … technologies, including AI," and hiring and retaining staff with expertise.¹⁴² The Federal Deposit Insurance Corporation also noted in its 2022-2026 strategic plan that it will expand its own use of suptech over the coming years, including ML.¹⁴³

Market regulators such as the CFTC and SEC have begun using deep learning tools to detect market manipulation and money laundering. In its primer on AI, the CFTC has suggested it could leverage AI to identify risk, perform market and risk surveillance, and identify market manipulation and abuse.¹⁴⁴ It has also suggested employing AI to evaluate the troves of data to which it has access—including from registrants, clearinghouses, and public data—and to perform systemic monitoring in ways that may forestall crises.

Similarly, the SEC purportedly first used NLP to determine whether it should have predicted the financial crisis and has reportedly incorporated AI into several of its risk assessment programs.¹⁴⁵ The SEC's use of NLP has evolved, and it has since begun using a form of unsupervised learning capable of reading documents, extracting insight, and identifying themes. Likewise, the agency uses these insights to train other models using supervised learning. The SEC claims that these methods combined may help it determine the likelihood of possible fraud when it reviews documents. According to the SEC and based on back-testing analysis, algorithms are five times better than random at identifying language that may indicate whether an investment advisor could merit enforcement referral.¹⁴⁶ After the flash crash in 2010, the SEC imposed requirements on national securities exchanges and the Financial Industry Regulatory Authority to "create, implement, and maintain" a consolidated audit trail (CAT) that could provide additional opportunities for AI-enabled suptech.¹⁴⁷ The CAT requires national securities exchanges, associations, and members to report various details of orders, providing the SEC with additional details and still more data, which it will use to improve market surveillance and likely use to develop, train, test, and then implement still newer models. Therefore, CAT was a product of a flash crash that was arguably the result of AI/ML-like technology and is expected to serve as a tool for the SEC to keep pace with various risks, including those from AI.¹⁴⁸ The extent to which market regulators are able to keep pace with the technologies used by regulated entities—and to develop their own techniques to manage the large quantities of data—may depend in part on how Congress evaluates their use and investments in such technologies.

Big Tech in Finance

Companies with large repositories of information may be poised to reap the benefits of that data and the models they may sustain. Big tech companies—technology-based businesses of considerable scale that have played a substantial role in transforming the internet economy, attracting billions of users—have large repositories of data and are developing their own AI models and products.¹⁴⁹ They have the financial capacity to make the necessary IT investments and may be best situated to capitalize on this investment by selling access to certain products or services.¹⁵⁰ This would represent a further consolidation of data and model creation and reinforce third-party dependency, potentially exacerbating concentration or systemic risk.

Another policy issue of the central role of data is that it may also create opportunities for big tech to increase its presence in financial services. Big tech companies currently have some presence in finance, mostly on the consumer products side. Their ability to accumulate and process data and AI/ML capabilities could make this transition easier, which would raise issues about regulatory arbitrage, as they are subject to uneven regulation for financial services. Separately, big-tech-related data privacy and a concentration of economic activity could be exacerbated if they successfully expand their presence in financial markets. For more information on this issue area, see CRS Report R47104, Big Tech in Financial Services, by Paul Tierno.

AI/ML and Financial Industry Employment

Since its origins, AI/ML has led to fears that it would disrupt the job market by replacing humans who were performing the jobs that would be automated. Some unsubstantiated recent projections suggest AI/ML may lead to a loss of 300 million jobs by 2035 across sectors.¹⁵¹ Research on the impact of past technological advances on jobs has been mixed. Broadly speaking, economists generally think that automated technologies replaced routine jobs but that those job losses can be offset by gains in other industries or by new jobs actually created by new technologies.¹⁵² The overall effect is often described as technologies having eliminated middle-income jobs and increased the portion of jobs that are either low or high paying.¹⁵³ Recent advances in AI/ML—including most recently of LLMs, such as ChatGPT—may increasingly challenge the conventional wisdom that white-collar jobs are safe from automation while only routine jobs and manual labor would be replaced. Instead, it is now widely believed that certain jobs requiring higher education may be at risk, including those in finance.¹⁵⁴

Use of AI/ML in finance seems poised to perpetuate the traditional dichotomy: that it will both boost and reduce demand for different types of workers. The AI/ML boom is expected to increase financial institution demand and competition for workers with AI/ML-specific skills.¹⁵⁵ One 2022 report suggested there may be as many as 100,000 AI/ML and related roles in banking and other financial institutions globally.¹⁵⁶ Whether or to what degree the technology will replace workers is up for debate, and it may be uneven across sectors. A 2023 survey of bank employees found that only 21% of those surveyed believed AI will replace many jobs in the banking industry, while 75% believe AI will change the nature of jobs but not replace human workers.¹⁵⁷ A slightly higher but similar number of global banking chief executives expect generative AI, in particular, to lead to a reduction of at least 5% in headcount in 2024.¹⁵⁸ Academic reports and empirical reviews also find that some financial services companies—such as asset management, in particular—have experienced job cuts and are likely to experience the largest number of job cuts in the near future.¹⁵⁹

Conclusion

As financial industry policymakers focus their attention on AI/ML in finance, they face competing pressures. Certain financial service providers and technology companies will tout the potential for AI/ML to lower costs, increase speed and accessibility, and improve accuracy and regulatory compliance. Consumer advocates and smaller firms may oppose adoption on the grounds that it introduces bias, risk, and potential for manipulation; eliminates jobs; and is unaffordable. Policymakers may decide to balance these competing priorities while assessing whether the existing regulatory structure is sufficient or whether one that is more closely tailored to the technological capacities of the evolving technology is necessary. Any legislative proposals considered by Congress will be evaluated by industry and consumers on whether it will lead to fair outcomes, not dampen the environment for innovation, and be based on quantifiable concerns but adaptable to future technology.