Cybersecurity Alert – Frontier Artificial Intelligence Models

Background

What are frontier AI models?

Frontier AI models are highly capable artificial intelligence systems that can perform complex reasoning, software engineering, and analytical tasks at a level that may exceed previous generations of AI technology. Such advanced AI systems can:

• Identify software vulnerabilities more quickly and at greater scale than traditional manual approaches;
• Analyze complex software environments and dependencies;
• Generate and test code to validate potential vulnerabilities;
• Assist security teams in prioritizing and remediating weaknesses; and
• Potentially enable threat actors to automate portions of the cyberattack lifecycle.

Recent developments

1. Anthropic’s Claude Mythos1

Anthropic recently announced Project Glasswing, a cybersecurity initiative built around its frontier AI model, Claude Mythos. According to Anthropic, the model demonstrated advanced capabilities in identifying previously unknown software vulnerabilities across operating systems, web browsers, and other software environments. Anthropic has reported that controlled testing showed the model could autonomously perform complex cybersecurity tasks, including vulnerability discovery and attack-path analysis, at a level that exceeded previous generations of AI systems.

Given the potential implications of these capabilities, access to Claude Mythos is currently restricted to a vetted group of technology companies, governments, critical infrastructure providers, and cybersecurity organizations participating in Project Glasswing. Participants include major technology and cloud service providers working collaboratively to identify and remediate software vulnerabilities before they can be exploited.

2. OpenAI’s Daybreak2

OpenAI recently launched Daybreak, a cybersecurity initiative that combines advanced AI models with security tooling and software engineering capabilities to assist organizations in identifying, validating, prioritizing, and remediating software vulnerabilities. The initiative is intended to help defenders automate portions of the vulnerability management lifecycle and improve the speed and effectiveness of security operations.

Daybreak includes participation from a range of cybersecurity, cloud computing, and technology organizations and reflects a broader industry trend toward the use of advanced AI systems to augment cybersecurity functions. Unlike highly restricted research programs, certain Daybreak services may be made available through controlled and verified access programs.

Why this matters for Dealer Members

As these capabilities become more widely available, firms should expect continued changes in both defensive and offensive cybersecurity practices. The implications for Dealer Members are significant:

1. Vulnerability exposure windows are shrinking.

The time between vulnerability discovery, disclosure, and exploitation may continue to decrease. Firms may have less time to assess, prioritize, and remediate vulnerabilities before they are targeted by attackers.

2. Vulnerability chaining increases risk.

Advanced AI systems may be capable of identifying complex attack paths by combining multiple lower-severity vulnerabilities that, individually, may not appear critical. As a result, vulnerability assessments relying solely on individual severity ratings may be insufficient.

3. There is increased pressure from zero-day vulnerabilities.

The speed and scale of AI-assisted vulnerability discovery may increase the frequency of newly identified vulnerabilities, including previously unknown weaknesses.

4. Third-party and supply chain risks are amplified.

Dealer Members increasingly rely on interconnected technology providers, cloud services, software vendors, and other third parties. AI-assisted vulnerability discovery may expose weaknesses across these interconnected ecosystems more quickly, increasing the potential for cascading operational impacts where vulnerabilities exist within critical suppliers or service providers.

5. Operational resilience becomes increasingly important.

As cyber threats evolve, operational resilience, including business continuity planning, cyber incident response, recovery testing, and crisis management capabilities becomes increasingly important. Existing operations and control processes may become strained or ineffective in a threat environment where attacks can occur at greater speed and scale.

6. AI-enabled social engineering and fraud is getting more sophisticated.

Frontier AI models are increasingly capable of generating highly convincing communications, synthetic identities, and impersonation attempts. These capabilities may increase risks relating to business email compromise, account takeovers, fraudulent payment instructions, help desk manipulation, and attacks targeting advisors, registered representatives, executives, and clients.

Risk management considerations

While the underlying principles of cybersecurity risk management remain unchanged, advances in frontier AI may increase the speed, scale, and sophistication of AI-enabled cyber threats and may require firms to assess whether existing governance, vulnerability management, third-party risk management, fraud prevention, detection, and incident response capabilities remain effective.

The examples below are intended to supplement current CIRO3 resources and highlight specific considerations arising from recent advances in frontier AI models.

1. Governance

Effective governance plays an important role in ensuring organizations understand and respond to emerging cyber risks.

Firms should consider:

• the potential impact of frontier AI developments materially affect the firm’s cyber risk profile and risk appetite.
• whether cybersecurity investments, staffing, insurance coverage, and technology strategies remain appropriate in light of evolving threats.
• whether the firm’s existing governance and escalation processes can support more rapid vulnerability remediation and incident response.

2. Vulnerability and exposure management

As AI-assisted vulnerability discovery accelerates, firms may have less time to identify and remediate weaknesses before they are targeted.

Firms should consider:

• Prioritizing remediation activities based on risk and exploitability rather than fixed patch cycles.
• Reviewing medium- and low-severity vulnerabilities that could be combined into more significant attack paths.
• Monitoring external threat intelligence and vendor advisories to identify emerging vulnerabilities affecting critical systems.
• Identifying and replacing unsupported or end-of-life technologies where feasible.
• Measuring and monitoring exposure windows, including the time required to identify, remediate, and contain vulnerabilities.

3. Third-party and supply chain risk management

The interconnected nature of technology ecosystems means vulnerabilities in one provider may affect multiple organizations.

Firms should consider:

• Engaging critical vendors regarding their vulnerability management, patching, and incident response capabilities.
• maintaining inventories of critical third-party services.
• Monitoring significant vulnerabilities affecting vendors, cloud providers, open-source software, and other technology dependencies.
• Reviewing outsourcing and third-party risk management practices to ensure they remain effective in a rapidly evolving threat environment.
• concentration risks arising from reliance on a small number of critical service providers.

4. Identity, access and protective controls

Strong preventive controls remain one of the most effective means of reducing cyber risk. Effective access management, network security, and data protection can help firms reduce the attack surface a Frontier AI model might access and limit the likelihood and impact of such attacks.

Firms should consider:

• phishing-resistant multi-factor authentication for privileged and high-risk accounts where feasible.
• Reviews of privileged access rights regularly and apply the principle of least privilege.
• Validating network segmentation and other controls designed to limit lateral movement within the environment.
• Secure cloud environments and verifying configurations against current security standards.
• Protection of sensitive data, credentials, encryption keys, and administrative accounts from unauthorized access.

5. AI-enabled fraud and social engineering controls

Frontier AI models may increase the sophistication and scale of impersonation, phishing, and social engineering attacks.

Firms should consider:

• controls for account takeovers, fraudulent instructions, and identity verification.
• Enhancing employee awareness and training programs to address AI-generated phishing and impersonation attempts.
• Enhanced verification procedures for high-risk or high impact transactions, payment requests, account changes, and privileged access requests.
• risks associated with synthetic identities, deepfakes, and impersonation of clients, advisors, executives, and service providers.

6. Incident response and recovery

Given the potential for frontier AI to increase the speed, scale, and sophistication of cyberattacks, Dealer Members should assess whether existing incident response and recovery plans can operate effectively in a compressed threat environment where attacks may occur more rapidly and affect multiple systems simultaneously.

Dealer Members should consider whether their incident response and recovery plans:

• Clearly define roles, responsibilities, escalation paths, and decision-making authority during a cyber incident;
• Include procedures for identifying, containing, eradicating, and recovering from cyber incidents affecting critical systems;
• Are tested regularly through tabletop exercises or simulations, including scenarios involving AI-enabled phishing, credential compromise, vulnerability exploitation, third-party compromise, and simultaneous attacks across multiple systems;
• Include communication protocols for internal stakeholders, clients, third-party service providers, regulators, law enforcement, and privacy commissioners, where applicable;
• Address recovery of critical business functions within defined recovery time objectives;
• Include procedures to preserve evidence, maintain logs, and support forensic investigation;
• Confirm that monitoring, logging, and alerting capabilities are sufficient to support timely detection, containment, and escalation; and
• Address applicable cyber incident reporting obligations, including reporting to CIRO where required.

Summary

The cyber threat landscape continues to evolve as advances in frontier AI accelerate the pace at which vulnerabilities can be discovered, exploited, and remediated. Dealer Members should proactively assess the impact of these developments on their cybersecurity and operational resilience programs and take appropriate steps to ensure that controls remain effective. Firms that are able to rapidly identify, prioritize, respond to, and recover from cyber threats will be better positioned to protect investors, safeguard market integrity, and maintain confidence in Canada’s capital markets.

Further information and resources on managing cybersecurity threats, including guides and webinars, are available on CIRO’s Cybersecurity & Technology site website.

• 1Anthropic, Project Glasswing: Securing Critical Software for the AI Era
• 2OpenAI, Daybreak
• 3Dealer Members should maintain cybersecurity and operational resilience programs that are commensurate with the nature, size, complexity, and risk profile of their business. CIRO’s Cyber Governance Guide, Cybersecurity Best Practices Guide for Dealer Members and Cyber Incident Management Planning Guide provide guidance on the governance, controls, processes, and practices that support an effective cybersecurity program.