Who Assures AI? Understanding Specific Threats and Counter Measures to AI Systems

In an era propelled by technological innovation, Artificial Intelligence (AI) emerges as a cornerstone of progress. However, alongside its promises of efficiency and advancement, come inherent risks that demand attention. Katie Hawkins, Principal Consultant with Leonardo’s UK Cyber & Security business, explores the intricate domain of AI assurance, shedding light on the specific threats to AI systems and the proactive measures necessary to counter them.

With advancements in computational power and data analytics, AI is already exceeding theoretical boundaries to become an integral part of our daily lives. From optimising supply chains to enhancing healthcare diagnostics, the potential applications are vast and far-reaching. Technological innovators such as Leonardo are at the forefront of harnessing AI’s potential to drive progress. Through initiatives focused on system autonomy, cybersecurity and industrial optimisation, we showcase the practical implementation of AI solutions.

Unveiling Threats to AI Systems

However, to fully realise the benefits of AI, it’s imperative to continuously identify, assess and mitigate potential threats. AI systems face an array of threats, each posing unique challenges to their integrity, security and ethical implications. At a high-level, threats include:

1. Cybersecurity Vulnerabilities: AI systems are susceptible to cyberattacks such as prompt injection and data poisoning attacks, jeopardising data integrity and system operations. Cybersecurity breaches can lead to unauthorised access, data breaches and disruption of AI functionalities.
2. Algorithmic Biases: Biases embedded within AI algorithms can result in discriminatory outcomes, perpetuating social inequalities and eroding trust in AI systems. For instance, in the criminal justice sector, an algorithm used in the US has been found to exhibit racial bias when predicting the risk of reoffending.
3. Adversarial Attacks: These involve manipulating input data to deceive AI systems, causing them to make incorrect decisions. Such attacks can bypass security measures, fool detection systems or manipulate AI-driven processes, posing significant risks to system integrity and security.
4. Model Vulnerabilities: Flaws or vulnerabilities in AI models can be exploited to manipulate system behaviour or extract sensitive information, posing security risks.
5. Misinformation and Manipulation: AI tools can magnify disinformation and misinformation, influencing political opinions and undermining social cohesion. Through techniques like deepfake videos, AI can deceive and manipulate at unprecedented scale.

Counter Measures – The Role of Assurance

In line with the UK government’s AI governance framework, the Department of Science, Innovation, and Technology (DSIT) has recently released an Introduction to AI Assurance. This guide serves as a comprehensive resource, providing accessible insights into assurance mechanisms and global technical standards. Its aim is to empower industry practitioners and regulators alike with the knowledge needed to effectively develop and deploy responsible AI systems.

Complementing this framework are organisations such as the Responsible Technology Adoption Unit and the Information Commissioner’s Office, which play pivotal roles in offering additional guidance. They focus on addressing algorithmic biases, promoting fairness, and ensuring data protection and privacy considerations in AI systems.

Moreover, the UK’s National Cyber Security Centre (NCSC) provides guidelines for secure AI system development, offering valuable insights into mitigating cybersecurity risks associated with AI technologies. These guidelines are categorised into four key areas within the AI system development life cycle:

• Secure Design – Outline the importance of understanding risks, threat modelling and incorporating security considerations throughout the system design process.
• Secure Development – Emphasise securing the AI supply chain, protecting assets, documenting data and models, and managing technical debt throughout the AI system’s lifecycle.
• Secure Deployment – Prioritise the implementation of robust infrastructure security principles, continuous protection of models and data, development of incident management procedures and ensuring responsible release practices.
• Secure Operation and Maintenance – Focus on monitoring system behaviour and inputs, following a secure-by-design approach to updates, and facilitating information sharing and lessons learned.

For each area, considerations and mitigations are suggested to help reduce overall risk to the organisational AI system development process.

Collaboration as the Key to Success

As highlighted by the DSIT, there will never be a silver bullet for AI assurance. Instead, a combination of assurance techniques must be applied across the lifecycle. Tackling the challenges of AI assurance necessitates collaborative efforts across diverse sectors. Governments, industry leaders and civil society must join forces to establish robust frameworks and regulatory standards for AI development. Through investment in innovative technologies and fostering a culture of collaboration, stakeholders can cultivate trust and accountability in AI-driven solutions, paving the way for responsible and impactful AI deployment.

When supporting our customers and partners, Leonardo has found that a sensible approach is to prioritise proactive measures such as secure by design, regularly updating systems to address vulnerabilities and conducting thorough risk assessments. Additionally, fostering collaboration among stakeholders to share best practices and insights can enhance collective defence against potential threats to AI systems.

Balancing opportunity and risk

Amidst rapid technological progress, AI presents transformative opportunities alongside significant risks, as demonstrated in this article which delves into the multifaceted realm of AI assurance, outlining specific threats like cybersecurity vulnerabilities, algorithmic biases and the spread of misinformation. Through fostering collaboration, investing in innovative technologies and establishing robust frameworks for AI development, stakeholders can effectively address risks and ensure the secure deployment of AI systems.