Generic risks and biases: Categories of risk

This sub-guideline is part of the guideline Generic risks and biases. Refer to the main guideline for context and an overview. For a discussion of risks that relate more specifically to the unique work of parliaments, refer to the guideline Risks and challenges for parliaments.

This sub-guideline explores new types of risk arising from the integration of AI that may not be familiar to parliaments and that, if not addressed effectively, can undermine democratic processes and public trust in parliamentary institutions.

AI literacy is an understanding of the basic principles, capabilities and limitations of AI – something that is crucial for informed decision-making about AI adoption and oversight in parliaments. It involves the ability to recognize AI applications, grasp fundamental concepts like machine learning and data analysis, and critically evaluate AI’s potential impacts. Without adequate AI literacy, users may misinterpret AI results, fail to recognize discriminatory patterns, become overly reliant on flawed AI systems, and overlook ethical and legal implications. This can lead to poor decision-making and potential harm.

AI systems used in parliamentary functions, such as for automated decision-making or policy analysis, can reflect and reinforce cognitive and other biases present in their training data. This can result in skewed policy recommendations and discriminatory legislative outcomes, adversely affecting minority groups, and undermining the principles of equality and fairness that underpin democratic institutions.

Parliamentary systems often handle sensitive personal and political data. Improper data-protection measures can lead to privacy infringements when using AI for data analysis and decision-making. Unauthorized access to, or misuse of, this data can compromise the privacy of citizens, MPs and other stakeholders, eroding trust in parliamentary processes.

AI systems, particularly those used in parliamentary settings, are potential targets for cyberattacks. These attacks can lead to the manipulation or theft of sensitive legislative data, but can also disrupt parliamentary operations or compromise the integrity of legislative processes. This poses significant risks to national security and public safety.

The opaque nature of AI decision-making – often termed the “black box” problem – presents challenges in parliamentary contexts where transparency and accountability are paramount. Decisions made or influenced by AI without clear explanations can lead to difficulties in holding the right entities to account for legislative outcomes, diminishing public trust in democratic institutions.

While AI can improve efficiency, the automation of administrative tasks within parliamentary functions can lead to job and task displacement, particularly for support and administrative staff. As AI becomes increasingly adept at handling routine tasks such as scheduling, document processing and data analysis, the need for human involvement in these roles may decrease. This reduction in demand can lead to workforce downsizing, resulting in unemployment and economic disruption for those affected.

Aside from the loss of jobs, the nature of remaining roles may change significantly. Tasks that were once performed by human workers may be automated, leading to a shift towards more complex, decision-oriented or creative responsibilities that require a higher level of expertise. This evolution in job tasks can be challenging for employees who may not have the skills or experience needed to adapt, creating further risks of job insecurity and potential displacement.

The shift towards AI-driven processes also has the potential to increase job polarization, where low-skill, routine jobs are automated, leaving a gap that may not easily be filled by existing employees. This could exacerbate social and economic inequalities, particularly if the affected workers are unable to transition into new roles that require different skills.

AI applications in parliamentary settings raise ethical questions, particularly regarding the delegation of decision-making authority. Relying on AI for policy recommendations, legislative drafting or constituent services can lead to ethical dilemmas, especially if AI decisions conflict with human values or lack the necessary contextual understanding. Different AI services may report varying values depending on the country in which the underlying model is defined and trained.

Shadow AI, which is related to the concept of shadow IT, can be defined as the unsupervised or unsanctioned use of generative AI tools within an organization or institution outside of its IT and cybersecurity framework. Shadow AI can expose organizations to the same risks as shadow IT: data breaches, data loss, non-compliance with privacy and data protection regulations, lack of oversight from IT governance, misallocation of resources, and even new risks stemming from a lack of understanding of the technology, such as the creation of AI models with biased data that can produce incorrect results.

Training and deploying AI systems demands massive computing and storage resources, often requiring the use of public cloud systems. In some cases, these cloud systems are located in a different country and are therefore subject to the laws and regulations of that country. Without appropriate risk-mitigation strategies, such as encryption or data minimization, it may be difficult for parliaments to maintain effective control over such AI systems.

The adoption of AI systems in parliamentary functions carries significant risks related to a lack of trust. One of the primary concerns is the complexity and opacity of these systems, which can lead to uncertainty about whether they are providing accurate and reliable information.

The absence of clear information on how these systems respect privacy or the nature of the data used for training further exacerbates distrust. Users may be concerned that their data could be misused or that the AI system’s decisions are biased or flawed owing to inadequate or biased training data. This lack of trust can hinder the effective integration of AI in parliamentary operations, as stakeholders may be reluctant to rely on systems they do not fully understand or trust.

The overall risk is that without trust, the benefits of AI may not be fully realized, as users may resist or underutilize these systems, potentially leading to inefficiencies and a failure to achieve the intended improvements in parliamentary processes.

The Guidelines for AI in parliaments are published by the IPU in collaboration with the Parliamentary Data Science Hub in the IPU’s Centre for Innovation in Parliament. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International licence. It may be freely shared and reused with acknowledgement of the IPU. For more information about the IPU’s work on artificial intelligence, please visit www.ipu.org/AI or contact [email protected].