Most types of attacks can be avoided or minimized by implementing good security practices. Nonetheless, some attacks targeting AI systems require specific measures. Recommended countermeasures are given below for the following types of attacks:
- Adversarial attacks
- Evasion attacks
- Transfer attacks
- Data poisoning attacks
- Model inversion attacks
- Membership inference attacks
- Distributed denial of service (DDoS) attacks
- Data manipulation attacks
- Misuse of AI assistants attacks
Adversarial attacks
Countermeasures for adversarial attacks, especially those targeting image recognition systems, include the following:
- Use trickier examples in the training phase. For instance, show the AI model lots of slightly altered images so that it learns not to be fooled by them.
- Add a little randomness (technically known as “noise”) to the images used in the training data set. That way, the model will learn to focus on the important parts of the image, not just on the small details that can be easily changed.
- Use stronger models: design the model so that it looks at the big picture (like a person’s overall shape) rather than just focusing on small details.
Evasion attacks
- Choose strong models that are less likely to be fooled by slightly altered inputs.
- Check or validate the input data system to ensure that it is clean and as expected. This can help to catch any abnormal or malicious inputs before they cause harm.
- Train the model using examples of these tricky inputs so that it learns to recognize and handle them properly.
- Regularly monitor how the model performs and update it to handle new types of attacks as they are discovered.
- If possible, and provided that the benefits outweigh the increased cost, use multiple models in combination so that if one model is fooled, the other models can still catch the problem.
Transfer attacks
Countermeasures for transfer attacks include the following:
- Train the model on a wide variety of data. This reduces the chances that an attack crafted on another model will work on the models that parliament is using.
- As with evasion attacks, use multiple models to make decisions – provided that the benefits outweigh the increased cost.
- During training, expose the model to adversarial examples (small, intentionally crafted changes in input designed to fool the model). This helps the model learn to recognize and defend against such attacks.
- If feasible, frequently update and retrain the model with new data. This can help to close any vulnerabilities that might be exploited in transfer attacks.
- Use techniques designed to make the model more resistant to attacks, such as smoothing or noise injection during training.
Data poisoning attacks
Countermeasures for data poisoning attacks include the following:
- Take care over who has access (physical or logical) to the training data set, enforcing robust user permissions.
- Carefully check the data before using it to train the model, including ensuring that the labels and data make sense. Proper sanitization is important for getting rid of data that may negatively impact the learning process.
- Evaluate the machine learning algorithms and check if they are designed to be less sensitive to corrupted data.
- Monitor the model’s performance after deployment in order to detect any unusual behaviour that might suggest it was trained on poisoned data.
Model inversion attacks
Countermeasures for model inversion attacks include the following:
- Differential privacy: This technique adds a small amount of random noise to the data or to the model’s outputs. The noise is carefully calibrated so that it does not significantly affect the model’s performance but makes it much harder for an attacker to extract precise information about individual data points.
- Data minimization: Only collect and use the data that is absolutely necessary for the model. The less sensitive data that is included in the training set, the less risk there is of exposing private information.
- Regularization: This technique can make the model less sensitive to specific data points, which reduces the risk of a successful inversion attack. It forces the model to generalize better, making it harder for an attacker to reconstruct specific inputs.
- Limitation of model access: Restrict who can query the model and how many queries they can make. If an attacker can only make a limited number of queries, it becomes more challenging for them to gather enough information to perform a model inversion attack. A robust account management system plays a key role in defending against this type of attack.
- Query auditing and anomaly detection: Monitor the queries made to the model and look for unusual patterns that might indicate an attack. If suspicious activity is detected, further queries from that source can be blocked.
- Adversarial training: Train the model with adversarial examples (inputs designed to trick the model) to make it more robust against various types of attacks, including model inversion attacks.
Membership inference attacks
Countermeasures for membership inference attacks include the following:
- Regularization: This approach makes the model less confident in its predictions, which makes it harder for an attacker to tell if a specific data point was included in the training data set.
- Differential privacy: This method involves adding noise to the data or to the model’s predictions to make it difficult for an attacker to distinguish between data that was included in the training set and data that was not.
- Model distillation: This method trains a simpler model to mimic the behaviour of a more complex model. The simpler model is less likely to give away specific information about the training data.
Distributed denial of service attacks
Countermeasures for DDoS attacks include the following:
- Use a content delivery network (CDN) to distribute the service across servers in different locations.
- Install a web application firewall to detect and block malicious traffic, including a DDoS attack, before it reaches the servers running the AI system.
- Increase server capacity. While this will not solve the problem itself, it will make it more difficult for the attacker to crash the entire system.
- Use externally sourced DDoS protection services to detect and mitigate DDoS attacks. These services can automatically identify and block malicious traffic, keeping a system running smoothly.
- Limit the number of requests a single user can make within a given time frame.
Data manipulation attacks
Countermeasures for data manipulation attacks include the following:
- Enforce the use of strong passwords and multi-factor authentication (MFA) to minimize the risk of unauthorized access.
- Regularly update all software (after proper testing in a controlled environment, as updates can sometimes introduce more problems into the system).
- Keep checking for the most recently discovered vulnerabilities and be ready to fight against them.
- Encrypt all data, and especially data that can lead to privacy issues. Thus, even if attackers are able to access the data, they will not be able to read it without the decryption key (or, at least, it will take a very long time before they can read it).
- Always back up all data properly to avoid data hijacking (which can make the service suddenly stop, negatively impacting parliament’s image among citizens and requiring the payment of a high ransom).
- Limit access to sensitive data, and monitor for unusual and suspicious activity such as changes to data or unauthorized logins.
Misuse of AI assistants attacks
- Countermeasures for misuse of AI assistants attacks include the following:
- Educate staff to exercise caution as to the information they provide to both personal and enterprise AI assistants.
- Where possible, avoid using AI assistants for AI project purposes.
- If this is not possible, check the privacy settings, as there may be an option to limit the data the AI assistant can access and store.
- Always review the AI assistant’s activity logs (if available) for any unusual behaviour.
The Guidelines for AI in parliaments are published by the IPU in collaboration with the Parliamentary Data Science Hub in the IPU’s Centre for Innovation in Parliament. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International licence. It may be freely shared and reused with acknowledgement of the IPU. For more information about the IPU’s work on artificial intelligence, please visit www.ipu.org/AI or contact [email protected].
