Using artificial intelligence and machine learning to detect malicious quantum circuits

Quantum computing brings many new opportunities but also introduces new types of security risks. One of these risks is that attackers can insert small changes, or trojans, into quantum circuits. This thesis explores how machine learning can be used to detect such changes. Two models were tested: a classical Random Forest (RF) model and a Quantum Support Vector Machine (QSVM). A new dataset was created using six quantum algorithms, including Grover’s algorithm, Shor’s algorithm, and the Quantum Approximate Optimization Algorithm (QAOA), with both clean and tampered circuits. Each model was evaluated using the standard classification metrics such as accuracy, precision, recall, F1 score, and the Receiver Operating Characteristic Area Under the Curve (ROC AUC). The RF model achieved an overall accuracy of 96.3%, with a precision of 95.8%, showing strong performance especially in shallow-depth circuits. The QSVM attained an accuracy of 88.1%, with a precision of 86.9%, and performed particularly well on more structured and deeper circuits, despite the current limitations of NISQ hardware. This study shows that both classical and quantum models can be useful for detecting quantum security threats. It also highlights the importance of designing future tools that can adapt to different circuit structures and support the security of emerging quantum computing systems.