Next-generation intrusion detection systems with LLMs : real-time anomaly detection, explainable AI, and adaptive data generation

Abstract

As the cybersecurity landscape continues to evolve, the need for effective real-time network anomaly detection mechanisms grows ever more critical. This study advances the traditional machine learning approaches in Intrusion Detection Systems (IDS) by exploring the diverse applications of Large Language Models (LLMs). Building on the Cybertrust framework, which guides the development of systems integrating Explainable, Actionable, and Interpretable Artificial Intelligence, this thesis extends the framework by focusing on three key aspects.

First, the HuntGPT system integrates Explainable AI (XAI) techniques and LLMs to improve the confidence of cybersecurity operations teams in evaluating AI-generated alerts. HuntGPT is a specialized intrusion detection dashboard that combines a Random Forest classifier with XAI frameworks like SHAP and Lime. Paired with a GPT-3.5 Turbo conversational agent, HuntGPT presents threat insights in an easily understandable format, enhancing user comprehension and interaction. Second, we evaluate the functionality and effectiveness of fine-tuned LLMs for classifying network anomalies. This involves comparing their performance to traditional machine learning approaches in terms of cost, speed, and ease of deployment. This investigation aims to prove the effectiveness of minimally preprocessed LLMs as intuitive and efficient candidates for seamless integration into cybersecurity infrastructure. Finally, we introduce SecGPT, an interactive LLM agent designed to streamline the penetration testing process and generate new attack profiles in dynamic environments. Paired with a GAN architecture, SecGPT synthesizes comprehensive data to improve model retraining and enhance IDS detection capabilities.

The study concludes with a comprehensive discussion of the results, highlighting the potential of LLMs in continuous data generation and their pivotal role in enhancing the explainability of AI solutions within intrusion detection systems. The findings contribute to advancing the understanding of LLMs in cybersecurity, paving the way for future research and practical implementations in live network anomaly detection scenarios.