Large language model based multi-agent system augmented complex event processing pipeline

Abstract

This thesis presents the development and evaluation of a Large Language Model (LLM) based multi-agent system framework for complex event processing (CEP) and video analysis. The primary goal is to create a proof-of-concept (POC) that integrates state-of-the-art LLM orchestration frameworks with publish/subscribe (pub/sub) tools to address the integration of LLMs with current CEP systems. Utilizing the Autogen framework in conjunction with Kafka message brokers, the system demonstrates an autonomous CEP pipeline capable of handling complex workflows. Extensive experiments evaluate the system's performance across varying configurations, complexities, and video resolutions, revealing the trade-offs between functionality and latency. The results show that while higher agent counts and video complexities increase latency, the system maintains high consistency in narrative coherence. This research builds upon and contributes to, existing novel approaches to distributed AI systems, offering detailed insights into integrating such systems into existing infrastructures. Future research directions include enhancing inter-agent communication, improving scalability, implementing adaptive resolution techniques, and exploring integration with emerging technologies like federated learning and edge computing. This thesis establishes a robust foundation for advancing real-time data processing and decision-making within distributed AI ecosystems.