Can AI Outsmart Firewall Errors? A Study on LLMs for Anomaly Generation and Detection

Modern network security relies on firewalls to block advanced cyber threats. Managing large firewalls with thousands of rules is difficult, and rule anomalies can weaken protection. Traditional algorithms detect these anomalies with speed and accuracy, but Large Language Models (LLMs) offer reasoning abilities that may reveal hidden patterns beyond fixed rules. This paper evaluates state-of-the-art LLMs for two tasks: generating firewall datasets with anomalies and detecting those anomalies. Although these models are built for general-purpose reasoning, they struggle with the structured logic required in firewall management. Future work should focus on adapting their reasoning capabilities to fit real-world firewall tasks.