Ensuring continuous security in CI/CD pipelines through LLM-assisted reduction of false positives in static analysis

Moayad Terro, Victoriia Korzhuk

Abstract


Static application security testing tools embedded in CI/CD pipelines generate considerable volumes of false positive alerts, which degrade developer trust and impede automated workflows. This paper describes an architecture where a large language model functions as a secondary triage component positioned downstream of a static analyzer within a CI/CD pipeline. Flagged code fragments along with contextual metadata are passed to the model, which classifies whether each reported finding reflects an actual vulnerability. Experiments on open-source Java and Python repositories show that this configuration reduces the false positive rate from approximately 61% to 17%, raising precision from 0.389 to 0.833 while retaining 91.3% of the true positives reported by the analyzers. The findings suggest that LLM-assisted filtering can be practically integrated into existing CI/CD workflows without substantial latency overhead.

Full Text:

PDF

References


Aloraini B. et al. An empirical study of security warnings from static application security testing tools // Journal of Systems and Software. – 2019. – Vol. 158. – Art. 110427.

Sadowski C. et al. Lessons from building static analysis tools at Google // Communications of the ACM. – 2018. – Vol. 61, no. 4. – pp. 58–66.

Wadhams Z., Reinhold A. M., Izurieta C. Automating static code analysis through CI/CD pipeline integration // 2024 IEEE International Conference on Software Analysis, Evolution and Reengineering-Companion (SANER-C). – IEEE, 2024. – pp. 119–125.

Chomątek Ł. et al. Decoding CI/CD Practices in Open-Source Projects with LLM Insights // Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering. – ACM, 2025. – pp. 1638–1644.

Heckman S., Williams L. A systematic literature review of actionable alert identification techniques for automated static code analysis // Information and Software Technology. – 2011. – Vol. 53, no. 4. – pp. 363–387.

Muske T., Serebrenik A. Survey of approaches for handling static analysis alarms // 2016 IEEE 16th International Working Conference on Source Code Analysis and Manipulation (SCAM). – IEEE, 2016.

Johnson B. et al. Why don't software developers use static analysis tools to find bugs? // 2013 35th International Conference on Software Engineering (ICSE). – IEEE, 2013. – pp. 672–681.

Christakis M., Bird C. What developers want and need from program analysis: an empirical study // Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering (ASE). – ACM, 2016. – pp. 332–343.

Ullah S. et al. LLMs cannot reliably identify and reason about security vulnerabilities (yet?): A comprehensive evaluation, framework, and benchmarks // 2024 IEEE Symposium on Security and Privacy (SP). – IEEE, 2024. – pp. 862–880.

Salem D. O. et al. AI-Driven Continuous Integration: Automating Code Review and Deployment with LLMs // 2025 10th International Conference on Fog and Mobile Edge Computing (FMEC). – IEEE, 2025. – pp. 268–274.

Baqar M., Naqvi S., Khanda R. AI-Augmented CI/CD Pipelines: From Code Commit to Production with Autonomous Decisions // arXiv preprint arXiv:2508.11867. – 2025. – URL: https://arxiv.org/abs/2508.11867 (accessed 20.05.2026).

Li Z., Dutta S., Naik M. IRIS: LLM-assisted static analysis for detecting security vulnerabilities // arXiv preprint arXiv:2405.17238. – 2024. – URL: https://arxiv.org/abs/2405.17238 (accessed 20.05.2026).

Far S. M. T., Feyzi F., Esmi N. Enhancing Static Analysis through LLM-Based Semantic Validation // 2025 11th International Conference on Signal Processing and Intelligent Systems (ICSPIS). – IEEE, 2025. – pp. 119–123.

Agrawal V. et al. LLM-Driven SAST-Genius: A Hybrid Static Analysis Framework for Comprehensive and Actionable Security // arXiv preprint arXiv:2509.15433. – 2025. – URL: https://arxiv.org/abs/2509.15433 (accessed 20.05.2026).

Ameen M. R. et al. QASecClaw: A Multi-Agent LLM Approach for False Positive Reduction in Static Application Security Testing // arXiv preprint arXiv:2605.01885. – 2026. – URL: https://arxiv.org/abs/2605.01885 (accessed 20.05.2026).

Patil A. A., Patil L. A., Rambola R. Toward Hybrid LLM-Assisted Vulnerability Detection for Continuous Integration: A Bias-Free Framework with a Context-Rich Dataset // 2025 IEEE DELCON-International Conference on Recent Smart Technologies in Engineering for Sustainable Development. – IEEE, 2025. – pp. 1–4.

Semgrep OSS [Electronic resource]. – URL: https://semgrep.dev/ (accessed 20.05.2026).

SonarQube Community Build [Electronic resource]. – URL: https://www.sonarsource.com/products/sonarqube/ (accessed 20.05.2026).

OpenAI. Introducing GPT-4.1 [Electronic resource]. – 2025. – URL: https://openai.com/index/gpt-4-1/ (accessed 20.05.2026).

Xia Y. et al. Beyond Static Pattern Matching? Rethinking Automatic Cryptographic API Misuse Detection in the Era of LLMs // arXiv preprint arXiv:2407.16576. – 2024. – URL: https://arxiv.org/abs/2407.16576 (accessed 20.05.2026).

Yang X. et al. Learning to recognize actionable static code warnings // Empirical Software Engineering. – 2021. – Vol. 26. – Art. 56.

Mittal A., Venkatesan V. Leveraging Generative AI for Proactive Security and Automated Remediation in Cloud-Native CI/CD Pipelines // International Conference on Software Engineering and Data Engineering. – Springer, 2025. – pp. 18–39.

Mittal A., Venkatesan V. Practical Integration of Large Language Models into Enterprise CI/CD Pipelines for Security Policy Validation: An Industry-Focused Evaluation // 2025 IEEE International Conference on Service-Oriented System Engineering (SOSE). – IEEE, 2025. – pp. 197–203.

Kisielewicz M., Kotzbach P., Kędziora M. Enhancing DevSecOps Through Large Language Model Integration: A Pipeline-Centric Approach // International Conference on Computational Collective Intelligence. – Springer, 2025. – pp. 124–137.

Brown T. B. et al. Language models are few-shot learners // Advances in Neural Information Processing Systems. – 2020. – Vol. 33. – pp. 1877–1901.

Wei J. et al. Chain-of-thought prompting elicits reasoning in large language models // Advances in Neural Information Processing Systems. – 2022. – Vol. 35. – pp. 24824–24837.

Kiashemshaki K., Torkamani M. J., Mahmoudi N. Secure coding for web applications: Frameworks, challenges, and the role of LLMs // arXiv preprint arXiv:2507.22223. – 2025. – URL: https://arxiv.org/abs/2507.22223 (accessed 20.05.2026).

Bhardwaj A. K. Enhancing DevSecOps with Generative AI for Secure Code Review and Threat Modeling // 2026 6th International Conference on Image Processing and Capsule Networks (ICIPCN). – IEEE, 2026. – pp. 912–920.


Refbacks

  • There are currently no refbacks.


Abava  Кибербезопасность ИТ-КОНГРЕСС ВМК МГУ 2026 СНЭ

ISSN: 2307-8162