Design and Implementation of a Web Application Vulnerability Scanner
Design and Implementation of a Web Application Vulnerability Scanner
Abstract
Web applications remain primary targets for attacks such as Cross-Site Scripting (XSS) and SQL Injection (SQLi), yet professional-grade scanning tools remain inaccessible to students, novice developers, and small organizations due to cost and required expertise . This project presents the design and implementation of a web application vulnerability scanner built on the Python Flask framework, following the OWASP Web Security Testing Guide (WSTG) framework . The system employs a modular architecture with three detection layers: URL crawling and endpoint extraction, passive scanning (security header analysis, sensitive file detection), and active scanning (XSS, SQLi, LFI, command injection). The scanner engine supports asynchronous background execution via Redis task queuing, with results stored in structured JSON format and presented through a user-friendly HTML interface. Multi-user access control supports scan history and report retrieval. Experimental validation against OWASP Juice Shop and WebGoat benchmark applications demonstrates effective detection of target vulnerability classes while maintaining reasonable performance overhead.
Keywords: Web Vulnerability Scanner; OWASP Top 10; Flask; Security Testing; Automated Scanning
________________________________________
Chapter 1: Introduction
1.1 Research Background
The widespread deployment of web applications has made them primary targets for attackers. According to OWASP, injection attacks, broken access control, and security misconfiguration consistently rank among the most severe web security risks. These vulnerabilities can lead to data breaches, account hijacking, and system compromise.
Traditional manual security testing, while accurate, is time-consuming and requires specialized expertise. Automated vulnerability scanners identify potential vulnerabilities by simulating attack payloads and analyzing response characteristics, making them indispensable tools in security testing workflows. However, existing open-source scanners suffer from several limitations: many focus on single vulnerability types or provide only command-line interfaces; reports lack readability and developer-friendly remediation guidance; architectures are insufficiently modular, hindering extension with new detection rules.
A recent study introduced ZeroProbe, an intelligent web-based vulnerability scanner integrating automated black-box detection with AI-based security analysis. The system achieved approximately 88% true-positive rate for XSS and 90% for error-based SQLi, with a mean scan duration of 18.4 seconds and a System Usability Scale score of 78.5 . This demonstrates that accessible, accurate vulnerability scanners can be constructed for educational and entry-level use.
1.2 Problem Statement
Current web vulnerability scanning systems face several critical challenges:
Limited Coverage: Many scanners support only a few vulnerability types, failing to comprehensively address OWASP Top 10 risk categories.
False Positives and Explainability: Automated scanning often produces false positives with insufficient evidence presentation and remediation guidance, increasing investigation costs for security teams.
Architectural Extensibility: Monolithic scanning engines struggle to integrate new detection logic or adapt to different scanning strategies.
User Accessibility: Professional-grade scanners are unfriendly to non-security experts, lacking intuitive web interfaces and clear risk-graded presentations.
1.3 Research Objectives
This project aims to design and implement a web application vulnerability scanner with the following objectives:
1. Implement a modular scanning engine based on the OWASP WSTG framework, supporting both passive and active scanning detection.
2. Cover core OWASP Top 10 vulnerability categories including security misconfiguration, injection, XSS, and sensitive information disclosure.
3. Provide a web management interface supporting scan task submission, progress tracking, and result viewing.
4. Generate structured reports containing vulnerability descriptions, evidence, risk levels, and remediation recommendations.
5. Support asynchronous background scanning to avoid blocking the user interface.
1.4 Thesis Structure
This thesis comprises five chapters. Chapter 1 introduces research background and objectives; Chapter 2 reviews related technologies and existing tools; Chapter 3 elaborates system design; Chapter 4 describes implementation details; Chapter 5 concludes and discusses future work.
________________________________________
Chapter 2: Literature Review
2.1 Web Application Vulnerability Classification
The OWASP Top 10 serves as the authoritative reference for web application security risks, covering broken access control, cryptographic failures, injection, insecure design, security misconfiguration, vulnerable components, authentication failures, software and data integrity failures, logging failures, and server-side request forgery.
The OWASP Web Security Testing Guide (WSTG) provides the most comprehensive testing scenario framework, organizing tests into twelve domains including configuration management, identity management, authentication, authorization, session management, input validation, error handling, weak cryptography, business logic, client-side, and API . Each test scenario includes summary, test objectives, testing methods, remediation recommendations, and tool references. WSTG identifiers follow the format WSTG-
2.2 Vulnerability Scanning Techniques
Web vulnerability scanning typically proceeds through three phases:
Pre-scanning Phase: Identifying scanning targets, crawling site structure, and extracting endpoints, parameters, and input forms. Crawler depth control and URL template normalization are key to improving scanning efficiency .
Scanning Phase: Executing passive and active security tests. Passive scanning analyzes HTTP responses to detect missing security headers, sensitive file exposure, and information disclosure without modifying requests. Active scanning injects test payloads to detect XSS, SQLi, command injection, and other vulnerabilities by analyzing response characteristics (error messages, behavioral changes, time differences).
Post-scanning Phase: Result interpretation, remediation workflow integration, and fix verification.
2.3 Analysis of Existing Scanning Tools
OWASP ZAP is the most widely used open-source web security scanner, providing passive scanning, active scanning, crawling, fuzzing, API testing, and WebSocket testing. ZAP supports scan policy management with configurable test strength, thresholds, and enabled rules.
WebScore is a full-stack vulnerability scanner inspired by OWASP Top 10. Built with a scalable Turborepo monorepo architecture, it separates frontend, API, and scanning engine into independent services. The system uses BullMQ/Redis for asynchronous scan task queuing, supporting automatic crawling and dynamic security scoring . Its check registry includes site-wide checks for cookies, JWT, CSRF, SSRF, CORS, and forced browsing.
ZeroProbe integrates automated black-box detection with AI-based analysis using OpenAI GPT-4o-mini with a local fallback engine for structured, human-readable reports. Evaluation against DVWA and bWAPP demonstrated approximately 88% true-positive rate for XSS, 90% for error-based SQLi, and 80% for time-based detection, with an overall false-positive rate of approximately 12.3% .
WVScan follows a modular architecture emphasizing separation of concerns, extensibility, and explainable findings. The scanner pipeline includes independent components for crawling, technology detection, and vulnerability scanning with a registry-based scanner architecture. Each scanner returns standardized findings containing title, severity, affected locations, payloads, evidence, and recommendations .
2.4 Scanner Capability Boundaries
Modern scanners excel at detecting vulnerability classes producing observable signals: