Comparison of commercial web application vulnerability scanners and open source scanners

Abstract

The paper compares eight vulnerability scanners based on two intentionally vulnerable applications. The comparison is performed using five criteria: accuracy, recall, Juden index calculation, web benchmark from WASSEC and OWASP. OWASP WebGoat and Damn Vulnerable Web Application (DVWA) are selected as the tested applications. Among the tested scanners there are three commercial scanners: Acunetix, HP WebInspect, AppScan, and five open source scanners such as: Arachni, IronWASP, Skipfish, OWASP ZAP, Vega. According to the results, it was concluded that commercial scanners are more effective in a number of criteria (including the list of threats). Some open source scanners (such as ZAP and Skipfish) can be characterized as originally targeted at certain types of threats. It is emphasized that there is no single security scanner that provides consistently high detection rates for all types of vulnerabilities. Based on the results of the review, it is claimed that the existing differences in the frequency of false-positive vulnerabilities (for both groups of scanners) are due to the fact that most commercial solutions have automated scanners, which are more effective than manual settings by the tester. It is obvious that the results of manual settings have a direct relationship with the actual level of the tester's competence, and largely determine the final results.