Comparison of commercial web application vulnerability scanners and open source scanners
Abstract
The paper compares eight vulnerability scanners based on two intentionally vulnerable applications. The comparison is performed using five criteria: accuracy, recall, Juden index calculation, web benchmark from WASSEC and OWASP. OWASP WebGoat and Damn Vulnerable Web Application (DVWA) are selected as the tested applications. Among the tested scanners there are three commercial scanners: Acunetix, HP WebInspect, AppScan, and five open source scanners such as: Arachni, IronWASP, Skipfish, OWASP ZAP, Vega. According to the results, it was concluded that commercial scanners are more effective in a number of criteria (including the list of threats). Some open source scanners (such as ZAP and Skipfish) can be characterized as originally targeted at certain types of threats. It is emphasized that there is no single security scanner that provides consistently high detection rates for all types of vulnerabilities. Based on the results of the review, it is claimed that the existing differences in the frequency of false-positive vulnerabilities (for both groups of scanners) are due to the fact that most commercial solutions have automated scanners, which are more effective than manual settings by the tester. It is obvious that the results of manual settings have a direct relationship with the actual level of the tester's competence, and largely determine the final results.
Downloads
References
Amankwah, R., Chen, J., Kudjo, P. K., & Towey, D. (2020). An empirical comparison of commercial and open‐source web vulnerability scanners. Software: Practice and Experience, 50(9), 1842–1857. https://doi.org/10.1002/spe.2870
El, M., McMahon, E., Samtani, S., Patton, M., & Chen, H. (2017). Benchmarking vulnerability scanners: An experiment on SCADA devices and scientific instruments. 2017 IEEE International Conference on Intelligence and Security Informatics (ISI). https://doi.org/10.1109/isi.2017.8004879
Alsaleh, M., Alomar, N., Alshreef, M., Alarifi, A., & Al-Salman, A. (2017). Performance-Based Comparative Assessment of Open Source Web Vulnerability Scanners. Security and Communication Networks, 2017, 1–14. https://doi.org/10.1155/2017/6158107
Chen, S. (2017, November 10). WAVSEP 2017/2018 - Evaluating DAST against PT/SDL Challenges. Security Tools Benchmarking. https://sectooladdict.blogspot.com/.
Acunetix | Web Application Security Scanner. (2022). Acunetix. https://www.acunetix.com/
The leader in Web application security assessment. (2011). Retrieved November 4, 2022, from http://www.hp.com/hpinfo/newsroom/press_kits/2011/risk2011/HP_WebInspect_data_sheet.pdf
HCL Software. (2021). Hcltechsw.com. https://www.hcltechsw.com/appscan
OWASP ZAP Tutorial: Comprehensive Review Of OWASP ZAP Tool. (2022). Www.softwaretestinghelp.com. https://www.softwaretestinghelp.com/owasp-zap-tutorial/
skipfish. (2017). Kali.tools. Retrieved November 4, 2022, from https://kali.tools/all/?tool=1256
Arachni - Web Application Security Scanner Framework. (n.d.). Arachni - Web Application Security Scanner Framework. https://www.arachni-scanner.com/
IronWASP - Инструменты Kali Linux. (n.d.). Retrieved November 4, 2022, from https://kali.tools/?p=1786
Vega Vulnerability Scanner. (n.d.). Subgraph.com. https://subgraph.com/vega/
OWASP. (n.d.). OWASP foundation, the open source foundation for application security. Owasp.org. https://owasp.org/
OWASP Benchmark. (n.d.). Owasp.org. https://owasp.org/www-project-benchmark/
Youden, W. J. (1950). Index for rating diagnostic tests. Cancer, 3(1), 32–35. https://doi.org/3.0.co;2-3">10.1002/1097-0142(1950)3:1<32::aid-cncr2820030106>3.0.co;2-3
The Web Application Security Consortium / Web Application Security Scanner Evaluation Criteria. (2009). Projects.webappsec.org. http://surl.li/dtuti