Comparison of commercial web application vulnerability scanners and open source scanners

  • Ivan Lakhtin V. N. Karazin Kharkiv National University
  • Dmytro Mykhailenko V. N. Karazin Kharkiv National University
  • Oleksii Nariezhnii V. N. Karazin Kharkiv National University https://orcid.org/0000-0003-4321-0510
Keywords: web application, vulnerability, testing

Abstract

The paper compares eight vulnerability scanners based on two intentionally vulnerable applications. The comparison is performed using five criteria: accuracy, recall, Juden index calculation, web benchmark from WASSEC and OWASP. OWASP WebGoat and Damn Vulnerable Web Application (DVWA) are selected as the tested applications. Among the tested scanners there are three commercial scanners: Acunetix, HP WebInspect, AppScan, and five open source scanners such as: Arachni, IronWASP, Skipfish, OWASP ZAP, Vega. According to the results, it was concluded that commercial scanners are more effective in a number of criteria (including the list of threats). Some open source scanners (such as ZAP and Skipfish) can be characterized as originally targeted at certain types of threats. It is emphasized that there is no single security scanner that provides consistently high detection rates for all types of vulnerabilities. Based on the results of the review, it is claimed that the existing differences in the frequency of false-positive vulnerabilities (for both groups of scanners) are due to the fact that most commercial solutions have automated scanners, which are more effective than manual settings by the tester. It is obvious that the results of manual settings have a direct relationship with the actual level of the tester's competence, and largely determine the final results.

Downloads

Download data is not yet available.

Author Biographies

Ivan Lakhtin, V. N. Karazin Kharkiv National University

student Computer Science Department (magistrate)

Dmytro Mykhailenko, V. N. Karazin Kharkiv National University

Computer science student (bachelor's degree)

Oleksii Nariezhnii, V. N. Karazin Kharkiv National University

Ph.D., Associate Professor,

References

Amankwah, R., Chen, J., Kudjo, P. K., & Towey, D. (2020). An empirical comparison of commercial and open‐source web vulnerability scanners. Software: Practice and Experience, 50(9), 1842–1857. https://doi.org/10.1002/spe.2870

El, M., McMahon, E., Samtani, S., Patton, M., & Chen, H. (2017). Benchmarking vulnerability scanners: An experiment on SCADA devices and scientific instruments. 2017 IEEE International Conference on Intelligence and Security Informatics (ISI). https://doi.org/10.1109/isi.2017.8004879

Alsaleh, M., Alomar, N., Alshreef, M., Alarifi, A., & Al-Salman, A. (2017). Performance-Based Comparative Assessment of Open Source Web Vulnerability Scanners. Security and Communication Networks, 2017, 1–14. https://doi.org/10.1155/2017/6158107

Chen, S. (2017, November 10). WAVSEP 2017/2018 - Evaluating DAST against PT/SDL Challenges. Security Tools Benchmarking. https://sectooladdict.blogspot.com/.

Acunetix | Web Application Security Scanner. (2022). Acunetix. https://www.acunetix.com/

The leader in Web application security assessment. (2011). Retrieved November 4, 2022, from http://www.hp.com/hpinfo/newsroom/press_kits/2011/risk2011/HP_WebInspect_data_sheet.pdf

HCL Software. (2021). Hcltechsw.com. https://www.hcltechsw.com/appscan

OWASP ZAP Tutorial: Comprehensive Review Of OWASP ZAP Tool. (2022). Www.softwaretestinghelp.com. https://www.softwaretestinghelp.com/owasp-zap-tutorial/

skipfish. (2017). Kali.tools. Retrieved November 4, 2022, from https://kali.tools/all/?tool=1256

Arachni - Web Application Security Scanner Framework. (n.d.). Arachni - Web Application Security Scanner Framework. https://www.arachni-scanner.com/

IronWASP - Инструменты Kali Linux. (n.d.). Retrieved November 4, 2022, from https://kali.tools/?p=1786

Vega Vulnerability Scanner. (n.d.). Subgraph.com. https://subgraph.com/vega/

OWASP. (n.d.). OWASP foundation, the open source foundation for application security. Owasp.org. https://owasp.org/

OWASP Benchmark. (n.d.). Owasp.org. https://owasp.org/www-project-benchmark/

Youden, W. J. (1950). Index for rating diagnostic tests. Cancer, 3(1), 32–35. https://doi.org/3.0.co;2-3">10.1002/1097-0142(1950)3:1<32::aid-cncr2820030106>3.0.co;2-3

The Web Application Security Consortium / Web Application Security Scanner Evaluation Criteria. (2009). Projects.webappsec.org. http://surl.li/dtuti

Published
2022-12-26
Cited
How to Cite
Lakhtin, I., Mykhailenko, D., & Nariezhnii, O. (2022). Comparison of commercial web application vulnerability scanners and open source scanners. Computer Science and Cybersecurity, (2), 41-49. https://doi.org/10.26565/2519-2310-2022-2-05
Section
Статті

Most read articles by the same author(s)