Highlights

Various government institutes and agencies worldwide use the platform as an assessment platform for vulnerability scanners, often as the main one.

Generic Application-Level Vulnerability Detection Module: a collection of features that attempt to identify generic exposures in the application layer, without prior knowledge about the application and its structure, and while potentially overcoming barriers along the way. This module is the primary focus of this research.

The purpose of the accuracy assessment is to see how effective each scanner is in detecting a variety of vulnerabilities, and to see whether or not the detection logic "settles" for simple scenarios, or covers a collection of common and advanced scenarios.

Since the latest benchmark, many open source & commercial tools added new features and improved their detection accuracy.

Syhunt dramatically improved their WIVET score (came out FIRST, along with 3 others), and slightly improved other scores as well (LFI, etc). — WAVSEP 2014

Test Cases

The various scanners were tested against the following test cases (GET and POST attack vectors):

• 816 test cases that were vulnerable to Path Traversal / LFI attacks.
• 184 test cases that included Hidden, Obsolete and Backup files.
• 108 test cases that were vulnerable to Remote File Inclusion (XSS via RFI) attacks.
• 66 test cases that were vulnerable to Reflected Cross Site Scripting attacks.
• 80 test cases that contained Error Disclosing SQL Injection exposures.
• 60 test cases that were vulnerable to Phishing via Unvalidated Redirect.
• 46 test cases that contained Blind SQL Injection exposures.
• 10 test cases that were vulnerable to Time Based SQL Injection attacks.

Additionally, the various scanners were tested against 43 different categories of false positive, covering Unvalidated Redirect, Obsolete/Hidden/Backup files, Path Traversal / LFI, Remote File Inclusion, Reflected XSS vulnerabilities and SQL Injection vulnerabilities.

Results

WIVET - Crawling Coverage

In order to assess the scanners attack surface coverage, the assessment included tests that measure the efficiency of the scanner's automated crawling mechanism (input vector extraction) , and feature comparisons meant to assess its support for various technologies and its ability to handle different scan barriers.

This section of the benchmark also included the WIVET test (Web Input Vector Extractor Teaser v3-rev148), in which scanners were executed against a dedicated application that can assess their crawling mechanism efficiency in the aspect of input vector extraction.

If you use scanners mainly in a point-and-shoot scenario, and prefer as much automation as possible, a high WIVET score will be the second most important feature you should follow.

Organizations and QA/DEV departments really need a tool with a high score in this category – especially in 2014 — WAVSEP 2014

Syhunt Dynamic 5.0 was able to achieve 94% crawling coverage, showing that Syhunt Dynamic's enhanced crawler is well equipped for handling malformed HTML, many kinds of relative links, finding certain patterns in HTML comments, and for performing a variety of browser emulation tasks, such as JavaScript execution and DOM emulation, user interaction simulation, session and cookie handling, code analysis, and more.

SQL Injection (Blind and Time-Based)

One of the most famous exposures (and powerful attacks) and the most commonly implemented attack vector in web application scanners — WAVSEP 2014

Syhunt Dynamic achieved 100% detection of SQL Injection vulnerabilities (both Blind and Time Based).

XSS (Cross-Site Scripting)

This section deals with the detection accuracy of Reflected Cross Site Scripting, a very common exposure which is the 2nd most commonly implemented feature in web application vulnerability scanners. — WAVSEP 2014

Syhunt Dynamic achieved 100% detection of XSS vulnerabilities in the testing, with zero false positives thanks to its advanced filters.

XSS (Cross-Site Scripting) & SQL Injection - Stacked

Syhunt, Acunetix, AppScan, Netsparker and WebInspect top the WAVSEP 2014 benchmark charts with 100 percent SQL Injection and XSS detection rates.

Hidden, Obsolete & Backup Files

This test assessed the capabilities of scanners to locate backup files with non-executable extensions, compressed versions of files and directories that developers may have forgotten, sequential files or copies of files and directories that are remnants of various development tests, and additional hazards that may lead to source code configuration disclosure. — WAVSEP 2014

File Inclusion (RFI & LFI) - Stacked

Note: QualysGuard WAS is absent from the file inclusion chart because there were issues when trying to use its LFI/RFI plugins.

Phishing via Unvalidated Redirect

At the time that the comparison was performed Syhunt Dynamic could not perform checks for unvalidated redirect. A plugin for detecting unvalidated redirects has been added later.