Because of the size of the OWASP Benchmark, you may need to give your tool more memory before it starts the scan. If its a Java based tool, you may want to pass more memory to it like this:
-Xmx4G (This gives the Java application 4 Gig of memory)
Checkmarx
The Checkmarx SAST Tool (CxSAST) is ready to scan the OWASP Benchmark out-of-the-box. Please notice that the OWASP Benchmark “hides” some vulnerabilities in dead code areas, for example:
if (0>1)
{
//vulnerable code
}
By default, CxSAST will find these vulnerabilities since Checkmarx believes that including dead code in the scan results is a SAST best practice.
Checkmarx’s experience shows that security experts expect to find these types of code vulnerabilities, and demand that their developers fix them. However, OWASP Benchmark considers the flagging of these vulnerabilities as False Positives, as a result lowering Checkmarx’s overall score.
Therefore, in order to receive an OWASP score untainted by dead code, re-configure CxSAST as follows:
CodeQL
To use the CodeQL command line interface (CLI) on Benchmark, first install CodeQL and its rules databases per: https://codeql.github.com/docs/codeql-cli/using-the-codeql-cli/ (i.e., codeql and the codeql-repo).
Once this is done, do the following:
You can also try different CodeQL query suites other than: java-code-scanning.qls. To get a list, run: ~/PATHTO/codeql/codeql database analyze owasp-benchmark –format=sarifv2.1.0 –output=results/Benchmark-CodeQL.sarif. When you do, a usage message will be displayed. Something like: “No queries were specified … one of these query suites might be what you want?” And then a list is displayed. 5 different query suites starting with ‘java-‘ were listed when we ran it. We determined that only 2 query suites include their Weak Random rule (java-security-and-quality.qls and java-lgtm-full.qls).
Kiuwan Code Security
Kiuwan Code Security wrote their own instructions for scanning the OWASP Benchmark. Refer to their step-by-step guide on the Kiuwan website.
Micro Focus (Formerly HP) Fortify
If you are using the Audit Workbench, you can give it more memory and make sure you invoke it in 64-bit mode by doing this:
set AWB_VM_OPTS="-Xmx2G -XX:MaxPermSize=256m"
export AWB_VM_OPTS="-Xmx2G -XX:MaxPermSize=256m"
auditworkbench -64
We found it was easier to use the Maven support in Fortify to scan the Benchmark and to do it in 2 phases, translate, and then scan. We did something like this:
Translate Phase:
export JAVA_HOME=$(/usr/libexec/java_home)
export PATH=$PATH:/Applications/HP_Fortify/HP_Fortify_SCA_and_Apps_17.10/bin
export SCA_VM_OPTS="-Xmx2G -version 1.7"
mvn sca:clean
mvn sca:translate
Scan Phase:
export JAVA_HOME=$(/usr/libexec/java_home)
export PATH=$PATH:/Applications/HP_Fortify/HP_Fortify_SCA_and_Apps_4.10/bin
export SCA_VM_OPTS="-Xmx10G -version 1.7"
mvn sca:scan
PMD
We include this free tool with the Benchmark for Java and its all dialed in. Simply run the script: ./script/runPMD.(sh or bat). If you want to run a different version of PMD, just change its version number in the Benchmark pom.xml file. (NOTE: PMD doesn’t find any security issues. We include it because its interesting to know that it doesn’t.)
SpotBugs
We include this free tool with the Benchmark for Java and its all dialed in. Simply run the script: ./script/runSpotBugs.(sh or bat). If you want to run a different version of SpotBugs, just change its version number in the Benchmark pom.xml file.
SpotBugs with FindSecBugs
FindSecurityBugs is a great plugin for SpotBugs that significantly increases the ability for SpotBugs to find security issues. We include this free tool with the Benchmark and its all dialed in. Simply run the script: ./script/runFindSecBugs.(sh or bat). If you want to run a different version of FindSecBugs, just change the version number of the findsecbugs-plugin artifact in the Benchmark pom.xml file.
Burp Pro
To scan, first crawl the entire Benchmark. To do a crawl, right click on Benchmark in the Site Map, select Scan–>Open scan launcher. Then click on Crawl and hit OK. Then save the project in case of a crash during the scan. Then select the /Benchmark URL and invoke: ‘Actively scan this branch’. Prior to all this, you might want to give Burp more RAM before you start it (e.g., launch it like so: java -Xmx2G -jar burpsuite_pro.jar).
ZAP
ZAP may require additional memory to be able to scan the Benchmark. To configure the amount of memory:
To run ZAP against Benchmark:
For faster active scan you can:
To generate the ZAP XML results file so you can generate its scorecard:
Things we tried that didn’t improve the score:
Interactive Application Security Testing (IAST) tools work differently than scanners. IAST tools monitor an application as it runs to identify application vulnerabilities using context from inside the running application. Typically these tools run continuously, immediately notifying users of vulnerabilities, but you can also get a full report of an entire application. To do this, we simply run the Benchmark application with an IAST agent and use a crawler to hit all the pages.
Contrast Assess
To use Contrast Assess, we simply add the Java agent to the Benchmark environment and run the BenchmarkCrawler. The entire process should only take a few minutes. We provided a few scripts, which simply add the -javaagent:contrast.jar flag to the Benchmark launch configuration. We have tested on MacOS, Ubuntu, and Windows. Be sure your machine or VM has at least 4M of memory.
Project Resources
Project Leaders
Project Information
Lab Project
Classification
Tool
Language
Java
License
GNU General Public License v2.0
Contributors
70
GitHub Stars
824
Downloads
0
OWASP is a nonprofit foundation improving software security through open-source projects, global communities, and education. All resources are free and open to everyone.
SlackOWASP, the OWASP logo, and Global AppSec are registered trademarks and AppSec Days, AppSec California, AppSec Cali, SnowFROC, OWASP Boston Application Security Conference, and LASCON are trademarks of the OWASP Foundation, Inc.
© 2026, OWASP Foundation Inc. All rights reserved.