📊 Security Testing Spectrum
No single testing method finds all vulnerabilities. An effective security testing program layers multiple approaches — automated tools for scale and speed, manual testing for logic and context. The tradeoff: coverage (breadth) vs depth (accuracy). Automated tools cover more ground; manual testing goes deeper on specific areas.
| Testing Type | When | What It Finds | False Positive Rate | Cost |
|---|---|---|---|---|
| SAST Static Analysis |
On commit / PR | Code-level flaws: injection, hardcoded secrets, insecure API usage | High (20–50%) | Low–Medium |
| DAST Dynamic Analysis |
On staging deploy | Runtime issues: XSS, SQLi, auth failures, misconfigs | Low (5–15%) | Medium |
| IAST Interactive Analysis |
During functional tests | Accurate runtime flows: data taint through app logic | Very Low | Medium–High |
| SCA Composition Analysis |
On dependency update | Known CVEs in third-party libraries | Low | Low |
| Manual Pentest | Pre-release / quarterly | Business logic flaws, chained attacks, zero-days | Very Low | High |
🔍 Static Analysis (SAST)
SAST analyzes source code (or compiled bytecode) without executing the program. It builds an Abstract Syntax Tree (AST), performs data flow analysis, and traces how untrusted data (taint sources) flows to dangerous operations (taint sinks) — identifying potential injection and other code-level vulnerabilities.
Commercial SAST Tools
- Checkmarx — enterprise-grade, broad language support, deep data flow analysis. Strong in financial services and regulated industries.
- Veracode — SaaS model, policy-based security gates, strong compliance reporting. Good for teams without security expertise on staff.
- SonarQube — open core with commercial tiers. Strong developer experience; integrates with most CI/CD platforms. Community edition is free.
Open Source SAST Tools
- Semgrep — fast, accurate, pattern-based analysis. Custom rules in YAML. Free tier covers most needs. Excellent for CI integration.
- Bandit — Python-specific. Scans for common security issues. Integrates into pre-commit hooks.
- SpotBugs + Find Security Bugs — Java bytecode analysis. Finds serialization issues, SQL injection, and more.
- ESLint security plugins — eslint-plugin-security, eslint-plugin-no-unsanitized for JavaScript/TypeScript.
- gosec — Go security checker. Fast, opinionated rules for common Go security mistakes.
IDE Integration & Developer Experience
- IDE plugins (Semgrep, SonarLint, Snyk) provide real-time feedback as code is written — the fastest feedback loop possible.
- Pre-commit hooks run SAST before code is committed — prevent vulnerabilities from ever reaching the repo.
- Developer-facing findings need clear remediation guidance — a finding with no "how to fix" is ignored.
- Reduce noise: tune rules to match your tech stack, suppress known-safe false positives with inline comments and suppression files.
# Semgrep rule: detect hardcoded credentials
# File: rules/hardcoded-secrets.yaml
rules:
- id: hardcoded-password
patterns:
- pattern: password = "..."
- pattern-not: password = ""
message: "Hardcoded password detected. Use environment variables or a secrets manager."
languages: [python, javascript, typescript]
severity: ERROR
metadata:
cwe: "CWE-798: Use of Hard-coded Credentials"
# Run Semgrep in CI:
semgrep --config rules/hardcoded-secrets.yaml --config p/owasp-top-ten ./src
# Run with auto-fix suggestions:
semgrep --config p/python --autofix ./src
🏃 Dynamic Analysis (DAST)
DAST tests the running application from the outside — like a real attacker would. It has no access to source code; it fires HTTP requests, observes responses, and infers vulnerabilities from the application's behavior. Low false positive rate because findings are confirmed in a live environment.
OWASP ZAP
- Free, open-source DAST tool maintained by OWASP. Gold standard for open-source web app scanning.
- Spider — crawls the application to discover all URLs and parameters.
- Active Scan — fires attack payloads against discovered parameters to detect injection, XSS, misconfigs.
- CI/CD integration — Docker-based ZAP scan (zap-baseline.py, zap-full-scan.py) runs in GitHub Actions, GitLab CI, Jenkins.
- API scanning — import OpenAPI/Swagger spec for targeted API fuzzing.
- Authentication scripts support modern login flows (form auth, OAuth2, NTLM).
Burp Suite
- Industry standard for manual and semi-automated web app testing. Community (free) and Pro (~$450/yr) editions.
- Proxy — intercepts and modifies all HTTP/S traffic between browser and application.
- Repeater — replay and modify individual requests for manual testing.
- Scanner (Pro) — automated active/passive scanning; significantly more accurate than free alternatives.
- Extensions (BApps) — rich marketplace: JWT Editor, Active Scan++, Autorize (access control testing), CSRF Scanner.
- SPA limitations — spider/crawler struggles with JavaScript-heavy apps; use browser-based crawl with authenticated session.
| Feature | OWASP ZAP | Burp Suite Pro |
|---|---|---|
| Cost | Free / Open Source | ~$450/year |
| CI/CD Integration | Native Docker support | Enterprise only (CI/CD) |
| API Support | OpenAPI / SOAP | OpenAPI / GraphQL |
| Scanner Accuracy | Good | Excellent |
| Manual Testing UX | Adequate | Best-in-class |
| Learning Curve | Moderate | Steep (worth it) |
🏋 Interactive & Runtime Analysis
IAST (Interactive Application Security Testing)
- IAST uses an agent embedded in the application runtime (JVM, .NET CLR, Node process) that instruments code execution during normal functional tests.
- Observes actual data flows — knows exactly which tainted input reached which dangerous sink, in context of real application logic.
- Very low false positive rate — findings are confirmed by actual execution traces.
- Contrast Security — leading IAST vendor; Java, .NET, Node, Python agents.
- Seeker by Synopsys — combines IAST with active verification.
- Requires functional test suite — IAST only observes code paths that are exercised.
RASP (Runtime Application Self-Protection)
- RASP is an agent that runs inside the production application and intercepts attacks in real time — blocking malicious execution rather than just reporting it.
- Can block SQL injection, command injection, and path traversal mid-execution before the dangerous operation reaches the database or OS.
- Performance overhead: typically 2–5% CPU overhead. Acceptable for high-value services.
- Acts as a defense-in-depth layer when patch deployment is delayed.
- Not a replacement for fixing the root-cause vulnerability — use RASP as a temporary mitigation while a proper fix is deployed.
Fuzzing
- AFL++ — coverage-guided fuzzer for C/C++ binaries. Mutates inputs to maximize code coverage, finding crashes and memory safety issues.
- LibFuzzer — in-process fuzzer from LLVM. Integrates with sanitizers (ASan, UBSan) for memory error detection.
- RESTler — stateful REST API fuzzer from Microsoft Research. Generates sequences of API calls to find logic flaws and crashes.
- APIFuzzer — reads OpenAPI spec and generates invalid/boundary inputs for each field.
- Fuzzing is especially valuable for parsers, serialization code, and any component that processes binary or structured external data.
⚙ Security Testing in CI/CD
DevSecOps integrates security testing into every stage of the CI/CD pipeline — making security continuous rather than a pre-release bottleneck. The goal is fast feedback loops with high signal-to-noise ratio.
Pipeline Integration Points
- On commit / PR — SAST (Semgrep, SonarQube), secret scanning (truffleHog, git-secrets), linting.
- On dependency update — SCA (Snyk, Dependabot), license compliance check.
- On merge to main — full SAST scan, container image scanning (Trivy), SBOM generation.
- On staging deploy — DAST (OWASP ZAP baseline scan), API security scan.
- Pre-production — full DAST active scan, manual pentest (quarterly or pre-major release).
Security Gates
- Blocking gates — fail the build on critical/high findings. Use sparingly and only on high-confidence rules.
- Advisory gates — report findings but don't block. Used during initial rollout or for medium/low severity.
- Track security debt: findings that don't block the build must be tracked in a backlog with SLAs (critical: 7 days, high: 30 days).
- Establish a clear false positive process — developers need a way to suppress false positives with documented justification.
CI/CD Platform Integration
- GitHub Actions — native CodeQL (free for public repos), Semgrep Action, Snyk Action, ZAP GitHub Action.
- GitLab CI — built-in SAST, DAST, container scanning, dependency scanning templates in Ultimate tier; available separately in lower tiers.
- Jenkins — plugins for OWASP ZAP, SonarQube, Dependency-Check; requires more manual configuration.
- Security findings should flow to the same ticketing system as bugs (Jira, GitHub Issues) — not a separate security tool that developers never check.
A Security Gate That Blocks Everything Gets Disabled
Developer experience matters. A SAST gate that generates hundreds of false positives for every commit will be disabled, bypassed, or ignored within weeks. Start with blocking only on secrets (zero acceptable false positives) and critical-severity, high-confidence rules. Expand coverage incrementally as you tune the rules and build developer trust in the tooling. High signal-to-noise ratio is more valuable than 100% theoretical coverage.