The problem SecretSifter addresses is structural. The dominant approach to secret detection today is so-called shift-left tooling: scanners such as TruffleHog, git-secrets, and detect-secrets that comb through source repositories and continuous-integration pipelines looking for committed credentials. These tools are effective on the surface they observe, but they are blind to credentials that reach production without ever appearing in a repository. Two mechanisms create this blind spot. First, build-time static substitution: bundlers like webpack, Vite, esbuild, and the Angular CLI compile environment variables directly into the deployed JavaScript bundle, so the credential exists only in the shipped artifact. Second, runtime delivery: configuration APIs and server-side-rendered state objects serve credentials to the browser on demand, bypassing any repository scan entirely. A companion measurement study published in IEEE Access quantified the gap on an enterprise corpus of 113 applications and found that a substantial fraction of production credentials are recoverable only from served content.

Existing runtime-adjacent tools, the paper argues, are too narrow to close that gap. File-oriented JavaScript scanners miss HTTP responses and headers; HTTP-template scanners miss file content; and none of them cover split webpack chunks, server-side-rendered blobs, JSON and XML bodies, and outbound headers in a single pass. SecretSifter was built to do all of these at once. Its architecture rests on a single detection core deployed as three editions that differ only in how they capture traffic: a Burp Suite extension for penetration testers working through a proxy, a Manifest V3 Chrome extension that attaches through the Chrome DevTools Protocol to watch responses inside the browser, and a standalone Windows and macOS desktop application that drives a real Chrome browser via Playwright, runs an embedded man-in-the-middle proxy, and imports HAR files. All three share the same engine and rule library, diverging only at capture, triage, and output.

The detection pipeline itself is layered. Content intercepted from live traffic is deep-copied off the proxy thread and dispatched by content type through three configurable scan tiers. Detection combines a library of 134 format-anchored vendor rules covering services such as GitHub, AWS, Stripe, OpenAI, Anthropic, and Slack, plus 52 context-gated rules that require a vendor keyword near the value; URL and query-embedded credential detection; database connection-string recognition; a context-gated generic key-value extractor wrapped in a false-positive suppression cascade; Shannon-entropy scoring with a default threshold of 3.5; and dedicated detection of hardcoded CryptoJS and AES passphrases, keys, and ciphertext. That last class matters: encrypted configuration blobs that pattern-only scanners see as opaque noise can be decrypted by SecretSifter when the key sits alongside them in the bundle. Coverage extends beyond initial bundles to split webpack and Next.js chunks, SSR state blobs such as NEXT_DATA, inline HTML scripts, bounded-depth JSON traversal, XML leaves, form-encoded OAuth token responses, and, when enabled, outbound request headers.

To keep the noise down, the tool employs a suppression cascade that filters framework directives, CSS-module hashes, object identifiers, reCAPTCHA keys, and hex strings lacking cryptographic context, along with cross-cutting handling of JWT recognition, opaque-bearer filtering, CDN blocklisting, and cross-request de-duplication. Optional AI-assisted triage supports four back ends: Burp’s native AI, Anthropic’s Claude, any OpenAI-compatible endpoint, and a cooperative mode in which an external LLM agent performs triage through the Model Context Protocol. Crucially, deterministic vendor rules rather than the model set the final severity, so risk ratings remain reproducible. When a remote back end is used, only a compact record of each finding is sent, never the full response body, and triage is disabled by default. Operators with strict data-handling constraints can run triage entirely locally through a self-hosted model served by Ollama or LM Studio.

The evaluation used GT-198, a benchmark of 198 scored values recovered from production JavaScript across 56 applications from a single authorized engagement: 161 static values, including 51 Azure APIM subscription keys and 33 Azure AD client secrets, and 37 CryptoJS-AES encrypted-configuration blobs. Ground truth was anchored by a Claude Opus 4.7 oracle with no access to any evaluated scanner, cross-validated by a second-vendor model at 89.9 percent high-confidence confirmation, and extended by a manual analyst pass. Ten systems were scored in default configuration: nine production scanners and the labelling oracle. SecretSifter detected 158 of 198 values, a recall of 79.8 percent, with 24 false positives and 86.8 percent precision, yielding an F1 score of 83.2 percent, the highest of any system in the set and marginally ahead of even the LLM oracle.

The comparison with individual tools is striking. TruffleHog, the strongest static scanner, managed 35.9 percent recall with high precision, leaving a 43.9 percentage-point gap. SecretFinder reached a comparable 31.8 percent recall but emitted 1,402 false positives, collapsing its precision to 4.3 percent, because generic UUID and base64 patterns matching webpack chunk hashes and build fingerprints saturate its output. Narrow parser-based scanners such as Titus and JSluice kept noise down but recovered far less. The CryptoJS-AES class proved decisive: out of the box, pattern-only scanners flagged none of the 37 encrypted blobs, while SecretSifter’s dedicated decryptor recovered all of them, the single largest component of its recall lead. A tightened analysis in which four rule-extensible baselines received a common six-rule overlay showed the strongest static scanners becoming statistically indistinguishable from SecretSifter on static text, meaning the software’s advantage rests on out-of-the-box coverage, decryption awareness, and noise suppression rather than claimed static-text superiority.

Qualitatively, the tool occupies ground no evaluated competitor touches. Only SecretSifter observes secrets that exist solely after client-side JavaScript executes, values in the rendered DOM and in-memory state, whereas the others inspect source bytes on disk, in a repository, or on the wire. The author is upfront about limitations: the benchmark comes from a single organization’s applications, the detection core and benchmark share an organizational context, and the author developed the tool under evaluation, with LLM-based benchmark construction and cross-vendor validation offered as mitigations and the competing interest declared. The recall figures should be read as in-context performance rather than an out-of-distribution estimate, and because the corpus contains real credentials it cannot be redistributed, a de-identified reproducibility bundle and a synthetic demo application called InsecureShield serve as the reproducible artifacts instead.

Beyond the benchmark, the engineering contribution is a reuse architecture: one detection core driving three deployment models through thin capture adapters rather than three forked codebases, with more than 90 regression-tested unit tests covering the rule library, entropy, and decryption logic. Bearer-authenticated REST and MCP interfaces, disabled by default, let an external LLM agent or CI pipeline run scans, manage rules, and drive triage without a graphical interface. The open-source release under the MIT license covers the Java detection core and its Burp capture layer, while the Chrome and desktop editions ship as packaged builds under a proprietary end-user license. The author positions the tool as reusable infrastructure enabling continuous runtime secret monitoring during penetration-testing and AppSec engagements, with future work targeting multi-organization benchmarking, expanded decryptor coverage, and deeper CI integration.

For defenders, the message is uncomfortable but clear: the credentials your repository scanners certify as clean may be sitting in plain sight in the traffic your users’ browsers download every day. As web applications increasingly assemble themselves at runtime from bundles, chunks, and configuration endpoints, security tooling that only reads source code is watching the wrong place. SecretSifter’s results suggest that watching the wire, with the right suppression and decryption machinery, can recover most of what the shift-left world misses, and that the era of assuming a clean repository means a clean deployment is coming to an end.

Subject of Research: Runtime detection of credentials exposed in live web application traffic

Article Title: SecretSifter: A runtime secret-detection engine for live web traffic

Article References: Gorijala, H. (2026). SecretSifter: A runtime secret-detection engine for live web traffic. SoftwareX, 36, Article 103087. https://doi.org/10.1016/j.softx.2026.103087

Image Credits: AI Generated

DOI: 10.1016/j.softx.2026.103087

Keywords: SecretSifter, secret detection, web security, API keys, runtime analysis, credential exposure, Burp Suite, Chrome extension, LLM triage, open-source software, penetration testing, SoftwareX