Skip to content
System One

teyhouse/jev-secret-detection

Measures how well Jev spots real secret credentials in file snippets. Each case is one Noul question and the probability is compared with the expected label, with no regex matching or provider verification. Reports accuracy, AUC, recall, precision, Brier score, review-band counts and latency.

Sample report output
Image from github.com

teyhouse/jev-secret-detection on GitHub

Stars
1
Forks
0
Language
Python
Last push
18 Sep 2026

Read from GitHub on . Counts change daily.

How it uses Jev

state = {"file_path": case.file_path, "content": case.content}
async with pool:
    start = time.perf_counter()
    result = await client.system_one(state=state, questions=QUESTIONS, model=MODEL)
    elapsed_ms = (time.perf_counter() - start) * 1000
response = result.raw_http_response
# Undocumented envoy header: time spent behind the API gateway, so it leaves out the network round trip.
server_ms = response.headers.get("x-envoy-upstream-service-time")
retries = int(response.request.headers.get("X-TypeSafe-Retry-Count", 0))
noul = result.nouls["contains_real_secret"].noul

View in main.py

Open on GitHub

PrimitivesNoul

More like this

Keep browsing