Initial commit: omp-security DSH plugin (oh-my-pi port) with docs and tests
This commit is contained in:
Agent committed
commit
e37410e516
9 files changed
+627
No files matched your search
@@ -0,0 +1,5 @@
|
||||
node_modules/
|
||||
*.bak
|
||||
*.orig
|
||||
*.log
|
||||
.DS_Store
|
||||
@@ -0,0 +1,73 @@
|
||||
# omp-security
|
||||
|
||||
[oh-my-pi](https://github.com/can1357/oh-my-pi)'s `security_scan` tool for the DeepSeek Harness (DSH). It plans and runs security reviews of the workspace with real scanners plus an **AI triage pass**, produces a **P0–P3 prioritized report**, and keeps findings so they can be validated later.
|
||||
|
||||
## Flow
|
||||
|
||||
```
|
||||
security_scan {"action":"preflight","target_kind":"working_tree"}
|
||||
→ Plan plan-3fa1c2 — working_tree at /repo
|
||||
Scope: 4 file(s): api.py, db.py, …
|
||||
Scanners: ✓ semgrep ✓ bandit ✓ gitleaks – osv-scanner (not applicable) …
|
||||
security_scan {"action":"start","plan_id":"plan-3fa1c2"}
|
||||
→ Security scan scan-2026-10-10-9b1e (operation op-77c0): completed; 3 finding(s) — 0 critical, 2 high, 1 medium …
|
||||
AI review (omni/claude-sonnet): Fix the shell injection in api.py first …
|
||||
1. [P1 high] subprocess call with shell=True identified, security issue.
|
||||
api.py:41 · bandit B602 · CWE-78 · id 4be1c0a9d2
|
||||
AI: tp — user input reaches the shell
|
||||
Fix: pass an argv list and shell=False
|
||||
```
|
||||
|
||||
| Action | Purpose |
|
||||
| --- | --- |
|
||||
| `preflight` | Pin the scope: `target_kind` = `repository` (default), `scoped_path` (+`include_paths`), `working_tree` (uncommitted changes), `ref_diff` (`base_revision`..`head_revision`). Optional `exclude_paths`, `scanners` subset, `ai_review`. Returns a `plan_id` and which scanners apply |
|
||||
| `start` | Run the plan in the background. Waits up to `wait_seconds` (default 20) and returns the report or progress plus an `operation_id` |
|
||||
| `status` | `operation_id` or `scan_id`; `wait_seconds`, `min_severity`, `max_findings` |
|
||||
| `cancel` | Stop a running operation |
|
||||
| `validate` | Record a verdict on a finding: `scan_id`, `finding_id`, `validation_status` (`validated`/`rejected`/`partial`/`unvalidated`/`error`), `validation_summary`, optional `validation_evidence` |
|
||||
| `list` | Recent scans |
|
||||
| `scanners` | Installed scanners and install hints |
|
||||
|
||||
**AI review:** a reviewer model (the agent's own model by default, or `reviewProvider`/`reviewModel`) gets:
|
||||
- every finding with surrounding code;
|
||||
- the diff (for `working_tree`/`ref_diff`) or the scoped files.
|
||||
|
||||
It marks each finding as true or false positive with a reason and a concrete fix, adjusts severity, and adds up to 10 issues the scanners missed. False positives are hidden from the report but kept in the stored scan.
|
||||
|
||||
Scans are stored as JSON in `~/.dsh/security-scans/`.
|
||||
|
||||
## Scanners
|
||||
|
||||
| Scanner | What it finds | Install |
|
||||
| --- | --- | --- |
|
||||
| semgrep | SAST rules for 30+ languages (`p/default`) | `pip install --user semgrep` |
|
||||
| bandit | Python security issues | `pip install --user bandit` |
|
||||
| gitleaks | Hard-coded secrets (git history for `repository` scans) | [releases](https://github.com/gitleaks/gitleaks/releases) → `~/.local/bin` |
|
||||
| osv-scanner | Vulnerable dependencies from lockfiles (npm, pnpm, yarn, pip, poetry, go, cargo, maven, …) | [releases](https://github.com/google/osv-scanner/releases) → `~/.local/bin` |
|
||||
| npm audit | npm advisories for `package-lock.json` projects | bundled with npm |
|
||||
|
||||
Missing scanners are skipped and listed in the preflight with install hints.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
dsh plugin --profile web add git+https://git.rizzy.dev/DSH-Plugins/omp-security.git
|
||||
```
|
||||
|
||||
Or in the Web UI, go to **Settings → Plugins → Install**. For development, use `link:` to a clone. Restart DSH afterwards.
|
||||
|
||||
## Configuration
|
||||
|
||||
```yaml
|
||||
- id: omp-security
|
||||
config:
|
||||
aiReview: true
|
||||
reviewProvider: '' # empty = the calling agent's provider
|
||||
reviewModel: '' # empty = the calling agent's model
|
||||
semgrepConfig: p/default # any semgrep --config value (registry pack, file, dir)
|
||||
maxReviewFindings: 60
|
||||
```
|
||||
|
||||
## Tests
|
||||
|
||||
`npm test` runs a scan of a deliberately vulnerable sample with bandit and gitleaks (stubbed reviewer), plus osv-scanner against a vulnerable `requirements.txt`.
|
||||
@@ -0,0 +1,9 @@
|
||||
- insert:
|
||||
- id: omp-security
|
||||
name: '@local/omp-security'
|
||||
config:
|
||||
aiReview: true
|
||||
# empty = use the calling agent's provider/model
|
||||
reviewProvider: ''
|
||||
reviewModel: ''
|
||||
semgrepConfig: p/default
|
||||
@@ -0,0 +1,303 @@
|
||||
/**
|
||||
* @local/omp-security — oh-my-pi's `security_scan` for DSH.
|
||||
*
|
||||
* preflight → start → status / cancel → validate, like oh-my-pi's native scans:
|
||||
* - preflight pins an immutable scope (repository, scoped_path, working_tree changes, or a
|
||||
* base..head ref_diff), detects languages/lockfiles and which scanners apply, returns a plan id;
|
||||
* - start runs the scanners in the background (semgrep, bandit, gitleaks, osv-scanner, npm audit)
|
||||
* and, when `ai_review` is on, a reviewer model triages the findings (true/false positive,
|
||||
* exploitability, fix) and adds issues the scanners missed in the scoped code;
|
||||
* - status returns phase and the prioritized report; findings persist under ~/.dsh/security-scans;
|
||||
* - validate records a human/agent verdict on one finding.
|
||||
*/
|
||||
import crypto from 'node:crypto'
|
||||
import fs from 'node:fs'
|
||||
import os from 'node:os'
|
||||
import path from 'node:path'
|
||||
import { createUserMessage } from '@deepseek-ai/dsh-llm'
|
||||
import { SCANNERS, SEV_ORDER, run, which } from './lib/scanners.js'
|
||||
|
||||
export const name = 'omp-security'
|
||||
export const inject = ['tools', 'llm']
|
||||
|
||||
const STORE = path.join(os.homedir(), '.dsh', 'security-scans')
|
||||
const ACTIONS = ['preflight', 'start', 'status', 'cancel', 'validate', 'list', 'scanners']
|
||||
|
||||
export function apply(ctx, config = {}) {
|
||||
const cfg = {
|
||||
aiReview: true,
|
||||
reviewProvider: '',
|
||||
reviewModel: '',
|
||||
maxReviewFindings: 60,
|
||||
maxReviewChars: 60000,
|
||||
semgrepConfig: 'p/default',
|
||||
...config,
|
||||
}
|
||||
fs.mkdirSync(STORE, { recursive: true })
|
||||
const plans = new Map()
|
||||
const ops = new Map()
|
||||
ctx.effect(() => () => { for (const o of ops.values()) o.abort?.abort() }, 'omp-security: operations')
|
||||
|
||||
const cwdOf = (exec) => exec?.agent?.session?.header?.cwd ?? process.cwd()
|
||||
const gitRoot = async (dir) => { const r = await run('git', ['rev-parse', '--show-toplevel'], { cwd: dir }); return r.code === 0 ? r.stdout.trim() : null }
|
||||
const save = (scan) => fs.writeFileSync(path.join(STORE, `${scan.scanId}.json`), JSON.stringify(scan, null, 1))
|
||||
const load = (id) => { try { return JSON.parse(fs.readFileSync(path.join(STORE, `${id}.json`), 'utf8')) } catch { return null } }
|
||||
|
||||
async function preflight(args, exec) {
|
||||
const cwd = cwdOf(exec)
|
||||
const root = (await gitRoot(cwd)) ?? cwd
|
||||
const kind = args.target_kind ?? 'repository'
|
||||
let files = null
|
||||
let diff = null
|
||||
let base = null; let head = null
|
||||
if (kind === 'scoped_path') {
|
||||
const inc = (args.include_paths ?? []).filter((p) => String(p).trim())
|
||||
if (!inc.length) throw new Error('scoped_path needs include_paths')
|
||||
files = inc
|
||||
} else if (kind === 'working_tree') {
|
||||
const r = await run('git', ['status', '--porcelain', '-uall'], { cwd: root })
|
||||
if (r.code !== 0) throw new Error('working_tree needs a git repository')
|
||||
files = r.stdout.split('\n').filter(Boolean).map((l) => l.slice(3).split(' -> ').pop()).filter((f) => fs.existsSync(path.join(root, f)))
|
||||
diff = (await run('git', ['diff', 'HEAD'], { cwd: root })).stdout
|
||||
} else if (kind === 'ref_diff') {
|
||||
if (!args.base_revision || !args.head_revision) throw new Error('ref_diff needs base_revision and head_revision')
|
||||
const rb = await run('git', ['rev-parse', '--verify', `${args.base_revision}^{commit}`], { cwd: root })
|
||||
const rh = await run('git', ['rev-parse', '--verify', `${args.head_revision}^{commit}`], { cwd: root })
|
||||
if (rb.code || rh.code) throw new Error('could not resolve base/head revision')
|
||||
base = rb.stdout.trim(); head = rh.stdout.trim()
|
||||
files = (await run('git', ['diff', '--name-only', '--diff-filter=ACMR', base, head], { cwd: root })).stdout.split('\n').filter(Boolean)
|
||||
diff = (await run('git', ['diff', base, head], { cwd: root })).stdout
|
||||
} else if (kind !== 'repository') throw new Error(`unknown target_kind ${kind}`)
|
||||
const exclude = (args.exclude_paths ?? []).map(String)
|
||||
if (files) files = files.filter((f) => !exclude.some((x) => f === x || f.startsWith(x.replace(/\/?$/, '/'))))
|
||||
if (files && !files.length) throw new Error('scope contains no files')
|
||||
const scanners = Object.entries(SCANNERS).map(([id, s]) => ({
|
||||
id, what: s.what, available: s.available(),
|
||||
applies: (kind === 'repository' || !['osv-scanner', 'npm-audit'].includes(id) || files?.some((f) => /(lock|requirements|go\.sum|Cargo|pom\.xml|Gemfile)/i.test(f))) && s.applies({ root, files }),
|
||||
install: s.install,
|
||||
}))
|
||||
const wanted = args.scanners?.length ? new Set(args.scanners) : null
|
||||
const selected = scanners.filter((s) => s.available && s.applies && (!wanted || wanted.has(s.id))).map((s) => s.id)
|
||||
const planId = `plan-${crypto.randomBytes(3).toString('hex')}`
|
||||
const plan = { planId, root, kind, files, exclude, base, head, diff, selected, aiReview: args.ai_review ?? cfg.aiReview, createdAt: Date.now(), owner: exec?.agent }
|
||||
plans.set(planId, plan)
|
||||
const scope = files ? `${files.length} file(s)${files.length <= 15 ? `: ${files.join(', ')}` : ''}` : 'whole repository'
|
||||
return [
|
||||
`Plan ${planId} — ${kind} at ${root}`,
|
||||
`Scope: ${scope}${exclude.length ? ` (excluding ${exclude.join(', ')})` : ''}${base ? `\nRange: ${base.slice(0, 8)}..${head.slice(0, 8)}` : ''}`,
|
||||
'Scanners:',
|
||||
...scanners.map((s) => ` ${selected.includes(s.id) ? '✓' : s.available ? '–' : '✗'} ${s.id}: ${s.what}${s.available ? (s.applies ? '' : ' (not applicable)') : ` — not installed (${s.install})`}`),
|
||||
`AI review: ${plan.aiReview ? 'on (reviewer model triages findings and inspects the scoped code)' : 'off'}`,
|
||||
`Start with security_scan {"action":"start","plan_id":"${planId}"}`,
|
||||
].join('\n')
|
||||
}
|
||||
|
||||
async function start(args, exec) {
|
||||
const plan = plans.get(args.plan_id)
|
||||
if (!plan) throw new Error(`unknown plan ${args.plan_id}; run preflight first`)
|
||||
const scanId = `scan-${new Date().toISOString().slice(0, 10)}-${crypto.randomBytes(3).toString('hex')}`
|
||||
const operationId = `op-${crypto.randomBytes(3).toString('hex')}`
|
||||
const scan = { scanId, operationId, planId: plan.planId, root: plan.root, kind: plan.kind, files: plan.files, base: plan.base, head: plan.head, phase: 'queued', startedAt: Date.now(), scanners: {}, findings: [], review: null }
|
||||
const abort = new AbortController()
|
||||
const op = { scan, abort }
|
||||
ops.set(operationId, op)
|
||||
save(scan)
|
||||
const header = exec?.agent?.session?.requestHeader?.()
|
||||
const route = { provider: cfg.reviewProvider || header?.config?.provider, model: cfg.reviewModel || header?.config?.model }
|
||||
op.promise = (async () => {
|
||||
try {
|
||||
scan.phase = 'scanning'
|
||||
await Promise.all(plan.selected.map(async (id) => {
|
||||
const t0 = Date.now()
|
||||
try {
|
||||
const found = await SCANNERS[id].scan({ root: plan.root, files: plan.files, signal: abort.signal, target: plan.kind, config: { semgrepConfig: cfg.semgrepConfig, exclude: plan.exclude } })
|
||||
scan.findings.push(...found)
|
||||
scan.scanners[id] = { status: 'ok', findings: found.length, ms: Date.now() - t0 }
|
||||
} catch (e) {
|
||||
scan.scanners[id] = { status: 'error', error: String(e?.message ?? e).slice(0, 500), ms: Date.now() - t0 }
|
||||
}
|
||||
save(scan)
|
||||
}))
|
||||
if (abort.signal.aborted) { scan.phase = 'cancelled'; return }
|
||||
dedupe(scan)
|
||||
if (plan.aiReview && route.provider && route.model) {
|
||||
scan.phase = 'reviewing'
|
||||
save(scan)
|
||||
try { await aiReview(scan, plan, route, abort.signal) } catch (e) { scan.review = { status: 'error', error: String(e?.message ?? e) } }
|
||||
}
|
||||
scan.phase = Object.values(scan.scanners).some((s) => s.status === 'error') || scan.review?.status === 'error' ? 'partial' : 'completed'
|
||||
} catch (e) {
|
||||
scan.phase = abort.signal.aborted ? 'cancelled' : 'failed'
|
||||
scan.error = String(e?.message ?? e)
|
||||
} finally {
|
||||
scan.finishedAt = Date.now()
|
||||
save(scan)
|
||||
}
|
||||
})()
|
||||
// Small scans finish quickly: wait briefly so the first status is useful.
|
||||
await Promise.race([op.promise, new Promise((r) => setTimeout(r, Math.min(args.wait_seconds ?? 20, 240) * 1000))])
|
||||
return statusText(scan, args)
|
||||
}
|
||||
|
||||
function dedupe(scan) {
|
||||
const seen = new Map()
|
||||
for (const f of scan.findings) {
|
||||
const key = `${f.file}:${f.line}:${(f.cwe ?? f.rule).toString().slice(0, 40)}`
|
||||
if (!seen.has(key)) seen.set(key, f)
|
||||
else { const g = seen.get(key); g.alsoReportedBy = [...(g.alsoReportedBy ?? []), f.scanner] }
|
||||
}
|
||||
scan.findings = [...seen.values()].sort((a, b) => SEV_ORDER.indexOf(a.severity) - SEV_ORDER.indexOf(b.severity))
|
||||
}
|
||||
|
||||
async function aiReview(scan, plan, route, signal) {
|
||||
const items = scan.findings.slice(0, cfg.maxReviewFindings)
|
||||
const ctxSnips = items.map((f) => {
|
||||
let code = ''
|
||||
try {
|
||||
const lines = fs.readFileSync(path.join(plan.root, f.file), 'utf8').split('\n')
|
||||
const a = Math.max(1, (f.line ?? 1) - 4); const b = Math.min(lines.length, (f.endLine ?? f.line ?? 1) + 4)
|
||||
for (let n = a; n <= b; n++) code += `${n}:${lines[n - 1]}\n`
|
||||
} catch {}
|
||||
return `[${f.id}] ${f.severity} ${f.scanner} ${f.rule}\n${f.title}\n${f.file}:${f.line ?? ''}\n${code}`
|
||||
})
|
||||
let extra = ''
|
||||
if (plan.diff) extra = `\nCHANGED CODE (diff):\n${plan.diff.slice(0, Math.floor(cfg.maxReviewChars / 2))}`
|
||||
else if (plan.files?.length) {
|
||||
for (const f of plan.files.slice(0, 20)) {
|
||||
try { extra += `\n=== ${f} ===\n${fs.readFileSync(path.join(plan.root, f), 'utf8').slice(0, 8000)}` } catch {}
|
||||
if (extra.length > cfg.maxReviewChars / 2) break
|
||||
}
|
||||
}
|
||||
const prompt = `You are a security reviewer. Triage the scanner findings below and inspect the code for real vulnerabilities.
|
||||
For each finding decide: "tp" (true positive), "fp" (false positive) or "uncertain", rate exploitability, and give a concrete fix.
|
||||
Also report up to 10 additional real issues you see in the provided code that scanners missed (injection, authz, path traversal, SSRF, deserialization, secrets, unsafe crypto, race conditions).
|
||||
Reply with ONLY JSON: {"triage":[{"id":"…","verdict":"tp|fp|uncertain","severity":"critical|high|medium|low","reason":"…","fix":"…"}],"new":[{"file":"…","line":0,"severity":"…","title":"…","reason":"…","fix":"…","cwe":"CWE-…"}],"summary":"2-4 sentences: overall risk and what to fix first"}
|
||||
|
||||
FINDINGS (${items.length}${scan.findings.length > items.length ? ` of ${scan.findings.length}` : ''}):
|
||||
${ctxSnips.join('\n').slice(0, cfg.maxReviewChars / 2)}
|
||||
${extra}`
|
||||
let text = ''
|
||||
for await (const ch of ctx.llm.stream({ provider: route.provider, model: route.model, messages: [createUserMessage({ content: [{ type: 'text', text: prompt }], source: { kind: 'omp-security' } })], maxTokens: 6000, signal })) {
|
||||
if (ch.type === 'text-delta') text += ch.text
|
||||
if (ch.type === 'finish' && ch.reason?.kind === 'error') throw new Error(ch.reason.error?.message ?? 'review model error')
|
||||
}
|
||||
const m = /\{[\s\S]*\}/.exec(text)
|
||||
if (!m) throw new Error('reviewer returned no JSON')
|
||||
const j = JSON.parse(m[0])
|
||||
const byId = new Map(scan.findings.map((f) => [f.id, f]))
|
||||
for (const t of j.triage ?? []) {
|
||||
const f = byId.get(t.id)
|
||||
if (!f) continue
|
||||
f.ai = { verdict: t.verdict, reason: t.reason, fix: t.fix }
|
||||
if (t.severity && SEV_ORDER.includes(t.severity) && t.verdict === 'tp') f.severity = t.severity
|
||||
}
|
||||
for (const n of j.new ?? []) {
|
||||
scan.findings.push({ id: crypto.createHash('sha1').update(`ai|${n.file}|${n.line}|${n.title}`).digest('hex').slice(0, 10), scanner: 'ai-review', rule: n.cwe ?? 'ai', severity: SEV_ORDER.includes(n.severity) ? n.severity : 'medium', title: n.title, file: n.file, line: n.line, cwe: n.cwe, ai: { verdict: 'tp', reason: n.reason, fix: n.fix } })
|
||||
}
|
||||
scan.review = { status: 'ok', model: `${route.provider}/${route.model}`, summary: j.summary }
|
||||
scan.findings.sort((a, b) => (a.ai?.verdict === 'fp') - (b.ai?.verdict === 'fp') || SEV_ORDER.indexOf(a.severity) - SEV_ORDER.indexOf(b.severity))
|
||||
}
|
||||
|
||||
function statusText(scan, args = {}) {
|
||||
const p = scan.phase
|
||||
const running = !['completed', 'partial', 'failed', 'cancelled'].includes(p)
|
||||
const live = scan.findings.filter((f) => f.ai?.verdict !== 'fp')
|
||||
const counts = SEV_ORDER.map((s) => `${live.filter((f) => f.severity === s).length} ${s}`).join(', ')
|
||||
const minSev = SEV_ORDER.indexOf(args.min_severity ?? 'low')
|
||||
const shown = live.filter((f) => SEV_ORDER.indexOf(f.severity) <= (minSev < 0 ? 3 : minSev))
|
||||
const max = args.max_findings ?? 40
|
||||
const lines = [
|
||||
`Security scan ${scan.scanId} (operation ${scan.operationId}): ${p}; ${live.length} finding(s) — ${counts}${scan.findings.length - live.length ? `; ${scan.findings.length - live.length} dismissed as false positive by AI review` : ''}.`,
|
||||
`Scanners: ${Object.entries(scan.scanners).map(([k, v]) => `${k} ${v.status === 'ok' ? `${v.findings}` : `ERROR (${v.error})`}`).join(' · ') || '(running)'}`,
|
||||
...(scan.review?.summary ? [`AI review (${scan.review.model}): ${scan.review.summary}`] : scan.review?.error ? [`AI review failed: ${scan.review.error}`] : []),
|
||||
...(running ? [`Still ${p}; call security_scan {"action":"status","operation_id":"${scan.operationId}"} again.`] : []),
|
||||
'',
|
||||
]
|
||||
for (const [i, f] of shown.slice(0, max).entries()) {
|
||||
const pr = { critical: 'P0', high: 'P1', medium: 'P2', low: 'P3', info: 'P4' }[f.severity]
|
||||
lines.push(`${i + 1}. [${pr} ${f.severity}] ${f.title}`)
|
||||
lines.push(` ${f.file}${f.line ? `:${f.line}` : ''} · ${f.scanner}${f.alsoReportedBy ? `+${f.alsoReportedBy.join('+')}` : ''} ${f.rule}${f.cwe ? ` · ${f.cwe}` : ''} · id ${f.id}`)
|
||||
if (f.ai) lines.push(` AI: ${f.ai.verdict}${f.ai.reason ? ` — ${f.ai.reason}` : ''}`)
|
||||
const fix = f.ai?.fix ?? f.fix
|
||||
if (fix) lines.push(` Fix: ${String(fix).split('\n')[0].slice(0, 300)}`)
|
||||
if (f.validation) lines.push(` Validation: ${f.validation.status} — ${f.validation.summary}`)
|
||||
}
|
||||
if (shown.length > max) lines.push(`… ${shown.length - max} more (raise max_findings or min_severity). Full report: ${path.join(STORE, `${scan.scanId}.json`)}`)
|
||||
return lines.join('\n')
|
||||
}
|
||||
|
||||
async function runAction(args, exec) {
|
||||
switch (args.action) {
|
||||
case 'scanners':
|
||||
return Object.entries(SCANNERS).map(([id, s]) => `${s.available() ? '✓' : '✗'} ${id}: ${s.what}${s.available() ? '' : ` — install: ${s.install}`}`).join('\n')
|
||||
case 'preflight': return preflight(args, exec)
|
||||
case 'start': return start(args, exec)
|
||||
case 'status': {
|
||||
const op = ops.get(args.operation_id)
|
||||
const scan = op?.scan ?? load(args.scan_id ?? '') ?? [...ops.values()].findLast((o) => o.scan.scanId === args.scan_id)?.scan
|
||||
if (!scan) throw new Error('unknown operation_id/scan_id')
|
||||
if (op && args.wait_seconds) await Promise.race([op.promise, new Promise((r) => setTimeout(r, Math.min(args.wait_seconds, 240) * 1000))])
|
||||
return statusText(scan, args)
|
||||
}
|
||||
case 'cancel': {
|
||||
const op = ops.get(args.operation_id)
|
||||
if (!op || ['completed', 'partial', 'failed', 'cancelled'].includes(op.scan.phase)) return 'Nothing to cancel.'
|
||||
op.abort.abort()
|
||||
return `Cancelling ${args.operation_id}.`
|
||||
}
|
||||
case 'validate': {
|
||||
const scan = load(args.scan_id ?? '')
|
||||
if (!scan) throw new Error(`unknown scan ${args.scan_id}`)
|
||||
const f = scan.findings.find((x) => x.id === args.finding_id)
|
||||
if (!f) throw new Error(`unknown finding ${args.finding_id}`)
|
||||
if (!['unvalidated', 'validated', 'rejected', 'partial', 'error'].includes(args.validation_status)) throw new Error('validation_status must be unvalidated|validated|rejected|partial|error')
|
||||
if (!String(args.validation_summary ?? '').trim()) throw new Error('validation_summary is required')
|
||||
f.validation = { status: args.validation_status, summary: args.validation_summary, evidence: args.validation_evidence ?? [], at: new Date().toISOString() }
|
||||
save(scan)
|
||||
const live = ops.get(scan.operationId); if (live) Object.assign(live.scan, scan)
|
||||
return `Finding ${f.id} validation is now ${args.validation_status}.`
|
||||
}
|
||||
case 'list': {
|
||||
const files = fs.readdirSync(STORE).filter((f) => f.endsWith('.json')).sort().reverse().slice(0, 20)
|
||||
return files.map((f) => { const s = load(f.slice(0, -5)); return `${s.scanId} ${s.phase} ${s.kind} ${s.root} — ${s.findings.length} finding(s)` }).join('\n') || 'No scans yet.'
|
||||
}
|
||||
default: throw new Error(`unknown action ${args.action}`)
|
||||
}
|
||||
}
|
||||
|
||||
ctx.tools.register({
|
||||
name: 'security_scan',
|
||||
description: 'Plan and run security reviews of the workspace. 1) preflight: pin the scope — target_kind repository (default), scoped_path (+include_paths), working_tree (uncommitted changes) or ref_diff (base_revision..head_revision); exclude_paths; optional scanners subset and ai_review — returns a plan_id and which scanners apply (semgrep SAST, bandit for Python, gitleaks secrets, osv-scanner and npm audit for vulnerable dependencies). 2) start {plan_id}: runs scanners in the background, then a reviewer model triages findings (true/false positive, exploitability, fix) and adds missed issues; returns early results and an operation_id. 3) status {operation_id or scan_id, wait_seconds, min_severity, max_findings}: P0–P3 prioritized report. Also cancel, validate {scan_id, finding_id, validation_status, validation_summary} to record a verdict, list (past scans), scanners (installed tools).',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
additionalProperties: false,
|
||||
required: ['action'],
|
||||
properties: {
|
||||
action: { type: 'string', enum: ACTIONS },
|
||||
target_kind: { type: 'string', enum: ['repository', 'scoped_path', 'working_tree', 'ref_diff'] },
|
||||
include_paths: { type: 'array', items: { type: 'string' } },
|
||||
exclude_paths: { type: 'array', items: { type: 'string' } },
|
||||
base_revision: { type: 'string' },
|
||||
head_revision: { type: 'string' },
|
||||
scanners: { type: 'array', items: { type: 'string', enum: Object.keys(SCANNERS) }, description: 'Subset of scanners to run.' },
|
||||
ai_review: { type: 'boolean', description: 'Triage with a reviewer model (default from config).' },
|
||||
plan_id: { type: 'string' },
|
||||
operation_id: { type: 'string' },
|
||||
scan_id: { type: 'string' },
|
||||
wait_seconds: { type: 'integer', description: 'start/status: wait up to this long for completion (default 20 for start).' },
|
||||
min_severity: { type: 'string', enum: SEV_ORDER },
|
||||
max_findings: { type: 'integer' },
|
||||
finding_id: { type: 'string' },
|
||||
validation_status: { type: 'string', enum: ['unvalidated', 'validated', 'rejected', 'partial', 'error'] },
|
||||
validation_summary: { type: 'string' },
|
||||
validation_evidence: { type: 'array', items: { type: 'object', additionalProperties: false, required: ['label', 'explanation'], properties: { label: { type: 'string' }, explanation: { type: 'string' } } } },
|
||||
},
|
||||
},
|
||||
output: { schema: { type: 'object', additionalProperties: false, properties: { text: { type: 'string' } } }, render: (_a, v) => [{ type: 'text', text: v.text }] },
|
||||
timeoutMs: 300_000,
|
||||
async execute(args, exec) {
|
||||
return { text: await runAction(args ?? {}, exec) }
|
||||
},
|
||||
})
|
||||
void which
|
||||
}
|
||||
+164
@@ -0,0 +1,164 @@
|
||||
// Scanner adapters: each returns normalized findings
|
||||
// {id, scanner, rule, severity: critical|high|medium|low|info, title, file, line, endLine, snippet, cwe, fix, url}
|
||||
import { execFile } from 'node:child_process'
|
||||
import crypto from 'node:crypto'
|
||||
import fs from 'node:fs'
|
||||
import path from 'node:path'
|
||||
|
||||
const HOME = process.env.HOME || ''
|
||||
export function which(cmd) {
|
||||
for (const d of [...(process.env.PATH || '').split(path.delimiter), path.join(HOME, '.local/bin'), '/usr/local/bin']) {
|
||||
if (!d) continue
|
||||
const p = path.join(d, cmd)
|
||||
try { fs.accessSync(p, fs.constants.X_OK); return p } catch {}
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
export function run(bin, args, { cwd, signal, timeoutMs = 600_000, env } = {}) {
|
||||
return new Promise((resolve) => {
|
||||
execFile(bin, args, { cwd, signal, timeout: timeoutMs, maxBuffer: 256 * 1024 * 1024, encoding: 'utf8', env: { ...process.env, ...env } }, (error, stdout, stderr) => {
|
||||
resolve({ code: error ? (typeof error.code === 'number' ? error.code : -1) : 0, stdout: stdout ?? '', stderr: stderr ?? '', killed: !!error?.killed })
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
const fid = (...parts) => crypto.createHash('sha1').update(parts.join('|')).digest('hex').slice(0, 10)
|
||||
const sev = (s) => {
|
||||
const t = String(s ?? '').toLowerCase()
|
||||
if (/crit/.test(t)) return 'critical'
|
||||
if (/high|error/.test(t)) return 'high'
|
||||
if (/med|warn|moderate/.test(t)) return 'medium'
|
||||
if (/low|note/.test(t)) return 'low'
|
||||
return 'info'
|
||||
}
|
||||
|
||||
function hasFiles(root, exts, files) {
|
||||
if (files) return files.some((f) => exts.includes(path.extname(f)))
|
||||
try {
|
||||
const stack = [root]; let n = 0
|
||||
while (stack.length && n < 20000) {
|
||||
const d = stack.pop()
|
||||
for (const e of fs.readdirSync(d, { withFileTypes: true })) {
|
||||
if (e.name.startsWith('.') || e.name === 'node_modules' || e.name === 'venv' || e.name === 'dist') continue
|
||||
const p = path.join(d, e.name); n++
|
||||
if (e.isDirectory()) stack.push(p)
|
||||
else if (exts.includes(path.extname(e.name))) return true
|
||||
}
|
||||
}
|
||||
} catch {}
|
||||
return false
|
||||
}
|
||||
|
||||
export const SCANNERS = {
|
||||
semgrep: {
|
||||
what: 'SAST rules for 30+ languages (OWASP top 10, injection, crypto misuse, …)',
|
||||
install: 'pip install --user semgrep',
|
||||
available: () => !!which('semgrep'),
|
||||
applies: () => true,
|
||||
async scan({ root, files, signal, config }) {
|
||||
const args = ['scan', '--json', '--quiet', '--disable-version-check', '--metrics=off', '--timeout', '30', `--config=${config?.semgrepConfig ?? 'p/default'}`]
|
||||
for (const ex of config?.exclude ?? []) args.push('--exclude', ex)
|
||||
args.push(...(files?.length ? files : ['.']))
|
||||
const r = await run(which('semgrep'), args, { cwd: root, signal, env: { SEMGREP_SEND_METRICS: 'off' } })
|
||||
let j
|
||||
try { j = JSON.parse(r.stdout) } catch { throw new Error(`semgrep: ${r.stderr.split('\n').filter(Boolean).slice(-3).join(' ') || `exit ${r.code}`}`) }
|
||||
return (j.results ?? []).map((x) => ({
|
||||
id: fid('semgrep', x.check_id, x.path, x.start?.line),
|
||||
scanner: 'semgrep', rule: x.check_id, severity: sev(x.extra?.metadata?.impact ?? x.extra?.severity),
|
||||
title: x.extra?.message?.split('\n')[0] ?? x.check_id, file: x.path, line: x.start?.line, endLine: x.end?.line,
|
||||
snippet: x.extra?.lines, cwe: [].concat(x.extra?.metadata?.cwe ?? []).join(', ') || undefined,
|
||||
fix: x.extra?.fix, url: x.extra?.metadata?.source ?? x.extra?.metadata?.references?.[0],
|
||||
}))
|
||||
},
|
||||
},
|
||||
bandit: {
|
||||
what: 'Python security linter',
|
||||
install: 'pip install --user bandit',
|
||||
available: () => !!which('bandit'),
|
||||
applies: ({ root, files }) => hasFiles(root, ['.py'], files),
|
||||
async scan({ root, files, signal }) {
|
||||
const args = ['-f', 'json', '-q', ...(files?.length ? files.filter((f) => f.endsWith('.py')) : ['-r', '.', '-x', './node_modules,./.venv,./venv,./.git'])]
|
||||
const r = await run(which('bandit'), args, { cwd: root, signal })
|
||||
let j
|
||||
try { j = JSON.parse(r.stdout) } catch { throw new Error(`bandit: ${r.stderr.slice(-300) || `exit ${r.code}`}`) }
|
||||
return (j.results ?? []).map((x) => ({
|
||||
id: fid('bandit', x.test_id, x.filename, x.line_number),
|
||||
scanner: 'bandit', rule: `${x.test_id} ${x.test_name}`, severity: sev(x.issue_severity),
|
||||
confidence: x.issue_confidence?.toLowerCase(), title: x.issue_text, file: path.relative(root, path.resolve(root, x.filename)),
|
||||
line: x.line_number, endLine: x.line_range?.at(-1), snippet: x.code, cwe: x.issue_cwe?.id ? `CWE-${x.issue_cwe.id}` : undefined, url: x.more_info,
|
||||
}))
|
||||
},
|
||||
},
|
||||
gitleaks: {
|
||||
what: 'hardcoded secrets (API keys, tokens, private keys)',
|
||||
install: 'download from github.com/gitleaks/gitleaks/releases into ~/.local/bin',
|
||||
available: () => !!which('gitleaks'),
|
||||
applies: () => true,
|
||||
async scan({ root, signal, target }) {
|
||||
const report = path.join(fs.mkdtempSync(path.join(process.env.TMPDIR || '/tmp', 'gl-')), 'r.json')
|
||||
const isGit = fs.existsSync(path.join(root, '.git'))
|
||||
const args = target === 'repository' && isGit
|
||||
? ['git', '--no-banner', '--report-format', 'json', '--report-path', report, '--redact', '.']
|
||||
: ['dir', '--no-banner', '--report-format', 'json', '--report-path', report, '--redact', '.']
|
||||
await run(which('gitleaks'), args, { cwd: root, signal })
|
||||
let j = []
|
||||
try { j = JSON.parse(fs.readFileSync(report, 'utf8')) } catch {}
|
||||
fs.rmSync(path.dirname(report), { recursive: true, force: true })
|
||||
return j.map((x) => ({
|
||||
id: fid('gitleaks', x.RuleID, x.File, x.StartLine, x.Commit ?? ''),
|
||||
scanner: 'gitleaks', rule: x.RuleID, severity: 'high', title: `${x.Description}${x.Commit ? ` (commit ${x.Commit.slice(0, 8)})` : ''}`,
|
||||
file: x.File, line: x.StartLine, endLine: x.EndLine, snippet: x.Match, cwe: 'CWE-798',
|
||||
}))
|
||||
},
|
||||
},
|
||||
'osv-scanner': {
|
||||
what: 'known vulnerabilities in dependencies (lockfiles: npm, pnpm, yarn, pip, poetry, go, cargo, maven, …)',
|
||||
install: 'download from github.com/google/osv-scanner/releases into ~/.local/bin',
|
||||
available: () => !!which('osv-scanner'),
|
||||
applies: () => true,
|
||||
async scan({ root, signal }) {
|
||||
const r = await run(which('osv-scanner'), ['scan', 'source', '--format', 'json', '-r', '.'], { cwd: root, signal })
|
||||
let j
|
||||
try { j = JSON.parse(r.stdout) } catch { if (/no package sources found/i.test(r.stderr)) return []; throw new Error(`osv-scanner: ${r.stderr.split('\n').filter(Boolean).slice(-2).join(' ') || `exit ${r.code}`}`) }
|
||||
const out = []
|
||||
for (const res of j.results ?? []) {
|
||||
for (const pkg of res.packages ?? []) {
|
||||
for (const v of pkg.vulnerabilities ?? []) {
|
||||
const group = (pkg.groups ?? []).find((g) => g.ids?.includes(v.id))
|
||||
const score = Number(group?.max_severity)
|
||||
const severity = Number.isFinite(score) ? (score >= 9 ? 'critical' : score >= 7 ? 'high' : score >= 4 ? 'medium' : 'low') : sev(v.database_specific?.severity)
|
||||
const fixed = (v.affected ?? []).flatMap((a) => (a.ranges ?? []).flatMap((rg) => (rg.events ?? []).map((e) => e.fixed).filter(Boolean)))
|
||||
out.push({
|
||||
id: fid('osv', v.id, pkg.package?.name, res.source?.path),
|
||||
scanner: 'osv-scanner', rule: v.id, severity,
|
||||
title: `${pkg.package?.name}@${pkg.package?.version}: ${v.summary ?? v.id}`,
|
||||
file: path.relative(root, res.source?.path ?? ''), fix: fixed.length ? `upgrade to ${fixed[0]}` : undefined,
|
||||
url: `https://osv.dev/vulnerability/${v.id}`, cwe: (v.database_specific?.cwe_ids ?? []).join(', ') || undefined,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
},
|
||||
},
|
||||
'npm-audit': {
|
||||
what: 'npm advisory database for package-lock.json projects',
|
||||
install: 'bundled with npm',
|
||||
available: () => !!which('npm'),
|
||||
applies: ({ root }) => fs.existsSync(path.join(root, 'package-lock.json')),
|
||||
async scan({ root, signal }) {
|
||||
const r = await run(which('npm'), ['audit', '--json'], { cwd: root, signal })
|
||||
let j
|
||||
try { j = JSON.parse(r.stdout) } catch { return [] }
|
||||
return Object.values(j.vulnerabilities ?? {}).map((v) => ({
|
||||
id: fid('npm', v.name, v.range),
|
||||
scanner: 'npm-audit', rule: (v.via ?? []).map((x) => (typeof x === 'object' ? x.source ?? x.url : x)).join(', '),
|
||||
severity: sev(v.severity), title: `${v.name} ${v.range}: ${(v.via ?? []).map((x) => (typeof x === 'object' ? x.title : `via ${x}`)).join('; ')}`,
|
||||
file: 'package-lock.json', fix: v.fixAvailable ? (typeof v.fixAvailable === 'object' ? `upgrade ${v.fixAvailable.name} to ${v.fixAvailable.version}` : 'npm audit fix') : undefined,
|
||||
}))
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
export const SEV_ORDER = ['critical', 'high', 'medium', 'low', 'info']
|
||||
@@ -0,0 +1 @@
|
||||
{ "meta": { "title": "OMP Security Scan", "description": "security_scan: preflight/start/status scans with semgrep, bandit, gitleaks, osv-scanner, npm audit and AI triage; P0-P3 report." } }
|
||||
@@ -0,0 +1 @@
|
||||
{ "meta": { "title": "OMP 安全扫描", "description": "security_scan：使用 semgrep、bandit、gitleaks、osv-scanner、npm audit 和 AI 分诊进行扫描，输出 P0-P3 报告。" } }
|
||||
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"name": "@local/omp-security",
|
||||
"version": "1.0.0",
|
||||
"description": "oh-my-pi style security_scan tool for DSH: semgrep, bandit, gitleaks, osv-scanner and npm audit plus AI triage, with prioritized reports and finding validation.",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"license": "MIT",
|
||||
"exports": { ".": "./index.js", "./package.json": "./package.json", "./locale/*.json": "./locale/*.json" },
|
||||
"files": ["index.js", "lib/*.js", "cordis.patch.yml", "locale/*.json"],
|
||||
"scripts": { "test": "node --test test/*.test.mjs" },
|
||||
"dsh": { "bundle": { "patch": "./cordis.patch.yml" } }
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
import test from 'node:test'
|
||||
import assert from 'node:assert/strict'
|
||||
import fs from 'node:fs'
|
||||
import os from 'node:os'
|
||||
import path from 'node:path'
|
||||
import { execFileSync } from 'node:child_process'
|
||||
import * as plugin from '../index.js'
|
||||
import { SCANNERS } from '../lib/scanners.js'
|
||||
|
||||
function harness({ llmText } = {}) {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'omp-sec-'))
|
||||
const tools = new Map()
|
||||
const llm = {
|
||||
async *stream() { yield { type: 'text-delta', text: llmText ?? '{"triage":[],"new":[],"summary":"stub"}' }; yield { type: 'finish', reason: { kind: 'stop' } } },
|
||||
}
|
||||
plugin.apply({ tools: { register(d) { tools.set(d.name, d) } }, effect() {}, llm }, {})
|
||||
const exec = { agent: { session: { header: { cwd: dir }, requestHeader: () => ({ config: { provider: 'p', model: 'm' } }) } } }
|
||||
const call = async (args) => (await tools.get('security_scan').execute(args, exec)).text
|
||||
return { dir, call, w: (f, s) => { fs.mkdirSync(path.dirname(path.join(dir, f)), { recursive: true }); fs.writeFileSync(path.join(dir, f), s) } }
|
||||
}
|
||||
|
||||
const vulnPy = `import subprocess, pickle, hashlib\n\ndef run(cmd):\n return subprocess.call(cmd, shell=True)\n\ndef load(b):\n return pickle.loads(b)\n\ndef h(x):\n return hashlib.md5(x).hexdigest()\n`
|
||||
const secret = 'AWS_SECRET = "' + 'AKIA' + 'Z7QWERTYUIOPASDF' + '"\naws_secret_access_key = "' + 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY' + '"\n'
|
||||
|
||||
test('scanners action lists tools', async () => {
|
||||
const h = harness()
|
||||
assert.match(await h.call({ action: 'scanners' }), /semgrep/)
|
||||
})
|
||||
|
||||
test('preflight → start → status finds python issues and AI triage applies', { skip: !SCANNERS.bandit.available(), timeout: 240000 }, async () => {
|
||||
const llmText = JSON.stringify({ triage: [], new: [{ file: 'app.py', line: 4, severity: 'high', title: 'Command injection via shell=True', reason: 'cmd reaches a shell', fix: 'pass a list and shell=False', cwe: 'CWE-78' }], summary: 'Fix the shell injection first.' })
|
||||
const h = harness({ llmText })
|
||||
h.w('app.py', vulnPy)
|
||||
h.w('creds.py', secret)
|
||||
execFileSync('git', ['init', '-q'], { cwd: h.dir })
|
||||
const pf = await h.call({ action: 'preflight', target_kind: 'scoped_path', include_paths: ['app.py', 'creds.py'], scanners: ['bandit', 'gitleaks'] })
|
||||
const planId = /Plan (plan-\w+)/.exec(pf)[1]
|
||||
assert.match(pf, /✓ bandit/)
|
||||
const st = await h.call({ action: 'start', plan_id: planId, wait_seconds: 200 })
|
||||
assert.match(st, /: completed;|: partial;/, st)
|
||||
assert.match(st, /B602|subprocess.*shell/i, st)
|
||||
assert.match(st, /pickle/i)
|
||||
assert.match(st, /AI review \(p\/m\): Fix the shell injection first/)
|
||||
assert.match(st, /ai-review/)
|
||||
const scanId = /Security scan (scan-[\w-]+)/.exec(st)[1]
|
||||
const fidm = /id (\w{10})/.exec(st)[1]
|
||||
assert.match(await h.call({ action: 'validate', scan_id: scanId, finding_id: fidm, validation_status: 'validated', validation_summary: 'confirmed by test' }), /validated/)
|
||||
assert.match(await h.call({ action: 'status', scan_id: scanId }), /Validation: validated/)
|
||||
fs.rmSync(path.join(os.homedir(), '.dsh/security-scans', `${scanId}.json`), { force: true })
|
||||
})
|
||||
|
||||
test('osv-scanner flags a vulnerable lockfile', { skip: !SCANNERS['osv-scanner'].available(), timeout: 240000 }, async () => {
|
||||
const h = harness()
|
||||
h.w('requirements.txt', 'django==2.2.0\n')
|
||||
const pf = await h.call({ action: 'preflight', scanners: ['osv-scanner'], ai_review: false })
|
||||
const st = await h.call({ action: 'start', plan_id: /Plan (plan-\w+)/.exec(pf)[1], wait_seconds: 200 })
|
||||
if (/osv-scanner ERROR/.test(st)) return // offline: tolerated
|
||||
assert.match(st, /django@2\.2\.0/i, st)
|
||||
})
|
||||
Reference in new issue
Block a user