TECHDEV · Open-source tool
logtrim
Your build log is 40,000 lines. Your agent's context window is not. logtrim sits in a pipe and shrinks verbose logs and tool output down to a token budget — keeping the beginning, the end, and every error line, while collapsing the repetitive middle into one-line markers that say what was dropped. Zero dependencies, plain Node.js, works on stdin or a file.
Node.js 18+ · Zero dependencies · MIT
Before / after
The bundled examples/sample.log is a 62-line build log (715 estimated tokens) full of progress spam, repeated warnings, and one real failure:
node bin/logtrim.js examples/sample.log --stats --budget 300Starting build: my-app@1.4.2 (node v24.20.0)
Resolving dependencies...
Installing 184 packages...
Compiling module 1/184: utils
Compiling module 2/184: config
... [10 similar lines omitted] ...
warning: unused variable 'tmp' in src/reports.js:42
warning: unused variable 'tmp' in src/reports.js:42
... [6 identical lines omitted] ...
✗ billing: 3 failing
Error: Cannot find module './currency' in /app/src/billing/rates.js
at Module._resolveFilename (node:internal/modules/cjs/loader:1140:15)
...
ERROR: upload failed after 3 attempts: connection reset by peer
Build finished with errors (exit code 1)Result: 62 → 23 lines, 715 → 251 tokens, 41 lines omitted — and the two errors, the stack trace, and the start of the log all survive. With the default budget (2000 tokens) the same file goes to 48 lines / 559 tokens, dropping only the collapsed repeats. Full output is in examples/trimmed-example.txt.
Sample log and trimmed output copied exactly from the logtrim README ("Before / after").
Quick start
# needs Node.js 18+
git clone https://github.com/techdevfun/logtrim.git
cd logtrim
npm link # makes `logtrim` available everywhere
# or run it without installing
node bin/logtrim.js --help
# trim a failing build before pasting it to your agent
npm run build 2>&1 | logtrim --budget 1500
# trim a saved log file, see what was cut
logtrim --budget 800 --stats deploy.log
# from stdin, explicit
cat deploy.log | logtrim --budget 800Commands copied exactly from the logtrim README ("Install" + "Usage"). Exit codes: 0 ok · 1 input file not found / unreadable · 2 bad arguments.
Options
| Option | Default | What it does |
|---|---|---|
--budget N | 2000 | Token budget for the output (~1 token = 4 chars, estimated) |
--head N | 40 | Lines always kept from the start |
--tail N | 60 | Lines always kept from the end |
--max-line N | 500 | Truncate lines longer than N chars, with a note |
--no-errors | off | Don't prioritize error lines when over budget |
--no-dedupe | off | Don't collapse repeated / similar lines |
--stats | off | Print {inputLines, outputLines, inputTokens, outputTokens, omittedLines} as JSON to stderr |
--help, --version | Show help / version |
Options table copied exactly from the logtrim README ("Usage").
How trimming works
- Collapse blank runs — 5 blank lines become 1.
- Collapse identical runs — 8 copies of the same warning become 2 copies plus
... [6 identical lines omitted] .... - Collapse similar runs — 12
Processed N/1000 recordslines become 2 plus... [10 similar lines omitted] .... "Similar" means the same shape with only numbers differing. Error lines are never collapsed this way. - Truncate absurdly long lines — a 3 KB single-line JSON blob gets cut to
--max-linechars with a[truncated, N chars]note. - If still over budget — keep head + error lines + tail, drop the middle, and mark each gap with
... [N lines omitted to fit B-token budget] .... Middle lines go first so the start and end survive. - Last resort — if error lines alone exceed the budget, they are truncated to share it. Output never exceeds the budget.
Steps from the logtrim README ("How trimming works"), not a copied snippet.
Why not just head / tail?
head -100 cuts the error at the end. tail -100 cuts the context at the start. grep -i error cuts everything that explains why the error happened. logtrim keeps all three: the setup, the failure, and the aftermath — and tells you exactly how much it removed.
From the logtrim README ("Why not just head / tail?").
Limits
- Token counts are estimates (characters / 4). That is accurate enough to decide what to keep, and it keeps the tool dependency-free. If you need exact tokenizer counts, pipe through a real tokenizer first and pass the result as
--budget. - "Similar" collapsing is shape-based (same line with only numbers differing) — genuinely different lines that look alike can be collapsed together.
- Trimming is destructive: the dropped middle is gone unless you re-run with a larger budget. Always keep the original log if you need a full audit trail.
Limits from the logtrim README's own notes.
Roadmap — not shipped yet
--format jsonpassthrough: keep JSON log lines intact instead of truncating mid-object--keep PATTERN: user-supplied regex for lines that must survive--watch: follow a growing log file and re-trim on change
Roadmap copied from the logtrim README; all three items are still open upstream.
License
MIT — see LICENSE in the repository.
Related tools
More open-source tools from TECHDEV: sessguard, runverdict, gheist-guard, vetgate, context-diet, ci-failure-detective, agent-seatbelt, devsecops-os, env-doctor, skill-audit, testguard, breakcheck, keymap, memgraph, ggufit, leaseguard, factfence, jsonfold
Have a say
Vote for this tool, suggest a feature, or discuss it with other operators on the community board.