The Anti-LLM Terminal Assistant
Giovanni Blu Mitolo just shipped TERMy, a terminal assistant that feels like a deliberate middle finger to the entire transformer stack. No embeddings. No machine learning. No LLMs. Just ~1000 lines of Python doing deterministic NLU (Natural Language Understanding) that runs on a Raspberry Pi Zero and responds in milliseconds.
The pitch is simple: you shouldn't need trillions of parameters to translate "activate the virtual environment" into a shell command. And after burning two months on subsidized API calls to Copilot for trivial operations, Mitolo decided to prove it.
The Failed Transformer Detour
Before going deterministic, Mitolo tried the obvious path: train a small transformer at home. He constrained himself to a 2010s-era gaming rig upgraded with 16GB RAM, an NVIDIA GTX 1050 Ti (4GB VRAM), and an i7-4790K. He built a framework from scratch, started with 100-200M parameter models similar to NanoGPT, added flash attention, even experimented with Mamba architectures.
The results were "creepy if not outright scary." His models hallucinated responses like "He's not a member of the world. He can't believe anything anymore. All of those animals are looking like excrements..." (I'm cleaning up the original expletive). They looped endlessly, rarely answered technical questions consistently, and would have needed a month of continuous training to maybe work.
He tried pivoting to local models via Ollama—ornith:9b, mistral:7b, cogito:14b. They occasionally worked but were too slow and unreliable for interactive terminal use, especially with only 4GB VRAM.
The Deterministic Pivot
Mitolo's breakthrough came from remembering the blockchain hype cycle, when everyone wanted to shoehorn distributed ledgers into problems that didn't need them. He set three hard constraints:
- No embeddings
- No machine learning
- No LLMs
Then he built a dataset format called NDF 0.0 (NPC-Forge Dataset Format) that captures everything needed for terminal command translation in self-contained JSON objects:
{
"category": "linux_files",
"input": ["list files", "list files and directories"],
"tools": [{
"name": "run_in_terminal",
"arguments": {
"command": "ls -lah",
"explanation": "Lists the files in the current directory.",
"goal": "Display current directory contents",
"mode": "sync"
}
}],
"message": "Done",
"thinking": ["That is quite simple!", "This is boring..."],
"permission": "yolo"
}
Each object is an atom of knowledge—category, input variations, tool calls, permission gating, even internal "thinking" traces. Want to teach TERMy Docker commands? Drop dataset_docker.json in the dataset directory. Instant Matrix-style knowledge upload.
Template Matching With Variables
The clever bit is handling variable extraction. For queries like "create file test.txt", TERMy uses template structures with semantic tags:
{
"structure": [
[{"tag": "<||vocab_create||>", "type": "vocab", "required": true},
{"tag": "<||vocab_file||>", "type": "vocab", "required": false},
{"tag": "<||file||>", "type": "filename", "required": true}]
]
}
Each tag like <||vocab_create||> maps to synonyms: "create", "make", "generate", "craft", "forge". Named entities get extracted via regex patterns. Tags can be required or optional, order-flexible.
Mitolo credits his friend Kevin for thinking through this structure, and you can see the influence of his compiler work on BIPLAN (his own programming language) in the design.
The Five-Step Pipeline
The NLU pipeline prioritizes cheap operations first, escalating to more expensive matching only when needed:
- Strip noise — Remove expletives, interjections, encouragement, thanks
- Sentiment analysis — Understand tone/context
- Exact Match — Very fast string comparison
- Template Match — Slower, uses the tag structure above
- Probabilistic Match — Slowest, handles typos and variations
Step 5 relies on classical NLP techniques:
- IDF (Inverse Document Frequency) to identify rare, meaningful words
- BOW (Bag of Words) to handle word order inversions
- IDF-weighted Levenshtein distance to safely tolerate typos
No neural networks. No training loop. Just deterministic text processing that feels like compiler design applied to natural language.
Permission Gating as a First-Class Primitive
Here's where TERMy gets interesting from a safety perspective: permission gating is hardcoded into the dataset. Every potentially destructive command requires "permission": "ask". The dataset creator decides what's safe to auto-execute ("permission": "yolo") versus what needs confirmation.
This is inherently safer than letting an unpredictable LLM run wild with shell access. Bugs can still exist in the implementation or dataset, but the attack surface is dramatically smaller. You're not hoping a prompt injection doesn't trick GPT-4 into running rm -rf /—you've explicitly enumerated what operations are allowed and under what conditions.
Cross-Platform Determinism
Mitolo wrote identical implementations in Python (for local OS environments) and JavaScript (for browser tabs or Node.js), both around 1000 lines. The classes FlintParser and FlintNPC handle the dataset format and NLU pipeline.
The JavaScript version runs client-side in any browser. Zero server calls. Zero tokens billed. The responses are instant because there's no model inference—just string matching, regex extraction, and dictionary lookups.
How This Compares to Established NLU
Mitolo positions TERMy against frameworks like Rasa, NLP.js, and ChatScript. Rasa and NLP.js are heavy, rely on ML classifiers, and require training pipelines. ChatScript is massive with a notoriously steep learning curve.
NPC-Forge (the framework underneath TERMy) requires zero training, uses a flexible self-documenting data format, and is small enough to run on a microcontroller. It's not trying to be a general-purpose conversational AI—it's optimized for the specific task of command translation.
The Copilot Hybrid
The meta move: Mitolo connected TERMy to Copilot as a deterministic pre-filter. Simple terminal operations get handled instantly by TERMy without burning tokens. Complex queries fall through to Claude or GPT-4.
This is the first deterministic agent harness I've seen in the wild, and I suspect we'll see more of this pattern. Why pay for inference when you can enumerate common cases? The LLM becomes a fallback for edge cases, not the first line of defense.
What This Means for the Agent Stack
TERMy is a data point in the emerging realization that we've been over-applying transformers. Not every problem needs trillion-parameter models. Sometimes you need a lookup table, a regex parser, and a permission system.
The dataset format is legitimately elegant. Self-contained JSON objects with input variations, tool calls, thinking traces, and permission gates feel like a natural unit of composition. You could version-control these, diff them, merge them, audit them—all the things you can't easily do with learned model weights.
Mitolo's framework is open-source at github.com/gioblu/NPC-Forge. The demo videos show millisecond responses, typo tolerance, and context tracking that would cost dollars in API calls with commercial LLMs.
The big question: how far can you push deterministic approaches before you need the flexibility of learned models? TERMy suggests the boundary is much further out than the current AI hype cycle would have you believe.
For terminal automation, it's not even close. Deterministic wins.