typesafe-computer-use
A native macOS computer-use loop built from local OCR, accessibility data, and bounded Jev actions.
What Jev decides
Which computer action, visible item, site, or off-screen accessibility control to use next.
The project reads the active Mac window deterministically, asks Jev which known action or target comes next, validates the action, and uses a separate writing model only when free text is required.
- Jev role
- Select the next bounded action and target from deterministic screen observations.
- Input
- The user goal, OCR blocks, accessibility controls, focused field, app, URL, time, and recent actions.
- Output
- Choice distributions for the action and target, plus Noul checks after generated text is entered.
- Decision frequency
- Once per computer-use step.
- Fallback
- Stop on low confidence, repeated no-ops, step limits, or missing permissions; free text uses a separate writer model.
How it works
Reported performance
Reported model cost per decision
0.0002 USD
Select one computer-use action from the same screenshot and goal.
Author reported · Source reviewedCaptured Sep 21, 2026
Reported model latency
0.13–0.38 s
One computer-use action decision.
Author reported · Source reviewedCaptured Sep 21, 2026
Overview
typesafe-computer-use combines ScreenCaptureKit, Apple Vision OCR, macOS accessibility, and deterministic action handlers. Jev selects among explicit actions and visible or accessible targets instead of receiving a screenshot and generating a plan.
The author reports low model cost and latency on a same-screen comparison. Those numbers are not an independent benchmark, and the repository documents important limits around icon-only controls, accessibility coverage, focus, and single-display capture.
- Language
- Python
- License
- MIT
- Author
- awlevin
Sources
Related projects
Distill
Route coding-agent work, reasoning effort, and context retention through bounded Jev decisions.
What Jev decides
Which configured model and effort should handle a call, what evidence deserves attention, whether another attempt is useful, and what context should be retained.
Jev-cu
Let Jev choose text-grounded computer-use actions while Codex reads and executes under local safety policy.
What Jev decides
Which visible text candidate and action to use next, whether the goal is complete, and whether the action is sensitive.
jev-model-router
Route Claude Code model tiers and reasoning effort with one parallel Jev classification.
What Jev decides
Which fast, balanced, or deep tier fits the work, how much reasoning it needs, and whether production risk requires escalation.