jev-desktop
Choose native desktop controls and compatible actions without placing the full accessibility tree in the calling agent context.
What Jev decides
Which operation and accessibility element fit the intent, whether the target is present, whether the action is destructive, and whether text is required.
The optional jev-desktop skill reads an agent-desktop snapshot, asks Jev for an operation and a target from compatible candidates, applies risk and confidence policy, executes through stable accessibility refs, and observes again.
- Jev role
- Choose one compatible native desktop action and target, then estimate presence, risk, and text requirements.
- Input
- The goal, an accessibility-tree skeleton or drilled region, compatible element actions, caller-supplied text availability, and recent steps.
- Output
- Choice results for operation and target plus Noul probabilities for presence, destructive risk, and text requirements.
- Decision frequency
- Once per step in the optional goal loop or once for a single-step resolution.
- Fallback
- Confirm under-threshold actions, abstain when the target is missing or ambiguous, stop when required text is unavailable, and validate operation compatibility in code.
How it works
Overview
jev-desktop is an optional Jev skill and script set inside the broader agent-desktop repository. agent-desktop itself is a Rust automation CLI and does not require Jev. The integration first reads a shallow native accessibility skeleton, can drill into truncated regions, and sends a bounded text description of controls to Jev. One request chooses an operation and a target while also judging whether the target is present, whether the action is destructive, and whether caller-supplied text is needed.
Only targets compatible with the chosen operation are accepted. Disabled and hidden elements are excluded, text must come from the caller, open sheets and menus take precedence, and destructive decisions require higher confidence. The repository skill documents offline tests; this directory did not run macOS desktop behavior or measure end-to-end latency.
- Language
- JavaScript and Rust
- License
- Apache-2.0
- Author
- lahfir
Sources
Related projects
typesafe-computer-use
A native macOS computer-use loop built from local OCR, accessibility data, and bounded Jev actions.
What Jev decides
Which computer action, visible item, site, or off-screen accessibility control to use next.
Jev Chat Assistant
An Android conversation copilot that uses Jev to interpret chats and rank reply candidates without sending for the user.
What Jev decides
The likely intent, conversation risk, reply need, best next action, unresolved tension, and which of three candidate replies fits best.
Distill
Route coding-agent work, reasoning effort, and context retention through bounded Jev decisions.
What Jev decides
Which configured model and effort should handle a call, what evidence deserves attention, whether another attempt is useful, and what context should be retained.