GhostHands
Let Jev pick GUI controls and execute input through real USB HID hardware.
What Jev decides
Which operation and screen element to use.
GhostHands pairs an indexed-control fast lane with a vision lane and sends mouse and keyboard actions through a Raspberry Pi Pico.
- Jev role
- Choose the next GUI operation and indexed control in the fast lane.
- Input
- The task and an indexed table of currently visible controls.
- Output
- A typed operation and element choice for the HID executor.
- Decision frequency
- For fast-lane GUI action steps.
- Fallback
- Screens without an indexed control table use the separate vision lane.
How it works
Overview
In the fast lane Jev chooses an operation and screen control from an indexed table with coordinates. A separate vision model handles screens without usable controls, while a Raspberry Pi Pico acts as a USB mouse and keyboard. This is a hardware-dependent GUI automation experiment; the vision lane uses other models rather than Jev alone.
- Language
- Python
- License
- MIT
- Author
- affirmitv
Sources
Related projects
typesafe-computer-use
A native macOS computer-use loop built from local OCR, accessibility data, and bounded Jev actions.
What Jev decides
Which computer action, visible item, site, or off-screen accessibility control to use next.
Jev Chat Assistant
An Android conversation copilot that uses Jev to interpret chats and rank reply candidates without sending for the user.
What Jev decides
The likely intent, conversation risk, reply need, best next action, unresolved tension, and which of three candidate replies fits best.
Distill
Route coding-agent work, reasoning effort, and context retention through bounded Jev decisions.
What Jev decides
Which configured model and effort should handle a call, what evidence deserves attention, whether another attempt is useful, and what context should be retained.