# Linux Computer Use for Codex

`linux-computer-use` is a practical substitute for native Computer Use when Codex needs to inspect and operate a Linux desktop application. It implements a controlled screenshot-act-verify workflow with deterministic shell tooling.

It supports:

- local Linux X11 and XWayland desktop sessions;
- Ubuntu and other Linux guests controlled from a macOS Parallels host;
- desktop screenshots;
- visible-window discovery and geometry;
- window focus;
- keyboard input, mouse movement, clicking, and scrolling;
- screenshot-backed smoke testing and UI QA;
- explicit safety boundaries for sensitive or consequential actions.

This project does not emulate or replace OpenAI's security model, visual reasoning, or native Computer Use plugin. It provides the mechanical desktop-control layer that Codex can call through the shell and pairs it with a skill that requires visual verification.

## Project layout

```text
LinuxComputerUse/
├── README.md
├── install.sh
└── linux-computer-use/
    ├── SKILL.md
    ├── agents/
    │   └── openai.yaml
    ├── references/
    │   ├── backends.md
    │   ├── safety.md
    │   └── troubleshooting.md
    └── scripts/
        └── linux-computer-use
```

The `linux-computer-use` child directory is the installable Codex skill. The repository root contains human-facing documentation and installation support.

## Requirements

For local Linux:

- Bash;
- an active graphical session;
- `xdotool` and `wmctrl`;
- one screenshot utility: `gnome-screenshot`, `grim`, `spectacle`, `scrot`, or ImageMagick `import`.

For a Linux VM controlled from macOS:

- Parallels Desktop and `prlctl` on the host;
- Parallels Tools in the guest;
- a running graphical Linux session;
- `xdotool` and `wmctrl` installed in the guest.

Ubuntu dependency example:

```bash
sudo apt update
sudo apt install xdotool wmctrl gnome-screenshot
```

The Parallels backend captures screenshots with `prlctl capture`, so it does not require a guest screenshot utility.

## Installation

Install into the active Codex home:

```bash
./install.sh
```

With no arguments, `install.sh` uses `${CODEX_HOME:-$HOME/.codex}/skills`. Pass one or more skills directories to install elsewhere:

```bash
./install.sh "/path/to/first/skills" "/path/to/second/skills"
```

Use `--source "/path/to/skill"` to install another local skill directory. Existing installations are moved to a sibling `skill-backups` directory before replacement; set `SKILL_BACKUP_DIR` to choose another backup location. The installer contains no machine-specific source, destination, or backup paths.

Restart Codex after installation so a new session discovers the skill. Invoke it explicitly with `$linux-computer-use`, or ask Codex to inspect, drive, or visually test a Linux desktop application.

## Quick start: Parallels Linux VM

Set the driver path:

```bash
SKILLS_DIR="${CODEX_HOME:-$HOME/.codex}/skills"
LCU="$SKILLS_DIR/linux-computer-use/scripts/linux-computer-use"
VM_NAME="<vm-name-or-id>"
GUI_USER="<linux-gui-user>"
CAPTURE_DIR="${TMPDIR:-/tmp}"
```

Check the VM and graphical session:

```bash
"$LCU" \
  --backend parallels \
  --vm "$VM_NAME" \
  --user "$GUI_USER" \
  doctor
```

Capture and inspect the display:

```bash
"$LCU" \
  --backend parallels \
  --vm "$VM_NAME" \
  screenshot "$CAPTURE_DIR/linux-before.png"
```

If the VM display is idle and the capture is black, wake it and capture again:

```bash
"$LCU" --backend parallels --vm "$VM_NAME" --user "$GUI_USER" wake
```

If the desktop is awake but the intended app is behind another window, `cycle-window` sends one physical Alt+Tab through Parallels. Inspect a screenshot immediately afterward.

List windows:

```bash
"$LCU" \
  --backend parallels \
  --vm "$VM_NAME" \
  --user "$GUI_USER" \
  windows
```

Interact with the selected application one action at a time:

```bash
"$LCU" --backend parallels --vm "$VM_NAME" --user "$GUI_USER" focus "<window-title-or-class>"
"$LCU" --backend parallels --vm "$VM_NAME" --user "$GUI_USER" type "example text"
"$LCU" --backend parallels --vm "$VM_NAME" --user "$GUI_USER" key Return
"$LCU" --backend parallels --vm "$VM_NAME" screenshot "$CAPTURE_DIR/linux-after.png"
```

Codex should inspect both images and verify the expected text. A successful process exit is not visual proof.

## Quick start: local Linux

```bash
SKILLS_DIR="${CODEX_HOME:-$HOME/.codex}/skills"
LCU="$SKILLS_DIR/linux-computer-use/scripts/linux-computer-use"
"$LCU" --backend local doctor
"$LCU" --backend local screenshot "${TMPDIR:-/tmp}/linux-desktop.png"
"$LCU" --backend local windows
"$LCU" --backend local geometry "<window-title-or-class>"
```

Run Codex from a terminal belonging to the graphical session so it inherits `DISPLAY`, `XAUTHORITY`, `WAYLAND_DISPLAY`, and D-Bus environment values.

## Command reference

Global options must precede the command:

| Option | Purpose |
| --- | --- |
| `--backend auto\|local\|parallels` | Select the execution backend |
| `--vm NAME` | Select a Parallels VM |
| `--user USER` | Select the guest GUI user |
| `--display DISPLAY` | Override the X11 display |
| `--xauthority PATH` | Override the Xauthority file |
| `--delay MS` | Control typing delay |

| Command | Purpose |
| --- | --- |
| `doctor` | Show target, session environment, and tools |
| `wake` | Wake an idle display without typing text |
| `cycle-window` | Send one Alt+Tab and require visual verification |
| `screenshot /absolute/path.png` | Capture the current desktop |
| `windows` | List visible X11/XWayland windows |
| `active-window` | Show the active window |
| `focus TARGET` | Focus a title match or explicit ID |
| `geometry TARGET` | Print window coordinates and size |
| `move X Y` | Move the pointer |
| `click X Y [BUTTON]` | Move and click |
| `type TEXT` | Type ordinary non-secret text |
| `key KEY...` | Send xdotool key names |
| `scroll up\|down [COUNT]` | Send wheel events |

Run `linux-computer-use --help` for the authoritative CLI help.

## Safety model

The skill requires Codex to:

1. verify the target before input;
2. capture the display before and after each meaningful action;
3. avoid secrets in command arguments;
4. pause before irreversible or consequential actions unless already authorized;
5. stop if focus, layout, or visible state is unexpected;
6. report what changed and where the evidence was stored.

The driver intentionally does not install packages, start privileged input daemons, handle credentials, bypass confirmations, or perform OCR-driven clicking.

## Wayland limitations

Input and window discovery use X11 tools. They work with X11 and XWayland applications, but do not reliably control native Wayland-only windows.

System-wide Wayland input injection commonly requires privileged `uinput` access. This project deliberately avoids enabling it automatically. Prefer XWayland, application test APIs, accessibility interfaces, browser automation, or a future native Linux Computer Use implementation.

## Validation

Validate the skill metadata:

```bash
python3 \
  "<skill-creator-directory>/scripts/quick_validate.py" \
  ./linux-computer-use
```

Resolve `<skill-creator-directory>` from the active Codex installation before running validation.

Check the scripts:

```bash
bash -n ./install.sh
bash -n ./linux-computer-use/scripts/linux-computer-use
./linux-computer-use/scripts/linux-computer-use --help
```

For an end-to-end smoke test, run `doctor`, capture a screenshot, list windows, focus a harmless application, perform one reversible input action, and capture a second screenshot.

## Troubleshooting

The skill contains targeted guidance in:

- `references/backends.md` for setup and backend selection;
- `references/safety.md` for authorization and sensitive workflows;
- `references/troubleshooting.md` for display, focus, screenshot, scaling, and Parallels failures.

Start with `doctor`. Most failures come from selecting the wrong GUI user, missing X11 environment values, targeting a native Wayland window, or using stale coordinates.