PalmClaw: A Native On-Device Agent Framework for Mobile Phones

Overview§

PalmClaw is an open-source framework that runs the entire agent loop (session management, memory, skill/tool execution) directly on a mobile device. Unlike prior GUI-only agents (e.g., AppAgent, DigiRL) that operate via screen coordinates and gestures, PalmClaw exposes device APIs as structured tools with typed arguments, return values, and explicit execution boundaries. This allows the LLM to invoke capabilities (e.g., send SMS, query contacts, set alarm) as discrete, verifiable actions.

Core Methodology§

The agent follows a standard ReAct loop but with a key difference: each tool call is defined as a function with a schema (name, description, parameters, returns). The LLM outputs JSON tool calls, which are executed on-device via platform-specific APIs (e.g., Android Intent or iOS SiriKit). PalmClaw enforces execution boundaries at both the action level (each tool is atomic) and session level (state is persisted locally).

Tool Definition§

Tools are defined using a decorator pattern:

# Example: defining a device tool
@device_tool(
    name="send_sms",
    description="Send an SMS message to a given phone number",
    parameters={
        "phone_number": {"type": "string", "description": "Recipient phone number"},
        "message": {"type": "string", "description": "Message content"}
    },
    returns={"type": "string", "description": "Message ID on success or error"}
)
def send_sms(phone_number: str, message: str) -> str:
    # Platform-specific implementation (e.g., Android SmsManager)
    return sms_manager.send(phone_number, message)

Agent Loop§

PalmClaw's agent loop is implemented as a state machine:

\begin{algorithm}[H] \caption{PalmClaw Agent Loop} \begin{algorithmic} \State \textbf{Input:} User request $R$, tool set $T$ \State $S \gets \text{new session}()$ \State $H \gets \text{load history}(S)$ \Repeat \State $A \gets \text{LLM}(R, H, T)$ \Comment{LLM generates action (JSON)} \If{$A$ is tool call} \State $O \gets \text{execute}(A, T)$ \Comment{atomic execution} \State $H \gets H \cup \{(A, O)\}$ \ElsIf{$A$ is final answer} \State \textbf{return} $A.\text{answer}$ \EndIf \Until{$A$ is final or max steps exceeded} \end{algorithmic} \end{algorithm}

Execution Boundaries§

Each tool defines:

• Precondition: device state required (e.g., network available, permission granted)
• Atomicity: tool completes entirely or rolls back
• Postcondition: expected state change (e.g., contacts.created)

This prevents the LLM from generating ambiguous action sequences common in GUI agents (e.g., swipe then guess location).

Key Results§

On a benchmark of 50 mobile tasks (e.g., "Send a text saying 'I'm on my way' to Mom"), PalmClaw achieves:

• Task success: 72.4% vs. 64.9% for best GUI agent (AppAgent) – relative improvement of 11.5%
• Completion time: median 8.2s vs. 160s – reduction of 94.9%
• Steps per task: 3.1 vs. 12.4

The dramatic time reduction comes from avoiding GUI latency (app loading, gesture sequences) and using direct API calls.

Implementation Details§

PalmClaw is built for Android using:

• LLM backbone: Gemma-2B-IT on-device via MediaPipe (quantized INT4)
• Tool registry: YAML schemas parsed at runtime
• Memory: SQLite for chat history and tool call traces
• Session persistence: SharedPreferences with encryption

The framework is open-source and supports custom tool addition via decorators. Future work includes multi-app workflows and privacy-preserving caching.

Equation Example§

Tool execution success probability $P$ given precondition $C$:

$$P(\text{success} \mid C) = \frac{\# \text{tool completions with }C}{\# \text{invocations with }C}$$

PalmClaw logs these to refine tool definitions automatically.