Open AutoGLM Phone Agent Skill by [ara.so](https://ara.so) — Daily 2026 Skills collection. Open AutoGLM is an open source AI phone agent framework that enables natural language control of Android, HarmonyOS NEXT, and iOS devices. It uses the AutoGLM vision language model (9B parameters) to perceive screen content and execute multi step tasks like "open Meituan and search for nearby hot pot restaurants." Architecture Overview Model : AutoGLM Phone 9B (Chinese optimized) or AutoGLM Phone 9B Multilingual Device control : ADB (Android), HDC (HarmonyOS NEXT), WebDriverAgent (iOS) Model serving : vLLM or SGLang (self hosted) or BigModel/ModelScope API Input : Screenshot + task description → Output: structured action commands Installation Prerequisites Python 3.10+ ADB installed and in PATH (Android) or HDC (HarmonyOS) or WebDriverAgent (iOS) Android device with Developer Mode + USB Debugging enabled ADB Keyboard APK installed on Android device (for text input) Install the framework Verify ADB connection Model Deployment Options Option A: Third party API (Recommended for quick start) BigModel (ZhipuAI) ModelScope Option B: Self hosted with vLLM Option C: Self hosted with SGLang Verify deployment Expected output includes a <think ...</think block followed by <answer do(action="Launch", app="...") . If the chain of thought is very short or garbled, the model deployment has failed. Running the Agent Basic CLI usage Key CLI parameters Parameter Description Default base url Model service endpoint Required model Model name on server Required apikey API key for third party services None device type adb (Android) or hdc (HarmonyOS) adb device id Specific device serial number Auto detect Python API Usage Basic agent invocation Custom task with device selection Direct model API call (for testing/integration) Parsing model action output ADB Device Control Patterns Common ADB operations used by the agent Midscene.js Integration For JavaScript/TypeScript automation using AutoGLM: Remote ADB (WiFi Debugging) Common Action Types The AutoGLM model outputs structured actions: Action Description Example Launch Open an app do(action="Launch", app="微信") Tap Tap screen element do(action="Tap", element="搜索框") Type Input text do(action="Type", text="火锅") Swipe Scroll/swipe do(action="Swipe", direction="up") Back Press back button do(action="Back") Home Go to home screen do(action="Home") Finish Task complete do(action="Finish", result="已完成搜索") Model Selection Guide Model Use Case Languages AutoGLM Phone 9B Chinese apps (WeChat, Taobao, Meituan) Chinese optimized AutoGLM Phone 9B Multilingual International apps, mixed content Chinese + English + others HuggingFace: zai org/AutoGLM Phone 9B / zai org/AutoGLM Phone 9B Multilingual ModelScope: ZhipuAI/AutoGLM Phone 9B / ZhipuAI/AutoGLM Phone 9B Multilingual Environment Variables Reference Troubleshooting Model output is garbled or very short chain of thought Cause : Incorrect vLLM/SGLang startup parameters. Fix : Ensure chat template content format string (vLLM) and mm process config with max pixels:5000000 are set. Check transformers version compatibility. adb devices shows no devices Fix : 1. Verify USB cable supports data transfer (not charge only) 2. Accept "Allow USB debugging" dialog on phone 3. Try adb kill server && adb start server 4. Some devices require reboot after enabling developer options Text input not working on Android Fix : ADB Keyboard must be installed AND enabled: Agent stuck in a loop Cause : Model cannot identify a path to complete the task. Fix : The framework includes sensitive operation confirmation — ensure confirm sensitive=True for purchase/delete tasks. For login/CAPTCHA screens, the agent supports human takeover. vLLM CUDA out of memory Fix : AutoGLM Phone 9B requires ~20GB VRAM. Use tensor parallel size 2 for multi GPU, or use the API service instead. Connection refused to model server Fix : Check firewall rules. For remote server: HDC device not recognized (HarmonyOS) Fix : HarmonyOS NEXT (not earlier versions) is required. Enable developer mode in Settings → About → Version Number (tap 10 times rapidly). iOS Setup For iPhone automation, see the dedicated setup guide:

open-autoglm-phone-agent