Google Gemini tests Android screen automation for in‑app task agents

Letting a model like Gemini literally drive the phone UI is a small but meaningful shift from chatbots to embodied digital agents. Up to now, most mainstream assistants have been glorified intent routers—they parse a command and call a well‑defined API. Screen automation breaks that constraint: the model observes and manipulates arbitrary app interfaces much like a human would, from tapping buttons to filling forms. That dramatically expands the action space, but also muddies reliability and safety, since the agent is now operating in unstructured, ever‑changing environments.([finance.sina.com.cn](https://finance.sina.com.cn/tech/digi/2026-02-04/doc-inhkrtwh2936393.shtml))

For the AGI race, this is an early glimpse of “phone‑native” agents that can execute multi‑step workflows across commerce, mobility, and productivity apps without human micromanagement. If Google hardens this from lab feature to product, it ratchets up competitive pressure on Apple, Microsoft, OpenAI and others to offer similarly capable device‑level agents. The same underlying mechanisms—screen understanding, tool use, persistent goals—are exactly what frontier labs are experimenting with in more general agent frameworks. The big open questions are around oversight and liability: Google’s own warnings in the strings (“Gemini may make mistakes; you are responsible for its actions”) hint at unresolved product‑law interfaces that will become acute as agents touch payments, identity and sensitive communications.