What is a desktop AI agent?

You want an AI working on your files so you can stop feeding documents into a chat window and copying the answers back out. But installing a desktop app doesn't tell you where that work goes. Andrew Ng put the hesitation plainly in his June 12, 2026 letter, Agents on the Desktop: "I do not use existing commercial desktop agents for highly confidential tasks, since I'm uncomfortable with some of their data-retention policies, which are often buried in obscure legalese and might change overnight with a new model". You shouldn't need to hunt through legal pages to understand what happens to a folder you just connected.

The short answer

A desktop AI agent is an AI program that carries out tasks using your computer's files and apps. It chooses steps and uses tools to do the work, rather than stopping at a written answer. Desktop access does not guarantee local execution: its maker decides which parts run elsewhere and what gets saved.

That makes it useful for work already sitting in folders. You can ask it to turn a set of notes into a document without uploading each note yourself. Ng describes the category this way: "Desktop agents not only chat with you but also read and edit local files, read/send messages, and provide scheduled deliverables like a daily news summary." Those capabilities depend on the tools and permissions each app provides.

How a desktop AI agent does the work

The model chooses an action, a tool carries it out, and the result gives the model something to work from on its next step. Ng writes: "The software that wraps around the LLM to implement a desired agentic system is called the agent harness". That surrounding software can give the model direct file access, command execution or connections to other apps, including MCP connectors. It can also provide screen control: reading screenshots or an accessibility tree, the app's description of its controls, then clicking and typing. A desktop AI agent can use both approaches within one task. Our computer use agent explainer covers the screen and action loop in detail.

Desktop AI agent vs browser agent vs chatbot

The distinction is where the software can act. A plain chatbot returns an answer for you to use. A browser agent acts within webpages. A desktop AI agent reaches files and apps through desktop access. Those categories can overlap inside one product, so judge the access granted to the task you are running.

Type	Where it acts	What it can reach	Where task data sits
Plain chatbot	The conversation	What you supply in chat	The conversation and the provider's storage, according to its policy
Browser agent	A browser, local or remote	Websites and files you upload or explicitly connect	The browser environment and any connected services
Desktop AI agent	Connected files and desktop apps	Whatever its tools and permissions allow	Local storage, cloud services or both, depending on the app

A remote browser cannot simply open a folder on your computer. Uploading a file or connecting a local tool changes that boundary. The broader chatbot vs AI agent vs AI operating system comparison explains which part of the work each category leaves to you.

Does a desktop AI agent run locally?

Some parts can run locally while others run in the cloud. Treat "local" as a claim about a specific part of the task. Reading a file from your computer does not establish where the model processes its contents or where the app keeps a copy afterward.

Check these separately before handing over a working folder:

• Where the actions happen. Establish whether the agent edits files on your computer, in a remote workspace, or through a connection between them.
• Where the model thinks. A tool can open a local document and send its contents to a cloud model. A local model changes that processing location.
• Where memory and records live. Find out where the app stores conversation history, task logs and file copies, and what its deletion policy covers.

A local model alone doesn't make an app offline. Remote tools still need a network connection; an app can also have separate update or analytics traffic. Before handing over confidential files, verify which parts of the task need a network connection and what they send.

Who decides where it runs

The app's maker controls the available execution options. Anthropic's Cowork help page gives a concrete example: "On October 6, 2026, new Cowork tasks on Pro and Max plans run in the cloud. The Only on your computer option in Settings > General was removed." It adds: "There's nothing to set up, and you can't switch back."

Anthropic explains the storage boundary too: "When using Cowork in the cloud, Claude's work runs on Anthropic's servers instead of your computer, and your sessions and files are saved to your Claude account." Local folder access remains limited: "Claude reaches only the folders you've connected, through the desktop app, and only while it's open."

Tasks started on your computer before October 6 stay there. For new tasks that run on your computer, Anthropic points people to Claude Code in the desktop app. That distinction matters when deciding which mode fits your work.

A hosted service can change how it runs. You may like the resulting product, but a setting you relied on is still the vendor's decision. The case for owning agent capability starts with that dependency: the place where your agent works and keeps its history should be under your control.

Are desktop AI agents safe?

Safety depends on the access you grant and the boundaries the software enforces. An agent that can edit a folder can damage work in that folder. A mistaken action has consequences beyond a bad answer in chat, especially when the agent acts through an account you already use.

Anthropic's computer use security guidance recommends:

• "Using a dedicated virtual machine or container with minimal privileges to prevent direct system attacks or accidents."
• "Avoiding giving the model access to sensitive data, such as account login information, to prevent information theft."
• "Limiting internet access to an allowlist of domains to reduce exposure to malicious content."
• "Asking a human to confirm decisions that might result in meaningful real-world consequences".

Installing an agent alongside your everyday files and signed-in apps does not automatically provide that separation. Start with a copied working folder and grant access for the task at hand. Use approval mode for consequential actions and inspect the action log before widening access. Our guide to AI agent permissions explains why a written instruction to be careful cannot replace an enforced boundary.

Why the agent still feels like a guest

A desktop AI agent still works inside an operating system built around a person. Screen control lets it operate apps whose controls expect human judgment. Folder grants and approval prompts let you lend it access. You remain responsible for deciding what it can touch and checking whether the work is right.

The 2024 OSWorld paper by Xie and colleagues recorded the difficulty of operating those interfaces: "While humans can accomplish over 72.36% of the tasks, the best model achieves only 12.24% success, primarily struggling with GUI grounding and operational knowledge." These are the 2024 baseline results. Scores have risen since. The old result describes an early evaluation, not the capability of today's agents.

Microsoft is addressing agent access inside Windows itself. Its agentic security documentation says: "We're creating the ability for agents in Windows to operate with dedicated agent accounts distinct from the user account". It describes Copilot Actions as an experimental preview and states: "The applications and actions driven by Copilot Actions run under the agent account instead of the account of the signed-in PC user."

Microsoft defines an agent workspace as "A contained environment where agents can work in parallel with a human user, enabling runtime isolation and granular permissions." That puts part of the responsibility into Windows. The operating system can distinguish the agent's work from yours instead of treating both as actions by the same user.

Where ERIKA fits

eFreedom's one product, ERIKA, is an operating system whose user is purely AI; its design puts the agent's identity, memory, permissions, action records and recovery into the OS. Our AI operating system explainer defines that layer, and why ERIKA is an operating system, not an app explains the decision to build there. To see how the pieces fit together, start at the ERIKA page.

Frequently asked questions

What is a desktop AI agent?

A desktop AI agent carries out tasks through access to your computer's files and apps. It uses a model to choose steps and tools to perform them. Its desktop access does not establish whether the task, model or saved history stays local.

Is a desktop agent the same as a computer use agent?

The terms overlap. Computer use describes how an agent operates a computer interface, such as reading a screenshot and clicking a control. A desktop AI agent may use that approach, direct file tools or connections to apps. It need not perform every action through a screen.

Do desktop AI agents run locally?

Some do, and some split work between your computer and cloud services. Check the location of the actions, model processing and stored history separately. Access to a local folder does not mean every part of the task runs on your computer.

Are desktop AI agents safe?

They require limits appropriate to the work. Restrict folder access, keep sensitive account information away from the model, require approval for consequential actions and review what happened. A separate environment with limited permissions reduces exposure; the desktop label itself provides no safety guarantee.

What is the difference between a desktop agent and a browser agent?

A browser agent acts within webpages. A desktop AI agent can reach connected files and apps beyond the browser. A remote browser needs an upload or an explicit connection to reach local files, while a desktop app's reach depends on the permissions you grant.

Sources

• Agents on the Desktop, Andrew Ng, The Batch, June 12, 2026.
• Use Claude Cowork on web, desktop, and mobile, Anthropic Claude Help Center.
• Computer use tool, Anthropic Claude Platform Docs.
• Windows 11 security book: Agentic security, Microsoft Learn.
• OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments, Xie et al., arXiv 2404.07972, April 2024.