AI Agents Are Coming for Your Computers

AI agents are no longer just passive tools—they’re becoming active decision-makers on our desktops. From automating workflows to managing tasks, these digital companions are stepping into roles once reserved for human operators. These bots are growing in sophistication and could soon take greater control of our computers, reshaping how we interact with technology. It’ll raise new questions about security, trust, and the balance between convenience and oversight.

Although not considered an agent by definition, Microsoft’s Recall was one of the earliest examples of artificial intelligence taking more active control of a laptop. The feature was touted as the equivalent of an on-device search engine, periodically snapping snapshots of your screen to help you find the content you need. Massive outcry over the exposed security flaws led to Microsoft not only making the feature off by default but also postponing the release multiple times.

However, such criticism hasn’t stopped others from toying with using AI agents to assume operational control of computers. In October, Anthropic released an upgraded Claude 3.5 Sonnet model capable of understanding and interacting with any desktop application. It can “imitate keystrokes, button clicks, and mouse gestures, essentially emulating a person sitting at a PC.” At least one security expert warns that Anthropic’s Computer Use functionality could be used to launch cyberattacks. I reached out to Anthropic to learn how it ensured Computer Use wouldn’t face the same blowback as Microsoft’s Recall feature but the company has yet to respond.

Motorola is another company that is experimenting with similar AI. Last month, it showcased AI capable of taking action on your behalf, opening the Uber app and requesting a ride. However, that example is a concept; Motorola states it won’t ship soon.

Google is also working on a project codenamed Jarvis that has somewhat similar functionality. Google offered a preview of what’s to come when it accidentally launched it to the Chrome store this week—it has since been removed. It would take over a web browser to accomplish tasks on a user’s behalf. Google’s Gemini AI would power it, and Project Jarvis could gather research, purchase a product, or even book a flight. The idea, like with Microsoft and Anthropic’s AI, is to automate away our everyday assignments.

After all, companies constantly remind us that AI agents aim to handle mundane, repetitive tasks, freeing us to focus on more creative pursuits.

Large Action Models (LAMs) form a crucial part of these features’ foundation. These AI systems interpret queries and respond by taking action. The most recognizable usage today is with the Rabbit R1 device. The hardware features a trainable AI agent that executes user requests. LAMs can be helpful across various applications, from the personal to the enterprise. We may see LAM-powered agents help us troubleshoot when our computers are broken, place an appointment on our calendar based on a recent email, or—like with what Google is doing—help book a flight.

From a business perspective, LAM agents can be used for customer service, marketing and sales, process automation, user interface and experience testing, and more. Salesforce is leveraging LAMs for its Agentforce platform. Shelby Heinecke, the company’s Senior AI Research Manager, tells me LAMs are what lie ahead for companies pursuing AI innovation:

“Large Language Models are basically Jack of all Trades. They’re great at everything. But as we go into this agent era, we need models that are still great at generating text, even different modalities. But we specifically need models that are experts at generating the next action. Absolutely, I think [LAMs] are the next phase.”

While we will see more AI agents capable of controlling our devices, we must not embrace such innovation blindly. They bring with them a unique set of benefits and risks. We must examine the potential vulnerabilities such agents might have if run on our computer, smartphone, or Internet of Things device.

Another point to consider is user readiness and the varying acceptance levels for AI agents in personal and business contexts. While some welcome these agents to offload repetitive tasks, others remain wary of relinquishing control. People want to be aware of an agent’s decision-making process. This highlights a crucial gap companies must address: making AI interactions transparent and allowing users to intervene when needed.

---

This Week’s AI News

🏭 AI Trends and Industry Impact

• What AI knows about you (Axios)
• VIDEO: AI on the frontlines: How technology is transforming health, defense, and beyond (CNN)

🤖 AI Models and Technologies

• Anthropic challenges OpenAI with affordable batch processing (VentureBeat)
• Meta unveils instant AI video generator that adds sound (The New York Times)
• Meet Mochi-1, the latest free and open-source AI video model (Tom’s Guide)
• Meta says it’s making its Llama models available for U.S. national security applications (TechCrunch)
• Hugging Face releases SmolLM2, bringing powerful models to the palm of your hand (VentureBeat)
• AMD releases OLMo, its first fully open 1B-parameter LLM series (My Two Cents)

✏️ Generative AI and Content Creation

• Generative AI has a massive e-waste problem (IEEE Spectrum)
• Google accidentally leaked a preview of its Jarvis AI that can take over computers (Engadget)
• How ChatGPT search paves the way for AI agents (MIT Technology Review)
• Despite its impressive output, generative AI doesn’t have a coherent understanding of the world (MIT News)
• Nvidia AI Blueprint makes it easy for any developer to build automated agents that analyze video (VentureBeat)

💰 Funding and Investments

• Amazon may invest further in Anthropic—as long as the OpenAI rival uses Amazon-developed silicon hosted on AWS to train its AI (TechCrunch)
• Perplexity reportedly in final stages of raising $500 million round at $9 billion valuation (CNBC)
• Robot AI specialist Physical Intelligence raises $400 million in financing from Jeff Bezos, OpenAI, and others (The New York Times)
• UnifyApps’ AI-powered app integration platform grabs $20 million from ICONIQ Growth (TechCrunch)
• AI startups will need “quality of revenue” to raise in 2025, seed investors warn (TechCrunch)

☁️ Enterprise AI Solutions

• ServiceNow advocates for “invisible” AI agents to ease worker adoption (VentureBeat)
• Microsoft unveils Magentic-One multi-agent infrastructure (VentureBeat)
• French health insurance company Alan unveils an AI health assistant for its 680,000 members (TechCrunch)
• SambaNova and Hugging Face make AI chatbot deployment easier with one-click integration (VentureBeat)
• Anthropic teams with Palantir and AWS to sell AI to defense customers (TechCrunch)
• Meet the startup that won the Pentagon’s first AI defense contract (VentureBeat)

⚙️ Hardware, Robotics, and Autonomous Systems

• Boston Dynamics’ latest videos show how the new Atlas robots handle practical tasks (IEEE Spectrum)
• Inside “Project Rodeo,” the Tesla effort pushing the limits of self-driving technology (Business Insider)
• Baidu reportedly getting ready to unveil AI-powered glasses to rival Meta’s Ray-Bans (Bloomberg)
• How Samsung fell behind in the AI boom leading to a $126 billion wipeout (CNBC)

🔬 Science and Breakthroughs

• How digital twins are transforming the world of water management (World Economic Forum)
• How AI is being used to respond to natural disasters in cities (TIME)
• Could AI help treat brain tumors in children? (BBC)
• The doctors pioneering the use of AI to improve outcomes for patients (Financial Times)

💼 Business, Marketing, Media, and Consumer Applications

• Nvidia passes Apple to become the world’s most valuable publicly traded company (CNBC)
• Salesforce to hire 1,000 salespeople as part of its AI agent push (Bloomberg)
• For some women building AI startups, finding community is the key to success (Business Insider)
• How General Mills found business value in generative AI (CIO Dive)
• Portrait of English mathematician Alan Turing created by humanoid robot fetches $1.3 million at auction (South China Morning Post)
• AI’s big legal showdown—and what it means for Doctor Strange’s hair (The Guardian)
• Gen Z, millennials are reportedly using AI for personal finance advice (CNBC)

⚖️ Legal, Regulatory, and Ethical Issues

• What Donald Trump’s election victory could mean for AI regulation (TechCrunch)
• AI startups and investors eagerly await less “red tape” under next Trump administration (Forbes)
• Ziff Davis study says AI firms rely on publisher data to train models (Axios)
• AI image generators need just 200 sample images to recreate an artist style perfectly—here’s how (Tom’s Guide)

🔎 Opinions, Analysis, and Editorials

• I’m the CEO of an AI company, and this is the BS behind AI (Fast Company)
• AI at maturity: Modern workflows in a post-computer era (OMERS Ventures)

🎧 Podcasts

• Alexa at 10: Amazon’s assistant is a winner and a failure (The Verge)

---

End Output

Thanks for reading. Be sure to subscribe so you don’t miss any future issues of this newsletter.

Did you miss any AI articles this week? Fret not; I’m curating the big stories in my Flipboard Magazine, “The AI Economy.”

Connect with me on LinkedIn and check out my blog to read more insights and thoughts on business and technology. 

Do you have a story you think would be a great fit for “The AI Economy”? Awesome! Shoot me a message – I’m all ears!

Until next time, stay curious!