Windows 11 AI Agent Launch Sparks Safety Concerns
Microsoft is accelerating the push for AI across its ecosystem as Windows 11 update version 26220.7262 introduces the new AI Agent, a feature designed to enhance system intelligence and automation. What surprised many users was the strong warning from Microsoft itself highlighting that the feature may cause model hallucination and introduce new forms of security risks that should be carefully considered before switching it on.
One of the biggest security concerns Microsoft emphasized is the risk of cross prompt injection, a vulnerability where hidden malicious instructions inside files such as PDFs or user interface elements can manipulate the AI Agent into performing unintended actions. Examples include hidden commands that could make the agent extract sensitive data, install malware, or execute tasks without the user’s knowledge.
Despite the concerns, Microsoft confirms that the AI Agent will not be enabled by default. Users must manually turn it on, and once activated, the feature becomes active across all accounts on that device. Microsoft also marks the AI Agent as experimental, and a detailed risk warning will appear during setup to ensure users understand the potential threats.
The company states that the AI Agent follows three core principles: making all agent actions transparent and verifiable, maintaining the highest security standards for data protection, and requiring user approval for any command involving personal information. However, Microsoft admits that these principles serve as guidelines rather than guarantees, as no AI system today can fully eliminate security risks or prevent hallucination.
The toughest challenge lies with everyday users who may not fully grasp the risks of such complex vulnerabilities. This shifts the responsibility of choosing whether to accept the risks directly onto the user. The move also reflects Microsoft’s strategy to push forward with AI development despite the structural flaws still present in the technology, such as unpredictable hallucination or the possibility of manipulation through hidden prompts.
Industry analysts believe the urgency behind this release comes from intense competition in the AI market. If Microsoft does not rapidly expand AI capabilities in Windows, it may fall behind other tech giants racing to integrate similar features. However, launching a feature with known vulnerabilities signals a shift away from traditional safety-first development and has raised concerns among users and cybersecurity experts worldwide.
 origin: PCGamer
