Microsoft, one of the leading tech giants, has upgraded its Copilot Vision, its AI tool for Windows. The new AI tool will be able to view your entire screen and could observe as well as analyse two apps at a time. But with this latest update for Windows Insiders, the AI can now see any open app, browser or even your whole desktop. With this major update, Microsoft Copilot Vision aims to transform itself into a more intuitive and responsive AI assistant for users, which will have the capability to understand the full context of on-screen activity.
It brings a new level of personalised support for editing documents, exploring creative projects or just browsing the web. By giving users full control over what Copilot can see, Microsoft balances powerful functionality with privacy, a crucial step as AI becomes more integrated into our daily digital lives.
How does it work? Like screen sharing, but smarter
Unlike Microsoft’s Recall feature, which is used to automatically capture snapshots of your screen, Copilot Vision is activated manually by the users.
Users will have to simply click the glasses icon in the Copilot app and choose what they want it to see, which is similar to the screen sharing feature, during a video call. This gives users more control over what’s shared with the AI.
Smarter assistance in real time
Microsoft says Copilot Vision can now offer suggestions, analysing visual content and coaching users aloud.
No matter if you are editing a resume, working on a design project, or even learning a new game, the AI assistant will be able to guide you with improvements, insights and live feedback.
From web browsing to phone cameras
Copilot Vision was first tested as an Edge browser assistant, which could view and respond to web content. It now supports mobile phone cameras as well, which will enable the user to AI what your camera sees via AI, and answer questions about it. It is further expanding its real-world usefulness, says The Verge.
Microsoft has also started with the Copilot for Gaming, which comes with an upcoming AI-powered assistant designed specifically for Xbox gamers. According to a report, the feature is currently being tested within the company by Microsoft employees, ahead of a wider rollout expected shortly.