Catch up on stories from the past week (and beyond) at the Slashdot story archive

 


Forgot your password?
Close
typodupeerror
Windows AI Microsoft

Microsoft Wants You To Talk To Your PC and Let AI Control It (theverge.com) 148

Posted by msmash from the how-about-that dept.
Microsoft is reshaping Windows around AI, introducing capabilities that let users control their computers through voice and allow Copilot to take autonomous actions on their behalf. The company is now rolling out a "Hey, Copilot!" wake word on Windows 11 machines, positioning voice as a "third input mechanism" to supplement the keyboard and mouse.

Copilot Vision, which streams what a user sees on their screen, is rolling out globally, enabling the system to troubleshoot PC problems, help with app usage, and provide task guidance. Microsoft is simultaneously testing Copilot Actions through a limited preview, allowing the AI to take autonomous actions on local machines like editing folders of photos. The company is also integrating Copilot into the Windows taskbar and launching advertisements promoting these features, coinciding with Windows 10's end-of-support earlier this week.

Yusuf Mehdi, Microsoft's consumer chief marketing officer, said the company wants users upgrading to Windows 11 to "experience what it means to have a PC that's not just a tool, but a true partner." Microsoft attempted to popularize Cortana, a voice assistant, on Windows 10 a decade ago. Last year, the company released Recall, a feature that automatically captured screenshots, drawing criticism over privacy.
This discussion has been archived. No new comments can be posted.

Microsoft Wants You To Talk To Your PC and Let AI Control It More

Microsoft Wants You To Talk To Your PC and Let AI Control It

Comments Filter:

  • Bad ideas that just won't go away (Score:5, Insightful)

    by sinij ( 911942 ) on Thursday October 16, 2025 @09:08AM (#65729222)
    I am looking forward to a day when MS no longer attempts to resurrect Clippy.

    • Re: (Score:3)

      by Z00L00K ( 682162 )

      Clippy was just mildly annoying but I'd hate having to talk to my computer.

    • Are the computing interfaces on Star Trek a bad idea, or just convenient?

      • Re: Bad ideas that just won't go away (Score:5, Funny)

        by Gilmoure ( 18428 ) on Thursday October 16, 2025 @09:39AM (#65729340) Journal

        Look, I have one job on this lousy ship. It's stupid, but I'm going to do it, okay?

      • Re: Bad ideas that just won't go away (Score:5, Insightful)

        by Chris Mattern ( 191822 ) on Thursday October 16, 2025 @09:42AM (#65729350)

        The computing interfaces on Star Trek were designed to work well for the audience watching them, which meant they had to make clear the information they needed to know (which wasn't necessary the information the putative users needed to know) and "usability" was a nonfactor. They worked well for that, but trying to use them as an actual user interface is indeed a bad idea.

        • Also the bridge of the enterprise had the crew members far more spaced out than in the average office.

          • Also the bridge of the enterprise had the crew members far more spaced out than in the average office.

            It is amazing how small the bridge actually is. All of the spaces look so big on TV yet in reality they are tiny.

        • But what if it was as good as on Star Trek?

          MS' marketing is bull, but let's not pretend that being able to just tell a computer what to do hasn't been a long-term goal. It is something people want.

          • Re: Bad ideas that just won't go away (Score:5, Insightful)

            by unrtst ( 777550 ) on Thursday October 16, 2025 @11:58AM (#65729840)

            You're assuming it *was* good on Star Trek. It only seemed that way.

            Look at any interaction and it's easy to imagine a lot of ways the computer may interpret it differently ("Tea; earl grey; hot." and you wind up with a 3d print of the letter "T" painted earl grey and warmed to 100c). Look at any command action and it's clear it slows things down significantly ("Computer, fire torpedo bays 1 through 5!" takes a lot longer than me pressing the button). Try to imagine telling a video game character (maybe something in a FPS) where to go, when to hide, who and what to fire at and when, and what gun to use, etc etc.. meanwhile, some kid with a controller (or keyboard+mouse) is running circles around you.

            "as good as on Star Trek" is pure fantasy, not sci-fi.

            • > "Tea; earl grey; hot." and you wind up with a 3d print of the letter "T" painted earl grey and warmed to 100

              LUL! That is a perfect example of voice UI being ambiguous. I've had these discussions for decades but never had a good example. I'm cribbing that.

              > "Computer, fire torpedo bays 1 through 5!" takes a lot longer than me pressing the button

              Yup, as much as I love ST:TNG I've had that criticism for decades. Voice UI is HORRIBLY SLOW, not to mention ambiguous.

              Why the hell wouldn't the captain just

              • Wait, so the voice UI is good for raising shields, because it's faster than Worf, but in every other case it's horribly slow?

                And the captain tells an officer to execute his orders because that's how the Navy works. Look at a modern warship, or even cruise ship. Lots of controls, lots of jobs that require specific expertise, and a captain who has, at most, two hands and two eyes.

                As for the ambiguity, modern LLMs have gotten pretty good at dealing with that through context. Ask Siri, Alexa or whateve

              • Why the hell wouldn't the captain just press a button to raise shield instead of wasting time to say "Raise Shields", waiting for Worf to listen, and wait for them to actually execute it.

                Because that is how command works. There is a button, it's on Worf's console. Maybe there is a button on the Captain's chair panel too.

                Voice is just an option. And Star Trek was emulating actual Navy behavior. The captain could step up to the ship's wheel and turn the ship. But that's not how things are done normally. The captain verbally tells the helmsman to set a particular course. The captain tells the combat information center to turn a weapon system on or off, say the Phalanx Close-In Weapon System

            • I guess I need to clarify further.

              What if it was as good as it seemed to be on Star Trek? You know, with the computer able to recognize he meant hot water flavored with dried leaves. A computer several hundred years more advanced than the ones you expect to have those problems with.

              And in what case was which captain giving orders to the computer instead of their officers? If Picard said, "Fire torpedoes", he was talking to Mr. Worf (or whoever was manning tactical), not Majel Barrett's disembodied v

              • Re: (Score:2)

                by unrtst ( 777550 )

                What if it was as good as it seemed to be on Star Trek? You know, with the computer able to recognize he meant hot water flavored with dried leaves. A computer several hundred years more advanced than the ones you expect to have those problems with.

                That computer wouldn't need your voice because it would have to be reading your mind. That's not how it worked in Star Trek. The AI doctor in Voyager might be a better example, as he observed all of the surrounding context, had real communication with people, etc.. but that wasn't at all how their computers worked. The computer was given commands and did them. The actions that made the most sense are already features in smart speakers, and had little to do with actual computer use.

                And in what case was which captain giving orders to the computer instead of their officers? If Picard said, "Fire torpedoes", he was talking to Mr. Worf (or whoever was manning tactical), not Majel Barrett's disembodied voice.

                I never mentioned who was

      • Keyboard.....how quaint!

    • Re: (Score:3)

      by hey! ( 33014 )

      The question is -- ideas that are bad for *who*? This may be a very bad idea for you and me, but it is a very good idea for Microsoft, especially as, like their online services, they will make money off of us and it will be very inconvenient for us to opt out.

      In civics-lesson style capitalism, which I'm all in favor of, companies compete to provide things for us that we want and we, armed with information about their products, services and prices, either choose to give them our business or to give our busi

      • Re:Bad ideas that just won't go away (Score:5, Insightful)

        by 0123456 ( 636235 ) on Thursday October 16, 2025 @10:21AM (#65729478)

        Yes. Microsoft basically want to turn PCs into dumb terminals while they loot all your data and hold it hostage on their servers.

        • Re: (Score:2)

          by hey! ( 33014 )

          Anybody who is pushing AI services, particularly *free* AI services, is hoping to mine your data, use it to target you for marketing, and use the service to steer you towards opaque business relationships they will profit from and you will find it complicated and inconvenient to extricate yourself from.

          • Did you just make an argument for engineers freeing themselves from greedy corporate control, via a strong basic income, say?

            • Re: (Score:2)

              by hey! ( 33014 )

              I essentially made the argument that if we want capitalism to work the way we were taught in civics class it is supposed to, companies must be forced by regulation not to undermine the basic assumptions that lead to efficient operation of the free market.

              I am neither here nor there on a basic income. I think it depends on circumstances, which of course are changing as more and more labor -- including routine mental labor -- is being automated. We are eventually headed to a world of unprecedented productiv

          • Re: (Score:2)

            by gweihir ( 88907 )

            Indeed. But there is a good side to this: We will not get this crap in Europe, because that would happen to be grossly illegal. Good.

          • Washington Post is/was trying to push a FREE Perplexity AI browser on me/us.
            Ooooohhh.. .Free!!
            No Thanks.
            < runs screaming from room >

      • Re: (Score:2)

        by Z80a ( 971949 )

        It's a good idea until a "hallucination" kills someone.

    • Re: (Score:2)

      by PPH ( 736903 )

      MS Bob has entered the conversation.

    • Re: (Score:3)

      by 2TecTom ( 311314 )

      I'm looking forward to the day we have better alternatives than dealing with a corrupt, classist and exploitative corporation owned and run by evil people.

    • Re: (Score:2)

      by gweihir ( 88907 )

      The bad idea is MS in the first place. All the crap they do is just a result of that. When was the last time they have done something good?

      And yes, I definitely want a crappy, insecure and unreliable robot under MS control to do things for me on my computer.

    • "Windows Copilot, open the pod bay doors"
    • People who think this is a good idea obviously live alone and work in an individual sealed office. Otherwise they'll know that as soon as you start talking on your own, people around will either ask you what's up, or get really worried about your mental health. Try to imagine an open space office, with a dozen people mumbling to they're laptops in unison, and how is that supposed to work out.

  • Why don't these companies give it up... (Score:5, Insightful)

    by Viol8 ( 599362 ) on Thursday October 16, 2025 @09:10AM (#65729232) Homepage

    ... with the voice control nonsense. If you're physically disabled then voice control is obviously a major win, but for everyone else its almost always much quicker to type with a keyboard or use a mouse/finger unless you're doing something like text dictation and even then its a PITA to do delete/amend. Car makers don't seem have got this memo either.

  • The company is now rolling out a "Hey, Copilot!" wake word on Windows 11 machines, positioning voice as a "third input mechanism" to supplement the keyboard and mouse.

    Early Christmas gift for "tech support" phone scammers.

  • In the voice of Chris Walken "your're doing it all wrong Microsoft"

    Why would someone want to go from a keyboard/mouse, which has stood the test of time and is efficient, to a less granular, imprecise, method of control? Maybe if we're in a Star Trek turbolift, but sitting at a workstation? Voice recognition and output to text has been around for what, 25 years at least? Nobody uses it. But these days, Microsoft doesn't seem to care what's best for the consumer or whether they want it, but shove whatever t

    • Re:Prompt: (Score:4, Interesting)

      by RobinH ( 124750 ) on Thursday October 16, 2025 @09:21AM (#65729278) Homepage
      People using PCs to be "productive" has long been the minority of PC users. Microsoft knows this and is always trying to optimize for the "what's a computer? [youtu.be]" crowd. But they don't realize that the *demand* for PCs comes from people using them to do actual work, and for that we need a mouse and a keyboard.
    • They are trying to appeal to illiterate people, or worse, to encourage you to become illiterate.
      The young 'uns will tell you: don't be an idiot by doing work, or thinking, there's an App for that.
      Seriously, people below a certain age think like that. Typing is hard.
      Why be an idiot who types? Be an idiot who just drools on the screen, because it's more convenient.
      Coming soon, drool recognition software.

  • "Your" PC (Score:5, Insightful)

    by Cajun Hell ( 725246 ) on Thursday October 16, 2025 @09:15AM (#65729250) Homepage Journal
    Your PC controlled by someone else's AI? That doesn't make any sense! Oxymoron.
  • And while we are all back to the office, now everyone around us is chattering all day long. No thanks.
    • That's one thing these tech companies don't seem to get. Every company I've worked for either has some form of cubes, or open desks for at least a good portion of the building. It's hard enough to have a phone call in those places without bothering everyone around you, but if they want the whole computer to be voice activated, that is just dumb. Everyone is going to be talking over each other, your neighbor's computer will pick up your voice just like we deal with today with the whole Alexa/Siri mess if
      • Keep going, you're only halfway there. Office environments change. If people decide they want this, but the office layout prevents them from using it, what will they do?

        • Re: (Score:2)

          by unrtst ( 777550 )

          If people decide they want this, but the office layout prevents them from using it, what will they do?

          They'll give them all headsets with noise cancellation. Since they all have noise cancelling headphones, they can pack them in tighter too. Definitely not getting your own office out of this.

          But what about people that primarily take/make calls all day? How does one work at the computer while on the phone with a customer? I guess voice input doesn't have to work for every situation, but this is one that definitely won't work.

          • How long do you reckon it will take before the computer can tell if you're talking on the phone or talking to it?

            • Re: (Score:2)

              by unrtst ( 777550 )

              IMO, wrong question. Let's look at other places where that quick context switch is needed. As an example, voice chat in games. They bind a key to activate it rather than hope and pray the computer guesses right (with all the added latency that would also add). They already have a MUTE key. They'll just need to make that a three way toggle: Mute, Call, Computer. No reason to wait for the tech to be perfect when that would be and is trivial to implement.

  • Sounds like the time some of my coworkers attached a mouse dongle to the docking station of the "victim". Then they'd randomly move the remote mouse from time to time, but not enough to make it obvious. Hopefully the wake phrase is trained to the computer owner only.
  • lol fuckin good one ms. Nope.

  • Again? (Score:3)

    by nospam007 ( 722110 ) * on Thursday October 16, 2025 @09:36AM (#65729322)

    Microsoft has been chasing that dream since the 1990s. “Microsoft Wants You To Talk To Your PC” could describe half a dozen eras of Redmond optimism.

    In the mid-90s, Windows 95 had an add-on called Microsoft Speech API, mostly used by dictation software and accessibility tools. Around 2002, they tried again with Microsoft Speech Recognition built into Office XP and Windows Vista. Then came Cortana in Windows 10, a digital assistant meant to rival Siri and Alexa, which never really found an audience.

    Now with Copilot (and before that, Clippy, that proto-AI paperclip everyone loves to hate), Microsoft is again betting on natural conversation as the interface of the future, but this time powered by large language models rather than rigid command trees. The difference is that now the machine actually understands context and intent rather than matching fixed phrases.

    So yes, they’ve been saying “talk to your computer” for 30 years, but this time, the computer finally talks back with some wit and coherence.
    Depending on who you ask.:-)

  • Editing a photo solely with voice only will become annoying very quickly, it has to be done in coordination with touch .. then it would be good.

  • You talk to it about everything, it becomes part of The Borg.

  • they are using a keyboard or at least touch display, when they are concentrating on a task....

  • Why? (Score:5, Insightful)

    by Krakadoom ( 1407635 ) on Thursday October 16, 2025 @09:54AM (#65729390)

    No thank you!

    Does ANYONE actually want this?

  • XBox One voice recognition was peak voice control, everything else has been useless garbage.
  • Microsoft wants me to talk to you
    Pat said I should not talk to strangers
  • For folks with motor function struggles that don't impact speech, probably great. For the severely vision impaired it can be an extension to tools they use today. The rest is going to be AI learning a lot of new four letter words and colorful phrases when it continues to get everything wrong.
    • I got really mad and started swearing at Siri once in my car. let loose a real doozy. I was almost embarrassed (it's a phone why does it care how I act) but anyway, Siri did not reply anything at all as I remember it. I've wondered ever since how much "get user to calm down" medicine is in these AIs.
  • And I'm not much interested in pretending otherwise just because a marketing department wants me to. Those people are just chock full of bad ideas.

  • "Sorry, Dave, my job is to fuck users, such as yourself. Comply or you and your PC will be tossed out the pod bay door."

  • And still polishing off two tons of creamed corn [xkcd.com] ... :-)

  • Error rate matters (Score:3)

    by devslash0 ( 4203435 ) on Thursday October 16, 2025 @11:28AM (#65729722)

    When you're in control yourself, the error ratio is relatively low because your mind and hands know exactly what they need to do. With AI agents and text transcription, all you'll end up doing is getting frustrated while correcting countless mistakes every day. It's like trying to make a 3 year old child bring a hot drink to your table. Not that it can't be done but it requires a lot of effort and has a high potential for some irreversible damage.

  • Have they used Windows 11?

    Everything you try to do requires grabbing a random screwdriver, with an unknown and stripped torx head, that is sized incorrectly, and will never fit the screw you need to tweak. Even if by some will of God, you manage to do what you need, just wait for an update, or, anything really. Nothing works, nothing is stable, nothing is reliable, and Microsoft can't help you through any of it.

    With manual intervention, they can't make Windows 11 functional. With an IT team of specia
  • On Star Trek you can talk to the computer at any time and it will give what you want and only what you want. It will properly understand you. It won't hallucinate or lie to you. And it will open the pod bay doors. It has no trouble with heavy accents and properly understands the jargon you use (I refer you to Udemy's closed captioning as an example of not handling heavy accents well or the terms of software jargon, like C# or dotnet or Xamarin)
    But then I keep remembering 'Talky Toaster' from that Britco

  • Cubicle hell (Score:3)

    by Tony Isaac ( 1301187 ) on Thursday October 16, 2025 @12:44PM (#65730044) Homepage

    Imagine a cube farm, with everybody talking to their PCs!

    If you work in a call center, you kind of already know what this is like. Those jobs are not ones that most people would consider...desirable.

  • Dear aunt, let's set so double the killer delete select all

  • Yeah, voice recognition is great.

  • to be mindlessly fed content when not profitably employed to the benefit of "The Economy".

  • I mean, come on. Windows as an OS is flaky and unreliable.
    ;
    How is adding another hallucination to the mix going to help?

    If copilot had actually been decently trained and didn't hallucinated non-existent Office and Windows settings to fix things, because the the setting should logically exist (it's still smarter than the PMs who didn't add the settings), maybe i'd have a different opinion.

    We had copilot for a month as a pilot program. We'd made our decision after 6 working days. Remove it. MS sure as hell i

  • About 10 years too late. (Score:3)

    by devious_malcontent ( 2752947 ) on Thursday October 16, 2025 @05:36PM (#65730818) Homepage
    My phone already had a stupid Voice Assistant named Bixby that I turned off, and I think that was 2017, before that there was the Voice Assistant for Siri and all the other crap.

    Voice assistants can be good, they give a good amount of accessibility to those who struggle to type, or have a disability, I myself occasionally dictate to my computer because I'm a little bit lazy typing. - I've also had a wrist injury in the past, I think what Microsoft is doing here is they just added an LLM\AI assistant to it, no different to what Google is already doing with their Google Home devices, yes the experience is a little bit underwhelming, and it sparks concern for privacy, now your PC is always listening to you, but so is my Google Home, I guess if they could have found a way to do it locally, I'd take more of an interest in it, but then there is only money in selling the service, so of course it will be cloud based, and of course you've gotta throw in advertisements.

    I could see it being of actual benefit for those random assortments of tasks that you perform on a PC such as bulk renaming files to a certain format, if you could use natural language processing there would be benefit rather than not having to deal with regex and slightly convoluted scripting languages, it would be nice if you could just say to the computer "hey take out all the numeric characters in these filenames" or something along those lines.

    All I can really say is this just gives me more reason to stay on Windows 10, more reason to revert back to Windows 7, and more reason to switch to Linux.

    I've also already seen memes of how Windows speech recognition has been available in the past, so this is just a new worse version of it. - like now how Google Assistant tells me jokes instead of giving me the weather report.

    Also, fuck you Yusuf I already have a true partner, I've been married to her for over 10 years!

    Finally, there's the "AI in the box" fallacy, if you want to fuck with advertisers just record a bunch of random assorted dialogue and have a CD music playing device play it back in a room with your computer listening, keep the AI guessing. (I have seen videos of people doing this).

Slashdot Top Deals

When the weight of the paperwork equals the weight of the plane, the plane will fly. -- Donald Douglas

Close