Become a fan of Slashdot on Facebook

 


Forgot your password?
Close
typodupeerror
×
AI Technology

ChatGPT 'Not Particularly Innovative' and 'Nothing Revolutionary', Says Meta's Chief AI Scientist (zdnet.com) 106

Posted by msmash from the how-about-that dept.
Much ink has been spilled of late about the tremendous promise of OpenAI's ChatGPT program for generating natural-language utterances in response to human prompts. The program strikes many people as so fresh and intriguing that ChatGPT must be unique in the universe. Scholars of AI beg to differ. From a report: "In terms of underlying techniques, ChatGPT is not particularly innovative," said Yann LeCun, Meta's chief AI scientist, in a small gathering of press and executives on Zoom last week. "It's nothing revolutionary, although that's the way it's perceived in the public," said LeCun. "It's just that, you know, it's well put together, it's nicely done." Such data-driven AI systems have been built in the past by many companies and research labs, said LeCun. The idea of OpenAI being alone in its type of work is inaccurate, he said.

"OpenAI is not particularly an advance compared to the other labs, at all," said LeCun. "It's not only just Google and Meta, but there are half a dozen startups that basically have very similar technology to it," added LeCun. "I don't want to say it's not rocket science, but it's really shared, there's no secret behind it, if you will." LeCun noted the many ways in which ChatGPT, and the program upon which it builds, OpenAI's GPT-3, is composed of multiple pieces of technology developed over many years by many parties. "You have to realize, ChatGPT uses Transformer architectures that are pre-trained in this self-supervised manner," observed LeCun. "Self-supervised-learning is something I've been advocating for a long time, even before OpenAI existed," he said. "Transformers is a Google invention," noted LeCun, referring to the language neural net unveiled by Google in 2017, which has become the basis for a vast array of language programs, including GPT-3. The work on such language programs goes back decades, said LeCun.
This discussion has been archived. No new comments can be posted.

ChatGPT 'Not Particularly Innovative' and 'Nothing Revolutionary', Says Meta's Chief AI Scientist More

ChatGPT 'Not Particularly Innovative' and 'Nothing Revolutionary', Says Meta's Chief AI Scientist

Comments Filter:

  • Unless we build it... (Score:4, Insightful)

    by dark.nebulae ( 3950923 ) on Tuesday January 24, 2023 @09:49AM (#63235312)

    Only then would it be cool and must have, just like our totally awesome Metaspace...

  • Duh (Score:5, Insightful)

    by battingly ( 5065477 ) on Tuesday January 24, 2023 @09:52AM (#63235326)
    Finding an engaging way to present new technology is often a key breakthrough. And yes, technology advances by building on the work of others.

    • Re:Duh (Score:4, Insightful)

      by JaredOfEuropa ( 526365 ) on Tuesday January 24, 2023 @10:01AM (#63235362) Journal
      Exactly, and ChatGPT was the first opportunity for many to interact with this technology themselves, and see what it is capable of. That in itself is pretty revolutionary.
      • Played around with it yesterday and its very accessible (well, when its actually accessible). Was able to answer several queries quickly that would have taken slightly more searching on Google--mostly on cooking and shopping lists. And I've seen Youtube videos where it literally writes articles based on a few key words and some ideas. I have to think half the blog articles I've read in the last month were spit out from it.

        I can't judge how advanced it actually is, but its useful. What exactly has Met

        • Re: (Score:3)

          by Rei ( 128717 )

          So long as you keep in mind that it'll occasionally (~10% of the time in my experience for reasonably complex queries) write confident answers that are entirely wrong, I find it useful. There's a lot of things you don't need perfect accuracy on (for example, on subject matter that you know enough to tell if it's wrong, or things where being wrong is fine or would become immediately clear without adverse consequences)

    • Re:Duh (Score:4, Insightful)

      by timeOday ( 582209 ) on Tuesday January 24, 2023 @10:32AM (#63235452)
      What's telling is how GPT3 didn't really take off until ChatGPT came along. In terms of what LeCun is calling "underlying technique," ChatGPT is a minor embellishment on GPT3. And yet, some threshold clearly was crossed.

      If there's anything the evolution of tech has taught us, it's that mastering 99% of the "underlying technique" guarantees little to nothing. Insert Xerox PARC, Nomad, MySpace, Slashdot...

  • Killer app (Score:4, Funny)

    by fleeped ( 1945926 ) on Tuesday January 24, 2023 @09:54AM (#63235340)
    Hurts when other make it, eh? Unlike the void in Meta :)

  • I wonder how hard the layoffs hit his group.

  • He's implying they have something equivalent, yet if he had they would have loved to have gotten the notoriety from it.

    Meaning this is plain ol' envy. "It's OK I guess" in more words.

  • The innovative part was the scale. Nobody built a system with such a large training set before or if they did the public did not get to play with it.

    That is a big deal. I would say its a big deal in the way MySpace and Facebook let the average smuck have something like 'their own website' was a big deal.

    • Exactly. Its really dumb take. Reminds me of takes about the ipod or the iphone. Its the complete package of lots of work that has been going on for a long time by many different groups and no part of it is really new. But man, you make money by packaging these things together into an easily used interface. Thats who gets recognized. Why doesn't Google/Meta have things like it already? Its not because they are afraid of making things up ( which lets be honest they do a billion times a day). Meta is focused

      • Re: (Score:2)

        by gtall ( 79522 )

        Google probably didn't field anything similar because they couldn't figure out how it would impact their cash flow. MS has nothing lose in investing in it because they haven't had a new idea in...well...they've never had a new idea, come to it. They just saw Google making money, it burns them to see other companies do well without MS's alleged software.

      • > They likely had it, then preemptively killed it.

        Of the 8 authors of the transformer paper, only one is still with Google today. The rest are working on their own startups. Probably they felt Google wasn't the best place to develop it.

    • I don't think that qualifies an "innovative". Throwing money at something is not innovation.

  • ChatGPT said the same thing about Yann LeCun.

    • Copying my communication with ChatGPT:

      explain why is Yann LeCun wrong
      I am not aware of any specific statement or belief that Yann LeCun, a well-respected computer scientist and AI researcher, is currently considered to be wrong about. Could you please provide more context or specify which statement or belief you are referring to?

      Yann LeCun claims that "ChatGPT is not particularly innovative". Is he correct?
      ChatGPT is a variant of the GPT (Generative Pre-training Transformer) model, which is a type of transf

  • Insults? Really? (Score:3)

    by zarmanto ( 884704 ) on Tuesday January 24, 2023 @10:12AM (#63235386) Journal

    I'm not completely sure if this is merely conventional wisdom at this point... but it seems to me that insults almost invariably say more about the person issuing them than they do about the target of their disdain.

    • The only speaking of insults or disdain so far is you. It isn't in the /. headline, nor in the story, nor in any of the topics.

      The article itself is quite accurate, this is a small iteration on existing technology. It doesn't insult anyone. The "innovation" was having the right amount of polish to go viral, getting lucky with a blend of factors that people loved.

      As a technical product, the pieces of tech behind it range from 2 to 25 years old, all of them iterations of past work. At it's core it is a high

  • If it's so easy, Yann, where's your model?

  • Much the same could have been said about the iPhone upon its introduction:

    "In terms of underlying [technology], [the iPhone] is not particularly innovative," said every phone manufacturer that was about to have their asses handed to them. "It's nothing revolutionary, although that's the way it's perceived in the public," they said. "It's just that, you know, it's well put together, it's nicely done."

    If he can't understand that "well put together" and "nicely done" can count as revolutionary, or that i

  • Sour grapes. That could completely change in a year or two, but this obvious product probably could have been released a year earlier with the same features by the likes of Google or Meta. It's making Nadella look like a genius for capturing the product without encumbering it with the Microsoft brand.

    LeCun might as well have said that the Meta C-Suite is braindead for wasting all of the research that his teams have done. Google even more so. Both companies are effectively monopolies in their spaces and

    • Google's only innovation: faster search results with fewer ads. I was at a competing search company at the time they launched and we got crushed because our front page was crapped up with numerous slow loading ads while google popped up instantly. This was critically important to their success but they've done nothing new in-house since. All purchased.

      Facebook innovation: nothing public facing I can think of. They did some nice back end work in several areas that users don't directly see. But no end us

      • The framework used by most AI research today come from FB - PyTorch. Almost any new model is in PyTorch first.

        • I consider that backend work but ok if you work in the field then yes that's important work. FB released a ton of great back end infrastructure projects, too, which I made good use of back in the day. But my consumer-customers had no idea I was using that stuff when pretty things appeared on their screens at home.

          In the context of the discussion where 'search' was one of the things we're talking about I was thinking in terms of consumer facing final product. Chatgpt mostly falls into that category, as we

  • Pressure from the inside? (Score:3)

    by itamblyn ( 867415 ) on Tuesday January 24, 2023 @10:26AM (#63235416) Homepage
    Yann LeCun has been railing against chatGPT enough that I suspect he is getting a lot of pressure from inside META about why they didn't launch something like this. The sad thing is that they DID launch https://galactica.org/ [galactica.org] but then pulled access to it almost immediately after.

  • This seems out of character (Score:3)

    by PacoSuarez ( 530275 ) on Tuesday January 24, 2023 @10:29AM (#63235434)
    I have followed Yann LeCun for a long time, and these comments seem out of character. In Machine Learning there is a long history of clever implementations of old ideas propelling the field forward. In this case he sounds a bit like Jurgen Schmidhuber, complaining about a recent success not being innovative and not giving proper credit to people who couldn't get similar ideas to work decades earlier. I hope Yann doesn't slip further in that direction.

    • I have followed Yann LeCun for a long time, and these comments seem out of character. In Machine Learning there is a long history of clever implementations of old ideas propelling the field forward. In this case he sounds a bit like Jurgen Schmidhuber, complaining about a recent success not being innovative and not giving proper credit to people who couldn't get similar ideas to work decades earlier. I hope Yann doesn't slip further in that direction.

      I think what he's trying to say is that a lot of the press has framed ChatGTP itself as a fundamental AI breakthrough when it's better understood as a well engineered integration of several existing technologies.

      I don't think he's meaning to come across as sour grapes, but it's one of those things that's difficult to communicate without sounding bitter or dismissive. At the same time he's seen all these people in the field come along and make fairly important contributions to natural language models, and th

      • Re: (Score:3)

        by ceoyoyo ( 59147 )

        Additionally, LeCun has lived through a couple iterations of somebody putting together an impressive AI demo, promising the moon, and discrediting the entire field.

        All the posters saying things like "it's just Eliza!" are idiots, but they're also products of previous hype. It's a bit incredible that a language model can do as much as they do, but there are also good arguments that learning solely from language has pretty big limitations.

      • Re: (Score:2)

        by olau ( 314197 )

        And he's not writing the headline. You need to separate what he actually said in a certain context, with the media distorting the story in the hunt for the most dramatic angle.

  • He's pretty much right -- using a corpus of existing language samples to decide which word to output next has been around for a long time.

    • Oh, you mean we have had Markov Word Salad for a long time? I want the GPT stake.

      • Re: (Score:1)

        by Shag ( 3737 )

        Markov word salad happens when you have a tiny corpus to calculate your next-word probabilities from. When you have however many yowzabytes of text GPT-3 is using, your odds of output that "makes sense" are higher. And yet, ChatGPT can still give answers that are perfectly formed and utterly wrong.

        Q: How big of a number can you divide by three and get two as an answer?
        A: You cannot divide any number by 3 and get 2 as an answer, since 2 is not a multiple of 3. The result of dividing a number by 3 will alwa

  • Well, if anyone knows (Score:4, Funny)

    by Foundryman ( 306698 ) on Tuesday January 24, 2023 @11:20AM (#63235612)

    Well, if anyone knows 'Not Particularly Innovative' and 'Nothing Revolutionary', it would be Meta.

  • Listen up (Score:3)

    by groobly ( 6155920 ) on Tuesday January 24, 2023 @11:26AM (#63235648)

    LeCun along with Geoff Hinton are the seminal people most responsible for the current state of so-called AI based on neural net learning. LeCun is not a blowhard. He knows whereof he speaks.

  • That's just for you though. It's like saying the iPod wasn't anything special either. All the parts had existed before. They just packaged it nicely /sarc

  • Sourpuss LeCun obviously doesn't know the difference between innovation and invention.

    To put it in terms even Metafolk might understand, Facebook wasn't inventive either. But it managed to drag everyone's grandparents online whereas MySpace and Friendster really didn't. Facebook brough the new to people: innovative.

    https://www.wired.com/insights/2015/01/innovation-vs-invention/ [wired.com]

  • I bet ChatGPT is incredibly good at creating cohorts of like-minded fools and useful idiots. It'll be instrumental in creating minion armies to do Skynet's dirty deeds. Don't look behind the curtain...skynet is sentient! Ignore the couple of soulless rich dudes behind that curtain.

  • Considering how the Metastasis' own currently heavily promoted as the best thing since sliced bread is nothing particularly innovative and nothing revolutionary.

    Seriously, the "metaverse" is basically VRChat without legs.

  • "In terms of underlying techniques, Yann LeCun is not particularly innovative," said ChatGPT when asked to comment.

  • I think it's a lot easier to think of chatgpt as a brilliant innovation if you aren't familiar with all the parts that were there before that were developed by others to make it. If you are familiar with those parts, you see that they did little work to put them together. Its like the iphone. Most people thought the iphone was revolutionary because touch screens were new and gyro controls were new. But if you had worked with that before then all you see is that Apple just took something they didn't inve

  • Yeah, this automobile thing isn't very innovative either. Sure you get around a little faster than horse and buggy, but not much and there's so much more infrastructure that's needed.

    In all seriousness, smart chatbots are clearly a major breakthough. It's not clear if chatGPT has an exclusive quality. I expect to see a whole industry of smart chatbots that can tailor to different domains. What's not clear is if chatGPT specifically will be around in 5 years, but something in the space will be.

    The real quest

  • Coming from the people who couldn't build Second Life with billions of dollars, I take that with a huge grain of salt.

  • There is something quite unique about second life through, sorry I mean meta.

  • Hi, I wrote an article "All Answers About GPT Chat Integration" https://cadabra.studio/blog/ab... [cadabra.studio] may I ask to take a look, and let me know your thoughts :)
  • Facebook calling another "technology" company "not particularly innovative or revolutionary" is the pot calling the kettle black.

    The only thing innovative about Facebook is how they engineer their UX to trigger just the right dopamine response in their users to addict them and keep 'em coming back.

    However, that innovation also happens to be just plain evil, so I can see why they might not shout it from the rooftops.

Slashdot Top Deals

Technology is dominated by those who manage what they do not understand.

Close