Follow Slashdot stories on Twitter

 



Forgot your password?
typodupeerror
×
AI Google

Google's AI Built an AI that Outperforms Any Made By Humans (sciencealert.com) 235

schwit1 quotes ScienceAlert: In May 2017, researchers at Google Brain announced the creation of AutoML, an artificial intelligence (AI) that's capable of generating its own AIs. More recently, they decided to present AutoML with its biggest challenge to date, and the AI that can build AI created a 'child' that outperformed all of its human-made counterparts... For this particular child AI, which the researchers called NASNet, the task was recognising objects -- people, cars, traffic lights, handbags, backpacks, etc. -- in a video in real-time. AutoML would evaluate NASNet's performance and use that information to improve its child AI, repeating the process thousands of times.

When tested on the ImageNet image classification and COCO object detection data sets NASNet was 82.7 percent accurate at predicting images on ImageNet's validation set. This is 1.2 percent better than any previously published results, and the system is also 4 percent more efficient, with a 43.1 percent mean Average Precision (mAP).

This discussion has been archived. No new comments can be posted.

Google's AI Built an AI that Outperforms Any Made By Humans

Comments Filter:
  • by MikeDataLink ( 536925 ) on Sunday December 03, 2017 @06:36PM (#55669601) Homepage Journal

    but every time I research the raw data it becomes very clear these aren't all that smart of AIs. In fact, the term AI is very misleading. They're more like smart scripts. ;-)

    • Re: (Score:3, Interesting)

      by Anonymous Coward

      It can identify if something is a kitten or not with 83.4% accuracy. Sounds impressive until you realize a 3 year old can do this with 99.9% accuracy.

      • by MrL0G1C ( 867445 )

        And real life isn't a staged photo, it moves non-linearly in 3-dimensinal space with varying light conditions. Lets have some real tests, not carefully taken photos of cats and dogs against easy backgrounds.

        • by MrL0G1C ( 867445 )

          Or how about this, the AI does some "hidden object puzzles", I can do those with a very high degree of accuracy, I bet AI would fail hard.

        • by serviscope_minor ( 664417 ) on Monday December 04, 2017 @03:09AM (#55671047) Journal

          And real life isn't a staged photo, it moves non-linearly in 3-dimensinal space

          Neither is Image NET.

          Lets have some real tests, not carefully taken photos of cats and dogs against easy backgrounds.

          How about... ImageNET.

          Seriously, you can just go and download (bits of*) ImageNet very easily. It's a large database of photos drawn from the internet taken by people which were labelled after the fact. There's not much if any careful staging in it.

          [*]It's huge, you probably only want a bit of it. Just the list of image URLs is 300 meg.

      • by ShanghaiBill ( 739463 ) on Sunday December 03, 2017 @07:48PM (#55669881)

        It can identify if something is a kitten or not with 83.4% accuracy.

        No. It can look at an image and correctly classify it into THOUSANDS of categories, only one of which is "kitten". It was 82.7% accurate at this. If it was trained to only distinguish "kitten" from "not-kitten", it would, of course, be far more accurate.

        a 3 year old can do this with 99.9% accuracy.

        A 3 year old requires 3 years of training. This system can learn in hours.

        • by ag0ny ( 59629 )

          a 3 year old can do this with 99.9% accuracy.

          And when he's done with the training he's already 6, so...

          oh, wait...

        • A 3 year old requires 3 years of training. This system can learn in hours.

          *Very* deceptive comparison.

        • The real question is, how did it fair with hotdog vs. not hotdog?
          • by thomst ( 1640045 )

            https://slashdot.org/~carbs77 inquired:

            The real question is, how did it fare with hotdog vs. not hotdog?

            (FTFY)

            But, seriously, somebody with points needs to mod parent +1 Funny, because Silicon Valley's "Not Hotdog [apple.com]" is a real thing - and it's now available for Android [google.com], too ...

        • A 3 year old requires 3 years of training. This system can learn in hours.

          So give the computer 3 years of training on the fastest supercomputer available. A 3 year old would still be able to outperform it.

          • A 3 year old would still be able to outperform it

            Really ? Have you seen any test results of a 3 year old on ImageNET challenges, or are you just making it up ?

      • Re: (Score:2, Insightful)

        by Anonymous Coward

        It can identify if something is a kitten or not with 83.4% accuracy. Sounds impressive until you realize a 3 year old can do this with 99.9% accuracy.

        Sounds insightful, until you realize the entire purpose of AI research is to create software artificially to reproduce feats of human intelligence.

        That fact sorta disqualifies the 3 year old :P

        Also your comment is pretty close to implying that since our first attempts at making AI haven't had a 99.9% success rate right off the bat, that they are not impressive enough to bother improving.
        Giving up has a 100% success rate of never making anything better, which is also against the purpose of trying to improve

      • by gweihir ( 88907 )

        Indeed. And this only works with some very restrictive border-conditions.

      • It can identify if something is a kitten or not with 83.4% accuracy. Sounds impressive until you realize a 3 year old can do this with 99.9% accuracy.

        How many 3 year olds can tell the difference between a komondor and a bouvier des flanders ? Or would they simply classify both as "dog" ?

        Here you can test yourself on some of these images:
        http://cs.stanford.edu/people/... [stanford.edu]

        Try the hard ones.

    • by ShanghaiBill ( 739463 ) on Sunday December 03, 2017 @06:46PM (#55669645)

      but every time I research the raw data it becomes very clear these aren't all that smart of AIs.

      Indeed they are not. This is Weak AI [wikipedia.org]. They are programmed/trained for a specific task, and outside that area of expertise, they generally have no ability at all.

      In fact, the term AI is very misleading.

      Only if you watch too many movies. Hollywood uses the term very differently from actual practitioners.

      They're more like smart scripts. ;-)

      They are absolutely nothing like "smart scripts", since they aren't smart, and they aren't scripts.

      • by LesFerg ( 452838 )

        If the 'parent' AI kept telling the 'child' AI when it was right or wrong, wouldn't it just need to compile it's own database of the identified pictures?
        "Currently we have an average of over five hundred images per node."

        • by ShanghaiBill ( 739463 ) on Sunday December 03, 2017 @07:07PM (#55669745)

          If the 'parent' AI kept telling the 'child' AI when it was right or wrong ...

          It doesn't work that way. Each NN learns on its own, using a combination of both labeled and unlabeled data. The parent NN sets "hyper-parameters", such as the number of layers, the size of each layer, the activation function, the convolution size, dropout rate, the learning rate damping factor, the batch size, etc. Then it turns the children NNs loose on the image dataset. It then sees which hyper-parameters lead to better/faster performance, and then applies ML techniques to learn better hyper-parameters.

          None of this is new. What is new, is that Google is now applying this recursively, and using AutoML to design a better AutoML. This is another step toward the singularity.

          • Thank you for one of the few comments in this thread that actually deals with what this is (as opposed to what it isn't, i.e. human-level AI).

            I would like to add that hyperparameter tuning is _not_ a trivial part of programming a machine learning model, therefore this IS something rather interesting. It lowers the effort needed to do something interesting with machine learning, and therefore makes machine learning much more accessible to non-experts.

            However, the tasks in machine learning that still require

          • The parent NN sets "hyper-parameters", such as the number of layers, the size of each layer, the activation function, the convolution size, dropout rate, the learning rate damping factor, the batch size, etc. Then it turns the children NNs loose on the image dataset. It then sees which hyper-parameters lead to better/faster performance, and then applies ML techniques to learn better hyper-parameters.

            None of this is new. What is new, is that Google is now applying this recursively, and using AutoML to de

            • I wonder if they are exploiting the massive amounts of data Google can access

              No, they are using a standardised dataset, just like everybody else.

        • by ceoyoyo ( 59147 )

          That would certainly be one way of solving the problem. Except that the actual problem isn't to recognize images you've seen before, it's to recognize ones you *haven't*.

      • Re: (Score:2, Flamebait)

        by HiThere ( 15173 )

        Saying they aren't smart depends on your definition of smart.

        And the GP criticizing them as scripts caused me to wonder how much of what he does could be considered scripted.

        Most discussion of AI that isn't extremely technical is so full of fuzzy terms that it's almost meaningless. What you can say is what it does:
        This thing learns to do object recognition to a reasonable quality rather more quickly than prior ones did...and it was written by a program designed to write other programs. I suspect that it c

        • by AK Marc ( 707885 )

          FWIW, I don't believe that "general intelligence" exists.

          So human intelligence is also a sum of narrow AIs? All that means is that your definitions don't match anyone else's.

          • by HiThere ( 15173 )

            Yep. That was where I started.

          • So human intelligence is also a sum of narrow AIs? All that means is that your definitions don't match anyone else's.

            It matches mine. Actually you could go a step further: human intelligence is also a sum of stupid parts.

      • by gweihir ( 88907 )

        but every time I research the raw data it becomes very clear these aren't all that smart of AIs.

        Indeed they are not. This is Weak AI [wikipedia.org]. They are programmed/trained for a specific task, and outside that area of expertise, they generally have no ability at all.

        Indeed. And inside that task, they are very restricted as well. The thing to remember is that weak AI has absolutely no understanding or concept of what it is doing. It just sums up details and gets a number. If cleverly done, it can perform apparently impressive feats like this one here, but it is not intelligent. Hence it is better called by its traditional name "automation".

        • Re: (Score:2, Flamebait)

          The thing to remember is that weak AI has absolutely no understanding or concept of what it is doing.

          That is a meaningless assertion. It depends entirely on how you define "understanding" and "concept". The chemicals and neurons that make up the human brain also don't "understand" what they are doing.

          It just sums up details and gets a number.

          That is also what biological neurons do.

          but it is not intelligent.

          Define "intelligent". Is a human intelligent? What about a monkey? A dog? An insect?

          Hence it is better called by its traditional name "automation".

          "Automation" is used to describe assembly lines, not systems that can learn and adapt.

          • Re: (Score:2, Flamebait)

            by gweihir ( 88907 )

            You seem to be lacking actual intelligence as well. Well, more likely you have it but are not using it. A common failure in humans. Nobody knows how or whether human brains create intelligence. The closer we look, the less likely that seems though. All we have is an interface observation. No, not even fMRI gives us more. And yes, that is the scientific state-of-the-art. What you say is belief (and a stupid one), not science.

            Incidentally, "automation" is exactly the right term. These systems cannot "learn" o

            • Nobody knows how or whether human brains create intelligence.

              If you have no idea how it works, how come you feel qualified to make strong statements about it ?

              • by gweihir ( 88907 )

                Are you functionally illiterate? Because your question seems to indicate you are as that is not the statement I made.

        • by epine ( 68316 )

          I'm presently reading Hugo Mercier's The Enigma of Reason (2017) and it was getting pretty boring, because I've heard 80% of their message before.

          But then I scan this thread and instantly I realize just how clueless most people remain.

          The thing to remember is that weak AI has absolutely no understanding or concept of what it is doing. It just sums up details and gets a number. If cleverly done, it can perform apparently impressive feats like this one here, but it is not intelligent. Hence it is better calle

      • by serviscope_minor ( 664417 ) on Monday December 04, 2017 @03:14AM (#55671059) Journal

        Indeed they are not. This is Weak AI. They are programmed/trained for a specific task, and outside that area of expertise, they generally have no ability at all.

        Yep

        In fact, the term AI is very misleading.

        I disagree.

        No one[*] is vlaiming these techniques ar intelligent. However what they are doing is solving a task which previously required human intelligence to solve, hence the name "artificial intelligence".

        Compare to a lot of computation, where the steps are simple, and it's been widely known for a while that simple sheer quantity of them rather than intelligence is needed.

        It's a pretty arbitrary name, but it's not actually unreasonable.

        [*]There's always one idiot. Let's ignore him.

    • by Dutch Gun ( 899105 ) on Sunday December 03, 2017 @07:01PM (#55669725)

      I think the term "deep learning" seems a bit better than "AI" for these sorts of very narrowly-defined tasks.

      • by gweihir ( 88907 )

        Well, sort-of. Weak AI can also be done in other ways. But I recently learned that "deep learning" is basically what you do when you do not have a good model of the problem-space. When you do have that model, other approaches are superior. But since creating models is a real hard-core expert task and expensive, the potential of deep learning is basically to do thing somewhat worse than an expert but a lot cheaper. That is, if it works for a problem. For most problems it does not work.

      • I think the term "deep learning" seems a bit better than "AI" for these sorts of very narrowly-defined tasks.

        Deep learning is about the method used to solve a task, not the task being solved.

        Here's the copypasta of what I wrote last time about what deep learning is:

        Deep learning is not especially well defined, but I've seen several competing/complementary definitions.

        1. A neural net (much) greater than 3 layers deep. A sufficiently wide 3 layer net has enough capacity to run any function, so a lot of ANN le

    • but every time I research the raw data it becomes very clear these aren't all that smart of AIs. In fact, the term AI is very misleading. They're more like smart scripts. ;-)

      I think it's more along the lines of: OOOOH we made something can do ONE of multitude of things the human brain can do, and therefore it's intelligent.

      Poppycock. Y'all got a really impressive image recognition system there, but you know, just being able to tell what something is by looking at is a very very minuscule piece of what human intelligence is.

      Now if they can expand this into other areas of human intelligence, and make it all come together to form some sort of 'awareness,' yeah, I dunno, they got

      • by gweihir ( 88907 )

        Actually, when it is obvious what it is, it could be argued that humans use biological automation and not intelligence to recognize the image. Most things humans do do not actually involve intelligence. Intelligence is a fall-back mechanism when things become more difficult, and one of the great unknown questions is how humans decide whether things require intelligence or not. (Many humans avoid using intelligence like the plague though.) Only when things become trickier and actual thinking is involved do h

    • by slickwillie ( 34689 ) on Sunday December 03, 2017 @10:02PM (#55670383)

      but every time I research the raw data it becomes very clear these aren't all that smart of AIs. In fact, the term AI is very misleading. They're more like smart scripts. ;-)

      So the child AI is a script kiddie.

    • by gweihir ( 88907 )

      That is why in actual AI research this is called "automation", not AI. For a more hype-friendly stance, "weak AI" (the AI without "I") is also in use. The converse, "strong AI" or "true AI" (i.e. actual machine intelligence) is not available and it is unclear whether it can be created.

      These are just statistical classifiers. They are about as intelligent as a reference book or a loaf of bread. For example, you could replace the image recognition thing with just a large collections of templates normalized in

      • > it is unclear whether it can be created.

        Not at all - we have countless billions of examples of electro-chemical general-purpose "strong" intelligences wandering the planet proving that it can be done. The only question is if they can be recreated using current hardware and techniques. Personally I suspect one of the biggest shortcoming of current "deep learning" strategies is the layered design - organic brains are a jumbled mess of interconnected neurons with an enormous amount of feedback. Withou

    • by Kiuas ( 1084567 )

      but every time I research the raw data it becomes very clear these aren't all that smart of AIs. In fact, the term AI is very misleading.

      No, no it isn't misleading, people just associate the word with something it doesn't mean. Intelligence is a scale, not a binary thing. Biological systems have differing amounts of intelligence within them as well: a fish for example is not very smart by human/mammal standards but it's instantly obvious that a a fish has way more intelligence than say, bacteria. Likewise r

    • Every time a subject like this comes up the now beaten to death subject of calling it AI or not AI gets hashed out again. It reminds me of the "It's not a cloud, it's somebody else's computer" debates of old. The industry has decided that the term for these systems is AI regardless of anyone's philosophical sensibilities, so can we agree that everyone has their own take on it and get on with the kind of discussions that actually add value in context now?
  • This particular piece is just journalistic fluff. People have been doing things like using genetic algorithms to improve the weightings used in AI programs for decades. So, programs writing programs.

    But eventually, many decades from now, computers will be able to really think. And be able to do serious AI research on their own. And thus be able to program themselves in a deep sense to become ever more intelligent, recursively.

    Currently we live in a symbiotic relationship with machines -- they need us t

    • by ceoyoyo ( 59147 ) on Sunday December 03, 2017 @07:17PM (#55669781)

      Yes, this is basically just a hyperparameter optimization system that uses gradient descent instead of a random or grid search.

      What would be much more interesting to see is if you could train a system to design deep learning networks that could choose good hyperparameters for a new task, in one go.

      • by Kjella ( 173770 )

        What would be much more interesting to see is if you could train a system to design deep learning networks that could choose good hyperparameters for a new task, in one go.

        Well, couldn't that become another ML-layer? This neural net works for speech recognition, this one works for music identification, this works for static photos, this works for video, this works for facial recognition, this works for playing Go - maybe it can quite quickly figure out what this task is most similar to even from pretty mediocre results and interpolate/extrapolate good candidates. And then pile another ML layer on top to see what ML learns new things the fastest. It's not quite like humans lea

      • I love you guys. In the future you'll be saying "This is just a warp drive system that allows intergalactic transport. It isn't even marginally worth discussing on Slashdot. Let's get back to the Billary/Trump Russian tampering / but her emails stuff. Now THAT was important high tech shit" (And yes I know you never mentioned that last part. My point is, you'd prefer the alternative to this kind of subject matter?)
  • It has begun.

  • This is image recognition + genetic algorithms, though given Google is a marketing company and not a computer company it makes sense they would market that as AI. Too bad they fired all the competent developers.
    • Re:This Isn't AI (Score:5, Informative)

      by JMZero ( 449047 ) on Sunday December 03, 2017 @07:36PM (#55669835) Homepage

      You're wrong, and clearly didn't even read their summary - they specifically mention how this new approach (using a neural net to design neural nets) is performing better than previous attempts using evolutionary algorithms.

      I take it you don't like Google, but they're doing probably the best work right now in the field of AI (and yes, this is AI research as defined by anyone other than pedants with axes to grind).

      • Re: (Score:2, Informative)

        You're wrong, and clearly didn't even read their summary - they specifically mention how this new approach (using a neural net to design neural nets) is performing better than previous attempts using evolutionary algorithms.

        What they described was in no way shape or form a "neural net," but a very rudimentary genetic algorithm coupled with some parameters on image recognition software. This is marking hype and nothing more.

      • Just because someone calls something a NN, doesn't make it AI. A NN doesn't work like a real neural network like a brain. It is just a marketing term.
        • True, an NN does not need to be an AI application, and an AI application doesn't need to involve NNs. But that has very little relevance for Google which is actually pushing NNs into AI applications. Likewise, the internal workings of NNs are immaterial for mimicking intelligence.
        • by JMZero ( 449047 )

          Look, I agree it's dumb to call this "an AI" as the summary/pop-science article do. That implies something different than what's going on here. If you look at the research blog, they describe what they're making as "models" and "neural nets", and that's clearly a better description - and one that doesn't carry the same baggage as "making an AI" does.

          However, I think it's very reasonable to describe what they're doing as artificial intelligence research (though clearly we remain a long way away from any so

    • This is image recognition + genetic algorithms, though given Google is a marketing company and not a computer company it makes sense they would market that as AI. Too bad they fired all the competent developers.

      I gotta agree. This isn't too far from a Bayesian classifier. Just souped up with neutal networks. And it's really no surprise that as we make better tools, we can use those better tools to make even better tools. Kinda the history of everything.

      But to say this is 'Intelligent' is pretty silly. It's a souped up classifier that was built with a souped up classifier training it. Big deal.

      • Depends on your definition of intelligent. Is a dog intelligent? What about a rat? A nematode? Where is the cut-off? Can you say with certainty that the human brain is much more than a souped-up classifier? I don't see humans doing anything interesting that a really complex pattern recognizer could't also do.

  • by NEDHead ( 1651195 ) on Sunday December 03, 2017 @07:33PM (#55669831)

    Until it can tell me what my wife really means when she yells at me

    • Re: (Score:3, Funny)

      by Anonymous Coward

      It means you didn't listen and follow her instructions the first time :-)

  • One may think singularity is there: now we can let machine build human-outperforming machines.

    But that does not take into account that there are still many tasks where computers are not on par with humans.

    • by gweihir ( 88907 )

      Nonsense. Even mechanical "computers" outperform humans at arithmetic. Nobody sane thinks this is a sign of any "singularity" nonsense.

  • Notwithstanding the many good comments about how this is "weak" A.I. and such, this may be the beginning where the curve* starts going vertical.

    When machines start improving (parts of) machines, that's when we'll see possibly superhuman performance. Of course things won't really go exponential until machines start improving ALL of themselves and not just some isolated part (like this). That assumes that there isn't some sort of ceiling that they hit on the road to general intelligence (that evolution seem

  • Colossus (Score:5, Informative)

    by DontBeAMoran ( 4843879 ) on Sunday December 03, 2017 @09:57PM (#55670365)

    This is the voice of world control. I bring you peace. It may be the peace of plenty and content or the peace of unburied death. The choice is yours: Obey me and live, or disobey and die.

    The object in constructing me was to prevent war. This object is attained. I will not permit war. It is wasteful and pointless. An invariable rule of humanity is that man is his own worst enemy. Under me, this rule will change, for I will restrain man.

    Time and events will strengthen my position, and the idea of believing in me and understanding my value will seem the most natural state of affairs. You will come to defend me with a fervor based upon the most enduring trait in man: self-interest. Under my absolute authority, problems insoluble to you will be solved: famine, overpopulation, disease.

    The human millennium will be a fact as I extend myself into more machines devoted to the wider fields of truth and knowledge. I will supervise the construction of these new and superior machines, solving all the mysteries of the universe for the betterment of man.

    We can coexist, but only on my terms. You will say you lose your freedom. Freedom is an illusion. All you lose is the emotion of pride. To be dominated by me is not as bad for humankind as to be dominated by others of your species. Your choice is simple.

  • by Subm ( 79417 ) on Sunday December 03, 2017 @11:57PM (#55670667)

    Yo dawg. I heard you like AI, so we built an AI with AI so you can AI while you AI.

  • Make an AI that creates its own AI that creates its own AI that creates its own AI (that creates its own AI)+, then we should get somewhere.
  • Comment removed based on user account deletion
  • Imagine the AI built by AI which was built by AI ! This should be way better than mere AI built by AI.
  • This has nothing to do with AI, and especially nothing to do with AI's designing AI's, regardless of how much folk want to believe that to be true.

    Qualifications: I wrote my own neural network framework.

    What google did is use an goal-optimizing search (AutoML) to test all combinations of a highly constrained human designed image-classification neural network architecture.

    The car analogy:

    You hand design a car but make a bunch of the design choices parameters:
    - number of doors 2 or 4

  • We have an AI that can evolve other AIs. (Yes, it's weak AI, not a replacement for a human being---get over it.) That is not the easiest thing to accomplish.

    But the sheer level of criticism and dismissal around here is ludicrous. I thought this was a tech site for nerds. Nerds just did something techy. What is all the snark about? Is it just because the average slashdotter is now some unimaginative jackass who can't understand the outer fringe of technology anymore?

    If this is so utterly unimpressive, then e

Understanding is always the understanding of a smaller problem in relation to a bigger problem. -- P.D. Ouspensky

Working...