Leaping the Uncanny Valley

reachums submits this glance at "the newest level of computer animation," intended to get past the paradoxical "uncanny valley" — that is, the way animated humans actually can appear jarring as the animation gets hyper-realistic. "This short video gives us a glimpse of what we can hope to see in the future of computer games and movies. Emily is not a real actress, but she looks like a real person, something we haven't truly seen before in computer animation."

We've heard this before

There was much talk about the uncanny value when Final Fantasy: The Spirits Within [amazon.com] came out after Square had promised for years that it would have realistic humans. A common criticism was that the human beings were real enough to inspire comfort for long enough that one would be then shaken by their lack of certain flexibility and the bloodlessness of their faces. Dr Aki was more creepy than sexy.

Re:We've heard this before

Dr Aki was more creepy than sexy.

"More creepy than sexy" -- four words that sum up most of the anime I've been subjected to. But if that's going to be the criteria under which we judge the depth of the uncanny valley, some people are going to navigate it a lot faster than others.

Yes, but does it even exist?

Well, a better question is if the uncanny valley really exists. Or rather, if it's really as simple as that valley, or we're actually looking at a more complex and multi-dimensional phenomenon.

And I'll attempt to build a framework to falsify it. It's a bit roundabout and I'll start by explaining the what and why of that framework, before all else. Bear with me, please.

First of all, before someone jumps in with the ever popular, "OMG, you're not worthy to question the high priests!" (err... "scientists"), the uncanny valley is just a hypothesis. A very compelling and well argued one, no doubt, but hardly a proven fact.

Second, before I get into the meat of the argument, the points chosen to represent it are highly debatable. E.g., is a zombie scary because of being close enough to the real thing to fall in the "uncanny valley", or because of the whole cultural meaning of death, undeath, corpses, etc?

When you look at each point individually, you can handwave and argue it to be wherever you want it, to support your hypothesis. It's called the Texas sharpshooter fallacy [wikipedia.org], after the fable of the sharpshooter who shot first and then painted a bullseye around the hole. You can "prove" anything in (pseudo-)science if you can do just that to the data: take a fuzzy and ill defined points and argue where they belong on your curve.

The "uncanny valley" paper does just that. We don't know the exact X coordinate on that graph for a zombie or a robot. It could be way right or way left, or whatever. So what really follows is that Mori decided a priori where they belong on that curve, and then places them at a point based on that. It's a textbook application of the Texas sharpshooter fallacy.

So what I'm going to do is an ad absurdum reduction of his curve.

I don't know the exact coordinates of any of my examples either, but, here's the important part: I don't need to pretend to. I'll just peg them between two other values, which, assuming the curve is correct, both fall in the valley or outside it, or some other position. Based on the reaction they caused, and, again, assuming that the curve were correct.

And due to the shape of the curve, if two points are in the valley, then everything between them is in the valley too. If two points are, say, both to the left of the valley, then a point between them should be on the left of the valley too. That is the important part.

So, let's build a counter-example: the FF movie was called a clear example of the Uncanny Valley. It's in the valley. Sony's Everquest 2 (particularly with the unnatural ambient bloom enabled) caused a similar reaction, and many euphemisms were used to describe just that: that that world looked disturbingly unnatural, especially if you pushed the graphics settings high enough. Classic example of entering the uncanny valley from the left, eh? So it's point 2 in that valley.

A point between them should, obviously, also be in the valley. That curve only has one dip, right?

Well, point #3 could be Oblivion. The graphics are better and more detailed than Sony's graphics in EQ, but don't even come close to the insane polygon counts and animations of the FF movies. It's between the two points. It should also be in the valley. It isn't. Nobody was repulsed by Oblivion's graphics. Or pick Crysis, or whatever newer high-end game, and you get the same curious behaviour. It ought to be in the valley, but it isn't.

Let's build another counter-example: so we're told that zombies are only repulsive because they're so close to humans as to fall in the uncanny valley. So logically, if you start with a zombie and move farther and farther away from human-like with it, eventually it exits the valley. Right? In fact, past a point it becomes outright _cute_ and appealing. Or ought to. I mean, that's the shape of that curve.

You probably realize already how absurd that statement is, but let's actually imagine it. Let's say we start with that corpse an

Re:How true was this?

Many of the examples you've given don't cross over into "uncanny valley" territory - nobody would get stuffed animals confused with real animals, or people. The things considered to be in the "uncanny valley" are generally attempts at photorealistic humans that get close enough to confuse the observer, and then turn disturbing when they act in an unnatural way. It is similar to your glass of water example, however this is cognitive dissonance on a higher level, because it's dealing with "real" people and sometimes "real" emotions.

Re:How true was this?

I'd say it is pretty hard to dispute. We need mechanisms for identifying someone with say, the early stages of leprosy or birth defects (finding the right mate, etc.).

Re:Uncanny in the other direction

There actually have been theory's put forward that there will be a second 'uncanny valley' when it comes to transhumanism.

In other words, as 'normals' begin modifying their appearance outside of what is considered normal, they will start to slide down that valley and become socially unacceptable till they get to the point where they are back on the other side of the valley and just accepted as 'freaks'.

Re:How true was this?

Actually, stuffed animals are a good example outside the uncanny valley. Remember, we are talking just about human appearance and action, not animal.

Regarding realistic paintings and statues - yes, they do look like humans, and I would say they reach past the valley on that point, but they don't act like humans. I would also suggest that it is easier for us right now to transcend the valley in appearance than it is to do so in action, if for no other reason because we've had more practice.

Wax figures are another good example. Madame Tussaud's wax figures are excellent representations of the human form - but again, they don't act human, which makes the valley easier to surpass. However, have you ever been to a bad wax museum? I have seen some pretty bad wax figures as well, and they trigger the uneasiness that this theory suggest an inanimate object in this valley would.

As far as cartoons, the physics-defying glass of water - these are all non-human representations, and thus not covered by this theory.

Re:How true was this?

I think you misunderstand the uncanny valley concept. In fact, your entire rebuttal is mostly a restatement of the concept itself.

The point is the more realistic something is, the more disturbing any 'defects' in it's simulation are. Stuffed animals don't breath, they don't move, they don't growl. Neither do paintings. These things may be realistic in the sense that they portray a snapshot of the thing they are based on, but they don't come anywhere close actually convincing you that they ARE the thing they were based on. I don't know of anyone who would mistake a stuffed bob cat sitting in someone's den or a museum as the real thing for more than a few seconds. Likewise, people pretty much know when they are looking at a painting or even a photo.

All of those things are on the 'safe' side of the valley. The problem comes when you start getting things that move, sound, and mostly act as if they are alive but clearly aren't. Your actor with the no-spill glass would be in the valley, so to would be photorealistic computer models that didn't have facial expressions when they spoke (ala FF).

And for the record, while the uncanny valley was popularized by talking about computer generated graphics, it was actually coined by a roboticst back in the 70's, and was based on an idea first presented by Freud in the 1910's.

Re:How true was this?

Important to note that the only part of the video that looks completely normal is the behavior of the head, the arms, the breathing of the torso-essentially everything done by the actual human. I'd be surprised if you felt absolutely no (at least confusion if not revulsion) watching the video. The face was interesting to me, but I was blown away by how realistic the hair looked, how realistic the arm movements--until I realized that that was all still a regular human being. Then, when I focused in on the face alone, it simply looked animated, and if not disgusting, at least completely out of place.

I think the general feeling, even if it isn't an all-out feeling of disgust, is one that things are...not...right. Ultimately, I think this is a pretty bad example, though, since Emily is touted as being "not real" but in fact the majority of the body language--the stuff we are tuned into almost subconsciously--is still human. I think if this video skips by any general feeling of revulsion, disgust, or out of place-ness, it is specifically because there are still physical human elements in it.

Re:We've heard this before

Actually, for me the lips were off. Most of the time they were fine, but sometimes they would get a little too wide, or their shape would be slightly off.

Re:We've heard this before

Just go to L.A. , there are much more fake looking lips on the women there

Not really animation

From what i understood, this is simply an easier kind of motion capture that works straight from video without the need for sensors etc. That's not the same as creative animation, you still need a real person talking and moving.

Re:Not really animation

Yea all this does is invalidate video as proof of anything. Now you create a clip of Barack Obama planting a UED in Iraq, or John McCain visiting a gay dance club.

All you need is video of somebody of similar build and you can put anybodies face on it.

Re:Not really animation

Exactly. This isn't precisely computer animation, it's motion capture minus a lot of steps.

Re:Not really animation

From what i understood, this is simply an easier kind of motion capture that works straight from video without the need for sensors etc. That's not the same as creative animation, you still need a real person talking and moving.

I haven't seen motion capture look this good. But anyway, the point of this is that you could, for example, use a cheap (read: plain) actress for the recording of scenes, and then animate the perfect, beautiful princess character using her mannerisms. Let's face it -- actors get paid the big bucks for their looks, and not so much their talent. There is a hell of lot of acting talent out there that doesn't necessarily have the right "look".

End of blah blah

Just as synthesizers were the end of "real" musicians, photography was the end of "real" paintings, etc.

They failed, and they're lying.

First off, they failed at getting passed the "uncanny valley". That video is still creepy looking.

Second, this isn't computer animation. It's just video processing. If you still need to do high resolution motion capture to produce your images, you haven't replaced the actor. You've merely edited their appearance in the performance. They didn't even bother to go so far as to take the captured motion and paste key bits of it together into the speech. They just had her sit there and say the whole thing, then "rendered" it.

Lame.

Not a good test

Motion capture a face and rerender it from the same viewpoint as a camera used to capture the texture and you'll trivially get something almost indistinguishable from the original. It's only a valid test if you change something significant: move the camera, change the lighting, change the facial features or change the performance.

Wow, quite amazing.

I am amazed at the quality of this animation: Still, I could see there was -something- wrong with her, but could not put my finger on it. (this was of course also influenced since I -knew- she was fake before watching the vid).

Btw, here's a direct link to the video: http://www.youtube.com/watch?v=bLiX5d3rC6o [youtube.com]
Be sure to tick the 'Watch in high quality' when the video opens (anyone knows a way to do that automatically in a link?)

Yup

I'd say it's past the uncanny valley. That's not to say that I can't tell it's fake. She looks a little fake. Something is wrong-- her face is too still or something. But she doesn't look like a zombie. She's not distractingly creepy. That's all they're really shooting for at the moment, right?

http://www.image-metrics.com/ has another demo

They have another demo on their Front Page [image-metrics.com]
And while it's extremely impressive, sadly it's definitely in the valley for me.

The demo is a streaming video ?!?

Yay ! Wonderful low-bandwidth youtube streaming video in all its glorious crap-quality !
The best way to show technical demos about photo-realism !

I can't wait to see the thumbnail sized 60%-quality jpeg screen caps, too !

I feel as much informed about the quality as when watching all those wonderful ads about hiddef screens on the TV.

---

Common, Image Metrics, can't you just post a descent hi-quality video file, so we can actually see what your technology looks like ?

Hasn't jumped the valley.

Heck, she doesn't even look as real as Celine Dion, let alone a real person.

Re:It's more convincing than...

Is a Turing test valid if the human is an idiot?

It's ridiculously clear from the video that the face was the "animated" bit.

Re:It's very close.

That's what I thought as well until I saw it in higher quality.

A higher quality version of the video can be found here [awntv.com].

It's not perfect, but it certainly is climbing high up out of the uncanny valley to say the least.