Kinect Hack Builds 3D Maps of the Real World

Lanxon writes "Noted Kinect-tinkerer Martin Szarski has used a car, a laptop, an Android smartphone and the aforementioned Xbox 360 peripheral to make a DIY-equivalent of Google Street View. The Kinect's multi-camera layout can be used to capture some fuzzy, but astonishingly effortless 3D maps of real world locations and objects. As we saw in Oliver Kreylos' early hack, you can take the data from Kinect's depth-sensitive camera to map out a 3D point-cloud, with real distances. Then use the colour camera's image to see which RGB pixel corresponds to each depth point, and eventually arrive at a coloured, textured model."

Wow

[MonsterTrimble]

That is a cool hack. I wonder if Microsoft had any idea the Kinect would become such a cool hacker piece of equipment?

Re:

[Is0m0rph]

MS has announced that releasing their own driver for the PC and Kinect in the future is likely.

Re:

[Zerth]

WTF are you talking about? Everything they've done since release has been encouraging to the Kinect hackers, and they've made quite a number of press releases to that effect.

Except for the first one, where they responded to news of PC Kinect drivers with "Microsoft will continue to make advances in these types of safeguards and work closely with law enforcement and product safety groups to keep Kinect tamper-resistant."

Re:

[bami]

That's just a knee-jerk reaction from the legal department.

MS Employee 1: Hey have you heard? They hacked the kinect to do some cool stuff.
MS Employee 2: Sweet
Lawyer 1: HACKED? THEY ARE HACKING OUR SYSTEMS, SUE SUE SUE.
Laywer 2: S&D tube one: fire!

Once they all got the memo that the hardware guys built the thing to be toyed around with (and seeing the hardware in the thing, they make some profit on it, selling it for $150, add that to the fact that it only has authentication for use with the xbox, and o

Re:

[Dunbal]

How do you expect them to use patents, exactly? If you paid for the Kinect and incorporate it in your own project, Microsoft doesn't acquire any "rights" over your work. What, you think that seat belt manufacturers get a cut from every commercial flight?

Re:

[tehcyder]

I'm sure the business model was to use this to drive XBox 360 sales. But if sales outside of this are significant enough (noticable), they just might let it keep going.

Don't be an idiot, the sales outside of use with the XBox 360 will be minuscule in number, but can be used to create some good PR with no real downside for Microsoft.

Actual blog post

[icebraining]

Link to the actual blog post: http://blog.decoratorpattern.com/2011/01/23/real-world-mapping-with-the-kinect/ [decoratorpattern.com]

Re:

[Idbar]

Also hackaday.com has some interesting projects about it. So far I remember this one [hackaday.com] among those trying to make the kinect portable to gather data.

Eh that's nothing

[kellyb9]

I like to play Kinect Sports when I'm driving.

Re:

[Umuri]

Umm, why a kinect? Any camera could give you basic visuals, and likewise an IR rangefinder would be much more useful at avoiding colissions on your path, the gps gives you position data.

I just don't see how this is new

Re:

[intellitech]

Not necessarily new, but cost-reduced and accompanying open-source code for effective use.

Re:

[IronHalik]

rc heli wit a real gun [youtube.com] Nuff said ;>

Re:

[sznupi]

Still here? ;) ("wit"?...hm, that did end up rather curiously)

Re:Scary applications

[TheSync]

an IR rangefinder would be much more useful at avoiding colissions on your path,

The Kinect is an "IR rangefinder", but a cheap mass-produced one. Unlike the more expensive laser scanned parallax or time-of-flight sensors, it uses a special diffraction grating to produce a "structured light field" for a 2D camera to measure localized parallax.

There are also cheaper pulsed IR time-of-flight depth sensors coming on the market for home use, these could have higher spatial resolution.

The cool thing is that these things don't cost tens of thousands of dollars.

Re:

[Missing.Matter]

The cool thing is that these things don't cost tens of thousands of dollars.

Yeah, that's the great thing about them, but the downside to this sort of sensor is that it doesn't scale well in size. That is, you can't use more than one kinect at a time, as the structured light fields from one will interfere with another. I suppose you can create some timing where one field will be off while another is on, but you can only have so many kinects in an area before this is impractical.

Of course it's not a problem for gaming in a living room, but it if you want to do any sort of multi robot

Re:

[tlhIngan]

Yeah, that's the great thing about them, but the downside to this sort of sensor is that it doesn't scale well in size. That is, you can't use more than one kinect at a time, as the structured light fields from one will interfere with another. I suppose you can create some timing where one field will be off while another is on, but you can only have so many kinects in an area before this is impractical.

Well, two Kinects do interfere, but not much - http://www.engadget.com/2010/11/29/two-kinects-join-forces- [engadget.com]

Re:

[TheSync]

the structured light fields from one will interfere with another

Indeed, I think a better solution is a time-of-flight sensor such as pulsed IR phase [mypublisher.be] or flash LIDAR [advancedsc...ncepts.com].
 

Re:

[Missing.Matter]

We use these [ifm.com] sensors on our robots. They aren't very high resolution (64x48) but the're accurate and provide good pictures over time. The time of flight design makes the more practical than the kinect for our applications, but they're much more expensive.

Re:

[techSage]

I believe that software algorithms are able to be used to "throw out" "dirty" data points so that your resulting 3D pointcloud is clean enough for most practical (non-scientific) uses. Or so I've been told by those performing such operations at certain national institutions. I think they know what they are talking about.

Re:

[batkiwi]

I think that you, like many people who enjoy bashing popular things, haven't actually done any reading to learn what you're talking about.

Kinect IS a camera with a 640x480 (not simply a swiviling spot rangefinder) IR rangefinder built in. And it's well under $200. And it supports USB using a protocol so simple an outsider, with no documentation, was able to build a driver in 1 week.

So what competes with it? Where is a 640x480 IR rangefinder for under $200 with an easy to use interface (not a single spot

Re:

[tehcyder]

why not just steal a fucking cruise missile and have done with it? moron.

Nifty!

[Dreth]

I can't imagine how this awesome will be with simultaneously-connected rotating Kinects.

Need for Speed Hot Pursuit - Kinect Version

[riskeetee]

The new version of Need for Speed Hot Pursuit uses the Kinect mounted on your car's dashboard. The chase scenes are way more exciting, and the cops use real bullets! Replayability, however, suffers greatly.

This is trivial

[Missing.Matter]

This is trivially done with ROS and the [slashdot.org]Kinect stack [ros.org]. I went out and bought a kinect and plugged them into my robot platforms, provided a transform between robot base and kinect sensor, and was done. It's a great application and anyone who owns a kinect should try it out, but at this point it's trivial and hardly worthy of a /. post.

Re:

[Anonymous Coward]

I love that "trivial".....like my dad could just go out and do it in 3 clicks.
Deutsche!

Re:

[Missing.Matter]

It takes maybe 4 commands to get this working.

sudo sh -c 'echo "deb http://code.ros.org/packages/ros/ubuntu [ros.org] maverick main" > /etc/apt/sources.list.d/ros-latest.list'
apt-get install ros-cturtle-ros
apt-get install ros-cturtle-kinect

That will get the Kinect working. Then you just need a GPS and the appropriate transform between the GPS antenna and the sensor (easiest to put the antenna on top of the kinect), and you're done.

And I don't see what me being German has anything to do with this.

Re:

[clyde_cadiddlehopper]

It appears that putting working links in slashdot posts is non-trivial.

Re:This is trivial

[kiwimate]

Well, nifty. Good for you. Instead of acting super cool and belittling what someone else has done, can't you just say "nice one, and if readers are interested in this then look at what I did"?

Re:

[Game_Ender]

His point is that the research field has been doing this (3D reconstruction with range sensors) for years, so long in fact they have created nice open source solutions for it. The tech news sphere, and most hobbyist it seams, are completely ignorant of this.

Re:

[Game_Ender]

Not only is it trivial, it doesn't even run any optimizations passes over the resulting point cloud in order to reduce error and produce nice surfaces. This process is very similar to the work done here [washington.edu] except the resulting solution will be quicker because you already have some decent depth information for the points.

Re:

[QuantumG]

It's a shame everyone is still looking for the low hanging fruit rather than putting in the hard effort required to make those 3d point clouds into actual surfaces and producing something that doesn't look like a pile of crap.

Also, describing this as "google streetview like" is just stupid. It's a true 3d point cloud, not a bunch of panoramic images.

 

Re:

[Missing.Matter]

There's a whole lot of stuff like that going on. For example, the whole ROS PointCloud Library is a major effort focused on surface reconstruction, object detection, etc. You just don't see front page /. posts on the really cool stuff.

Videogame peripherals

[Dusthead Jr.]

The thing that I find cool about modern video gtame peripherals is that they seem more standardized than in the past (with the exception of the Sega Genesis/Atari 2600 controllers.) They either use USB or Bluetooth, and to the 360's credit you can still use its proprietary wireless pad on a PC.

Re:Videogame peripherals

[VortexCortex]

The thing that I find cool about modern video gtame peripherals is that they seem more standardized than in the past (with the exception of the Sega Genesis/Atari 2600
controllers.)

Perhaps the connection is more standard, but I also see a fair amount of non standard shit going on, and not just in video game peripherals. The bundled Kinect doesn't come with the adapter so you can connect it to your PC (or older Xbox360). Don't fool yourself for one moment -- If MS could have ignored the existing XBox360 interface your Kinect they would have gladly made it more difficult to connect a Kinect to your PC.

Xbox360 headsets have nonstandard pin-outs, so you can't use other PC headsets on on XBox or vise versa. Garmin's GPS uses a proprietary USB cable (having a small resistor across two pins), and refuses to charge when connected to a PC or via standard USB plugged into a voltage inverter. Zune uses a standard AV connection (Camcorder 3.5mm), but they swap the audio & video jack pins so that you have to plug white to yellow & vise versa, (many assume only the more expensive Zune brand cable works). Even the power cable on a Apple G5 I serviced last week was non-standard (-_-) instead of ('.'). The iPhone/iPad Touch & Zune use encrypted protocols (keys changed on each firmware upgrade) so you can't use them without the crappy bundled media manager software.

They either use USB or Bluetooth, and to the 360's credit you can still use its proprietary wireless pad on a PC.

The 360's wireless is NOT bluetooth. You must use MS's proprietary wireless receiver. The controller will not work though a blutooth receiver (built-in or external). The play&charge USB cable is for charging only, and won't allow you to connect it to your PC.

Also, What's the use of a standard plug (USB), if the protocol or other proprietary quirks are introduced to make it incompatible (ala Zune, iPad Touch, Garmin resistor).

I have to disagree with you on the standardization trend; To me, it seems that more companies are figuring out how to proprieterroize the "standard" connections; less of my "standard" cables work with my devices.

PS: Just because the end that plugs into your PC is a standard USB connector, doesn't mean the end that plugs into your device isn't a proprietary shaped USB connector...

Re:

[Missing.Matter]

The bundled Kinect doesn't come with the adapter so you can connect it to your PC (or older Xbox360).

This wasn't a result of any sort of lockdown attempt. The Kinect has motors in it, which draw more power than USB can provide, so it uses a proprietary connector for the XBox. The adapter to use on the old Xbox requires an AC connection to provide the additional power.

The 360's wireless is NOT bluetooth. You must use MS's proprietary wireless receiver.

Which costs all of $10 on ebay. Anyway, it seems more efficient with power than USB; I've been using the same two batteries in my controller f

Re:

[Missing.Matter]

And by "more efficient with power than USB" I meant bluetooth.

wordplay (n): A pun often missed by grammar nazzis

[VortexCortex]

proprieterroize (V): the introduction of proprietary or incompatible features to an otherwise standard product for the express purpose of limiting the customers' purchasing options in order to artificially drive up the product's cost.

A portmanteau of proprietary and terrorize often used to describe vendor lock in strategies.

"Although the two cables appear identical, I can't use a standard USB cable to charge my GPS; I have to use Garmin's proprieterrorized USB cable instead."

----

Appropriation would mean tha

1MB test database?

[Yvan256]

Why is that website asking permission to create a 1MB "html5 test db"? I can understand doing some testing, but on a live website?

Awesome.

[__aaelyr464]

I love seeing hardware being hacked and used for purposes other than what it was designed for. To me, that is the essence and lifeblood of true innovation and engineering.

Sidenote: why is this article tagged "microshit"? Really?

Re:

[RudeIota]

[blockquote]Sidenote: why is this article tagged "microshit"? Really?[/blockquote]

It's tagged !microshit, which means NOT microshit. ! is a reference to programmatical methods for representing "NOT" equal to.

Next logical Steps....

[Anonymous Coward]

Build a 3D model of your entire neighborhood.
Convert it for use in your FPS of choice (Onslaught/ Tower defense style gameplay a plus).
Add realistic weapon drops and "spawn" points.
Drop in a few hundred AI bots, track their movements to see which positions are most/least effective.

Congratulations, you now have a battle-plan for the zombie apocalypse.

Game Map Makers Rejoice...

[pRtkL xLr8r]

Wouldn't it be great - tweak this with GPS data or something similar, and then find some building that you think would be great for your multiplayer game. Simply walk around the building with a laptop and a Kinect, and after a few sweeps the software models the interior which you could use to import into your map editor...

Re:

[noidentity]

Not a good idea, unless the game involves fluffy bunnies and rainbows.

Meh.

[kuzb]

Call me when a kinect or kinect-like device builds a 3D model that doesn't look like complete ass.

How long until Google's driving down the street w/

[dwightk]

Kinects?