Photosynth Demo

A couple of days ago Microsoft labs released a demo of their new Photosynth software on the web. Photosynth allows the aggregation of social picture networks (a la Flickr) into a completed image in addition to allowing a level of depth to image browsing previously unavailable. There is also a very impressive video of the demo available.

Some impressive things

[kiwicmc]

Unlike the first set of posters I managed to get over my self importance and watched a couple of seconds of BMW ads to see the actual video.

I liked the initial viewing of large quantity of hi-res images and the smooth zoom. The aggregation of many thousand flickr images of the Notre Dame (including one of a poster on a wall) into a 3-D image was fantastic.

C

One step forward!

[Sectrish]

At least now someone at Microsoft seems to know _what_ to buy, this is some pretty amazing technology. I just hope that someday it will be available to other OS'es too.

The Humane Environment

[toQDuj]

This zoom-ability of the first part has a lot in common with the ideas behind Jef Raskin's The Humane Environment http://en.wikipedia.org/wiki/Archy/ [wikipedia.org].

The second part, however, shows marvellous stuff. Especially if what I think he did, was search for patterns in images, and compare those for unique objects to collect a library of images of a single object.

This guy and supposedly his group shouldn't work for Microsoft in my opinion, but would perhaps feel more at home in a fundamental science laboratory. But I think my opinion on this is slightly partial.

B.

Re:Huh?

[timeOday]

I don't think this technology has that much to do with social picture networks in particular, I'm not sure using it to index images is all that compelling. What would be more useful is inputting some images from different angles (or a video) and getting back a .3ds texture-mapped geometric model. Reconstruction of gometry from imagery has been a big research topic for ages but I'm not aware of any effective, user-friendly software to do it.

Just looking at that

[goldcd]

rather fabulous demo, I realize that that would tie in beautifully with the surface computing MS showed last week (which was lovely as a tech demo with little immediate use).
Vista is 'nice' but it's just a progression of what we already know - these tech demos give me a big warm fuzzy futuristic feeling inside :)
If nothing else it shows that MS is innovating again (at last) - Ball's back with Apple and Google now - "Make me more impressed!"

Re:One step forward!

[evohe80]

One thing that amazes me of Microsoft is how, having so many bright people at MS reasearch, most of their stuff is so bad, and/or lacks innovation. (I know part of this came from some other company they bought, but some of it is original from MS, I've read a paper related to this technology).

Every single paper I've seen from MS research is great. Well done!

(from someone developing computer vision on linux)

Data aggregation

[jemenake]

Near the end of his presentation, the guy sums up the technology as taking all of these separate images from various sources on the net and figuring out how they all interlink to present a larger, more coherent picture. He got applause.

My first thought was about the U.S. government's "total information awareness" project, where they're trying to take lots of separate pieces of info (which are already available to law enforcement) and interlinking them all together to provide a more coherent picture... but most people consider that to be evil.

Granted, the government isn't doing it with vacation photos, but the idea, of finding pieces of data that are related and finding out *how* they're related, is the same. The difference in people's reaction to it, I can only attribute to the fact that people see the photosynth guy as good, and the government as evil. But I don't agree that the goodness or evilness of an action is solely determined by the goodness or evilness of who's doing it. The U.S. gov't tries this and fails. It expects that it can invade foreign countries and install friendly governments and torture people because it's "the good guys", yet the soviet union did those same things during the cold war and we admonished them for it because they were "the bad guys".

So, where am I going with this rant? My point is this: You can't blame somebody for connecting the dots. In fact, that seems to be one of the things that we, as humans, are particularly good at. So, if you think that this photosynth thing is fine, then I think you've got to grant that the TIA project is fine. Now, you could argue that some particular bits of information shouldn't be available, but the piecing it together to form a more coherent picture... I can't come up with an argument against it that I consider defensible. Sure, it makes me uncomfortable, but that's not an "argument".

Vast Desktop...

[Slur]

Actually, as I looked at the demo, I couldn't help feeling like all that virtual space was looking like a damn nice desktop environment. Nevermind the part of the demo with a flat-on scrolly-zoomy desktop, as nice as that would be (Seems obvious in a way too... And wouldn't it be nice if Leopard had that instead of "Spaces" ?). But imagine the notion of opening up an application and instead of just popping up a new window it creates a new space - within the desktop virtual space - and brings you into it. You can always pull back and move around to another window or workspace, but while in it you'd be totally immersed.

I dunno, I just like the notion of immersive environments, especially for conceptual learning. I think we're going to see a prevalence of this kind of interface in the near future.

Re:I tried to WTFA

[cheater512]

Which were then manually screened to weed out the crap ones.

Re:I tried to WTFA

[Anonymous Coward]

Actually I'd say the typical MS portion comes from the fact that this was originally intended to build 3D scenes from a common photoset. Since it can't do that very well (and various incanations of this are already been done for free on the web) it's instead been reconfigured into a social tool - taking advantage of the database it generated instead.

I'd be a bit more impressed if the performance of the photo panning software is maintained on consumer hardware. However, keep in mind that even this isn't new, as a very similar technique is being used to give the appearance of higher polygon counts in games for years. Procedural geometry counts etc.

Re:It's here! Web 2.0 is HERE!!!

[adisakp]

I could never understand why we use raster-imaging for computer games because it's a squillion times quicker than ray-tracing, but nobody had applied the same idea to other applications.

I don't think that basic rasterizing engines are the limit. The limit is that the source data for all these pictures are tens or hundreds of gigabytes (and in the future, conceivably terabytes). Somewhere in the assembly and cross-correlation of all this data, they have to be generating LOD's (levels of detail) and dynamically loading / managing MIP-maps to keep the loaded dataset to a reasonable level. This is the hard part since "reasonable level" for loaded imageset size is probably currently a couple hundred megabytes or much less. You can probably load more data into RAM but try maintaining a 60FPS refresh with a gigabyte of textures - especially on a laptop or basic computer.

Once you've done this you can use a variety of display techniques... the main reason to use basic texture-mapping / flat rasterization is that sources are photos which are basically a pre-lit "flat" textures.

However, if you can generate a 3-D model and can separate lighting / color information (perhaps using combinations of day and night pictures or varying lighting from different photographs), it would be then possible to perform simple ray-tracing or other hybrid renderers -- think how cool it would look to have a dynamic artist's sketchpad with these images "penciled" in realtime. There are already high-frame-rate (near-realtime) ray tracing demos already out there for CELL and X86 that render moving images at a lower-res for higher-interactive frame rates and then when not-moving, render high-quality image stills that are quite impressive.

Re:mod parent up

[sych]

hmm... I notice that in that demo, it's running on Gnome. But now, Microsoft Photosynth "runs only on Windows XP SP2 and Windows Vista".

linux & mac can try the old demo

[cyanescent]

there's a demo that is OS agnostic (java applet) on the washington Phototourism page [washington.edu].

Re:Some impressive things

[Korvar]

Step 1) Get lots of photos of a given subject

Step 2) Process these photos and find "similar points"

Step 3) Start correlating points on separate photograps

With enough points in common on two or more photographs, you can begin to get an idea of the 3D relationship between the points, and also the cameras taking the photographs.

There are applications that allow you to do Step 2 manually (the clearest example of the process I found was http://www.3dphoto.dk/UK/technique-UK.htm [3dphoto.dk]), but Photosynth appears to do it automagically, which is the cool part.

Re:Windows only.

[naoursla]

Yep. I run Windows 2003 Server at work and it doesn't work on that either. I am pretty sure the Photosynth team wants it to run on more platforms. This is still a new product that is barely out of the research stage.

Re:I tried to WTFA

[SpryGuy]

A friend of mine asked, "Doesn't that violate about a billion copyrights?"

I shrugged. Can someone take my photos on Flikr and use them to create new content without my approval?

Re:I tried to WTFA

[BeanThere]

I did WTFA, and I obviously paid more attention than you did, because the most interesting part of the demo is preceded by a comment along the lines of "I'd like to jump straight to one of Miller's original datasets and this is from an early prototype of Photosynth that we first got working this summer" ... I repeat myself again, they developed the cool technology AND THEN AFTERWARDS it was bought into by Microsoft.