Follow Slashdot blog updates by subscribing to our blog RSS feed

 



Forgot your password?
typodupeerror
×
Graphics Software

Computer Scientists Scour Your Holiday Photos 156

Barence writes "Hundreds of thousands of images on Flickr are being used to teach a program to determine the geographic location of an image, simply by looking at it. The program attempts to mimic the way that humans can deduce the location of an image by searching for visual clues, such as similarities to pictures or locations they have seen previously. In its current state it can guess the location of a photo to within 200km, 16% of the time — extremely accurate given the complexity of the problem."
This discussion has been archived. No new comments can be posted.

Computer Scientists Scour Your Holiday Photos

Comments Filter:
  • 16% percent of the time it works every time.
  • by tomalpha ( 746163 ) * on Wednesday June 18, 2008 @09:02AM (#23838477)

    The paper [cmu.edu] referenced in the article has an interesting density map of where their 20 million source photos were taken (ok, so they only ended up using 200 or so of these). It says it uses a logarithmic scale, and seems to imply that the vast majority of photos available to them on Flickr were taken in one of only a handful of locations:

    • London
    • Paris
    • New York
    • Washington
    • Los Angeles
    • Tokyo

    Ok so there are a couple more than this, and my geography is appalling, but these seem to be the only areas that are are coloured red.

    • by elguillelmo ( 1242866 ) on Wednesday June 18, 2008 @09:19AM (#23838775)
      then... if there are 6 sources of pictures, by blindfold guessing you'll get it right 16.66..% of the time
      • by SteveAyre ( 209812 ) on Wednesday June 18, 2008 @09:36AM (#23838999)
        So it's actually less accurate than if it just guessed? :)
      • Re: (Score:3, Informative)

        There are way more than 6 sources, check the map.
      • Re: (Score:1, Informative)

        by hagnat ( 752654 )
        yeah, but only if the program chose the same city 100$ of the times, which i guess the guy who created this program would realize its malfunctioning and then fix it.

        Assuming that, from the 200 pictures set used to validate the program, the same amount of pictures were provided for each city (32 pics of each city) and that the program chosen then in the same proportion (the program identified [correctly or not] each city 32 times), you would have 16.6% * 16.6% = 2.75% chance of correctly identifying the city
        • Re: (Score:3, Interesting)

          I'm not with you in the argument. Assuming there are just 6 cities, and that the proportion from each is the same: 1/6, if you guess randomly you are right 1/6 of the time. It's just like a die... Then, if there are zillions of sources but only six cities amount for most of the pictures, then randomly guessing among them will get you close to this 1/6...
          • Re: (Score:3, Insightful)

            by hagnat ( 752654 )
            exactly, its like a d6 die... if i have to identify one (and only one) picture i'll have a 1d6 chance of getting it right

            but when you have to roll this for 200 cities, also chosen by a 1d6 roll, you have two dies being rolled 200 times, and you want to know how many times both dies have the same value
            • Re: (Score:3, Funny)

              by danaris ( 525051 )

              No, I think you're misunderstanding how this works.

              Given that there are 6 locations where all (or nearly all) of the pictures being used were taken,

              Deriving from this that the probability of getting any 1 of these right by random guessing is 1 in 6,

              Given that the accuracy averaged over 200 pictures is slightly less than 1 in 6,

              The computer does slightly worse than chance.

              However, my guess is that the first given is not entirely correct ;-)

              Dan Aris

            • but when you have to roll this for 200 cities, also chosen by a 1d6 roll, you have two dies being rolled 200 times, and you want to know how many times both dies have the same value

              Ummm... yeah. And it's still 1/6 times. You failed statistics, didn't you?
            • by raehl ( 609729 )
              but when you have to roll this for 200 cities, also chosen by a 1d6 roll, you have two dies being rolled 200 times, and you want to know how many times both dies have the same value

              And you do realize that if you throw two dice, there are 6 results out of 36 possible where the dice come up with the same number...

              Also known as ONE IN SIX!
        • If there are 6 groups, then the chance of randomly guessing the correct group is 16.66%, not 2.75%.

          Let's say we pick the same city every time.

          We have a picture. It's from one of 6 locales. We pick a locale C. What are your chances that you're right? 1 in 6.

          We have another picture, and pick locale C again - chances? 1 in 6.

          Another picture, we pick locale C, and again, chances are 1 in 6.

          You seem to understand this concept when you say the chances are 1 in 6 when we pick the same location each time.

          So, l
      • by Rich0 ( 548339 ) on Wednesday June 18, 2008 @10:39AM (#23839995) Homepage
        If you picked a random point on the globe, and I picked a random point on the globe, then they would be within 200 miles of each other a few percent of the time. If the only logic used by the software was to determine whether or not any land was visible it could probably increase that probability significantly - the earth doesn't have that much dirt poking out of the oceans. 200 miles is a VERY large area of land.
      • by Falkkin ( 97268 ) on Wednesday June 18, 2008 @11:23AM (#23840739) Homepage
        If you RTFP, Figure 6 shows the difference between the performance of the algorithm and random guessing. It's pretty significant.
      • Re: (Score:3, Funny)

        then... if there are 6 sources of pictures, by blindfold guessing you'll get it right 16.66..% of the time
        If they're smart, they'll only ever guess one of two points:

        point a) Halfway between NY and Washington DC
        point b) Halfway between London and Paris

        This should give them a better than one-in-three chance of being correct to within 200km, as long as their program can take a decent stab at guessing which of the two sets is the more likely....
  • by stainlesssteelpat ( 905359 ) on Wednesday June 18, 2008 @09:03AM (#23838501)
    Dude, that's as accurate as my girlfriends map navigation. *sigh*
  • by 14erCleaner ( 745600 ) <FourteenerCleaner@yahoo.com> on Wednesday June 18, 2008 @09:06AM (#23838553) Homepage Journal
    I'll guess...New York City, without even looking at the pictures that should get me in that ballpark.
    • Re: (Score:3, Funny)

      by Bearpaw ( 13080 )
      Wait, don't tell me, let me guess where you're from ...
      • Re: (Score:1, Funny)

        by Anonymous Coward

        Wait, don't tell me, let me guess where you're from ...
        Anyone from NYC calls it "the city", so obviously not from there. But the rest of the world is inconsequential anyway. For instance, LA is just the syllable between SO and TI.
      • Wait, don't tell me, let me guess where you're from ...

        I could show you a picture of it, and you'd probably have better a than 16% chance of guessing. (Or you could just look at my profile.)

  • What I need (Score:4, Interesting)

    by Intron ( 870560 ) on Wednesday June 18, 2008 @09:07AM (#23838583)
    ... is a program that will remember the names of the people in the photos.
  • heh (Score:5, Funny)

    by Kingrames ( 858416 ) on Wednesday June 18, 2008 @09:07AM (#23838591)
    Show it a picture of the andromeda galaxy and throw its statistics way off.
  • <strike>extremely accurate</strike>an interesting start.
  • I'll no longer have to spray paint "I was here" for people to know where I was.
  • by SlashTon ( 871960 ) on Wednesday June 18, 2008 @09:10AM (#23838637)
    of the time...

    (Not counting those rich bastards who can afford taking a holiday on the ISS).

    • 1/2 of the circumference of the earth is just over 20,000 km, so you are within that distance by surface travel 100% of the time.

      OK, maybe you meant the through-the-earth distance. At the poles, that's 6,356.8 * 2 = 12,713.6 km. At the equator, it's slightly more.
      • by slart42 ( 694765 )
        he meant guessing the center of earth each time. That will usually get you a distance of ~6378km from any photo location.
      • OK, maybe you meant the through-the-earth distance.

        His math is OK. He meant his guess is "the center of the Earth" for every unknown location.

      • Yes, I meant through the Earth distance and yes I did manage to use the radius.

        That will teach me to post before drinking my coffee...
    • The ISS being only about 350km above the ground, and orbiting the earth more than ten times a day, it's likely that during a week on the ISS, that toursit would be within 2000km of any arbitrary place within a latitude range around the equator (sorry, I don't remember how large that range is).
  • by UnknowingFool ( 672806 ) on Wednesday June 18, 2008 @09:13AM (#23838681)
    Just like all statistics getting a good sample population is very important. If this program were to sample the /. population, it would come to one of two conclusions.
    1. We have no holidays as we don't socialize.
    2. We all live within 1.0 km of a basement. :P
    • Re: (Score:2, Funny)

      Just like all statistics getting a good sample population is very important. If this program were to sample the /. population, it would come to one of two conclusions.
      1. We have no holidays as we don't socialize.
      2. We all live within 1.0 km of a basement. :P
      But it couldn't sample the population here, on account of the tinfoil hats.
    • Re: (Score:3, Funny)

      by raehl ( 609729 )
      That was much funnier before I started working out of my basement.
    • by raddan ( 519638 )
      Hey, I resent that. I live in the attic now.
  • by Bombula ( 670389 ) on Wednesday June 18, 2008 @09:14AM (#23838695)
    The Photosynth multi-resolution and image-recognition tech demonstrated at TED looked cooler if you ask me:

    metacafe link here [metacafe.com] and TED link here [ted.com].

    • Re: (Score:2, Interesting)

      And has already been covered on /. over a year ago [slashdot.org]. Hell, it's been used to solve a crime in a popular police drama, so it is officially old news.
      • Pretty sure he didn't say, "ZOMG! LOOKS AT THIS NEW TECH!" He just said it was cooler... and provided a link so you'd know what he was referring to. Nothing wrong with that. Personally, I happen to agree with him. That was the first thing that came to mind when I read about this new project too.
  • by Anonymous Coward on Wednesday June 18, 2008 @09:15AM (#23838707)
    Machine is shown hundreds of thousands of holiday pictures from Flickr.



    Scientists surprised to discover it is possible for a machine to loose will to live.

  • Source code (Score:5, Funny)

    by RandoX ( 828285 ) on Wednesday June 18, 2008 @09:18AM (#23838751)
    It's a for loop that spits out "Your mom's basement".
    • by fitten ( 521191 )
      LoL... I don't have any mod points or you'd get one :)
  • This is very hard (Score:5, Insightful)

    by mzs ( 595629 ) on Wednesday June 18, 2008 @09:19AM (#23838765)
    Look at this set of pictures:

    http://htmlhelp.com/~liam/Hawaii/Kauai/WaimeaCanyon/ [htmlhelp.com]

    Would you know simply by looking at the photos without the sign that this was not say the grand canyon? The whole correct to 200 km aspect is troublesome when the state of the art in computer vision cannot yet even answer that this is a picture of a canyon.
    • Re: (Score:1, Informative)

      by Anonymous Coward
      I've already modded you so posting anonymously to preserve the mod.

      To answer your question, yes, I would but only because I would know what to look for. In your case, the walls are not steep enough, too much vegetation and no thin grey haze hanging over the canyon.

      Regardless, your point is still valid.
      • by fitten ( 521191 )
        I was going to post exactly the two points about vegetation and the steepness of the walls. But as you also say, there are many similar looking places all over the world. Hiking pictures in a forest, for example, would be almost impossible to match, even if you could narrow the lattitude/altitude down by the species of plants seen in the image unless the plants were unique to a particular forest/mountain/etc.
    • by cheebie ( 459397 ) on Wednesday June 18, 2008 @09:33AM (#23838955)

      Would you know simply by looking at the photos without the sign that this was not say the grand canyon?


      Yes, because there aren't 746 helicopters flying over it.
    • Would you know simply by looking at the photos without the sign that this was not say the grand canyon?

      Quite possibly. Although in this case, the answer was given away in the URL and so claiming "I would have known" is not really a credible claim. (I've been to Waimea Canyon recently.) I do a LOT of hiking and backpacking, and I tend to study the areas I am traveling through very closely. I surprised myself the other day when a blurry photograph of a trail was shown on the evening news and I identified

      • I said out loud, "That looks like the Wildwood trail a few miles north of Germantown Road." And of course, it was. And that wasn't an isolated event.

        I don't think I'm special for being able to do this. It's just paying attention to detail, like the particular ratio of the populations of certain kinds of plants, what kind of moss is growing on the trees, the quality and appearance of the trail surface, how the trail slopes. These things stick permanently in my brain.

        You probably are also not special in havin

        • You probably are also not special in having selective memory and/or reinforcement bias, which would lead you to recall that particular success, or others like it, and forget or disconsider failures. And we /. readers are also not specially prepared to estimate how many TV Viewers had the exact same reaction as you, except they erred the pictured location and then decided *not* to write about it (selection bias).

          My ability to recall terrain is interesting enough to me that I pay attention when I try to d

    • Waimea is pretty distinctive and I have no problem recognising it. I can't imagine confusing it with the Grand Canyon. I suspect an algorithm like SIFT [wikipedia.org] could also easily distinguish between them, if enough of both canyons was pre-processed.
    • That actually looks nothing like the grand canyon to me. The geography is distinctly different in those photos. Too much vegetation, the plants I see don't appear to by typical of the area, not steep enough drops, too many rolling hill-like areas, no jagged edges, not enough color layering in the rock, among other things.
    • There's an upcoming paper coming from MIT on this topic, Recognition of Natural Scenes from Global Properties: Seeing the Forest Without Representing the Trees [mit.edu] that proves this isn't as hard as you might think.

      To sum up this massive paper in a very small (and likely highly imprecise) nutshell, building models up from basic objects (the traditional method) is only one way to approach this. Using this method, you are correct; it's impossible to understand what a canyon is. Using the new global properties

    • Re: (Score:3, Interesting)

      by Facegarden ( 967477 )
      I would know it was Waimea Canyon because I've been there, and to the Grand Canyon, and as others have said the amount of vegitation and steepness of the walls differentiate the two very clearly, and i know that canyons that large are rare (on earth, above water), so it is unlikely to be some other place i just haven't been. :)
      But i know a lot more than computers.

      This all brings up an interesting point though... When you're in an unfamiliar place, but your friend knows the area, they can always tell you whe
    • Re: (Score:3, Insightful)

      by SpinyNorman ( 33776 )
      Would you know simply by looking at the photos without the sign that this was not say the grand canyon?

      Yes, because the GC doesn't have so much vegetation growing down the sides.

      The whole correct to 200 km aspect is troublesome when the state of the art in computer vision cannot yet even answer that this is a picture of a canyon.

      The two things arn't related. You don't need to know it's a canyon to be able to locate it - you just find the closest match in your database and give that as the location. You only
  • by stoofa ( 524247 ) on Wednesday June 18, 2008 @09:19AM (#23838767)
    OsamaBinLaden2001 has deleted his account
  • by jesdynf ( 42915 ) on Wednesday June 18, 2008 @09:20AM (#23838783) Homepage
    200km, 16% of the time? I guess that sounds sorta neat... except that 84% of the time, it's off by more than 200km. Now, we know that the earth's circumference is 40000km, and it follows that nobody can ever be more than 20000km from any location on Earth.

    So 16% of the time, it's accurate to within one percent of the TOTAL RANGE OF ERROR. The other 84% of the time, you're on your own. I wonder if I could manage that kind of accuracy just by sampling colors, classifying them by terrain, and then just picking a likely spot at random.
    • by SQLGuru ( 980662 ) on Wednesday June 18, 2008 @12:05PM (#23841453) Homepage Journal
      Surface area of a sphere = 4*pi*r^2
      Radius of the Earth = 6 378.1 kilometers (from Google: http://www.google.com/search?hl=en&rlz=1T4ADBS_en__230US231&q=radius+of+earth [google.com] )

      Surface area of Earth: 510,065,600 km2 (http://www.google.com/search?hl=en&rlz=1T4ADBS_en__230US231&q=surface+area+of+earth)

      Percentage of surface area that is land: 29.2% (http://www.physicalgeography.net/fundamentals/8o.html)
      Surface area of Earth that is land: 148,940,000 km2 (same source)

      Area of a circle = pi*r^2
      Radius of "target" = 200km
      Area of target = 125663.7km2

      Number of "target" areas that could fit on the surface of the Earth covered by land (assuming too few landmarks to identify pictures take over water, so they will be excluded): 1185.2

      Chance of being right by pure dumb luck - 1 in 1185.2

      Layne
    • Re: (Score:3, Insightful)

      by raddan ( 519638 )
      Your post made me think of something-- the Earth is really BIG! But I'm a nerd, so I had to prove it to myself.

      The surface area of the Earth, not counting water, is 510,072,000 km^2, according to Wikipedia. So, that's roughly 5.1 x 10^14 m^2. Again, according to Wikipedia, there are currently 6.67 x 10^9 people on Earth. That translates to about 7.65 x 10^4 m^2 for each person! In terms that Americans can understand, that's roughly 14 football fields (or to choose a landmark close to me, a little bi
  • I only skimmed the article but at least we are getting some good scientific use out of all the social networking gobbledegook we have floating around out here.
  • by RichMan ( 8097 ) on Wednesday June 18, 2008 @09:26AM (#23838863)
    From the looks of the test selecting London all the time would have a
    1/6 chance = 16.67% chance.

    They need better double blind testing and a more diverse set of geographical locations.
    • by zappepcs ( 820751 ) on Wednesday June 18, 2008 @09:49AM (#23839173) Journal
      It's a bit worse than that I think. Trying to identify location of a picture by looking at it in the way that humans do requires that you know the location. As an example of why this implies intimate knowledge to be useful, everyone knows of the big statue of liberty. Not just anyone can guess that your holiday picture with the non-descript base in the background was taken at the base of the statue of liberty. The same goes for > 90% of other places in the world.

      Another example: The forests on planets on the show Stargate One, are they in Missouri, Montanna, Canada? Just looking at them will not necessarily tell you anything unless you are intimately familiar with the actual location.

      A photo in Syntagma Square in Athens may look like it was taken in Central Park in NYC if not enough of the background was included. It will take huge amounts of data and photos to get anywhere close to what a human can do at this job, and even then it is limited to only what it has seen before.

      Other knowledge plays a part too. London bridge is now in Arizona (I think) as it was moved brick by brick and re-assembled. Seeing the bridge does not now mean you know where it is .... it's a trick question. The point is that you need additional information as well. A picture that is a beautiful park setting that has a kangaroo in it? is it in Australia, or a zoo? Additional information is required.

      Hats off to them for working on it. It's a tough problem.
      • Re: (Score:3, Interesting)

        by darkstar949 ( 697933 )
        Wikipedia to the rescue - http://en.wikipedia.org/wiki/London_Bridge [wikipedia.org] - London Bridge is actually pretty interesting as well as the original one was demolished, the second one was moved to Arizona, and the third one currently standing in London was built between 1967 and 1972.
      • Re: (Score:3, Interesting)

        by Bat Country ( 829565 )

        I'd imagine great work could be done by examining light intensity and coloration (atmospheric red shift) vs date stamp on the image (working from RAW with some camera data), they could guess the latitude fairly accurately. By similar methods you could figure out pollution levels, thus narrowing the sample range further.

        Additionally comparing geometry could help factor out region with plant recognition fairly well also. You're not going to see a saguaro in Kentucky unless you're in a botanical garden. They

    • Re: (Score:3, Interesting)

      by klenwell ( 960296 )

      From the looks of the test selecting London all the time would have a
      1/6 chance = 16.67% chance.

      Indeed, not very impressive for London.

      Look at this guy's [wikipedia.org] claim for basic audio analysis:

      "Simply phonetics. The science of speech. That's my profession; also my hobby. Happy is the man who can make a living by his hobby! You can spot an Irishman or a Yorkshireman by his brogue. I can place any man within six miles. I can place him within two miles in London. Sometimes within two streets."

      And that was almost a century ago!

      • Re: (Score:3, Insightful)

        by Woundweavr ( 37873 )
        Accents used to be much more pronounced pre-radio/mass media. I know in Boston, individual streets had slight variations so that you could tell the neighborhood of a person by their accent. However, now that a large percentage of the words we hear are from movies/TV/radio our accents get washed out.
    • Double blind testing for a computer? I think you misunderstand the point of double blind testing.
  • ...by lots of photos of amateur, um, "naturist" pics?

    "Hey, Frank? Why are there giant palm trees in Washington D.C.? And why is the Washington Monument pink no...

    Oh, never mind."
  • by Anonymous Coward
    Surface Area of Earth
    510,065,600 km2
    *.33 = 170021866 \\Estimate 2/3 of earth is ocean
    / 3.14 * 200^2

    =8%
  • by BMonger ( 68213 ) on Wednesday June 18, 2008 @09:51AM (#23839209)
    I propose we all take pictures with blue screen in them (not the whole background, just "enough") and then write a script to randomly replace the blue screen with alternative locations every time the picture loads.
  • I hope they can extend this to identifying the location down to the nearest street. It's possible to do this if there are some obvious hints like a postcode on a street nameplate. Having a webpage address or telephone number on a shop display can help too. Even a original shop name, or a unique combination of high street stores side-by-side can help.
  • The problem with geo-tagged flickr photos is that in many places the detail on the maps and aerial shots provided isn't defined enough to allow an accurate placement.

    The even bigger issue is that, although some cameras now have GPS, the majority of geo-tagged shots are placed manually by humans who often get it wrong or deliberately place their photos onto a more popular location just to increase their traffic.
  • by erroneus ( 253617 ) on Wednesday June 18, 2008 @10:17AM (#23839633) Homepage
    I'd like to present this with Moon landing pictures to see where the moon landing was staged! (hahaha... love it)
  • In its current state it can guess the location of a photo to within 200km, 16% of the time â" extremely accurate given the complexity of the problem."

    60% of the time, it works every time. They've tested it.
  • You could probably get within 200km better than 16% of the time if you always guess the photographer's home city.

    I also wonder how well you could do by checking the photo's timestamp, then examining the shadows to determine the sun angle.

    I once correctly guessed "Quebec" in a National Geography Bee when asked in which Canadian province a picture of a particular attraction lay. My clue? The sign on the front of the bus was in French.
    • I also wonder how well you could do by checking the photo's timestamp, then examining the shadows to determine the sun angle.


      You need accurate timestamps to calculate that.
      My camera does not adapt automatically to DST and I forgot to make the change myself.
      Moreover, I do not think of changing its clock either when I go visiting other countries...

      "hey, sunlight at midnight, must be in North Pole"
  • by hedu ( 1215514 ) on Wednesday June 18, 2008 @11:33AM (#23840933)
    Reminds me of the experiment done in a Dutch military lab a couple of years ago. They trained a neural network to recognize whether a photograph taken out on a country road had a military vehicle in it or not.

    The system recognized the photos from the training set perfectly, but did no better than random on images fed to it that were taken at different times.

    Turns out all the training shots with a military vehicle in it had been taken on a sunny day, and the control shots without one had been taken when it was overcast. The system had been trained to recognize a different thing from what they intended!
    • by querist ( 97166 )
      That's the "joy" of artificial neural networks (ANNs) - they will focus on the things that change and will learn to ignore the "noise". In your example, the lighting was the "key" that was consistant in the two types of pictures.

      My research involved ANNs (specifically, Self-Organizing Maps, but I've worked with the more "traditional" feed-forward-back-prop nets I suspect you were discussing here). From my own experience, what the parent post said is quite probable.

      I guess the folks in the Dutch military did
  • Do you think it helps or hurts that my photos on Flickr have titles like "Tokyo - Ueno park"?
    • Re:It helps.. (Score:4, Insightful)

      by gstoddart ( 321705 ) on Wednesday June 18, 2008 @12:29PM (#23841817) Homepage

      Do you think it helps or hurts that my photos on Flickr have titles like "Tokyo - Ueno park"?

      For the researchers, it probably helps. They chose pics that had either GPS or location information -- so they could manually verify where the photos originated.

      If they started out with a bunch of pics they didn't have any location information about ... they'd never be able to measure their results. ;-)

      Cheers
  • Is it in sample performance ( using the training set ) or out of sample performance ( using unseen images ) ?
    How were the picture selected ? Pick the geographical center of the US east coast each time and you should get a decent result for example⦠TFA says it's 20 times better by random, but do they mean purely random (New York and the middle of the Pacific equally weighted) or random based on the distribution of the geographical locations of the set ?

    What is the distribution of the results ? It
  • Google (Score:4, Interesting)

    by sckeener ( 137243 ) on Wednesday June 18, 2008 @01:50PM (#23843099)
    Google should get behind this. I think their Picasa would benefit from it.

    Generate some autotags.

    What would be nice also is if they had a feature where if you labeled someone in a picture, if you uploaded another picture with that person in the picture, the program would prompt to auto tag.

    I've been going through old family photos and it would save so much time if the programs I am using autolabeled based off details in the picture.
  • Does it cue on the date, time, and sunlight conditions? Because that's how I would write it. A computer could easily narrow the geographical location of a photo that way, but humans don't really do that without a conscious effort.
  • If I post a few photos from a recent endoscopy, can this program guess where I had it done?

    Or, at least, tell me if my piles are recovering?

"Being against torture ought to be sort of a multipartisan thing." -- Karl Lehenbauer, as amended by Jeff Daiell, a Libertarian

Working...