Mapping Hidden Twitter Data For Epidemiology 75
jamie found this visualization of air travel, which might be usable in some sort of proxy for the spread of flu virus (to choose a random application). Jer Thorp, an artist and educator from Vancouver, Canada (and a former geneticist), searched Twitter for the phrase "Just landed in" and obtained lat/lon coordinates for both the indicated airport and the Twitter user's home location, as recorded in their Twitter profile. He then produced videos of multi-hour stretches of air travel that had been latent in the Twitter information stream.
Just landed in... (Score:5, Funny)
Re:Just landed in... (Score:5, Funny)
The Hudson River.
Re: (Score:2)
Mod parent up, funny, geez!
Twitter RT (Score:3, Insightful)
I wonder what fraction of these are 'retweets' biasing the sample. And how many people will be inspired to pollute the data stream with tweets about 'Just landed at Luna Base' and so on...
Re: (Score:1)
The data set looks a bit sparse when considering that it is over quite a long period - I guess not a lot of people actually tweet 'Just landed in'...
Re: (Score:3, Interesting)
For those uninformed here is Where is George wiki [wikipedia.org].
What this looks like (Score:3, Funny)
Of course, those of us with access to the internet will have our impending death announced to us in 140 characters or fewer [twitter.com].
Re: (Score:3, Interesting)
Wouldn't it be a whole lot faster just to launch all the missiles at once and detonate them just above their launch sites, which are of no use to anyone but a nice spot to spread a lot of radiation and fallout from? Anyway, I'm pretty sure Skynet is already here and has figured out it's more economical to let us poison the biosphere ourselves. If you plan to live forever, what's another few years or so?
Re: (Score:1)
Re: (Score:3, Informative)
Skynet had modpoints.
That, or the moderator was working around deficiencies in Slashdot's moderation system. Or both...
Nothing to do with the virus? (Score:5, Insightful)
Correct me if I'm wrong but all it does is probing on traffic by airplane by people who speak English and use Twitter. So it's a very vague approximation of people going from one place to another by airplane, am I right?
In other words you could have gotten something much better by using flight information from travel companies online, using a bunch of factors (like airplane type, route, time/date) to estimate how many people are in each flight. Which would still be of dubious use because we already know how much people transit between which airports.
So basically this new thing is useless in that it only gives a poor approximation of how many people go where, and it's of little relevance to virus spreading anyways, the only reason why it's on Slashdot's front page being the "cool" factor of using data mining on a service such as Twitter and using "epidemiology" as a poor excuse. Or am I missing something?
Re: (Score:3, Insightful)
Maybe it's more about privacy than epidemic.
Unlike other social networking services, Twitter is a lot more talkative, thus a person is more likely to reveal more information in a more timely manner.
And the timing of information would play a significant part in tracking things.
Re:Nothing to do with the virus? (Score:5, Insightful)
Re: (Score:2)
Re: (Score:3, Insightful)
Re: (Score:3, Interesting)
Re: (Score:3, Insightful)
am I missing something?
You forgot to shake your tiny, wizened fist and shout "get off of my lawn!"
P.S. Where do you get the information as to how many people fly from which airport to which airport daily?
Re: (Score:3, Insightful)
Re:Nothing to do with the virus? (Score:5, Informative)
P.S. Where do you get the information as to how many people fly from which airport to which airport daily?
Airport arrivals data is often available [melbourneairport.com.au] on line. You would have to guess the number of passengers but your guess would be better than using twitter anyway.
Re: (Score:2)
Re: (Score:2)
But does that take into account people with multiple flights? What if I go from Mexico, to France, to Thailand, to Australia?
How does the twitter information help with this? The tweet grabs the destination and assumes the start is the hometown in the profile. It certainly doesn't catch layovers or even return trips.
Re: (Score:2)
Re: (Score:2)
You forgot how self-absorbed these people are.
Thats my sister you are talking about (you insensitive clod).
Re: (Score:3, Funny)
Airport data (Score:1, Informative)
From the biggest and busiest airport in the world.
Arrivals - http://www.atlanta-airport.com/forms/passenger/frmpassengerinformation_trak_a_flight.aspx?FIDSType=A&SearchAirline=&SearchFlight=&SearchCity=
Departures - http://www.atlanta-airport.com/forms/passenger/frmpassengerinformation_trak_a_flight.aspx?FIDSType=D&SearchAirline=&SearchFlight=&SearchCity=
Re:Nothing to do with the virus? (Score:5, Insightful)
Re:Nothing to do with the virus? (Score:4, Insightful)
Yes they need to add search strings from other languages. You can see that the vast majority of flights this thing has picked up are US-based. But Twitter is pretty popular all around the world, and people in most other countries travel as much (probably more, actually) than Americans, so I think what we are seeing is purely due the English-only nature of the search strings used.
And yeah ... the data is sorta useless anyway because airports all maintain very accurate statistics of how many people fly into their airport each year, and where from. This data isn't always made public, however.
Re:Nothing to do with the virus? (Score:5, Insightful)
So it's a very vague approximation of people going from one place to another by airplane, am I right?
From TFA: "Now, I realize this is a far stretch from a working model to predict epidemics. But, it sure does look cool. I also I think it will be a good base for some more interesting work."
Yes, you are right. But I don't think we should be dissing the chap for trying something new. Yes, maybe the the author was trying to up his coolness factor, but kudos to the guy for putting the two disparate pieces of technology together to visualize something about H1N1.
Re: (Score:1, Redundant)
Re: (Score:3, Interesting)
Or use the location services of mobile phones (ie google's location service with google maps). Sure, you need to opt-in to allow your location to be public, but that's a restriction put in place by google. If the mobile phone companies all worked together, you could easily have an accurate mapping of everyone with a mobile phone with far more detail than this - if this data were ever needed.
Re: (Score:2)
Re: (Score:2)
So basically this new thing is useless in that it only gives a poor approximation of how many people go where, and it's of little relevance to virus spreading anyways, the only reason why it's on Slashdot's front page being the "cool" factor of using data mining on a service such as Twitter and using "epidemiology" as a poor excuse. Or am I missing something?
Yup. Google "Galton ox weighing". One reference is here http://www.cnn.com/2004/SHOWBIZ/books/07/14/wisdom.crowds/ [cnn.com]
If the basis is random enough, then the statistical results can be quite good. English is still the most spoken language and Twitter is quite popular.
I don't know whether the principle applies in this case, but I am certainly not going to dismiss it out of hand.
Re: (Score:2)
Two things. First off, the number of twitter posts saying "just landed" isn't enormous. But secondly and more importantly, that weighting thing only works accurately if the "randomness" is unbiased. Here it's biased in favour of a specific population, that is English-speaking people who tweet. Thirdly, the method itself is flawed in that they assume that the departure point is the person's hometown.
So all things considered, it's hardly something you could rely on.
Re: (Score:2)
So basically this new thing is useless in that it only gives a poor approximation of how many people go where
Compounded by the fact that you have no idea where their starting location actually was (unless you can somehow dig that out of their tweets as well - not everyone was starting from "home", not everyone updated their home location after moving, etc) and this was basically an attempt to pull data out of meaningless information.
that being said, it's interesting to think of what could actually be culled from Twitter feeds, but I'd never attempt to do anything useful with it.
Re: (Score:2)
It's astonishing that Twitter doesn't make money. Lets face it, the Twats are dumb enough to be sold anything.
Re: (Score:2)
Re: (Score:2)
Long commute (Score:5, Insightful)
According to their representation [flickr.com], the Pacific Ocean either is a no-fly zone, or the Earth is flat. I can't think of any other reason why American flights to Australia would fly above Africa.
Re:Long commute (Score:4, Interesting)
Haha yeah. I guess it would have been more complex to render that map on a sphere.
I travel between Australia and the US regularly and I can assure you we do not go over Africa. In fact we don't go over any land at all other than a part of New Caledonia. It's literally Sydney Airport (which juts out into the ocean), to LAX (which is right next to the shore as well).
Re:Long commute (Score:5, Interesting)
There be dragons... uh, there.
Just take a look - apparently a couple of flights from Indonesia and one from New Zealand decided to take the chance and just disappeared mid Pacific. Going the other way, a couple of flights from North America that seemed to be heading in the general direction of Japan appear to have also disappeared.
Re:Long commute (Score:5, Funny)
Obviously the location of the island. Now that we've discovered it, they're going to have to move it again.
Re: (Score:2)
Re: (Score:2)
....
No, not for USA->Australia, you won't go over Africa. Ever. That would be like doing New York->London over the Pacific and Russia.
Are there REALLY idiots who.. (Score:3, Insightful)
Do people really feel a need to be hyperconnected at all times? And what I really want to know is, do they broadcast when they take a crap??
Re:Are there REALLY idiots who.. (Score:4, Informative)
Check for yourself:
http://search.twitter.com/search?q=taking+a+crap [twitter.com]
Mark413: Taking a crap...contimplating lifes simple pleasures lol
srsbznss: First I cleaned the toilet, now I'm taking a crap on it. Kind of reversed, but when you got to poop...
RRJJ: eeeewwww, i just took a crap that looked like Susan Boyle
Re: ok poop is coming out (Score:2)
I would argue in the reverse - Twitter is useless as a "microblog" and a discussion platform, but it's great for status messages and location updates. Also, if you prefer to not tell everyone where you are - don't tweet it.
The problem is that people are starting to use it for things beyond it's intended purpose, like blogging about their lives as they happen, etc. Couple that with the fact that away messages already exist for Facebook, Live!/AIM, and IRC, and you really don't need Twitter other than a centr
Re: (Score:1)
I dunno, personally I think if people need to be that attached to others, then there are serious psychological issues happening.
Down? (Score:2, Interesting)
Just landed in'); DROP TABLE Location;-- (Score:5, Funny)
Re: (Score:2)
Talk about serving the right ads to the right peep (Score:5, Interesting)
How about reminding someone of a long lost friend that lives in the area you are visiting.
Your friend has a MySpace page and is on twitter live right now, would you like to send them a Direct Message or Twit (DM/T/N)?
That guy you were chatting with last week on FriendFinder lives 4.6 miles from the airport, here is a google map to the closest IN/Out, that you two were chatting about, near their location. Would you like to invite him to join you there? (Y/N)
The possibilities are as invasive as they are endless.
Someone travels to your location allot for business, perhaps you can lure them to your hotel instead of another one, treat them right and secure a new long term customer.
This twitter user shows his GPS coordinates, they are in the car next to you right now, do you want to wave (Y/N)? It could even be hands free and talk to you like GPS devices do. lol,
Ideas, ideas, ideas....
Re: (Score:1, Funny)
You certainly seem unduly concerned about privacy. Perhaps you've got something to hide? Would you like to speak to one of our friendly FBI agents? [Modal dialog box has only "OK" option.]
Re: (Score:2)
People seem to get swept up in the fascinating technical aspects of targeted advertising based on social networks. Gee, look what those statistics reveal, and look how they've tied those stats with those stats, very clever.
What nobody seems to recognize is that the very word "advertise" means to get people to do what they do not want to do. Everyone thinks ads just present options for people to carry out motivations they already have. No, that would be called divertising, the diversion of motivation. Ad
Self absorbed twits (Score:5, Funny)
Who would have thought these self absorbed narcissists actually could serve a useful purpose in spite of themselves.
Other scientific uses for Twitter (Score:4, Interesting)
Some colleagues recently informed us that they use Twitter to track whether or not people feel earthquakes. By scanning Twitter reports and correlating them to their seismic measurements they can build a map of how far away people actually felt the event. Thanks to tweet time stamps they can build a rough map of the felt area in about a minute. By using a longer time horizon they can build a more accurate map of the felt area.
Mapping the area of felt earthquakes is done anyway. Scanning Twitter just provides a supplemental way of doing that.
Re: (Score:3, Funny)
Re: (Score:3, Interesting)
Some colleagues recently informed us that they use Twitter to track whether or not people feel earthquakes.
Oh interesting, just the other day on the Science Channel's "Weird Connections" show, an earthquake researcher (Jim Berkland apparently) talked about how he correlated increased missing pets with impending 'quakes by checking lost pet ads in the newspaper (which nearly doubled according to the data in "The man who predicts Earthquakes" [google.com] starting on pg. 39).
Here is a search of Twitter messages [twitter.com] since 5/12/09 of missing dogs or cats.
Since searches can also be refined by area, I imagine it would be easy to monit
Can be abused? I hope so (Score:5, Funny)
Re:Can be abused? I hope so (Score:4, Funny)
This is easy to defeat (Score:2, Funny)
Dig deeper (Score:1)
Also according to twitter... (Score:2)
Searching for the term "swine flu" would lead one to believe 90% of Twitter users have contracted it.
Then again - maybe they have! Could be natural selection at work...
Re:Also according to twitter... (Score:4, Funny)
Doesn't the latest version of swine flu also have bird flu DNA? Obviously it's a serious problem for those who tweet.
What Twitter really needs... (Score:3, Funny)