Screenshot of a Topsy search for the hashtag #occupymilwaukee, during the week of the Oct. 15 protest.
In trying to finish a Storify on a long-gone event, I did some research into digging up old tweets. Twitter is a moving target and most of the information I found is incomplete or outdated. So I thought I'd share what I've been able to figure out.
The problem
Twitter has done well. Really well: to succeed as well as it has is probably the dream of every tech start-up, every internet entrepreneur, every venture capitalist in social media. But Twitter has its wrinkles. You can't use the site for long without being familiar with the Fail Whale, for example.
One of Twitter's weaknesses is its search. If you've ever used it, you probably have realized it's limited. And if you have saved searches for uncommon hashtags, you're familiar with results you got only days ago disappearing.
Apparently, the underlying design of the site doesn't allow the site to search back more than a week or so, although Twitter has vowed to improve it. And the API — the way external sites, like Storify, grab or post tweets — only allows access to the past 3200 tweets of a user.
Some here and now solutions
Not to worry, you still have a few tricks up your sleeve. Google, unsurprisingly, works okay. I can retrieve a small sample of the tweets during October 2011 that include '#JOUR1550,' the hashtag used for one of my journalism classes, with this search. I know there were far more tweets than that, but it's better than nothing. To limit by date like I did, go to the sidebar on the left and choose your preferred span or type in a specific range using "Custom..."
Better than Google for finding old tweets is Topsy. It turns up more results than Google for the searches I've done. Using the advanced search, you can also limit results to a date range. When searching an individual's stream, the results are good: it found all my tweets over this past summer.
Snapbird is good for searching a particular user, including yourself. The best use of this may be to get more tweets from users you've identified as tweeting what you want. So if you can identify a few users who tweeted an event more or less constantly, you should be able to dig up their entire feed for the day you're interested in.
I found three other services that look promising, but really aren't that useful. Google has a customized search that is less useful than the previous Google trick — it's harder than just going to Google's webpage and doesn't have the date selection feature. Another is FriendFeed which might find tweets others don't, but it's in a very confusing format. Finally, we have TweetScan, which, in my tests, doesn't turn up much of anything.
But what about the future?
It turns out, if you plan to use tweets in advance, you have a lot more options. (Shocker, right?) For one thing, you can favorite tweets or add them to a curation tool like Storify. You could also copy and paste the link to the tweets into a file (click the date underneath the tweet, then copy the URL). Those links are permanent: People have dug up functioning links to tweets from 2006.
You can also archive your own tweets. The problem is that this only saves the tweets available when you start archiving, although if you have few enough tweets, that could be all of them. There are many of these, so I'm going to dedicate an entire post to them.
Over the long term, your tweets are safe. Even though they can't be found by search, all tweets are available via link (see the one above), so Twitter isn't deleting them. I suspect it will eventually improve its search or allow users to access all of their own tweets. Even if Twitter destroys all its data or goes bankrupt, the Library of Congress has an archive of the site. It isn't searchable right now, but I imagine they will be eventually.
In a medium that's all about the immediate, it isn't a surprise it's so hard to find things older than a week. But several tools do a pretty good job of finding past tweets anyway. Topsy, Snapbird and Google are the best bets at the moment. The future will probably bring more and better options, but for now the best option is to be proactive — archive tweets as soon as you know you want to keep them.
The problem
Twitter has done well. Really well: to succeed as well as it has is probably the dream of every tech start-up, every internet entrepreneur, every venture capitalist in social media. But Twitter has its wrinkles. You can't use the site for long without being familiar with the Fail Whale, for example.
One of Twitter's weaknesses is its search. If you've ever used it, you probably have realized it's limited. And if you have saved searches for uncommon hashtags, you're familiar with results you got only days ago disappearing.
Apparently, the underlying design of the site doesn't allow the site to search back more than a week or so, although Twitter has vowed to improve it. And the API — the way external sites, like Storify, grab or post tweets — only allows access to the past 3200 tweets of a user.
Some here and now solutions
Not to worry, you still have a few tricks up your sleeve. Google, unsurprisingly, works okay. I can retrieve a small sample of the tweets during October 2011 that include '#JOUR1550,' the hashtag used for one of my journalism classes, with this search. I know there were far more tweets than that, but it's better than nothing. To limit by date like I did, go to the sidebar on the left and choose your preferred span or type in a specific range using "Custom..."
Better than Google for finding old tweets is Topsy. It turns up more results than Google for the searches I've done. Using the advanced search, you can also limit results to a date range. When searching an individual's stream, the results are good: it found all my tweets over this past summer.
Snapbird is good for searching a particular user, including yourself. The best use of this may be to get more tweets from users you've identified as tweeting what you want. So if you can identify a few users who tweeted an event more or less constantly, you should be able to dig up their entire feed for the day you're interested in.
I found three other services that look promising, but really aren't that useful. Google has a customized search that is less useful than the previous Google trick — it's harder than just going to Google's webpage and doesn't have the date selection feature. Another is FriendFeed which might find tweets others don't, but it's in a very confusing format. Finally, we have TweetScan, which, in my tests, doesn't turn up much of anything.
But what about the future?
It turns out, if you plan to use tweets in advance, you have a lot more options. (Shocker, right?) For one thing, you can favorite tweets or add them to a curation tool like Storify. You could also copy and paste the link to the tweets into a file (click the date underneath the tweet, then copy the URL). Those links are permanent: People have dug up functioning links to tweets from 2006.
You can also archive your own tweets. The problem is that this only saves the tweets available when you start archiving, although if you have few enough tweets, that could be all of them. There are many of these, so I'm going to dedicate an entire post to them.
Over the long term, your tweets are safe. Even though they can't be found by search, all tweets are available via link (see the one above), so Twitter isn't deleting them. I suspect it will eventually improve its search or allow users to access all of their own tweets. Even if Twitter destroys all its data or goes bankrupt, the Library of Congress has an archive of the site. It isn't searchable right now, but I imagine they will be eventually.
In a medium that's all about the immediate, it isn't a surprise it's so hard to find things older than a week. But several tools do a pretty good job of finding past tweets anyway. Topsy, Snapbird and Google are the best bets at the moment. The future will probably bring more and better options, but for now the best option is to be proactive — archive tweets as soon as you know you want to keep them.
RSS Feed