Python: Get Twitter Tweets using ‘Get Old Tweets’ Package

This article shows how you can get/fetch Tweets from Twitter API using a very useful Python package named “Get Old Tweets“.

You can perform different tasks using the GetOldTweets, like:

– Searching Tweets of any particular user and between any dates
– Searching Tweets for any particular hashtag or any text and between any dates
– Get Tweets by location
– Get Top Tweets
– Different parameters of any tweet can be fetched like username, date, retweets, mentions, hastags, etc.

Limitation of Twitter’s Standard Search API

Overview of Twitter Search API is present here: https://developer.twitter.com/en/docs/tweets/search/overview

Currently, there are three tiers of Twitter Search API: Standard, Premium, and Enterprise.

The Standard Twitter Search API is FREE but this can only search/fetch tweets published in the past 7 days. To get tweets published on more prior dates, you need to pay for either Premium or Enterprise APIs.

GetOldTweets package is useful in this case when you want to Freely get/fetch tweets older than 7 days.

Install GetOldTweets

– Clone the GetOldTweets GitHub Repository

– Then, go inside the cloned repo and run the following command on terminal

pip install -r requirements.txt

This will install the expected package dependencies.

Get/Search Tweets

You can run the Main.py python file to see the example output. This file contains the example code to fetch/search tweets using any particular username or search text.

Here’s the full code of Main.py file:

import sys
if sys.version_info[0] < 3:
    import got
else:
    import got3 as got

def main():

	def printTweet(descr, t):
		print(descr)
		print("Username: %s" % t.username)
		print("Retweets: %d" % t.retweets)
		print("Text: %s" % t.text)
		print("Mentions: %s" % t.mentions)
		print("Hashtags: %s\n" % t.hashtags)

	# Example 1 - Get tweets by username
	tweetCriteria = got.manager.TweetCriteria().setUsername('barackobama').setMaxTweets(1)
	tweet = got.manager.TweetManager.getTweets(tweetCriteria)[0]

	printTweet("### Example 1 - Get tweets by username [barackobama]", tweet)

	# Example 2 - Get tweets by query search
	tweetCriteria = got.manager.TweetCriteria().setQuerySearch('europe refugees').setSince("2015-05-01").setUntil("2015-09-30").setMaxTweets(1)
	tweet = got.manager.TweetManager.getTweets(tweetCriteria)[0]

	printTweet("### Example 2 - Get tweets by query search [europe refugees]", tweet)

	# Example 3 - Get tweets by username and bound dates
	tweetCriteria = got.manager.TweetCriteria().setUsername("barackobama").setSince("2015-09-10").setUntil("2015-09-12").setMaxTweets(1)
	tweet = got.manager.TweetManager.getTweets(tweetCriteria)[0]

	printTweet("### Example 3 - Get tweets by username and bound dates [barackobama, '2015-09-10', '2015-09-12']", tweet)

if __name__ == '__main__':
	main()

Export Tweets to CSV file

This package also provide a feature to export searched tweets into a CSV file. The output file is named output_got.csv.

The command line script Exporter.py is used for this purpose.

To use this script you can pass the following attributes:

username: Username of a specific twitter account (without @)
since: The lower bound date (yyyy-mm-aa)
until: The upper bound date (yyyy-mm-aa)
querysearch: A query text to be matched
near: A reference location area from where tweets were generated
within: A distance radius from “near” location (e.g. 15mi)
maxtweets: The maximum number of tweets to retrieve
toptweets: Only the tweets provided as top tweets by Twitter (no parameters required)
output: A filename to export the results (default is “output_got.csv”)

Examples:

# Example 1 - Get tweets by username [barackobama]
python Exporter.py --username "barackobama" --maxtweets 1

# Example 2 - Get tweets by query search [europe refugees]
python Exporter.py --querysearch "europe refugees" --maxtweets 1

# Example 3 - Get tweets by username and bound dates [barackobama, '2015-09-10', '2015-09-12']
python Exporter.py --username "barackobama" --since 2015-09-10 --until 2015-09-12 --maxtweets 1

# Example 4 - Get the last 10 top tweets by username
python Exporter.py --username "barackobama" --maxtweets 10 --toptweets

Hope this helps. Thanks.