Python: Get Twitter Tweets using Tweepy Library

This article shows how you can get/fetch Tweets from Twitter API using a popular and easy to use Python Library named “Tweepy“.

You can perform different tasks using the Tweepy library like:

– Searching Tweets for any particular user
– Searching Tweets for any particular hashtag or any text
– Getting any user’s data like name, location, friends count, etc.
– Posting Tweets for the authenticated user
– Getting Twitter Trends information
– Fetch and send direct message (DM) by the authenticated user
– Get user’s timeline statuses
– And much more

Full features can be found in the documentation page of the Tweepy library.

In this article, we will mainly focus on searching tweets using the Tweepy library.

Overview of Twitter Search API is present here: https://developer.twitter.com/en/docs/tweets/search/overview

Currently, there are three tiers of Twitter Search API: Standard, Premium, and Enterprise.

We will be using the Standard Twitter Search API which is FREE. This can search for tweets published in the past 7 days. To get tweets published on more prior dates, you need to pay for either Premium or Enterprise APIs.

User Authentication to Twitter API

Getting Application Keys and Access Tokens

First of all, we need to create user authentication to the Twitter API.

Twitter uses OAuth to provide authorized access to its API.

To generate an OAuth access token, you need to:

– Login to your Twitter account
– Go to https://apps.twitter.com and create a new app.
– After you complete creating the new app, you can get the get the application’s tokens and keys from the “Keys and Access Tokens” tab.

Once we have the consumer keys and access token for our newly created Twitter Application, we can move forward on installing Tweepy library and writing python code to fetch tweets.

The following is needed in the code:

– Consumer Key (API Key)
– Consumer Secret (API Secret)
– Access Token
– Access Token Secret

Installing Tweepy Library

To install Tweepy, you need to run the following pip command:

pip install tweepy 

Writing code

Now, let’s write the Python code to fetch tweets using the Tweepy library.

Installing Key and Tokens

import tweepy
 
# consumer keys and access tokens, used for OAuth
consumer_key = 'YOUR-CONSUMER-KEY'
consumer_secret = 'YOUR-CONSUMER-SECRET'
access_token = 'YOUR-ACCESS-TOKEN'
access_token_secret = 'YOUR-ACCESS-TOKEN-SECRET'

# OAuth process, using the keys and tokens
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
 
# creation of the actual interface, using authentication
api = tweepy.API(auth)

Post a new Tweet

You can use the following code to post a new tweet through your account.

api.update_status('Test')

Get information about yourself

The me() method creates the user object and returns the user whose authentication keys were used.

user = api.me()

print('Name: ' + user.name)
print('Location: ' + user.location)
print('Friends: ' + str(user.friends_count))

'''
Output:

Name: Mukesh Chapagain
Location: Kathmandu, Nepal
Friends: 1652
'''

Get information about other users

You can also get other user’s information.

The get_user() function gets the user object for any particular Twitter user handle.

Here, we use twitter user “spiritualsatya”:

user = api.get_user('spiritualsatya')

print('Name: ' + user.name)
print('Location: ' + user.location)
print('Friends: ' + str(user.friends_count))

'''
Output:

Name: Spiritual Satya
Location: 
Friends: 207
'''

Get friends of the authenticated user

Print 5 friends:

user_friends = user.friends()
for friend in user_friends[:5]:
    print (friend.screen_name)

Searching/Fetching Tweets

Here, I will show some examples of fetching tweets using any particular text query.

– search tweets for text “Nepal”
– search in language “ne” (i.e. Nepali language)
– and limit the search result to 5

search = tweepy.Cursor(api.search, q="Nepal", lang="ne").items(5)

– search recent/latest tweets containing text “Nepal”
– regardless of any language
– search result type can be: mixed, recent, popular
– mixed = recent + popular

search = tweepy.Cursor(api.search, q="Nepal", result_type="recent").items(5)

• search english language tweets

search = tweepy.Cursor(api.search, q="Nepal", result_type="recent", lang="en").items(5)

• print the searched tweets

for item in search:
    print (item.text)

• Other different parameters of the tweet can be fetched like created date, retweet count, user location, etc.

search = tweepy.Cursor(api.search, q="Nepal", result_type="recent", lang="en").items(5)

for item in search: 
    # print tweet text
    print (item.text)
    
    # print tweet created date 
    print (item.created_at)
    
    # print retweet count of the tweet
    print (item.retweet_count)
    
    # print the username who published the tweet
    print (item.user.name)
    
    # print the location of the user who published the tweet
    print (item.user.location)
    
    # print the language code of the tweet
    print (item.metadata['iso_language_code'])
    
    # print the search result type
    print (item.metadata['result_type'])
    
    # print the device/source from which the tweet has been published
    # e.g. Twitter for Android, Twitter for iPhone, Twitter Web Client, etc.
    print (item.source)

– print list of hashtags present in the tweet
– print empty list when no hashtag is present

for item in search:
    # print list of hashtags present in the tweet 
    # print empty list when no hashtag is present
    hashtags = item.entities['hashtags']
    if hashtags:
        ht = [ht['text'] for ht in hashtags]
        print ht
    else:
        print hashtags 

– print list of users mentioned in the tweet
– print empty list if no user is mentioned in the tweet

for item in search:
    # print list of users mentioned in the tweet 
    # print empty list if no user is mentioned in the tweet 
    user_mentions = item.entities['user_mentions']
    if user_mentions:
        mentions = [user['screen_name'] for user in user_mentions] # name, id, screen_name
        print mentions      
    else:
        print user_mentions

Hope this helps. Thanks.