With what's happening in Twitter, many people are considering moving to Mastodon. Like Twitter, Mastodon also has an API that can be used to create many useful application, bots, to analyze data, respond to notification or simply post some statuses.
In this article we will explore all the things Mastodon API can do and how you can use it to build applications with Python.
Setting Up
First of all, we need an API access to Mastodon to build anything. We have two options that are suitable for development. We can either run our own local instance of Mastodon or we can use a bot-friendly public instance.
If you want to run a local instance, then you can use the repository I created, which has all the info you need and will walk you through the setup, all you need is Docker (and docker-compose).
If that's too much hassle for you, you can use bot-friendly instance such https://botsin.space/, you simply register there as on any other instance, and then you should be able to set up an app.
When choosing which one of these options is better for you, consider following pros and cons:
If you set up local Mastodon with Docker, then you have generally no limitation - you get administrator access/permissions to Mastodon internals, full application scopes and no rate-limiting. On the other hand you might run into some issues, for example with media uploads (images, videos) which require you to provision storage (S3 bucket). By default, you also won't have access to federated data (timeline, accounts) - you decide whether that's an issue during development/testing or not.
If you choose bot-friendly public instance, then you get the opposite - all the features such as federation or media uploads will work fine, you will however have limited permissions, access and rate-limits.
Create an Application
Regardless of whether you choose to use local instance or bot-friendly one (in the following section we will assume the latter), you will need to create an app. To do so, you can use both web interface and CLI/code. Here we will do it via web UI.
To create an app, navigate to application page and click New application. The created app should look something like this:
You will notice that here I specified Redirect URI as http://localhost:8080. You can leave the urn:ietf:wg:oauth:2.0:oob in your app - both options will work for local testing - we will explore their differences later on when we authenticate using OAuth.
With the application created, we want to test if the credentials work. The simplest way to do so, is to make request using curl with the Your access token field as Bearer token to - for example - post a status (Toot):
curl -X POST 'https://botsin.space/api/v1/statuses/' \
-H "Accept: application/json" \
-H "Authorization: Bearer <ACCESS_TOKEN>" \
-F "status"="Toot with cURL" \
| jq .
{
"id": "109400295593975107",
"uri": "https://botsin.space/users/pythonbot/statuses/109400295593975107",
"url": "https://botsin.space/@pythonbot/109400295593975107",
"content": "<p>Toot with cURL</p>",
"account": {
"id": "109375732335027944",
"username": "pythonbot",
"acct": "pythonbot",
"url": "https://botsin.space/@pythonbot",
}
}
This makes requests on behalf of the user that created/installed the application. If you want to act on behalf of other users, you need to get authorization from them (we will do that in the next section).
With that said though, you don't actually need to authenticate to read public data from Mastodon:
curl 'https://botsin.space/api/v1/timelines/public?limit=1' | jq .
[
{
"id": "109400295593975107",
"created_at": "2022-11-17T14:01:27.512Z",
"visibility": "public",
"uri": "https://botsin.space/users/.../statuses/...",
"url": "https://botsin.space/@.../..."
"replies_count": 0,
"reblogs_count": 0,
"favourites_count": 0,
"content": "<p>Toot with cURL</p>",
"account": {
"id": "...",
"username": "...",
"acct": "...",
"url": "https://botsin.space/@..."
"statuses_count": 1,
},
}
]
Here we read the public timeline of botsin.space instance. There are a couple more things you can access without credentials, such as accounts or instances. For more info see docs.
Using The API
Now that we have access to API and we confirmed that we can make some sample calls, let's write some proper code with Python. While making raw requests like with curl would work, it's more convenient to use Mastodon's Python client library. To install it, run:
pip install Mastodon.py
Like with examples earlier, first thing we need to do is authenticate:
import mastodon
from mastodon import Mastodon
m = Mastodon(access_token="...", api_base_url="https://botsin.space")
m.toot("Test post via API using OAuth")
This is equivalent to the authentication used earlier with curl. All we need to access the API on behalf of the bot account is access token and API base URL.
Alternatively, you can put the credentials into file and read them from there:
# Authenticate the app
m = Mastodon(
client_id="clientcred.secret"
)
# clientcred.secret:
# RIM......aPk (client_id)
# -3z......Muc (client_secret)
# https://botsin.space (api_base_url)
# pythonbot (client_name)
In this case however, the file doesn't contain the access code, so we will use it as an opportunity to test out authorization of other users:
url = m.auth_request_url(redirect_uris="http://localhost:8080", scopes=['read', 'write', 'follow'])
# https://botsin.space/oauth/authorize?client_id=<CLIENT_ID>&\
# response_type=code&\
# redirect_uri=http%3A%2F%2Flocalhost%3A8080&\
# scope=read+write+follow+push&\
# force_login=False&state=None
# Login as a user
m.log_in(
"pythonbot@botsin.space",
redirect_uri="http://localhost:8080",
code="<AUTHORIZATION_CODE>",
scopes=['read', 'write', 'follow'],
to_file="pythonbot-token.secret"
)
m = Mastodon(access_token='pythonbot-token.secret')
m.toot("Test post via API using OAuth")
The above is the "right way" to do it if you want other users to use the application. We use the auth_request_url to generate URL that we can send to the user, where they will authorize the application. Because we specified the redirect_uris here, we need a server running on the address so that Mastodon can redirect the user there. For testing, you can spin-up a server using docker run --publish 8080:80 nginx.
If you (or a user) then click the authorize button on the page from url, you will be redirected to http://localhost:8080 with the code in URL (and in nginx logs). Also, for real world application, make sure to include state argument with random, non-guessable value to preserve security.
If you didn't change the default redirect URI (urn:ietf:wg:oauth:2.0:oob) during app creation, then you can't pass redirect_uris to the command(s) above, in that case the user will be presented with:
And they will need to copy the code manually into the application config, which is useful for local testing.
Also note the use of scopes argument in both auth_request_url and log_in methods above - the scopes argument defaults to ['read', 'write', 'follow', 'push'], which means that if your application - like the example one here - has more limited permissions, then the authentication will fail, if the scopes are not provided explicitly.
The above has shown how you can authenticate to a public instance, if you however chose to use a local instance of Mastodon (with self-signed certificate), then you need to pass session object to the methods which turns off certificate verification, which is absolutely not secure, but it's fine for development:
import requests
session = requests.Session()
session.verify = False
# Authenticate the app
m = Mastodon(
client_id="clientcred.secret",
session=session,
)
# clientcred.secret:
# veM......kE4
# fWP......uUA
# https://localhost
# local-pythonbot
# Login as a user (with Password)
m.log_in(
"admin@localhost",
password="users-password",
to_file="usercred.secret"
)
m = Mastodon(access_token="usercred.secret", session=session)
m.toot("Test post via API...")
In this snippet we also show that you can also use user's password for authentication, passing it directly to the log_in method.
Regardless of which method you chose, you now have an authenticated bot or user and can - for example - post a status using the client library:
print(m.status_post("Same as 'toot' method."))
# {
# 'id': 109370260599659421,
# 'replies_count': 0,
# 'reblogs_count': 0,
# 'favourites_count': 0,
# 'content': '<p>Same as 'toot' method.</p>',
# 'account': {
# 'id': 109359234895957150, 'username': 'admin', 'acct': 'admin',
# 'display_name': '', 'followers_count': 0, 'following_count': 0, 'statuses_count': 3
# },
# 'media_attachments': [], 'mentions': [], 'tags': [], 'poll': None
# }
In this example we use status_post method rather than the toot method, because it has more options, such as post scheduling or replying to other statuses:
from datetime import datetime, timedelta
now = datetime.now()
schedule = datetime.now() + timedelta(minutes=10)
# Scheduled:
status = m.status_post("This will be posted 10min from now.", scheduled_at=schedule)
# Reply:
m.status_post("This is a reply to previous Toot.", in_reply_to_id=status["id"])
And it also has ability to create media posts:
metadata = m.media_post("image.png", "image/png")
# Response format: https://mastodonpy.readthedocs.io/en/stable/#media-dicts
m.status_post("This post contains previously uploaded image.png", media_ids=metadata["id"])
Uploading media posts is a two-step process. First we upload just the media, e.g. image or video in specified format. Then we use the returned metadata to attach the media to the status using media_ids argument. Be aware that this tries to upload image into S3 bucket, which won't work with the default configuration on the local instance.
Similarly to media, you can also attach polls to posts using make_poll method in conjunction with status_post(..., poll=some_id).
Another basic feature of the API that you might want to use is search:
from pprint import pprint
result = m.search(q="some_friend", result_type="accounts")
pprint(result)
# {'accounts': [{'acct': 'some_friend',
# 'display_name': '',
# 'followers_count': 0,
# 'following_count': 0,
# 'id': 109370544417433130,
# 'statuses_count': 0,
# 'username': 'some_friend'}],
# 'hashtags': [],
# 'statuses': []}
You have the option to search for accounts, hashtags or statuses, by default you get all of it. Be aware that full-text search is an optional feature of Mastodon, so the status search might not work on the instance you use.
In the previous example we searched for accounts based on a name, so now that we found some other user(s), we can use the result of the query to follow them:
pprint(m.account_follow(result["accounts"][0]["id"]))
# {'blocked_by': False,
# 'blocking': False,
# 'domain_blocking': False,
# 'endorsed': False,
# 'followed_by': False,
# 'following': True,
# 'id': 109370544417433130,
# 'muting': False,
# 'muting_notifications': False,
# 'note': '',
# 'notifying': False,
# 'requested': False,
# 'showing_reblogs': True}
All we need to do is pass the account ID to account_follow which returns updated "relationship" with the other user. Be aware that the follows are not always instant and it might take a bit to appear or might even require approval from the other user.
Now that you followed someone and possibly sent them a message(s), you can also read the conversation via API:
pprint(m.conversations())
# [{'accounts': [{'acct': 'some_friend',
# 'username': 'some_friend'}],
# 'id': 1,
# 'last_status': {'account': {'acct': 'admin',
# 'username': 'admin'},
# 'content': '<p><span class="h-card"><a '
# 'href="https://localhost/@some_friend" '
# 'class="u-url '
# 'mention">@<span>some_friend</span></a></span> '
# 'Hello friend!</p>', # <- The actual message
# 'id': 109370650516300327,
# 'mentions': [{'acct': 'some_friend',
# 'id': 109370544417433130,
# 'username': 'some_friend'}]},
# 'unread': False},
# ...
# ]
The content in the above output is the actual message, it's however wrapped in HTML which makes it quite unreadable. The conversation also forms kind of a linked list which makes it little hard to navigate in the JSON. To make it a bit more readable you can use something like:
from bs4 import BeautifulSoup
for message in m.conversations():
print(BeautifulSoup(message["last_status"]["content"], "html.parser").text)
indent = 4
current = message["last_status"]["in_reply_to_id"]
while current is not None:
parent = m.status(current)
html = BeautifulSoup(parent["content"], "html.parser")
print(indent*" " + html.text)
indent += 4
current = parent["in_reply_to_id"]
print()
# @some_friend Yet another message in thread.
# @some_friend This is reply to earlier message.
# @some_friend Hello friend!
# @some_friend What's up?
Here we use BeautifulSoup to parse the HTML and grab the actual message content. The above output shows 2 threads (conversations) with the same user. First with 3 messages the other with one. Notice that they're in reverse order - the latest one on top.
After a while you will end-up following quite a few users, and you will also get bunch follows back. You can use the following to view the follower data:
user = m.account_search("some_friend@localhost")
m.account_following(id=user[0]["id"])
m.account_followers(id=user[0]["id"])
m.account_relationships(id=user[0]["id"]) # Mutual follow
We first need an ID of an account, which we get by searching for it using account_search. We can then use the ID to check who they're following, who's following them and with whom they have mutual follows. For each of these methods a user dictionary is returned.
If you want to build a bot, then you will need a way to listen and respond/react to events. You can do that using the StreamListener class:
class Listener(mastodon.StreamListener):
def on_update(self, status):
print(f"on_update: {status}")
# on_update: {'id': 109371390226010302, 'content': '<p>Listening to Toots...</p>',
# 'account': {'id': 109359234895957150, 'username': 'admin'}, ...}
def on_notification(self, notification):
print(f"on_notification: {notification}")
# Follow notification:
# on_notification: {'id': 7, 'type': 'follow',
# 'account': {'id': 109370544417433130, 'username': 'some_friend'}, ...}
m.stream_user(Listener())
To react to any events that comes your way, you need to create a class that inherits from mastodon.StreamListener and to override some of its methods. Here we override on_update method which is triggered when new status is posted, and on_notification which listens to all of your notifications, such as mentions or follow. You could use this to build bots like Unroll/ThreadReader that we know from Twitter.
After creating the class, all we need to do is start listening with stream_user(Listener()). Be aware that this is a blocking call, so you might want to run it in thread in the background.
Final piece of API that we will take a look at is pagination:
statuses = m.timeline(timeline="home", limit=3)
while statuses:
statuses = m.fetch_next(statuses) # Could also pass in "statuses._pagination_next"
for s in statuses:
print(BeautifulSoup(s["content"], "html.parser").text)
This particular example will give you all the Toots as well as direct messages of the logged-in user, in batches of 3 posts.
Pagination becomes very useful when you start reading/processing a lot of data, such as all the statuses from public timeline. It's also necessary to not exceed Mastodon's rate-limit, which 300 request per 5 minutes.
Where to Install?
When you finish writing the code, then comes the time to install the application/bot onto some Mastodon instance.
If you've been on Mastodon for a bit, then you know that where you create your account does matter and same is the case for bots or API access.
Let's say that you're going to have the bot continuously listen to events/notifications. For that, installing it on any bot-friendly instance should suffice, as it can receive events (mentions, messages) from users on other instances. If you however want the bot to act on behalf of other users, then the bot must be installed on the same instance where the user is, so that they can authorize it.
Also, if you're only scraping or analyzing non-public data - probably running the application locally - then you will need to install the application on the instance where the data is.
So, the bottom line is:
- Bot can only authorize users on instance where app exists,
- Bot can read federated timeline, regardless of where it's installed,
- Bot can receive notifications (mentions, follows) from other instances,
- Certain features are limited only to your instance, for example hashtag search
Closing Thoughts
In this article we explored a lot of features of the Mastodon API, but there are more things you can do with it, so be sure to check out docs of the Mastodon's Python library as well as the official Mastodon docs which has a lot of information about how to use the API.
Also, while many bots and application can be useful - as we know from Twitter - many also can be quite annoying and obnoxious. So, let's try to keep Mastodon a nicer place than Twitter, at least in terms of what we use its API for.