Understanding ActivityPub Part 3: The State of Mastodon

This article is part of the series Understanding ActivityPub, which takes a look at the ActivityPub protocol through the lens of real-world examples. The protocol exchanges are taken from ActivityPub.Academy, a modified Mastodon instance that shows ActivityPub messages in real time (see the announcement post).

The following articles are available.

Introduction

One key selling point of Mastodon, or of any Fediverse platform, is that it is distributed. It means that no actor or company is in full control of the platform. But distribution comes with challenges. For it to not be completely inefficient, information is copied between instances. When you view a profile of a user on a different instance, your instance will fetch that profile and store a copy locally. When you create a post, your instance will send copies to the instances of all of your followers. It will be further distributed by people boosting it, leading to even more copies. This would not be an issue if the information was immutable, but it’s not. Your profile can change. People will interact with your post by liking or replying to it. Distribution makes it practically impossible that these changes are propagated properly to all instances, which means that two Mastodon instances can have a different state of the same piece of information. In this article, we will investigate how Mastodon handles this state. More specifically, we will investigate the state of a Mastodon post on different instances.

You may have noticed that sometimes when you see a boosted post in your timeline, it has no or very few replies, but when you open up that same post on the original page, a whole conversation took place. The counters showing the number of boosts and likes are even more inconsistent. They are almost always higher on the original page.

Take for example my post on the previous article in Understanding ActivityPub. At the time of writing, it was boosted six times and liked twelve times, and has three messages in the reply thread. If you view the same post on hachyderm.io, it shows only five boosts, no likes, and two messages in the replies. On k8s.social it shows one boost, no likes, and no replies. (I cannot link to the post on the other instances directly. But you can search for my full handle @crepels@mastodon.social on those instances and then select the post from my profile.)

Let’s find out how these discrepancies come to be. We will see that it all comes back to how ActivityPub works.

Understanding Replies

We will start by investigating replies. As in the previous posts, we will use actors from different instances who interact with one another: @alice@activitypub.academy, @berta@techhub.social, and @canon@mas.to. We will create different social graphs depending on the example.

Replies are forwarded to followers

To start with, we use the following social graph.

This means that both @berta and @canon follow @alice, but there are no other follower connections. We now let @alice create a post and let @berta reply to it.

The first two activities are from @alice, sending the new post to her followers. We already saw this pattern in Part 1: Protocol Fundamentals.

The third activity contains @berta’s reply. It is sent by @berta to the shared inbox on activitypub.academy. Looking at the source, we see that it is addressed to the public collection (since we sent the reply as a public message; see Part 1: Protocol Fundamentals for more details), and cc’d to @alice and to @berta’s followers. Let’s take note of the last fact: @berta sends her reply to all of her followers.

Now the context of the last activity is very interesting. The Create activity itself is identical to the one in the first activity. But the sender and the receiver are very odd! Let’s investigate them one by one.

The sender is @alice, even though @berta is listed in the actor field of the activity. If you have read Part 1: Protocol Fundamentals, you might think that this shouldn’t be possible. If other actors could send activities on @berta’s behalf, they could do all kinds of nefarious things. That’s why instances attach a Signature header to every activity, to ensure the authenticity of the sender. The signature is created using the private key of the actor of the activity. But for this activity, this does not work. activitypub.academy does not have access to @berta’s private key. Still, the message is not discarded on the other end (we will see this in a second). How can that be? If we take a look at the source of the activity, we see that it has its own signature field, created with @berta’s key. When @berta’s instance sent the Create activity, it attached this signature. This is what allows any other actor to distribute this activity. We can still be confident that the message originally came from @berta and was not tampered with, otherwise the signature would not match.

The receiver is also interesting. The activity is sent to the shared inbox of mas.to. So far, every activity that we have seen that was sent to a shared inbox still had a dedicated recipient on the other end, usually mentioned in the cc field. This could be either a specific user, or a collection of followers, if one of the followers was on that instance. But here it is neither. The cc field contains @alice and @berta’s followers. @alice is not on mas.to, and neither are any of @berta’s followers. But one of @alice’s followers is. When @alice’s instance sends @berta’s reply to all instances of her (@alice’s) followers, even though it is not addressed to any particular user, the instances still process it and can show it as a reply to the post (in the source of the activity, you can see the replyTo field of the Note object, which the instance uses to match the reply to the original post).

The parent instance always receives the reply …

We saw that @berta’s reply was addressed to @alice. A user will always be addressed if they are @-mentioned, and that happened for @alice in @berta’s reply. What happens if we change that? Will @alice still be addressed because @berta is replying to her post? Here is the full Activity Log when @berta replies a second time, but now without the @-mention.

It might seem like there is a mistake. There is only one activity, which is activitypub.academy distributing @berta’s reply to @alice’s followers. But the activity that notified activitypub.academy of @berta’s reply in the first place seems to be missing! Well, that’s not actually the case. The activity was still sent to activitypub.academy, but it was not addressed to @alice (you can check the to and cc fields in the source). The Activity Log only shows activities that were either authored by @alice, or were sent to @alice (either by sending it to her personal inbox, by adding her actor ID or a followers collection that she belongs to to the to, cc, bto, or bcc fields). That’s why @berta’s doesn’t show up here. (I might add an option in the future to show all activities processed on the instance.)

But @berta sending the activity is good news. It means that the reply is always sent to the instance of the parent post, even if it is not sent to the author of the post directly. As we saw in the first case above, this allows activitypub.academy to show the reply whenever someone views @alice’s post.

… but the grandparent instance might not

Since we now know that the reply is always sent to the instance of the parent post, the next question would be if this is also the case for the grandparent instance. What I mean by this is: when @berta replies to @alice and then @canon replies to @berta, will @alice’s instance receive @canon’s reply? As it turns out, it won’t. When we try this by letting @canon reply to @berta’s Hello alice reply with Hello berta (again, not tagging anyone), then this reply will be visible on techhub.social, but it doesn’t appear on activitypub.academy (and the Activity Log remains empty). This means that even though @alice started the thread, she doesn’t have the full picture of the conversation. And here we are looking at the simplest possible example. For a post that initiates several reply threads, it may very well happen that no instance has the full picture.

Mastodon fills in gaps in threads

Let’s stay with the example for one more test. What happens when @canon replies to his own reply, but now tags @alice?

We see that @canon’s instance sent the reply to @alice (since she was @-mentioned). We might expect that there is now a hole in the conversation. The instance has @alice’s original post, @berta’s reply, and @canon’s reply to himself, but the middle part, @canon’s reply to @berta should be missing. But when we check the post, @canon’s first reply is there! It appeared when the instance processed his second reply. This is a Mastodon feature. It checks the inReplyTo field of each post. If it doesn’t have the corresponding message in the database, it fetches it from the other instance using the ActivityPub data. (In the previous post, we saw that Lemmy does something similar.) This works recursively as well. This means that while you might miss the end of a reply thread, you should never miss the beginning or parts in between.

As a side note, we see in the Activity Log that the reply is not forwarded by activitypub.academy. This makes sense, since it was not a reply to one of @alice’s posts. But now @berta is out of the loop. She doesn’t follow @canon and @alice didn’t forward the reply, so it is never sent to her instance.

Mastodon actively fetches some replies

We just saw that Mastodon looks backwards from any message it becomes aware of and fills in gaps that it encounters. But it also looks forward to some degree. When it sees a message for the first time, it checks if there are already some replies, and if so, it fetches them from the ActivityPub data. (This happens for example if your instance learns about a post because someone on a different instance boosted it.) There are some constraints though. It only fetches replies that were authored by users on the same instance as the original message, and it fetches at most five replies.

To try this out, we will use a fourth actor, @blaine@techhub.social, and we will use the following social graph.

This means that @berta and @blaine follow @alice, and @canon follows @berta. We then let @alice create a new post, and let her and @blaine reply to it. Finally, we let @berta boost @alice’s post. (I won’t include the Activity Log here since it doesn’t reveal anything new. The fetching of the replies happens in the background.) When we check @alice’s’ post on mas.to, we see that @alice’s reply was fetched, while @blaine’s was not.

Tying it all up

We can now collect all of our findings in a flowchart that describes whether we can see a reply to a post. This is inspired by Per Axbom’s great flowchart The visibility of a toot which describes in which timeline a Mastodon post will show up. The reply flowchart is a lot more involved though. This is a reflection of the fact that the visibility of replies is complicated, which explains some of the confusion that people have with Mastodon.

Note: There are a few other things that affect the visibility of a reply that I won’t cover in depth in this post and that don’t appear in the flowchart. For example, if users or other instances are blocked by you or your instance, this can affect the visibility of replies. If you enter the URL of a post in the search bar, your Mastodon instance will fetch it for you (this only works when you are logged in). Your instance might also use a Fediverse relay to pull in more posts, especially when it is a smaller instance.

What does this mean in practice?

The flowchart above has eight different decisions, but the last four are more or less edge conditions. Usually, whether you see a reply or not boils down to whether someone on your instance follows the poster or the replier or whether they are even on your instance. Very roughly, this means that the more users are on your instance, the more likely you are to see a given reply; but there’s still a good chance that you don’t see all replies. On the flip side, it means that if you are on a small instance, you are less likely to see replies. In the extreme case, you are the only user on your instance (some people host their own Mastodon instance). You then will only see replies if you personally follow the poster or the replier. For posts that you only see because one of your followers boosted it, you will usually see no replies at all. You can find out more what it means to be on a single-person instance on Julia Evans’s blog.

Understanding Like and Boost counters

After the replies investigation, the investigation of the like and boost counters will be a lot simpler. Let’s find out why these counters almost always show different information on different instances.

Likes

The simpler case is that of likes (or Favourites, as Mastodon calls them). We’ll use the simple social graph from above, where @berta and @canon follow @alice.

Now we let both @alice and @berta like the Hello World post from @alice. Here is the full Activity Log.

The is only one Like that @berta sent to @alice. This Like is not forwarded to any other instances (compare this with Lemmy, which would send Announce activities to distribute the Like to all followers, see Understanding ActivityPub - Part 2: Lemmy). @alice’s Like is not sent out at all, it stays local at activitypub.academy. This shows that the information flow of Likes only goes from the liker to the likee, which means that only the instance where the post originates knows about all Likes. Every other instance only knows about the Likes that were issued by its users. We can verify this by viewing the post on all three instances. On activitypub.academy, it has two likes, on techhub.social it has one, and on mas.to zero.

We can summarize this result in the following simple flowchart.

Boosts

For boosts, we include @blaine again, with him and @berta following @alice and @canon following @berta.

We let @alice create a new post and then boost it with all four actors. Here is the Activity Log.

We see @alice’s boost sent to her followers on techhub.social, followed by the Announce activities from @berta, @blaine, and @canon, since Mastodon always sends them to the author of the post. What we don’t see here is the Announce that @berta sends to @canon, but we know that it must have been sent as well. Now activitypub.academy knows about all four boosts, techhub.social knows about three (it doesn’t know about @canon’s boost), and mas.to only knows about two (@berta’s and @canon’s boost). And those are exactly the numbers that @alice’s post shows on the respective instances. More generally, only the instance where the post originates knows about all boosts (since every boost is always sent to the author of the post). Every other instance only knows about boosts that were either sent from a user on that instance, or that were sent from a user on a different instance who has followers on that instance.

The flowchart for boosts looks similar to the one for likes.

Can this be improved?

Missing replies on a post just because you are on a “wrong” instance is clearly not ideal. Seeing strange numbers on the like and boost counters might not be very important in most cases, but if you care about those, you probably prefer to see the correct numbers. So the question is, can the situation be improved?

Currently (at the time of writing, the instances I used were on version 4.1.4 or 4.1.6), Mastodon mostly displays information that it knows about through the ActivityPub protocol (we have seen some exceptions, like where Mastodon actively fetches gaps in reply threads). In other words, it relies on information being pushed to it. It could instead also try to pull information, like it currently does when filling the gaps in threads. One approach to this is FediFetcher, a script that fetches missing replies to (some of the) posts on your instance. A similar approach could be taken to fetch the actors that liked or boosted a post.

Also note that in this post, I only looked at the information that the Mastodon web app displays. The official Android app does the same. But there are other Mastodon apps that fetch information from the original instances directly, which alleviates some of the problems.

There are also ways to use ActivityPub differently to improve the situation. We already saw in the previous post how Lemmy achieves that all followers of a community always have an accurate view of all comments and likes. The community acts as a central actor. All comments, even replies, are sent to this community, which then distributes it to its followers. Lemmy doesn’t have the concept of boosting, which makes the problem of distribution a lot simpler. But even in the world of microblogging platforms like Mastodon, a similar concept can be used. As was pointed out by Mario Vavti, in Hubzilla, the toplevel poster acts as a central actor. All replies are sent to this actor who is responsible to deliver them to the participants of the thread. This means that at least all instances which have participants of the thread are always in the loop. But there might still be instances which don’t have any participants in the conversation and don’t have followers of any participants; for example, they might have learned about the post because someone boosted it. Hubzilla solves this by allowing these instances to register as thread listeners. They will be notified by the original instance about replies as well. This comes at the cost of performance though, so this option is disabled by default.

Conclusion

In this post, we have seen how replies, likes, and boosts are implemented on the level of the ActivityPub protocol. We have seen how this can lead to two instances showing different replies and different boost and like counters for the same post.

The research for this topic was a lot of fun. Before I started this, I noticed the discrepancy of the like and boost counters and found them really confusing. Now that I understand the reason, I find it both satisfying (because the answer is fairly simple, and we can see how it uses information directly from ActivityPub) and dissatisfying (I’m not sure why I should care how many people on my instance liked a given post). In the case of replies, I had a vague idea that missing replies were mostly caused when a post was boosted after the replies were written. As we have seen, this couldn’t be further from the truth. The time of the boost has almost nothing to do with the visibility of replies, and in the rare case where there is a connection, a late boost can actually lead to more replies being on your instance. Seeing the full picture with all edge conditions was quite eye opening (and I might still have missed something; if that’s the case, please let me know), and it could explain some of the confusion that people have with Mastodon.

Overall, I’m really happy that this was all possible by running some experiments and looking at the protocol interactions. I also looked at the source code to confirm some assumptions, but I would have never reached the same level of understanding by looking at the source code (or the ActivityPub spec) alone. I’m a firm believer that one of the best ways to learn about a topic is to play with it, look at examples, and try to break it down into the simplest base cases. So far, working on Understanding ActivityPub has only confirmed this belief.

—Written by Sebastian Jambor. Follow me on Mastodon @crepels@mastodon.social for updates on new blog posts.