In our last post, we asked what it would take to build a permissioned data app without AppViews. In this post, we will go ahead and actually build one. Along the way, we will highlight the new protocol features we had to puzzle out in order to get there. We hope that by speculatively building out prototypes before permissioned data is officially supported, we can uncover both underlying requirements and hidden opportunities for the protocol. 

To recap from last time, we are focused on building a permissioned data service (our pear node) that provides the right APIs for local-first personal apps to be built on top. Importantly, we are solving for the use case of small-world applications. In part one, we went deep on why we think cutting the AppView out of the picture makes sense for our use case. 

So given all that, can it actually be done? Let’s find out.

Initial assumptions

Since we are in uncharted territory here, let's lay out the smallest possible set of assumptions about how permissioned data will work explicitly. Then from first principles, we will work out what else we need as we build towards a complete calendar app. Just like Bluesky, we hope to take no steps backwards—an end user would be none-the-wiser that this calendar app was actually built on a completely different protocol and stack than what they are used to for GCal or Outlook.

We started out with these assumptions:

• Each user’s repo is the canonical source of truth for their data. A user cannot write into someone else's repo.

• Simple access control rules sit in front of users’ permissioned data on their PDS, preventing users from accessing other people’s data that they are not authorized to see, and granting particular DIDs permission to read a given record.

• From the last post: AppViews and other third-parties cannot aggregate users’ non-public data. (how we plan to actually enforce this is a post for another time)

We want to highlight the choice we made to deal with client apps (e.g. a website in a browser) exclusively. This might be a surprising decision, so we dedicated a separate blog post to it:

Data model

Before diving into the implementation, let’s write out some pseudo-lexicons for our app. Our actual implementation was based off of community.lexicon.calendar.* . We've simplified it here to make the important parts clear:

event: {
    title: string
    description: string
    start: datetime
    end: datetime
}

invite: {
    invitee_did: string
    event: uri
}

rsvp: {
    event: uri
}

Sending invites

The beginning of any calendar interaction starts with someone creating a new event, and then adding a bunch of their friends or colleagues. Let’s say Alice’s birthday is coming up, and she is inviting her friend Bob. The client code needed to support this is simple enough, just a couple lines of code are needed to request  event and invite records to be written to Alice’s PDS, with permissions granted to Bob. Now, if Alice creates an event and invites Bob, Bob can query the events collection to see everything right?

Not exactly. Though Bob has permission to see the record, his client has no idea that there is a new event record available to view. Without an AppView in the mix, Bob’s client would need to query Alice’s PDS to see that he was actually invited to the event. But so far, neither Bob’s client nor PDS has any way of knowing that Bob was granted permission to see something new.

Clearly, we need some way of telling another user's PDS when something is available to fetch. Our first thought was to build a "notifications" API. The PDS could provide an API for sending notifications to another PDS. With this API in hand, our calendar frontend could submit notifications to each of the invited users when the event is created. Then, the invitee’s client could see the notification and display the event it just learned about.

We implemented a prototype of this, but it turned out to be quite clunky. Having apps handle the notifications increased client complexity quite a bit, and it introduced the risk that different clients might use the system in unpredictably different ways, making reliable guarantees difficult.

Our next stab took a more restricted approach. We decided that a better model for this was PDS-to-PDS permissions sync. If someone writes a record to their PDS granting you permission, your PDS would get a small ping about it. An example minimal ping could be as simple as this:

{
    uri: habitat://did:web:alicexyz/community.lexicon.calendar.event/birthday123
}

Note that this sync mechanism is not like the firehose: PDS'es communicate directly with each other in a verifiable way. Now we have all the pieces to put together the basics of our API:

network.habitat.repo.getRecord(...)
network.habitat.repo.listRecords(...)
network.habitat.repo.createRecord(...)

In our last post, we used the term “fetch-the-world” to describe our way of fetching all the records from the many sources that we care about in one go, and then doing any post-processing (joining, filters, etc) on the client side. This API design makes writing client code super easy, by allowing us to easily snag all events, invites and RSVPs from every PDS that granted us access to one of these with a single listRecords call:

const events = habitatClient.listRecords([“community.lexicon.calendar.event”,“community.lexicon.calendar.invite”, “community.lexicon.calendar.rsvp”] )

Voila, my invitees can now see my invite! We have just learned something: assuming AppViews are out of the picture, some form of PDS-to-PDS communication will be needed, in order for users to know about new records they care about.

Handling RSVPs

At this point, our calendar app supports creating events, inviting people, and allowing those people to fetch those events. But now that my friends can see the invite I sent them, they will want to tell me and everyone else whether they can make it or not. Once again, this seems pretty easy to do - the RSVP-ing user  can just write a community.lexicon.calendar.rsvp record to their repo, and give permissions to everyone involved (which sends their PDS’es a notification under the hood). But wait! The invitee’s client has no idea who else was invited.

At this point, we considered a couple solutions:

1. 1.

   Denormalize the invite list into the event record 🤮

2. 2.

   Create an API to query who has permissions to see the original event, and add all of those users to the RSVP.😵‍💫

3. 3.

   Implement reusable access control lists for permissions on multiple related records across PDS’es 🫡

(1) is straightforwardly bad, because denormalizing the data creates all sorts of headaches. The event record’s internal list of users would have to match the actual invite records written out and the permissions being enforced on each PDS. 

(2) Seems like it works, but it has a catch. If anyone is added to the invite after the rsvp is sent, then the newly invited person will not end up having permissions to see the existing RSVPs.

(3) Is similar to (2), but it abstracts the literal list of invitees into a single reference to a group-like object. If you are in the group, you will be able to see any record that the group was given access to. Even if you are added to the group late, you will still be able to access all the same records as everyone else in the group.

We ended up going with (3), and we called these groups “cliques”. There’s a lot more details we had to work out in the implementation, but here is a quick and dirty rundown:

• A “clique” is fundamentally an access control list stored on someone’s PDS, that can be referred to by anyone else’s PDS. In this sense, they act as a convenient way of delegating permissions to another user.

• Cliques could grant different scopes to users (`read_clique`, `edit_clique`), but for now we will assume we only care about reads. If a user is a clique member, they will be permitted to read all records that grant access to that clique. This means that records can now grant permissions to either a simple user DID, or a clique reference.

• Cliques plug into the permissions sync mechanism we described in the previous section. Being added or removed from a clique will result in your PDS syncing these permission changes. Note that it’s safe if the sync lags: if a clique owner removes you from a clique, you will be denied from fetching records that grant permission to that clique, immediately, but your PDS might still think you have access to the record even if you don’t for a little while, resulting in unnecessary fetches. This is a subtle but important nuance, and it’s by design. 

• When a user is added to an already-existing clique, their PDS should backfill knowledge about records granting permission to that clique by querying the PDS of the user who owns the clique.

Several conversations about grouping mechanisms have been ongoing in the ATmosphere. It seems that by attacking the question of permissioned data from different angles we are converging on some primitives that will be broadly useful! This is cool.

Now, our calendar app is much more feature complete! We can create events, invite people to them, and see whether they RSVP’ed! Each record associated with the event grants permission to the clique created by the event owner, so everyone who was invited to the event can see all the relevant records.

Invitees of invitees

In our first example, Alice invited Bob to an event. Now, what if Bob wants to add someone to the event as well?

We will need to expand on our implementation of cliques. So far, we have been very careful to avoid situations where a user can make writes to another user’s repo. This constraint dramatically simplifies complexity, for example by reducing the chance of write conflicts. However, for this use case, we will need to tweak this constraint by adding scopes to cliques. We debated doing this because allowing writes to another user’s repo feels like a concession, but given that permissions are by nature oftentimes shared and the complexity of other approaches we considered, we felt that this was a valid compromise to make.

So now, Bob has a way to bring a +1 to the party 🎂.

Conclusion

We’ve now finished a very basic calendar app! It turns out that a small-world permissioned data app is achievable without an AppView in the middle, if we make the PDS and the client responsible for a bit more. 

There are definitely some open questions with our approach:

• How can a non-owner edit an event?

• Could you make invitees invisible on the event until they RSVP?

• How do we handle the performance of permissions sync and fan-out as the scale of apps grows?

We will leave these questions for future posts. Of course, none of the protocol elements we discussed are close to final. But just by going through the exercise of building an app, we’ve learned a ton of new things. While the solutions may change, the problem classes we’ve explored will continue to be relevant.

The meta-lesson here is that building prototypes of real products is a very effective way to jumpstart thinking about the future. At each step of building the app, we encountered new challenges that forced us to speculatively create protocol features. Needless to say, there were a lot of points in this exploration where different technical decisions could have been made. For example, we explored using UCAN tokens rather than simple ACLs, but decided not to go with them because the revocation case became significantly harder. We would love to hear from you if you would have chosen to do something different.

We’re beginning to formalize our APIs: take a sneak peek here if you want to think even more about permissioned data. 

🌱

That's all for now (or is it? 👀). See y'all at AtmosphereConf!