Hi, Steemit! We're Textile. Here's a deeper look at the tech behind our Threads protocol.

Written by Carson Farmer & Sander Pick

Download the app, take a picture, share!

Recently,  we’ve started writing more about the technologies underlying Textile  Photos that help keep your photos (and likes and comments, etc) safe and  secure on the decentralized web. In our previous post, we talked about the encryption process Textile Photos,  with a focus how Textile delivers end-to-end encrypted photo sharing.  Today’s post is a follow-up (though it should also be sufficiently  detailed to stand on its own), this time highlighting how Textile  coordinates private photo sharing among groups of users, a feature we  call Threads.

Why we built it
We  designed Threads to allow groups of users to share photos securely and  privately, without any centralized, authoritative database. We also made  sure it all works well offline, that its possible to recover lost data,  and that its easy to add new members.

What makes Threads different
Threads  allow private groups to post photos and interact over a decentralized  network, maintaining complete control over their own content. Textile  operates in a completely zero-knowledge framework. Private by design.

Why Threads are exciting
Because  photos are just the first step. Today, Threads allow users to share a  photo with other Thread in a secure, decentralized way. Threads can  facilitate secure sharing, coordination, and storage of many types of data over a decentralized network. Upgradable by design.

On  the surface, you can think of each Thread like a decentralized  database, shared between specific participants. We built Threads into  the fabric of Textile (see what we did there 😉) because group members  need a record of who shared what photo, and when. But, once we created  Threads, we realized just how powerful a concept this was — for those  familiar with mobile app development, think Realm or Firebase but without the centralized server.

To  really understand what Threads brings to the table, you really need to  understand Threads themselves. So let’s dig a bit deeper into how  Textile conceptualizes and implements Threads, and how that helps keep  your photos (and likes and comments, etc) safe and secure on the  decentralized web. We’ll start by highlighting the specific requirements  we had when developing Threads, and then break down each of these  requirements into the specific solutions that we came up with. Along the  way, our CTO Sander Pick will highlight how those various solutions  came about, and why we think our approach is in the best interest of our  users.

The experience

Textile Photos  allows small, decentralized, private groups to share photos, send  messages, and engage with each other. That’s the experience, so it has  to ‘just work’.

Requirements

To drive the Textile user experience, we identified five key features needed in the sharing protocols:

1. A mechanism to share and receive state updates within a group of n users
   To  enable photo sharing (and other common interactions such as likes and  comments) among a group of friends and/or colleagues, some concept of a shared state is required.
2. A way to ensure the shared state stays resilient to peers dropping out or latency issues
   Since  we’re operating in a mobile environment, we have to expect peers to  continually drop ‘offline’ due to coverage issues, app back-grounding,  battery optimizations, and a whole slew of other reasons for a mobile  device to be cut off from a network.
3. A way to avoid state conflicts with other members of the group
   On  top of the requirements above, when peers do come back online, we don’t  want any state changes that were made by other members of the group  while they were disconnected from the network to conflict with their own  local changes.
4. A mechanism to recover the full state from the network as a whole
   Another  important consideration in the mobile world is that the number of users  (out of n) that are online at any given time is generally unknown, and  quite possibly zero. To reiterate, we want a decentralized shared state,  but it has to work even when you are the only member online.  This means we have to assume the full group state may not ever be  directly accessible (i.e., downloadable) from a single group member.  This is in contrast to something like Bitcoin, where new nodes are able to download the full blockchain from any connected peer.
5. Way to link updates via their content, rather than where they are stored
   Since we are building on top of the IPFS network, and would like to eventually support a Filecoin-based  future in which users can select from a multitude of decentralized  storage providers, Threads need to embrace content addressing, rather  than location addressing. This makes it easy to grow and change the  underlying network, without affecting data access and sharing.

With these requirements in mind, let’s break down our solutions into their individual components…

Solutions

1. Handling Updates — use a peer-to-peer network with structured updates

First things first: how do we handle state updates between a set of distributed peers? This is mostly about peer-to-peer (p2p) networking.  And when it comes to communicating between heterogeneous network  devices (computers, phones, IoT devices, etc), we actually need many different types of network protocols.  That way, no matter what type of device we are talking about — be it a  phone, desktop computer, browser, or Internet-enabled fridge — it is  able to communicate with other devices located in the same room, or on  the other side of the planet.

At Textile, we use the super amazing libp2p  library for our networking needs. Libp2p is a networking stack and  library (you might have heard it called a protocol suite) modularized  out of the IPFS project,  and bundled separately for other tools to use. Essentially, libp2p does  all the heavy network lifting so that we can focus on our core task:  exchanging updates between communicating peers.

Decentralized, peer-to-peer networks are radically different types of communication networks.

Libp2p  was a pretty natural choice for us. The stack includes all the crypto  and networking protocols we need to deliver messages to group members,  and the libp2p developer community is super responsive and excited about  the power of p2p interactions. Easy choice.

The  other really nice thing about using the libp2p library is it comes  packed with many useful cryptography tools and functions, keeping  communications secure. For instance, all p2p communications over the  Textile network use the secio stream security transport.  This way, all connections use secure sessions provided by libp2p/secio  to encrypt all traffic, whereby a TLS-like handshake is used to setup  the initial communication channel.

Like many IPFS-based projects, Textile uses Protocol Buffers  for over-the-wire communication, and advanced cryptographic algorithms  to secure those messages. Essentially, each update to the shared group  state is just an encrypted Profobuf message with two parts: a header  with author and date info, and a body with the type-specific data. These  pieces are sent in their own inner-’envelope’ which contains a link to  the encrypted message and the Thread ID. This inner-envelope is then  signed by the sender and placed into the wire ‘envelope’ along with it’s  signature. You can read more about some of the cryptographic tools  Textile uses in this previous article. You can also check out how we structure our Protobuf messages, learn a bit more about how secio works, plus check out some recent updates to message encryption while you’re at it.

2. Network Resilience — support offline messaging so peers can come and go

If you are at all familiar with libp2p, then you might be thinking “ah libp2p has a pubsub layer that would be perfect for exchanging updates to a group of connecting peers”.  And while you’d certainly be right, there are a few key limitations  that makes using pubsub for something like Textile Photos pretty  cumbersome. On top of this, while pubsub is super nice for things like  chat rooms or distributed services, it is a ‘fire-and-forget’ messaging  protocol, meaning that once a peer publishes a message, it is up to its  peers to ensure they are listening for the right message at the right  time. To circumvent this, some pubsub systems introduce message echoing,  to ensure a message stays in the system long enough to be picked up by  the peers who might need it. However, this can lead to really noisy  network traffic, and is really just a band-aid over a larger issue.

Our  initial POC involved pubsub and always-online room echoers… not  scalable or particularly decentralized. A real solution to distributing  state has to involve direct messaging with an offline mechanism.

So this starts to get at our second requirement, that the shared state stays resilient to peers dropping out.  We need to assume peers might not be around to receive important  messages in ‘real-time’, which is a common problem with p2p systems.  Right now, Textile addresses this problem by enabling what you might  call offline messaging. Since we’re already using IPFS for data storage and communication,  we wanted to take advantage of some of the core technologies driving  IPFS. In particular, we (currently) use a special fork of the Kademlia-based distributed hash table (DHT) used by IPFS, that allows us to post messages for a peer directly in the DHT. For those unfamiliar with DHTs, they are a hash table  where the data is spread across a network of nodes or peers. And these  peers are all coordinated to enable efficient access and lookup between  nodes in a decentralized way. You can read more about this kind of stuff  in our previous article about how IPFS peers find, request, and retrieve content (and each other) on the decentralized web.  So, when a peer we want to communicate with is offline, rather than  blindly sending them a message that will never be received, we post a  message to Textile’s DHT, and they can then retrieve that message the  next time they come online again. Conceptually simple, and works pretty  well in practice.

p2p  network with custom Textile DHT overlay. Peers post (key, value)  messages (value) with a key specific to their intended recipient, and  this key is broadcast and available to entire network; though only the  intended recipient is able to decrypt the actual message content. Based  on Figure 1–2 from this thesis.

There  are still some issues with our current approach, including that it is  difficult/impossible to remove messages from the DHT manually. Indeed,  it can start to get a bit messy when left-over offline messages have to  be retrieved each time a peer comes back online… imagine a peer that  goes in and out of service frequently, this could lead to a lot of  network traffic and wasted CPU cycles. So, we’ve implemented an alternative  to this DHT-based offline messaging system that does not suffer from  these limitations (and also allows us to participate in the public IPFS  network), while still remaining decentralized and scalable in the  long-term. This new approach should be released soon, after more testing  and evaluation. You can follow along with this progress as part of the move towards a Cafe-based setup (see also What’s Next).

3 & 4. Avoiding Conflicts & State Recovery —use a CRDT to keep an immutable history across peers

Ok, so our next requirement and its associated solution have received a great deal of research and development attention over the years. The question of “how to avoid state conflicts with other members of a group?”  comes up when working collaboratively on documents, updating shared  databases, etc. For the purposes of updating a shared Thread of photos,  it turns out that an operation-based CRDT  that supports append-only operations is pretty much all you need to get  going. You can think of Textile’s CRDT (which shares some ideas with ipfs-log)  setup as an immutable, append-only tree that can be used to model a  mutable, shared state between peers. Every entry in the tree is saved on  IPFS, and each points to a hash of previous entry(ies) forming a graph.  These trees can be 3-way and fast-forward merged.

Speaking  of forks and joins, for those familiar with git and other similar  system, you might be thinking this sounds a lot like a git hash tree, Merkle DAG, or even a blockchain.  And you’d be right! The concepts are very similar, and this buys us  some really nice properties for building and maintaining a shared state.  By modeling our shared Thread state in this way, we benefit from tried  and tested methods for allowing a peer to incorporate other peers’  updates into their state while maintaining history (via fast-forwards  and three-way merging for example).

At the end of the day, a Thread is just a git-like hash tree of updates with a deterministic merge policy. Simple.

So  what does this look like in practice? Currently — because things might  change as we make improvements to the underlying implementation — each  Thread in Textile Photos is essentially a chain of updates, where each  update represents some specific action or event. For instance, when you  create a new Thread, under-the-hood you are actually creating a JOIN update on a new Thread chain. Similarly, when you update the Thread via a new photo (DATA update), comment, or like (ANNOTATION update), you’re actually updating that Thread chain. After each modification, the HEAD of the Thread will point to the latest update.

Building on top of these ideas, we also have concepts such as an INVITE, which points a new peer to a given point on the Thread chain, or a MERGE, which happens when the current HEAD  is not contained in an incoming update’s parent list for some reason  (maybe the peer doesn’t know about it because they were offline). If two  peers are merging the same sub trees,  all they need to do to ensure the update resolves to the same hash is  a) include the same date b) exclude author info. To get the same date,  they both follow a rule: choose the latest of the parents for the date  (in practice they add a little bit extra on to keep it ahead of both  parents).

To give you a better idea of what exactly we’re talking about, consider the following set of operations: User A creates a new Thread, and adds a Photo. They then externally invite User B (sent via some other secure communication channel), who eventually joins the Thread. But before User B is able to join the Thread, User A adds another Photo, moving the Thread’s HEAD forward. By the time User B joins the Thread, they’d end up with a Thread sequence that looks something like this:

Thread  join example. Solid arrows point towards the ‘parent’ of a given  update, over-the-wire communications are indicated with a 📶-style  arrow, and messages that are rebroadcast (e.g., via the welcome message)  are indicated with a dashed arrow. Similarly, merges point to both  their parent updates.

Here,  we see the merge happening at the end of the sequence because the  bottom peer is joining via an external invite that is no longer HEAD , forcing them to merge the most recent DATA update with their own JOIN update. But since merge results are deterministic (given the same parents), both peers create the MERGE update locally, and do not broadcast them to avoid trading merges back and forth.

A more complete sequence is given in the following figure. Suppose User A  goes ‘offline’ (e.g., their phone goes to sleep, they shut down the  app, they lose their data connection, etc), and in the mean time, both Users A and B update the Thread, with User A adding an ANNOTATION update, and User B adding a new Photo (DATA update). Now, when User A comes back online, there is a conflict, and both Users create a MERGE update to remedy this. A MERGE update has two parents, in this case, the DATA and ANNOTATION update from the different users. As always, the HEAD continues to point to the latest update (which in the example below eventually becomes an ANNOTATION from User B). Once both peers are online again, the more straightforward update and transmit mode of operation can continue.

More  complex Thread interaction where one or more peers are temporarily  offline. Note that an external invite is the same as a normal invite,  but the invite details are encrypted with a single use key, which is  sharable with the invite update location.

The  same properties that make hash trees or blockchains useful for  developing a shared, consistent (consensus-driven) state, also makes it  possible to address our fourth requirement: the ability to recover the full state from the network as a whole.  Because each Thread update references its parent(s), given a single  point on the Thread chain, we can trace back all the way to the  beginning of the Thread. For example, at any point along the sequence in  the above figures, a peer can trace back the history of the Thread, as  indicated by the solid arrows. This works particularly nicely when a  peer JOINs a thread, even at a point prior to the current HEAD. They can simply JOIN, and any existing Thread member can send them the latest HEAD  (even via offline messages if needed). From here, they can explore the  entire history of the Thread with ease. This is all really similar to  git commit speak, in which one only needs to know about a single commit  to be able to trace back the entire history of a code project; it’s also  essentially how blockchains work.

5. Content Addressing — store everything on IPFS and get ready to scale

As  we alluded to earlier, each update to a Thread is backed by an IPFS CID  hash (i.e., they are content addressable chunks of data on IPFS). This  means where the data is stored  is no longer relevant… IPFS will find it on the network via it’s hash.  This helps us address our fifth requirement, that we have a way to link updates via their content, rather than where they are stored. We’ve covered this topic a lot in the past, but for the uninitiated, the next paragraph provides a summary of how content addressing on IPFS works (pulled from this previous article).

Rather than referencing a file or chunk of data by its location (think HTTP), we reference it via its fingerprint. In IPFS and other such systems, this means identifying content by its cryptographic hash, or even better, a self-describing content-addressed identifier (multihash). A cryptographic hash is a (relatively) short alphanumeric string that’s calculated by running your content through a cryptographic hash function (like SHA). For example, when the (unencrypted) Textile logo is added to IPFS, its multihash ends up being QmbgGgWW3vH7v9FDxVCzcouKGChqGEjtf6YLDUgSHnk5J2. This ‘hash’ is actually the CID (Content IDentifier) for that file, computed from the raw data within that PNG. It is guaranteed to be cryptographically unique to the contents of that file, and that file only. If we change that file by even one bit, the hash will become something completely different.

Now,  when we want to access a file over IPFS (like the above logo), we can  simply ask the IPFS network for the file with that exact CID, the  network will find the peers that have the data (using a DHT), retrieve  it, and verify (using the CID) that it’s the correct file. What this  means is we can technically get the file from multiple places  because as long as the file matches the hash, we know we’re getting the  right data. Which brings us to the solution to our final requirement…  use IPFS! For now, Textile is maintaining a network of large,  homogeneous, volunteer nodes (we call them Cafes)  to ‘pin’ and store content on IPFS. It is important to note here that  the other nodes doing the pinning are the same as the nodes on your  phone — Textile Nodes that offer a pinning service to other peers. Soon,  we’ll allow users to elect their own Cafe nodes, add even add  additional nodes for redundancy. All this could eventually be driven by  Filecoin for even greater scalablility and flexibility.

What’s Next?

So  there you have it. Five solutions to five requirements for seamless,  secure, decentralized photo sharing and backup. Easy 😉. And at a  conceptual level, the Textile Thread protocol is  relatively simple: blocks of operations chained together to produce a  beautiful Thread of photos. But there’s a lot of complexity going on  under-the-hood that has required a lot of experimentation, testing, and  limit pushing, especially on mobile. And our journey isn’t over yet.

The Textile team  is still hard at work iterating, updating, and improving upon what we  already have working. For example, we’ll soon to moving to a new offline  messaging system that allow us to drop the custom DHT fork, and move  back to the public IPFS network. On top of this, our move to more powerful backup  and recovery capabilities has us taking new approaches to security,  profile management, offline interactions, and much much more. On top of  these changes, the team is actively working to modularize the Threads  concept and code into its own stand-alone package, which should provide  developers with something akin to a Realm and/or Firebase layer for  decentralized mobile applications!

If you are interested in learning more about this stuff, reach out over Twitter or Slack,  or pull us aside the next time you see us at a conference or event. We’re happy to provide background, thoughts, and opinions on how we  think the future of decentralized apps will play out. In the mean time,  don’t forget to check out our GitHub repos  for code and PRs that showcase our current and old implementations. We  try to make sure all our development happens out in the open, so you can  see things as they develop. Additionally, if you haven’t already, don’t miss out on signing up for our waitlist, where you can get early access to Textile Photos, the beautiful interface to Textile’s Threads.