Unraveling A Tangled Web; Roll Your Own Records; atmo

I’ve got ATproto on the mind, this week, and the hump day edition of the block should make a great reference for your own hacking on ATproto.

TL;DR

(This is an AI-generated summary of today’s Drop using Ollama + llama 3.2 and a custom prompt.)

• A walkthrough of analyzing the current extent of the Tangled.sh, ATproto-based, decentralized social coding network.
• ATproto enables custom data repositories beyond social media posts, demonstrated through a “bookshelf” record example using Bash with curl, allowing users to create, update, delete, and query custom records with their own schemas (https://atproto.com/guides/lexicon)
• atmo.lol is a decentralized URL shortener that stores links on your ATproto identity, creating shortened URLs in the format “atmo.lol/handle/short” that redirect to original destinations while maintaining user control over link management (https://www.atmo.lol/)

Unraveling A Tangled Web

The Tangled team continues to refine their nascent, distributed (via ATproto) social/collaborative coding platform and I am keen to keep an eye on it. So, I set up a jetstream consumer that subscribes to the following collections:

• sh.tangled.repo
• sh.tangled.feed.star
• sh.tangled.graph.follow
• sh.tangled.publicKey
• sh.tangled.repo.issue.comment

and shunts any messages to a RedPanda topic where I can siphon them off and process them in real-time or whenever the spirit moves.

You can spy on that activity without any code by occasionally hitting https://tangled.sh/ since they update the main page with similar events.

As a refresher, jestream messages look like this:

{
  "did": "did:plc:hwevmowznbiukdf6uk5dwrrq",
  "time_us": 1742203972061888,
  "kind": "commit",
  "commit": {
    "rev": "3lkktwh4czs2e",
    "operation": "create",
    "collection": "sh.tangled.graph.follow",
    "rkey": "3lkktwh2ubi22",
    "record": {
      "$type": "sh.tangled.graph.follow",
      "createdAt": "2025-03-17T09:32:50Z",
      "subject": "did:plc:o3bh4ov7x4ypfz4xg5redtki"
    },
    "cid": "bafyreicyxf4fp4smmsxk7th3r6kia5czkvjt5lh7kpcop7rigdczyznwvm"
  }
}

My goal for today’s journey was to collect actor and target DIDs and enumerate their knots (git repos). By monitoring this slice of the jetstream over time, we’ll be able to measure the adoption growth as well as the human social graph.

Rather than connect to RedPanda directly, I grabbed a point-in-time count of collected records using RedPanda’s CLI:

docker exec -it redpanda-0 rpk topic consume tangled --offset start --num 0

then exported the records in ndjson format:

docker exec -it redpanda-0 rpk topic consume tangled --num 238 -f "%v\n" > /tmp/tangled.json

These days, I would normally do the rest of this work with DuckDB, but we need to make a few HTTP calls to retrieve additional metadata, so we’re going to use the {tidyverse} R variant.

We’ll first grab any actor DIDs we can find:

msgs <- jsonlite::stream_in(file("~/Data/tangled.json"))

actors <- unique(msgs$did)

msgs$commit |> 
  filter(
    collection == "sh.tangled.graph.follow"
  ) |>
  pull(record) |> 
  filter(!is.na(subject)) |> 
  pull(subject) |> 
  unique() -> followed

msgs$commit |> 
  filter(
    collection == "sh.tangled.feed.star"
  ) |>
  pull(record) |> 
  filter(!is.na(subject)) |> 
  pull(subject) |>
  gsub("^at://|/sh.*", "", x = _) |> 
  unique() -> starred

Combined and de-duplicated, that amounts to ~100 unique DIDs. We’ll use https://plc.directory/ to get each account’s handle and PDS FQDN, as we’ll need to grab a list of their top-level records. We’ll be using this function to do so:

get_tangled_repo <- function(host, did) {
  
  httr::GET(
    url = sprintf("%s/xrpc/com.atproto.repo.listRecords", host[1]),
    query = list(
      repo = did,
      collection = "sh.tangled.repo",
      limit = 100
    )
  ) |> 
    httr::content(as = "text") |> 
    jsonlite::fromJSON()
  
}

And, here are the steps:

c(actors, followed, starred) |> 
  unique() |> 
  sprintf("https://plc.directory/%s", x = _) |> 
  map(httr::GET, .progress = TRUE) |> 
  map(httr::content, as = "text") |> 
  map(jsonlite::fromJSON) |> 
  map(\(.x) {
    tibble(
      did = .x$id[1],
      handle = .x$alsoKnownAs[1],
      pds = .x$service$serviceEndpoint[1]
    )
  }) |> 
  list_rbind() |> 
  mutate(
    repos = map2(pds, did, get_tangled_repo, .progress = TRUE),
    records = map(repos, "records")
  ) |> 
  select(-repos, -did) |> 
  filter(lengths(records) > 1) |> 
  unnest(records) |> 
  unnest(value) |> 
  janitor::clean_names() -> xdf
## Rows: 147
## Columns: 10
## $ handle      <chr> "at://icyphox.sh", "at://icyphox.sh", "at://icyphox.sh", "at://…
## $ pds         <chr> "https://pds.icyphox.sh", "https://pds.icyphox.sh", "https://pd…
## $ uri         <chr> "at://did:plc:hwevmowznbiukdf6uk5dwrrq/sh.tangled.repo/3lkoko7u…
## $ cid         <chr> "bafyreibek6tak5ep37jhih3jxipav6wpc3traumdiuu3lyckrgz4nnxloi", …
## $ knot        <chr> "localhost:5555", "localhost:5555", "localhost:5555", "localhos…
## $ name        <chr> "local", "local", "_foobar", "kubernetes", "hmmm", "legit", "an…
## $ type        <chr> "sh.tangled.repo", "sh.tangled.repo", "sh.tangled.repo", "sh.ta…
## $ owner       <chr> "did:plc:hwevmowznbiukdf6uk5dwrrq", "did:plc:hwevmowznbiukdf6uk…
## $ added_at    <chr> "2025-03-18T22:57:50+02:00", "2025-03-18T22:57:36+02:00", "2025…
# $ description <chr> NA, NA, NA, NA, NA, "web frontend for git (tangled's grandpa)",…

Let’s see who’s here:

xdf |> 
  pull(handle) |> 
  unique()
##  [1] "at://icyphox.sh"                 "at://stephanetrebel.bsky.social"
##  [3] "at://oppili.bsky.social"         "at://mchal.lol"                 
##  [5] "at://nelind.dk"                  "at://chadtmiller.com"           
##  [7] "at://technoduck.me"              "at://patrick.sirref.org"        
##  [9] "at://jcmdln.bsky.social"         "at://j3s.sh"                    
## [11] "at://mmatt.net"                  "at://fxbrottier.com"            
## [13] "at://xeiaso.net"                 "at://steveklabnik.com"          
## [15] "at://davidcel.is"                "at://bladee.bsky.social"        
## [17] "at://alternatebuild.dev"         "at://stephen.jennings.io"       
## [19] "at://bobbby.online"              "at://journcy.net"               
## [21] "at://56quarters.xyz"             "at://isaaccorbrey.com"          
## [23] "at://seabre.com"                 "at://theoparis.com"             
## [25] "at://ducky.ws"                   "at://mary.my.id"                
## [27] "at://hrbrmstr.dev"               "at://melontini.me"              
## [29] "at://soopy.moe"                  "at://gune.ninja"                
## [31] "at://fogtype.com"                "at://narendasan.com"            
## [33] "at://brycemecum.com"             "at://sethvincent.com"           
## [35] "at://semitext.xyz"               "at://increasing.bsky.social"    
## [37] "at://shi.gg"                     "at://bryantluk.com"             
## [39] "at://atpota.to"                  "at://tangled.sh"                
## [41] "at://anil.recoil.org"           

Bluesky/ATproto gets a bad rap for “not being federated”. Sure, there are far fewer folks running their own PDS instances and there are Mastodon instances, but that will change over time. And, I was pleasantly surprised to see seven non-Bsky ones in just this tiny sample:

xdf |> 
  pull(pds) |> 
  unique() |> 
  curlparse::domain() |>
  psl::apex_domain() |> 
  table(dnn = "domain") |> 
  sort() |> 
  as.data.frame.table(
    responseName = "ct"
  )
  domain ct
## 1 chadtmiller.com  1
## 2      icyphox.sh  1
## 3       mchal.lol  1
## 4       mmatt.net  1
## 5        rof1.net  1
## 6       soopy.moe  1
## 7        zio.blue  1
## 8    bsky.network 20

We can also see if this sampled Tangled population is embracing decentralization when it comes to Knots (self-hosted git + Tangled interfaces):

xdf |> 
  pull(knot) |> 
  table(dnn = "knot") |> 
  sort() |> 
  as.data.frame.table(
    responseName = "ct"
  )
##                         knot ct
##  1     bigmoves-knot.fly.dev  1
##  2        code.theoparis.com  1
##  3           knot.recoil.org  1
##  4  enanan.staging.soopy.moe  2
##  5   knot.alternatebuild.dev  2
##  6            knot.nelind.dk  2
##  7        knot.technoduck.me  2
##  8           knot2.mmatt.net  3
##  9         knot.hrbrmstr.app  4
## 10            git.recoil.org  5
## 11            knot.mchal.lol  5
## 12          knot.kelinci.net  6
## 13            localhost:6000 19
## 14            localhost:5555 37
## 15          knot1.tangled.sh 57

So, the answer is “sort of”.

Our final safari for today is to enumerate the repos, which we can do in full, here, since this sampled slice is still small:

xdf |> 
  distinct(
    handle, name, description
  )

• (@56quarters.xyz) mtop
• (@alternatebuild.dev) mcp-servers: damn i got this in a bad state somehow and can’t delete i guess
• (@alternatebuild.dev) mcpsrvrs
• (@alternatebuild.dev) test: sanity check using hosted knot
• (@anil.recoil.org) helloworld
• (@anil.recoil.org) helloworld2
• (@anil.recoil.org) knot-docker: Docker compose for running your own Tangled Knotserver
• (@anil.recoil.org) ocaml-jmap
• (@anil.recoil.org) opam-repository
• (@atpota.to) atpota.to
• (@atpota.to) bluesky-brand-playbook
• (@atpota.to) cred.blue
• (@atpota.to) flushes.app
• (@bladee.bsky.social) darkfeed
• (@bobbby.online) blog
• (@bobbby.online) bookmarker
• (@bobbby.online) tasty
• (@bobbby.online) tasty-bookmarking
• (@bobbby.online) tasty_phx
• (@bryantluk.com) helloworld
• (@brycemecum.com) alembic
• (@chadtmiller.com) stuff
• (@davidcel.is) dotfiles
• (@ducky.ws) test-repo
• (@fogtype.com) nebel:
• (@fxbrottier.com) certs-helper
• (@gune.ninja) atproto-pgp-keyserver
• (@gune.ninja) bluesky-avatar-updater
• (@hrbrmstr.dev) 47-watch: Keeping an eye on the Don
• (@hrbrmstr.dev) dailydrop: Daily Drop example repo
• (@hrbrmstr.dev) hrbrthemes
• (@hrbrmstr.dev) my-first-self-hosted-knot-repo: My first self-hosted Knot repo!
• (@hrbrmstr.dev) skygrep
• (@icyphox.sh) _foobar
• (@icyphox.sh) ahstasht
• (@icyphox.sh) another
• (@icyphox.sh) asht
• (@icyphox.sh) ashtatstttash
• (@icyphox.sh) bropls
• (@icyphox.sh) buh
• (@icyphox.sh) dotfiles: my nix dotfiles
• (@icyphox.sh) dummy
• (@icyphox.sh) hellobsky
• (@icyphox.sh) hmm
• (@icyphox.sh) hmmm
• (@icyphox.sh) k8s
• (@icyphox.sh) kubernetes
• (@icyphox.sh) legit: web frontend for git (tangled’s grandpa)
• (@icyphox.sh) local
• (@icyphox.sh) lol
• (@icyphox.sh) lolol
• (@icyphox.sh) newrepo
• (@icyphox.sh) site: my website at https://anirudh.fi
• (@icyphox.sh) tesbt
• (@icyphox.sh) test: my test repo
• (@icyphox.sh) testa
• (@icyphox.sh) testrepo
• (@increasing.bsky.social) simple-left-right
• (@isaaccorbrey.com) hello_world
• (@j3s.sh) .files
• (@j3s.sh) test
• (@j3s.sh) vore.website: beebo
• (@jcmdln.bsky.social) ansible-collection-openbsd: An Ansible collection for OpenBSD – https://galaxy.ansible.com/ui/repo/published/jcmdln/openbsd/
• (@journcy.net) tidal: messing around learning tidal
• (@mary.my.id) aglais
• (@mary.my.id) anartia
• (@mary.my.id) atcute
• (@mary.my.id) bluesky-embed
• (@mary.my.id) boat
• (@mary.my.id) knot-docker
• (@mchal.lol) atmo.lol: A decentralized URL shortener.
• (@mchal.lol) garduino
• (@mchal.lol) jedo: A simple and intuitive CLI “Projects” directory manager.
• (@mchal.lol) jellydonut: Remind me to delete this once deleting repos is added!
• (@mchal.lol) qwertycarduino
• (@mchal.lol) site
• (@melontini.me) test
• (@mmatt.net) CSCI-2170
• (@mmatt.net) mmatt.net
• (@mmatt.net) thisisatest1
• (@narendasan.com) e-ink-poster: A poster using an e-ink panel with software written in Rust
• (@narendasan.com) test-repo
• (@nelind.dk) test
• (@nelind.dk) http://www.nelind.dk
• (@oppili.bsky.social) a-rep
• (@oppili.bsky.social) a-repository
• (@oppili.bsky.social) another repo
• (@oppili.bsky.social) another-another-test-2
• (@oppili.bsky.social) arstarstarstarst
• (@oppili.bsky.social) bcc
• (@oppili.bsky.social) branch-testing
• (@oppili.bsky.social) bt-2
• (@oppili.bsky.social) bt-3
• (@oppili.bsky.social) bt-4
• (@oppili.bsky.social) foobar-testing-repo
• (@oppili.bsky.social) gixsql-flake
• (@oppili.bsky.social) gixsql-flake-2
• (@oppili.bsky.social) lurker: selfhostable, read-only reddit client
• (@oppili.bsky.social) main-branch-testing
• (@oppili.bsky.social) main-branch-testing-2
• (@oppili.bsky.social) one-last-time
• (@oppili.bsky.social) opam-repo-test
• (@oppili.bsky.social) opam-test-1
• (@oppili.bsky.social) opam-test-3
• (@oppili.bsky.social) plonkli: atproto pastebin service: https://plonk.li
• (@oppili.bsky.social) plonkli: git remote add origin git@localhost:6000:oppili.bsky.social/plonkli this is a very long description of this repository
• (@oppili.bsky.social) tangled-test
• (@oppili.bsky.social) test-local-repo
• (@oppili.bsky.social) test-local-repo-2:
• (@oppili.bsky.social) trunk-repo-branch
• (@oppili.bsky.social) txn-2
• (@oppili.bsky.social) txn-3
• (@oppili.bsky.social) txn-test
• (@oppili.bsky.social) txn-test: asdfasdfasdf
• (@patrick.sirref.org) opentrace
• (@seabre.com) dotfiles
• (@seabre.com) resume
• (@seabre.com) test2
• (@semitext.xyz) osrx
• (@sethvincent.com) pizzatown
• (@shi.gg) bun-broken
• (@shi.gg) mellow-web: The weeb for the next gen discord boat – Wamellow
• (@soopy.moe) explod
• (@soopy.moe) gensokyo
• (@soopy.moe) knotserver-module
• (@stephanetrebel.bsky.social) test
• (@stephen.jennings.io) 2padlocks
• (@steveklabnik.com) hello-world
• (@tangled.sh) core: Monorepo for Tangled — https://tangled.sh
• (@tangled.sh) myown
• (@tangled.sh) site
• (@technoduck.me) iss_locator.rs
• (@technoduck.me) staticrustator
• (@theoparis.com) bevy-os
• (@xeiaso.net) hypercloud

That’s 135 repos!

It’s super cool to see folks jumping into this Tangled part of the ATproto-verse.

You can find the R code here: https://atmo.lol/hrbrmstr.dev/drop-263 (we’ll be explaining atmo in a bit) and the data it uses here: https://rud.is/dl/tangled.json.

Roll Your Own Records

As we’ve droned on about before, the AT Protocol (ATporoto) powers Bluesky and offers far more than just social media posts. It’s a decentralized protocol that enables custom data repositories with flexible schemas. This means you can build applications that store and retrieve structured data beyond typical social media content. While I’ve showcased some of those used before, it’s time to walk through how you can hack on ATproto to store your own subversive content/apps, such as:

• Digital bookshelves tracking reading history and reviews
• Music collection catalogs with ratings and listening history
• Recipe collections with ingredients and preparation notes
• Location-based check-in systems
• Knowledge bases with interconnected information

Let’s walk through creating a (lame) custom “bookshelf” record using Bash with curl to demonstrate the core concepts.

You will need:

1. A Bluesky account
2. An app password (created in Bluesky settings)
3. Basic command line knowledge
4. curl and jq installed

We’ll use Bluesky app passwords (so, really, just “tokens”) for authentication. (NOTE: this is a toy example and you should not write out seekrits to files even temporarily in a real app. I did it this way so you can comment out the cleanup/deletes to see what the payloads look like.)

#!/usr/bin/env bash

set -e
trap cleanup EXIT

TEMP_DIR=$(mktemp -d)
AUTH_FILE="${TEMP_DIR}/auth.json"
RECORD_FILE="${TEMP_DIR}/record.json"

cleanup() {
  echo "Cleaning up temporary files..."
  rm -rf "${TEMP_DIR}"
}

HANDLE="your-handle.bsky.social"
APP_PASSWORD="your-app-password"
COLLECTION_NAME="is.rud.bsky.bookshelf.book"
RECORD_TYPE="book"

# Create auth payload
cat > "${AUTH_FILE}"  "${RECORD_FILE}" << EOF
{
  "repo": "$DID",
  "collection": "$COLLECTION_NAME",
  "rkey": "$RECORD_TYPE-$(date +%s)",
  "record": {
    "\$type": "$COLLECTION_NAME",
    "title": "The Great Gatsby",
    "author": "F. Scott Fitzgerald",
    "rating": 5,
    "review": "A classic American novel that explores themes of wealth, class, and the American Dream.",
    "readDate": "$TIMESTAMP",
    "createdAt": "$TIMESTAMP"
  }
}
EOF

The record structure includes:

• repo: Your DID
• collection: The namespace for your data type
• rkey: A unique key for this record
• record: The actual data with required $type field

Now we publish the record to your ATproto repository:

# Send the create record request
CREATE_RESPONSE=$(curl -s -X POST "https://bsky.social/xrpc/com.atproto.repo.createRecord" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ACCESS_TOKEN" \
  -d @"${RECORD_FILE}")

# Extract record identifiers
URI=$(echo "$CREATE_RESPONSE" | jq -r .uri)
CID=$(echo "$CREATE_RESPONSE" | jq -r .cid)

After successful creation, you’ll receive:

• a URI: The canonical identifier for your record
• a CID: The content identifier (hash of your record data)

For production applications, you should define a lexicon schema for your collection. This is a formal definition of your data structure that helps with validation and interoperability.

Beyond creation, you’ll likely need to:

• Update records using com.atproto.repo.putRecord
• Delete records using com.atproto.repo.deleteRecord
• Query records using com.atproto.repo.listRecords

which you can extrapolate from this script as you peruse the Atmosphere.

ATproto allows for complex data relationships through references between records, enabling rich, interconnected data structures.

Since we used curl for our examples, the concepts transfer easily to any programming language with HTTP capabilities:

• Python: Use the requests library
• JavaScript: Use fetch or axios
• Go: Use the standard net/http package
• Rust: Use the reqwest crate
• R: Use httr or httr2

By understanding these fundamentals, you can build applications that leverage the decentralized, interoperable nature of the AT Protocol beyond just social media posts.

The create & delete example scripts can be accessed vi my Plonk/paste instance.

atmo

atmo.lol (knot) is a decentralized URL shortener that creates short URLs. Unlike traditional URL shorteners, all your links are stored on your own ATproto identity, meaning you—and only you—can create, manage, and delete them.

When you create a shortened URL with atmo.lol, it’s stored in a new collection called “lol.atmo.link” on your ATproto identity. Each record contains just two simple values:

• short: Your chosen shortened path
• url: The original destination URL

To use a shortened link, simply enter: https://atmo.lol/<handle>/<short>

For example: https://atmo.lol/hrbrmstr.dev/drop-263 redirects you to yesterday’s Tidy Tuesday post.

The full process:

1. Resolves your handle into a DID
2. Locates your PDS (Personal Data Server)
3. Checks its cache for the record
4. If needed, queries your PDS for the corresponding record
5. Redirects visitors to your original URL
6. Caches the result for 30 minutes to improve performance

Currently, atmo.lol offers a range of powerful tools for URL management. Their backend webserver efficiently handles all redirections, ensuring your shortened links work seamlessly.

For those who prefer command-line interfaces, atmo.lol provides a comprehensive CLI tool that gives you complete control over your shortened URLs. This allows for quick creation and management directly from your terminal.

They’ve also implemented full did:web support, embracing decentralized identifier standards for enhanced security and interoperability across the web.

You can poke under the covers of the shortened URL (above) by navigating to my top-level identity and looking for lol.atmo.link.

Tap on that and you’ll see drop-263. This is the full URL for that: https://pdsls.dev/at://did:plc:hgyzg2hn6zxpqokmp5c2xrdo/lol.atmo.link/drop-263, and this is the JSON record:

{
  "url": "https://dailydrop.hrbrmstr.dev/2025/03/18/drop-623-2025-03-18-typographywresistance-tuesday/",
  "$type": "lol.atmo.link"
}

This is a great example of a small, focused app. Unfortunately, short URLs are evil things, so — perhaps — use this code to riff from vs actually use short URLs.

FIN

Remember, you can follow and interact with the full text of The Daily Drop’s free posts on:

• 🐘 Mastodon via @dailydrop.hrbrmstr.dev@dailydrop.hrbrmstr.dev
• 🦋 Bluesky via https://bsky.app/profile/dailydrop.hrbrmstr.dev.web.brid.gy

Also, refer to:

• this post, and
• this post

to see how to access a regularly updated database of all the Drops with extracted links, and full-text search capability. ☮️