The Machine is bleeding, or maybe we are. The sum of human knowledge and culture has been scraped, ingested, and sprayed across the internet. To borrow a Vervaekeism, pay attention to the words I’m using. Words are intentional, multi-valent, the result of a mind struggling to reveal itself.

The “tokens” an LLM outputs are none of these things. Although it hijacks “language” and bends it to its uses, the LLM cannot produce words, only “content” — pure, decontextualized, decompositional liquid. In other words, “slop”.

Slopitechture

Post-industrial sludge is not a good building material. Sure, you might be able to squirt it in the joints and empty spaces to fill up space or glue surfaces together. You might even be able, with sufficient discipline, to create some kind of cellular superstructure to hold the slop in with. But slop software has as much in common with software craftsmanship as a 3D printed building has in common with architecture.

In both cases, we countenance the perversion of our discipline in the name of “efficiency”. If it reduces labor and material costs, how can we argue? While you might not want to live in a goop house yourself, at least it benefits the poor who couldn’t otherwise afford one. Stained glass, wooden beams, and brick fireplaces are relics of a time gone by. We may feel nostalgic for the past, but we must leave it behind in the name of Progress.

But do we know what we’re giving up? The gradual phasing out of wood-burning stoves means people can no longer heat their own homes. The use of industrial materials in construction means people can no longer build or repair their own houses. If I must depend on industrial inputs to keep the machine that is my home running, am I really free, or have I been “domesticated” in a new sense?

Slopware Development

Slop coding is the same kind of transition, from beauty, purpose, meaning, and elegance, to “output". It is the displacement of human skill, ingenuity, and attention to detail by the brute force volumetric deposition of gunk.

This is the tyranny of the average. Slop code is by definition, exactly as good as what a moderately competent developer can produce in his specific area of expertise. It will be far “better” in any given circumstance than code produced by novice programmers, programmers working in a different industry or language, or the non-programmer.

To that I say: wonderful. I’m not an elitist. If a shop owner, analyst, or mom can whack together custom software to solve their particular problem, great. I’ll even use it too: to work in a language or domain I’m not personally familiar with, stand up a prototype, write a quick utility.

But do we know what we’re giving up when we apply this style of programming? The proliferation of average is the sacrifice of the weird. Typescript displaces Clojure. Idiosyncratic coding styles are swallowed under a wave of monotone. Deep understanding of a problem is replaced by a rat’s nest of exponential backoff, try/catch, and convergent interfaces. Minds skim across the surface of understanding like rocks on a lake. Programming pearls are cast before swine.

If you’re a professional programmer, try to go a week without using an LLM and you’ll notice just how much you rely on it. Admit it, you don’t read documentation anymore, do you? When was the last time you actually looked at a page of search results instead of just asking Kagi Assistant or Perplexity for the answer? You mean I have to use my hands to type? That’s like a baby’s toy!

This isn’t just about the atrophy of skills through disuse, although it is that. It’s about our newfound ability to just not bother. Why bother read documentation for half an hour when the LLM already knows how to use the tool? Why bother reviewing the code, when it’s probably good enough, and can be re-generated if not. Why bother designing anything when average is good enough?

But if we don’t bother, we don’t care, and no fire hose of manure can build a house when the architect is asleep in his office.

Upslopping

Call this up-skilling if you like. Tell me all about your polecats and how if you yell at them with just the right tinge of paternal disappointment they won’t light your factory on fire or delete your hard drive. Tell me about how if I just pay Anthropic $1,000 every week (you don’t really think your “pro” subscription is still going to be there after the bubble pops, do you?) I’ll be able to do the work of ten sad, unfulfilled developers instead of only one.

I can’t help but feel that all this hype is just cope for people who like to be dominated by their own tools. LLMs are suitable to particular types of tasks, for which average and fast are unqualified endorsements. The kind of task which was already easy, but which is now easier. Pounding out reams of boilerplate with a single prompt is an exhilarating experience, but it’s still just reams of boilerplate.

When I look at the hype surrounding LLM-assisted coding, I don’t see people writing ground-breaking new products. I see clones of clones: derivative, unoriginal, average work. But the hard parts are still there. So you cloned Slack — now execute on a go-to-market strategy. We’ve always felt the draw to work on easy tasks and busywork instead of the important stuff. LLMs juice that impulse, making it almost impossible to resist. I can build 10 complex projects in a month! Who cares if no one uses them and I have no intention of marketing, supporting, or maintaining them?

And at the end of the day, nothing can replace code except for code. We seem to have all forgotten that code is not written for machines, it’s written for humans. Code is language — a medium of communication intended to be shared between people. This is why we have higher-level languages — because it’s a waste of your short life to sit staring at the screen of your hex editor.

This is the same argument AI evangelists will use in favor of LLM-generated code. The claim is that the “prompt” is a new kind of higher-level programming language which frees us from the drudgery of syntax. Anyone who thinks otherwise is a technophobic luddite.

But this just doesn’t hold up under scrutiny. Constraints are not limitations, they are affordances which focus the attention and intention of the builder. Whatever problems programming language design causes through the “foolish consistency” of “little minds”, it more than makes up for by establishing conventions that structure our thought. This is why programmers learn new languages — to learn to think in new ways.

The idea of “prompt as source code” sounds nice. But think for two seconds about this. Your compiler is a massively expensive thinking machine in a data center somewhere. Are you really going to spend $100 every time you want to update your specification so you can rebuild your app? Are you really going to tolerate the massive variability in output that will occur when the LLM one-shots your project from scratch over and over?

Even if we solve these problems (and we will, for example by asking the agent to do a comprehensive diff between the prompt and the code in order to make incremental, directed changes) a specification is still a specification — i.e., code.

It’s easy for programmers to underestimate just how much source code matters. Syntax is not mere ceremony; boilerplate is meaningful. We get so used to our coding conventions that they become invisible to us. Maybe some of this implicit information can be captured in a prompt, or replicated by an LLM. But for projects that aspire to maturity, the developer will always need a way to reach underneath the level of the conceptual to specify how as well as what. The failure of purist declarative programming is evidence enough of this.

For me at least, getting my hands dirty also helps me to think. If I don’t look at code and think about how it fits together, I’m never quite able to comprehend the problem I’m addressing, or what the proper solution is. I know that this is less of a thing for people with management or engineering experience, but I don’t think it can be fully discounted. The fact is, the farther you are from the code, the farther you are from the code.

Anyway what would it look like to bring this vision of LLM-as-compiler to fruition? Well, a new programming language, of course. But this time it’s better than before, because it’s slower, more expensive, non-deterministic, and doesn’t run on your own hardware! Sounds like a winning combination to me.

Sloponomics in One Lesson

Notice what I’m not saying. I’m not saying LLMs are useless, that they are a fad, or that I won’t use them. They are an almost ideal tool for non-experts (and all of us are non-experts in most areas). What I am saying is that their use has a cost. That there are trade-offs to technical choices. That many of these trade-offs are invisible, and only show up at scale, and over time.

It’s pretty universally agreed upon at this point that we are in the midst of an AI bubble. The disruptive model of Silicon Valley has itself been disrupted, and now every tech company on earth is in a mad dash to realize the promises of the machine mind. This means that we are in a period of growth, not consolidation. In order to gain market share, everyone is moving as quickly as possible to give away as much as possible. Think about where we will be when OpenAI, or Anthropic, or Google, or Microsoft captures a decisive share of the market and consumers lose the ability to choose between providers. Someone has to pay for all those data centers.

I don’t think LLMs are going to get prohibitively expensive, pricing out these companies’ current customers. But I do think there will be an effort to cut costs by switching work loads to cheaper models, resulting in the gradual enshittification of LLM tools along with everything else in our declining dollar economy. For those of us who hand-pick our models or run them ourselves, well, we’ll pay for what we get. For the rest, it doesn’t take much imagination to anticipate how LLM providers are going to abuse their ability to intermediate our conversations with the bots, from injecting content directly into conversations or software products, to massaging our chat histories into the highest-resolution advertising (and propaganda) profile ever witnessed.

The bigger point here though is not an economic one. It’s that whether in terms of the economy, our skills, or our culture, there’s no going back. Our decisions have consequences; the technologies we create and adopt have an effect, not just in terms of getting done the work we aim them at, but also in terms of unintended second-order consequences. People and tools exist together in a complex, ecological system. We may put our oar in to try to influence the development of this ecosystem, but the technological millieu influences us comprehensively. The technological world creates the circumstances upon which our very existence as technological man is predicated. A shift as large as the one we are undergoing now will not be inconsequential.

There are unique risks involved in this particular technological shift too. The possibilities of model collapse mean that the returns on efficiency of LLMs may be diminishing, or even reversing, as the machines are fed with their own output. The atrophy or abandonment of human skills plays into this too, because if organic activity is displaced by artificial, then the amount of new content on which LLMs can be trained is rapidly declining.

Already, StackOverflow’s traffic has declined to 10% of where it was only 5 years ago. TailwindCSS recently laid off 75% of their team because no one is finding out about their premium offering from their documentation any more. There’s a chance that we are entering a dark age of innovation in software. If LLMs are only able to regurgitate reconstituted ideas from the past, they can’t be considered a source of new ideas.

One counter-argument of course is that LLMs make it easy to learn new subjects, increasing the spread of knowledge. I think this argument is over-stated. LLMs can help someone get to basic competency quickly, but cannot replace deep study. Innovation still has to come from a deep well of experience, which is comprised not just of knowledge, but also of pain, frustration, and nights spent awake imagining possible futures.

Another counter-argument is that LLMs will allow disciplined programmers to curate their attention, directing it toward solving interesting and hard problems. This may be the case in some resource-scarce environments like start-ups, but in most cases the gains will be marginal. Chores are rarely a real excuse, and can themselves be a source of inspiration; under-performance is more often attributable to complacency, incentives, and sources of external friction like corporate bureaucracies than mere drudgery. And for the occupational “innovators” in academia, well, they’re already pursuing the cutting edge in their micro-niche with their full attention.

The Technological Slopciety

A helpful paradigm for looking at the problems of tools and technology which I’m currently obsessed with is called “conviviality”, coined by Ivan Illich in his eponymous book. Here’s how he defines the concept:

Tools foster conviviality to the extent to which they can be easily used, by anybody, as often or as seldom as desired, for the accomplishment of a purpose chosen by the user. The goal of such a tool is to serve a society “in which modern technologies serve politically interrelated individuals rather than managers”.

In other words — freedom.

At first glance, LLMs seem like an ideally convivial tool. Even I am excited for the possibility that laymen may be able to build better tools for themselves than any vendor might be able to offer. But there’s one key point in that definition that does not apply to a slop society: we cannot use LLMs as seldom as we might desire.

As work and communication become increasingly LLM-shaped, both through the direct adoption of LLM tools and through the consequent reshaping of public spaces to accommodate them, society itself will become less human-shaped, to the point at which even those of us who have carefully maintained our skills against atrophy will no longer have an outlet for them. Just as agricultural implements have been adapted for use by tractors rather than by farmers, so our information landscape will adapt for the alien mind of the machine, rather than for our wet eyeballs and trembling tympanic membranes.

This is already happening — the pull request that the TailwindCSS team rejected was for the addition of text-only documentation aimed explicitly at LLMs. How long until people simply stop writing documentation for humans entirely? Might we eventually learn how to speak the LLMs' native language, and write our documentation using that? Humans may eventually lose the ability to even read without an LLM.

The trajectory of the LLM society is decisively anti-convivial. More quickly than ever, we are reaching the point in the adoption of a new technology which Illich describes as its “negative” range:

When an enterprise grows beyond a certain point on this scale, it first frustrates the end for which it was originally designed, and then rapidly becomes a threat to society itself. These scales must be identified and the parameters of human endeavors within which human life remains viable must be explored.

This is tyranny of a kind that does not come from governments, but from technology itself. What Illich calls “radical monopoly”:

Radical monopoly exists where a major tool rules out natural competence. Radical monopoly imposes compulsory consumption and thereby restricts personal autonomy. It constitutes a special kind of social control because it is enforced by means of the imposed consumption of a standard product that only large institutions can provide.

If we have any desire for freedom and independence in an increasingly digital society, we have to find a way to protect ourselves from this radical monopoly. Self-hosting our own LLMs will not suffice, because even they will contribute to the reshaping of society and our exclusion from it.

Rage Against The Slop

In the face of a globally advancing tsunami of machine turds, I can only offer sheer bloody-minded curmudgeonliness, like someone who pays all their expenses in cash in an effort to stave off the advance of the global financial surveillance panopticon. That is the knife that we need to bring to this gunfight.

It’s impossible to be anti-LLM, just like it is impossible to be anti-tree. The existence of trees is just a fact, and so is the existence of LLMs. Instead of thinking about LLMs on their own terms — derascinated, platonic, monadic — we should instead think in terms of instances. One cannot abolish all trees, but you can certainly cut one down, or plant one.

An individual tree differs from the idea of a tree in almost every way. The idea of a tree is determined, abstract, normative. The tree in my front yard is constantly changing, is concrete, contextual. It was a rotten elm, it is now a rotting stump.

The challenge for us in the age of the LLMs is to not give ourselves up to the apparent determinism of the machine. Stop thinking about the abstract promised or dreaded future, open your eyes and look at the LLM sitting in front of you on your desk or in your hand. Is this a tree you want to water, to fertilize, to see grow? Does it need some pruning in order to bear fruit in your life? Or does it need to be cut to the ground.

Every prompt not written, every agent not installed, every token not purchased is a decision, a vote for who you want to be, and the home you want to live in. Resistance against the machine is not a simple matter of bugging out to the back country, but of creative subversion, malicious compliance, and dynamic equilibrium. It is the design and use of new tools and technologies which mediate the old ones. And it is the refusal to give up your soul to that of the machine.

A question I have been asking myself for several years now is this: how can the internet, a medium which thrives on materialistic reductionism, context collapse, attention harvesting, and censorship, be reformed so that its power can be leveraged in support of human flourishing instead? In this talk, I hope to present a way to think about engaging with digital communication technologies in a way that safeguards our freedom from the tendency of both governments and corporations in a digital environment to intermediate, manipulate, and extract value from our communication.

My thesis here is quite narrow: there are a few particular digital tools which can be applied in Ivan Illich's terms of "conviviality" to reform the internet in service of human flourishing.

Before I explain what that means though, there are a few general points I think it would be helpful to address.

Let me start by defining my end goal of "human flourishing". This can mean many things to many people, but my inspiration comes from the English folk hero Ned Ludd, who (according to legend) smashed two knitting frames in a fit of passion in 1779. The term "Luddite" is often used as a derogatory term for simple technophobes, but the reality is that the Luddites were not against technology as such, but instead advocated for the use of technology in such a way as to reinforce what Craig Calhoun calls the "moral economy", "a system built around community bonds, local economics and human-scale systems". (Kingsnorth, 280) Human flourishing is the balance of freedom and responsibility in a context of relational belonging.

I also want to give a brief summary of Marshall McLuhan's "media ecology" to frame my argument. It's common to think of tools as moral to the extent that they are used for a particular purpose. But tools are not simply a blank canvas for human intention; rather, tools are designed for a particular purpose, and do not exist in isolation. When a new tool is created, certain values are embedded into its shape, which in turn interact with the pre-existing technological millieu. This complex combination of different tools and the people using them creates an environment in which the use of a given tool is mediated by another.

When evaluating the use of a given tool, we have to keep in mind not the instrumental use of the tool to achive certain ends only, but also the formal way in which that tool - in combination with every other relevant tool or intention - formally shapes us. The ultimate end of a given tool is not the achievement of a particular task, but the modification of the technological environment in which we exist, and which inevitably affects our ability to understand the world, act in it, and assign value.

The adoption of a certain type of tool therefore always creates a certain type of person as a result. This relationship is not always straightforward, and so can be hard to see. But it should be obvious that modern technology has had its part in creating modern man - a materialist, who cares only about quantifiable goods, who is unable to discern the value of good work done well, and who is unable to preserve traditional values and beliefs.

But this is not a one-sided transformation: technique may itself be deployed in a way that cultivates a different kind of person. This is incredibly difficult because of technique's ability to absorb and translate any criticism leveled against it into its own terms - whether as satire, or entertainment - but even Jacques Ellul admits, "if we do not even consider the possibility of making a stand against these determinanants, then everything will happen as I have described it, and the determinants will be transformed into inevitabilities."

So, what can we do to recover the moral economy in the face of technological progress? Amid the doomerism I have found one ray of hope: Ivan Illich's book Tools for Conviviality. In it, he defines conviviality as "the opposite of industrial productivity", and as "individual freedom realized in personal interdependence". (Illich, 11)

For Illich, the "moral economy" is not merely an ideal irretrievably lost to the past, but one which can be recovered in part through judicious application of technology. He states, "Tools foster conviviality to the extent to which they can be easily used, by anybody, as often or as seldom as desired, for the accomplishment of a purpose chosen by the user". (Illich, 22)

With this definition in mind, we can look again at our technological society and see many places where conviviality remains the norm: in mechanics' shops, farmers' markets, forestry, woodworking, fabrication, even certain types of information technology. We can also discern that the use of convivial tools produces a different kind of person from the office worker, burger flipper, or denizen of the assembly line - they tend to be more resourceful, more resilient, more comfortable with risk and better able to manage it, and most relevantly, masters of their tools rather than mastered by them.

The logic of the machine

Before we can propose an agenda of reform by identifying convivial tools in a digital context, it would be good to know first where we stand as regards the problem of our industrial and digital environment.

Jacques Ellul is the authority on this question, and so I will simply borrow his definition of "technique": it "is the translation into action of man's concern to master things by means of reason, to account for what is subconscious, make quantitative what is qualitative, make clear and precise the outlines of nature, take hold of chaos and put order into it".

Technique is a holistic, self-perpetuating paradigm that seeks to collapse all meaning into bare efficiency. Technique is everything central planning wishes it were. It is the governing spirit of an apparently decentralized economy, which converts independent actors to its own values so that they can participate in its quest to optimize away friction in industrial processes for maximum output.

This orientation toward efficiency naturally results in two characteristics that combined describe much of the dysfunction of our time: scale and centralization.

Scale is the growth of an enterprise (or a government) to expand both vertically and horizontally. "Vertical integration" allows for maximally efficient mapping of inputs to outputs, while horizontal growth allows for the suppression of competition and the creation of synergies between differerent types of products. The result of scale is centralization, in which every process is controlled either directly or indirectly by some few powerful actors.

But scale has its own problems. Changing the scale at which an activity occurs results in a qualitative difference in what is actually happening, which introduces complex consequences, fragility, and bureaucratic waste. Illich explains:

To each social environment there corresponds a set of natural scales. This is true for the primary group, for the production unit, for the city, the state, and the organization of men on the globe. To each of these social environments there correspond certain characteristic distances, periods, populations, energy sources, and energy sinks. In each of these dimensions tools that require time periods or spaces or energies much beyond the order of corresponding natural scales are dysfunctional.
He continues:

There is a form of malfunction in which growth does not yet tend toward the destruction of life, yet renders a tool antagonistic to its specific aims. Tools, in other words, have an optimal, a tolerable, and a negative range.
Changing the scale at which a tool operates can cause it to cross from one of these ranges into another, first negating its benefits, then reversing them.

Illich uses the example of the automobile, which at its "optimal" scale simply gave people the ability to travel farther, in less time. But as infrastructure was built up around the car, we arrived at the "tolerable" scale - a point at which people came to be obligated to travel in order to reach the same kind of destination they once could achieve on foot or at home.

As society crystallized around this scale, people ended up spending more time spent traveling, at a greater cost. Worst of all, we cannot now return to localism, because neither the local community nor the local economy exist any longer. This is the "negative" range.

The result is what Illich calls "radical monopoly". Radical monopoly is a monopoly not just of an industry by a single brand, but of a way of life by a single industry. Illich elaborates:

Radical monopoly exists where a major tool rules out natural competence. Radical monopoly imposes compulsory consumption and thereby restricts personal autonomy. It constitutes a special kind of social control because it is enforced by means of the imposed consumption of a standard product that only large institutions can provide.
Illich cites many other examples of radical monopoly in his book, including medicine, education, undertakers, and law. To that list, I would add the internet.

As an extension of industrial technology, digital technology shares its orientation towards efficiency, but more so. Liberated from the constraints of the physical world, digital technology need not concern itself with material science or structural engineering. Its strength is instead in proliferating, broadening, and amplifying channels for information transfer.

Because it is untethered from all but the most abstract physical limitations, the marginal cost of software naturally approaches zero. This makes for a unique competitive landscape in which the only way to win is to charge nothing. And because inventory costs nothing (at least, in comparison with the scale of the enterprise), internet businesses can scale up indefinitely.

This makes monopoly easier to achieve, and more important to survival than ever before. The problem is the revenue model - if the product is given away for free, how can the business make money?

This problem is frequently solved by lowering prices in order to achieve monopoly, then raising prices once the market has no alternative to turn to. This was the playbook of Uber and AirBnB, companies which create a market, inject themselves into its private transactions, then siphon off revenue in exchange for provided efficiency.

In some cases though, businesses don't have a way to charge fees for their service directly, so they resort instead to extracting value in terms of what Nicholas Carr calls the "hyperreality" - an informational abstraction over reality that displaces reality itself. In it, people are monetized by being reduced to "profiles": abstracted, quantified versions of real people, which are valuable to the extent that they produce two important commodities: engagement, and data.

In this "information economy", the traditional vendor/customer relationship is transformed into one in which software vendors take on the role of brokers who mediate parties in economic transactions or social interactions. This gives them the ability to harvest users' attention and data for whatever use the data brokers' customers might have.

When confronted with the level of access digital platforms have into our lives, it's easy to dismiss the threat because we "have nothing to hide". But it is not the data of an individual that is really valuable, it is the data attributable to a "profile" - a demographic, which can then be targeted with advertisements, social experiments, and political propaganda. This is a tragedy of the commons, in which the complacency of the individual about digital privacy fuels the machine of surveillance captialism.

The data broker business model is anti-convivial. It introduces into economic transactions or social interactions a misaligned third party which intermediates "personal interdependence", subordinating it to industrial productivity. These tools cannot be used for private purposes. And of course, it only gets worse at scale.

In this talk I'm mostly focusing on businesses, but the same can be said of governments as well, either in partnership with businesses, or on their own. Central bank digital currencies are in vogue among developed countries as an effective way to surveil citizens for the purpose of coercing them to behave in a certain way. Corporations' incentive structures not only affect legislation through lobbying, but can also form a bridge for authoritarian policies to cross borders: if a transnational corporation complies in one area, it reduces the barriers in place for compliance in another.

It is clear to me that digital communication technology has exceeded its "optimal" scale by colonizing the internet, and establishing a radical monopoly over communication as a whole. The shape of digital communication as it stands today no longer serves the interests of the people who use it. Rather, just as in an industrial economy people are reduced to "consumers", in a digital economy people are reduced to "sessions", "views", and "clicks". To the extent that we inhabit this digital environment, we are quantified, digested, abstracted, and instrumentalized as fungible grist for ends of the machine.

This was not always the case, and there remain pockets of fun and freedom on the internet that have held out against the advance of surveillance capitalism simply by virtue of being small. With the advent of LLMs, however, even these are quickly disappearing, as their contents are scraped, digested, and used to fuel the chatbots. This, combined with the proliferation of AI-generated "slop" content has caused many people to retreat to "private" digital spaces, known as the "cozyweb". By and large, these places are not immune to the intermediation of the ubiquitous "platform", but they do at least serve as a refuge for people wishing to communicate with a particular, scale-bounded selection of real people.

This revealed preference for privacy, familiarity, and natural scales should be encouraging: people recognize the dysfunctionality of the "open" internet, and want to scale down their online presence. However, while this impulse is healthy, the average internet user isn't equipped to follow this impulse very far.

Digital conviviality

Whether you are equally pessimistic about the purported benefits of the internet doesn't really matter. The reality is that every one of us already lives in its digital environment to some extent. For all of us, it is imperative that we find a way to be "in" the internet, but not "of" it.

This is where digital tools for conviviality come in. Whereas most tools are oriented at making new activities possible or existing activities more efficient, we need a different kind of tool: the kind that says "no". The danger of the digital environment is that it will abridge distances and dissolve distinctions. The tools needed to carve out a habitable space within such a hostile environment will differ greatly from the tools we are used to thinking of as useful.

Here are a few of the attributes of digital communication as it exists that I would like to be able to say "no" to:

• The ability for platforms to intermediate my private communication
• The ability for platforms to "lock in" my usage - thereby monopolizing my attention
• The tendency of digital communication to optimize for my engagement over value provided to me
• The tendency of digital communication to de-contextualize communication, transforming it into "content"

Here are the corresponding "yes"es which digital convivial tools should allow us to say:

• I want my private communication to be only between me and the people it is intended for
• I want to switch platforms without losing my data or the relationships it facilitates (known as credible exit)
• I want to receive real value in exchange for money, not content in exchange for attention and data
• I want my digital communications to enrich, not detract from, real relationships

There are a myriad of tools which support some combination of these values in practical, directed ways: adblockers and privacy browsers reclaim our attention; VPNs protect our privacy against service providers and ISPs; bitcoin's digital scarcity can defend us against capricious monetary policy; various proxy services can obscure our physical addresses, credit card numbers, and more; certain services exist which retroactively clean up our digital footprint.

But I want to focus in particular on two techniques which are fundamental in supporting digital conviviality: open source software, and asymmetric encryption. These tools fit Illich's definition of "conviviality" in that they can be used voluntarily for private ends, but they also go beyond mere conviviality in that they have the potential to subvert the architecture of the corporate internet into one more systematically conducive to individual freedom.

Open source software

Open source software is software (or software protocols - standards that allow multiple programs to talk to each other) that is legally available for anyone to read, use, and modify. All internet standards are necessarily open in some sense, which is why the internet is described as an "open" protocol. Other examples include linux, Blender, VLC, and Firefox.

Openness is a double-edged sword. As Paul Kingsnorth puts it, "'Openness' is both the aim and the core value of the age of globalism". He elaborates:

Open is good, closed is bad. Why? Because closed things can't be harvested, exploited, or transformed in the image of the new world which the machine is building. 'Open' things, on the other hand; well, they're easy prey.
This pattern applies to software as well. A tactic used by large software companies to destroy open source competition is known as "embrace, extend, extinguish", in which a large company adopts a project, dedicates massive resources to developing it, then breaks compatibility, leaving their version of the project as the only viable option for users.

At the same time, big companies execute this kind of attack for a reason. Even though openness results in vulnerability, it also empowers its users in ways that proprietary software doesn't. The ability to understand, fix, modify, and compose software projects is a super power - we need look no further than companies like Zapier that exist solely to help people glue different software services together.

Openness is also how God made the world. Most people are used to thinking of technology in terms of power that can be used to achieve political ends. And yet God "makes His sun rise on the evil and on the good, and sends rain on the just and on the unjust" (Matthew 5:45). Digital tools can be misused, just like the affordances God himself put into creation. But open-source software creates an opportunity for individuals to learn responsibility and competency, in turn transforming its users one interaction at a time into the kind of people who have mastery over their tools, rather than being mastered by them.

Convivial tools as a whole are in fact "open" by definition. Imagine if when your car battery died you had to buy a whole new car, either because you weren't legally allowed to open the hood, or because the car was designed in such a way as to obscure how it actually worked. Every tool usable by a non-expert shares this in common — it is intelligible. Software is no different.

The idea of open source frequently leads to the idea that you have to read and understand the source code of every program you run in order to do it "right". But this verification can be mediated in a number of ways: reputation is hard to build and easy to destroy; software vendors might choose to align their business model with their users to gain trust; technical friends can be relied on to give reasonable recommendations. It's also possible through the magic of LLMs for non-technical users to create, modify, and evaluate open source software themselves (although I can't claim that LLMs are themselves convivial tools).

Choosing to use open source software (and supporting the developers who build it) is an investment in tools that empower their users rather than extracting value from them. In terms of the four values I mentioned earlier, open source software provides assurance that user privacy is respected; it will never lock you in to a proprietary data format or platform; and it is paid for on a voluntary basis, which means any revenue the developer makes is directly correlated with value received.

To give a concrete example, I own a jailbroken Google Pixel phone, which I bought on eBay instead of buying through my phone carrier. This allows me to run an Android fork called GrapheneOS instead of stock android, which sandboxes all the Google processes and offers ad- and surveillance-free alternatives for many of them, as well as allowing me to remove all the bloatware. I use several alternative app stores like Obtainium, F-Droid, and ZapStore, which promote free and open source apps published directly by the developer, rather than apps that use advertising or surveillance-type business models published with Google's blessing.

Doing this sounds daunting, and it does take some doing, but in the context of the progressive degredation of corporate solutions, the end result is refreshingly clean. And anyway, ease of use can't be our primary goal if we wish to become competent and responsible tool users.

Asymmetric cryptography

Running open source software isn't always possible though. No matter how principled someone is about using open source software on their own devices, they'll still be compelled to use resources provided by unaligned third parties if they want to take advantage of digital communication networks.

This is where my favorite digital tool for conviviality comes in: asymmetric cryptography.

Even in terms of computing, this technique is relatively new. Discovered in 1976 by Whit Diffie and Martin Hellman, it revolutionized the field of cryptography, which until then was exclusively "symmetric" - that is, the same key (a secret number used to convert a message to enciphered text and back) was used to both encode and decode a message.

This is how every cipher has worked for thousands of years - from the Caesar Cipher to the Nazis' Enigma machine. Asymmetric cryptography made it possible for the first time to send an encrypted message to someone without first communicating the key, even if the attacker was listening to the entire exchange.

If the endpoints of a communication channel are secure, encryption makes it possible to use untrusted infrastructure to create a secure bridge between individuals, regardless of how secure the intervening networks are. Because those networks are unlikely to be under the control of the people using them, encryption can be thought of as converting anti-convivial systems into convivial ones.

Asymmetric cryptography also makes possible a concept known as "digital signatures". In contrast to encryption, which reduces the amount of information shared by users, digital signatures instead add additional information to communications - namely, proof that a given message came from a particular person.

This doesn't sound very useful, but it is actually vital for supporting individuals' rights to "credible exit". If a service that stores information is also relied upon to authenticate it (in other words to prove that it was published by a particular person), that information is not portable. As a result, people that rely on access to that information are stuck on that platform.

But if we know the cryptographic identity of the person who published a particular piece of information, we can rely on its digital signature to validate its authorship, regardless of how we get ahold of it.

This matters, because it is social platforms' hold on user-generated content (tweets, emails, blog posts, podcasts, music), which gives them the ability to exploit their users' attention and data without accountability. But if I can use third-party software to either publish or access that content, the platform is demoted from a party with a stake in my activity to a mere hosting service which can be discarded in favor of another.

Unfortunately, in practice asymmetric cryptography has generally been used to protect communications between corporations and governments rather than between individuals. This is a result of its history as a way to secure financial transactions on the internet, which normalized its capture by a hierarchical bureaucracy of certificate issuers.

Even in cases where encryption is commonly used to secure communications between individuals, it usually comes with some important caveats. Any system controlled by a third party can be changed at any time to introduce backdoors, and in practice many "end to end encrypted" systems, like those provided by X, WhatsApp, and Telegram are not designed to protect the user from the service provider itself.

There are two main reasons for this. First, it's simply not in the interest of tech platforms to fully relinquish control over their users' content, in large part due to legal risk. Second, users themselves are accustomed to convenience, and the hardness of cryptography implies a significant trade-off in this area.

There are systems that attempt to give users the benefits of raw encryption, mostly notably PGP by Phil Zimmerman, but they have never reached widespread adoption for these same reasons. In the last few years though, as applied cryptography has matured through the growth (and speculative crash) of numerous cryptocurrencies, encryption and the alternative networking architectures that facilitate its use have been getting more attention.

Digital signatures are also finally getting the attention they deserve as protocols like scuttlebutt, nostr, and many others encourage users to take direct control of their cryptographic identities rather than delegating their management to service providers. In the long run, this technology has the potential to rewire the internet itself so that platforms are forced to be accountable to and aligned with their users.

Conclusion

These are only a few examples of digital tools for conviviality, and there remains significant uncertainty regarding their adoption and potential subversion, particularly by governments hoping to implement authoritarian policies using the internet as a hook. For example, digital signatures rely on cryptographic identities, which is not so far off from the dystopian possibilities of social credit scores. These risks have to be taken seriously if we are to adopt these tools, especially where large scale adoption makes them impossible to opt out of.

Convivial tools are a bottom-up approach to incrementally regaining control over and responsibility for our own lives and communities. They are not a panacea, or a comprehensive system, or a revolution, and that is exactly the point. They make small-scale resistance against the machine just slightly more feasible. But they exist within a dynamic equilibrium, in which reform and capture, centralization and decentralization are in a constant struggle.

To close, I'd like to leave you with a quote from the 1993 Cypherpunk Manifesto, which embodies many of the ideals I've been advocating for here.

We cannot expect governments, corporations, or other large, faceless organizations to grant us privacy out of their beneficence. It is to their advantage to speak of us, and we should expect that they will speak. To try to prevent their speech is to fight against the realities of information. Information does not just want to be free, it longs to be free. Information expands to fill the available storage space. Information is Rumor's younger, stronger cousin; Information is fleeter of foot, has more eyes, knows more, and understands less than Rumor.

We must defend our own privacy if we expect to have any. We must come together and create systems which allow anonymous transactions to take place. People have been defending their own privacy for centuries with whispers, darkness, envelopes, closed doors, secret handshakes, and couriers. The technologies of the past did not allow for strong privacy, but electronic technologies do.

We the Cypherpunks are dedicated to building anonymous systems. We are defending our privacy with cryptography, with anonymous mail forwarding systems, with digital signatures, and with electronic money.

In closed systems, rules are set in stone, and can't be changed by anyone except the owners of the system. In open systems, rules are established based exclusively on what Steven Johnson calls the "physics" of the system - the internal organizing principles that distinguish the system from other systems, and which give it meaning and utility.

This latter type of rule also exists in all but the most rigid systems, but are counter-balanced by another type of rule: rules can be changed capriciously, and enforced arbitrarily. When participating in a system run by someone else, it's impossible to predict from one moment to the next what the consequences of your actions are. This is why fiat currency creates so much chaos, and why civil law necessarily suppresses the rights and responsibilities recognized by common law.

But I want to draw a parallel here to another type of system: games.

Games fall into one of our two categories of sytems: games which are a mutual fiction shared between two players (an open system), or a game which requires a platform in order to play (most video games).

An example of the former is that of chess. The rules are well-known and inflexible, leaving very little room for arbitrary enforcement. And yet, there is infinite room for creativity, both within the boundaries of the game, and through the modification of the rules to create something entirely new.

The voluntary modification of the rules of a game is one of the affordances that exists within an open system. Changing the rules changes the game into something somewhat but not entirely different. When is chess no longer chess? When it's 3D? 4D? 5D? Every compounding dimension will still bear some resemblance to the normative version of the game.

This modification of rules by the players of the game is distinct from arbitrary modification of rules by the "owners" of a game. Take Runescape for an example: Jagex is well-known for selective enforcement of additional rules, and modification of game mechanics in order to engineer a more perfect environment for players (and their business model).

And yet no matter how heavy-handed a system administrator's oversight might be, there are always affordances that users can take advantage of.

In "Everything Bad Is Good For You", Steven Johnson identifies the behavior of discovering and exploiting these affordances as "probing":

Probing often takes the form of seeking out the limits of the simulation, the points at which the illusion of reality breaks down, and you can sense that's all just a bunch of algorithms behind the curtain.

As a paradigmatic example, he explains that PacMan

had its rules, which were so simple you could express them in three sentences... But experienced PacMan players soon discovered that the monsters roamed the maze in predictable ways, and if you followed a certain course—literally called a "pattern"—you'd complete the level without losing a man every time you played. Patterns weren't build into the official rules of the game; they were a legacy effect of the limited computational power of those arcade machines, and the predictable way in which the monsters' behavior had been programmed. To detect those limitations, you had to probe the PacMan game by playing it hundreds of times, experimenting with different strategies until one sequence revealed itself.

The abstraction of PacMan is different from its implementation. The idea of the game was simple, but the implementation had enough incidental complexity that was accessible in subtle ways to the user that they were able to exploit it in creative ways.

Another great example of this is the many ways to run DOOM, including on a pregnancy test. The designers of the pregnancy test did not intend for this to be possible, and yet here we are.

Nostr, like all systems, is also a game. It has certain rules that are more or less set in stone (using the secp256k1 curve for example), but which can be remixed to create new, subtly different sytems. Within the boundaries of nostr (however you might choose to define those), the ways its various features can be exploited are endless.

In some ways, this is less fun than attempting to circumvent rules put in place by omnipotent administrators. At the same time, we have endless affordances which are possible by the reconfiguration and remixing of different features.

Want to efficiently sync events between relays? Invent a sync protocol. Want to go back to trusting servers and get deniability? Strip event signatures. Want to manage relays? Add an API. Don't like kind 1 replies? Make kind 1111 "comments". Want to hijack kind 1 for your own purposes? Auto-repost your new events to kind 1. Want to crash clients? Make a relay that serves non-string tags. Want to play chess? Do it.

Every NIP is a possibility for how the basic building blocks of nostr can be reconfigured. And because nostr is an open system, anything goes. Everyone on nostr has voluntarily chosen to enter into an adversarial environment where the only rules that exist are the ones the users bring to the table. This makes all of us "hackers", by virtue of the code we write, or the software we choose to run.

You have the power to create new data types, overload old ones, add an API to your relay that no one else supports (or lobby for adoption), open a PR to your favorite client or fork it, protect your users' privacy or expose it, experiment with new UI patterns or re-implement the old ones, do good or evil, nurture the protocol or trash it.

This is not to say our choices don't have consequences. There are truly bad ideas, and attacks are possible in so far as the utility and logic of the system that has emerged so far can be subverted by users or software implementations. But there's no need to be completely self-serious. Yes, we need some apps that have actual business models, and the discipline to execute on them. But the culture of nostr is first and foremost one of experiementation in a spirit of fun and creativity.

So go play!

Likewise man himself, being part of nature, has also lost his mystery — no longer are we made in the image of God; instead, our race represents only a transitory phase between what came before and what comes next. Our value exists only within the context of a relentless materialistic progressive logic which battles entropy by greedily hoarding orderly power. But such logic has a fatal fallacy — entropy always wins.

Shintoism — the distinctively Japanese form of animism — appeals strongly to me. Their worship of nature and its multitude spirits created a culture of incredible beauty, realized in honor culture, craftsmanship, and ritualism. But this crystalline beauty proved fragile when Matthew Perry arrived in Japan in 1853. Since then (and particularly since WWII) Western materialistic and consumerist values have infected Japanese animism, causing its fruit to wither on the branch. Centuries of tradition have been shed, leaving the Japanese without much hope for the future.

In The Society of the Spectacle, Debord argues that the "spectacle" (also described as "hyperreality" by Nicholas Carr) is a form of non-living which colonizes life itself, evacuating it of meaning by converting direct lived experience into pure mediated imagery. For Debord, the spectacle is the culmination of a bourgeois consolidation of power, which takes on a life of its own, expanding the proletarian underclass to the whole of society in the absence of revolutionary resistance.

But the revolutionary subversion of materialism he advocates is no less fatalistic than the optimistic progressivism of materialism itself. It is not the Hegelian synthesis it claims to be; it is merely an antithesis, orienting human action towards immediate, rather than deferred entropy. It is itself the premature collapse of the system it criticizes, and is therefore essentially oriented to it.

Materialism is a disease of modernity, which kills its host, either through the ennui of hopelessness, or the auto-immune response of critical theory, which indiscriminately seeks to dissolve every structure of power, whatever its predicate. These reactions cannot kill the disease, but they will make the host sicker.

I cannot guess what the synthesis of these forces will be, but I do know one thing that neither materialism nor Marxism can touch. In the words of 1 Corinthians 13:

Love never fails. But whether there are prophecies, they will fail; whether there are tongues, they will cease; whether there is knowledge, it will vanish away. For we know in part and we prophesy in part. But when that which is perfect has come, then that which is in part will be done away.

That which is perfect has come, and will come again. Man's pride comes before his fall, and the beauty of nature may be marred by it. But the God of transcendent love, in whom nature finds both its origin and its purpose, is also the God who upholds creation, and has purchased it by his very own blood. He is not dead, he is alive. His perfect love endures forever, and it is this love which governs every current of history, teaching us the error of our ways and pointing ever more clearly to Christ, who is

the image of the invisible God, the firstborn over all creation... All things were created through Him and for Him. And He is before all things, and in Him all things consist. And He is the head of the body, the church, who is the beginning, the firstborn from the dead, that in all things He may have the preeminence. For it pleased the Father that in Him all the fullness should dwell, and by Him to reconcile all things to Himself, by Him, whether things on earth or things in heaven, having made peace through the blood of His cross.

This perfect love is the only place we can hide from the storms of life. He is the Rock, cloven to make a hiding place for everyone who feels "weary and heavy laden":

Rock of Ages, cleft for me,
Let me hide myself in Thee
...
Nothing in my hands I bring,
Simply to Thy cross I cling

Digital signatures are essential to making Nostr work. The goal of Nostr is to break down walled gardens by subverting one of their key value propositions: content authentication. Or, in other words, the ability to know that a particular person said a particular thing.

This is a challenge in the digital world because information can be copied or fabricated at will. Simply saying that someone authored a particular piece of content doesn't make it so. When you go to Twitter and you load up a tweet, you only know that tweet is real because you trust Twitter. And if someone takes a screenshot of that or copies the text and emails it to you, then you have even less assurance that what's been presented to you is authentic.

What this means is that data that is not cryptographically signed is tightly coupled to custody. The only person who can reliably attest to the authenticity of a given piece of information is the person who can trace its provenance from the author, through storage, and to your device. This is very convenient for social media platforms — reliance on unsigned data means that they are needed. There has to be a single trustworthy custodian in order for unsigned data to work. The same is true of search results on Google; you don't know that search results are any good unless Google says they are.

What signed data gives us is the ability to know that something is true without having to trust anyone. If I create a note on Nostr and use my private key to sign it, anyone can verify the signature using the hash of the event and my public key (which is attached to the event). This lets them know that the event was created by the person who has access to my private key, i.e., me.

A Nostr event can thus be sent over an untrusted communication channel without the recipient losing the ability to know that it was me who signed it. As long as they know my public key, I can email a Nostr event, I can send a Nostr event over a peer-to-peer communication or over Bluetooth or over the LAN, or I can print it up and send it by mail. No intermediary can stop me without securing a monopoly on my communication.

Publicity Technology

The business model that fuels today's social media platforms is predicated on the capture of user data for their exclusive monetization. The user has become the product. Our data is used in a focused way to create targeted advertisements, or in the aggregate to understand and anticipate user behavior.

Signed data solves only half of this problem — it actually worsens the problem to the extent that data is public and accessible to anyone who wants to analyze it for patterns. Designing digital identity also has an incredible amount of complexity involved, and must be approached with caution. From Philip Sheldrake's essay, Human identity: the number one challenge in computer science:

Put starkly, many millions of people have been excluded, persecuted, and murdered with the assistance of prior identity architectures, and no other facet of information technology smashes into the human condition in quite the same way as digital identity[...] This should give anyone involved in digital identity cause to consider the emergent (i.e. unplanned) consequences of their work.

When designing systems that make use of digital identity, it's important to work from a conception of identity not as objective, but as subjective — that is, defined not by a set of static attributes, but by the dialectical contexts and relationships the person behind the identity participates in. The former kind of identity allows others to act upon the identity; the latter allows the person who own the identity to act.

Cryptographic identity doesn't automatically make this distinction, but can be used in either way. If the goal is user empowerment, a system of identity that is crafted to protect the digital freedoms of the user must be carefully designed.

Because identity is intended to be shared in a social setting, Nostr is not really "privacy technology". Rather, Nostr is "publicity technology".

When you create an event and you send it to untrusted custodians (particularly if left unprotected by access controls or encryption) you are advertising something about yourself to the entire world. All the data included in an event and all the metadata that can be harvested by observers and middlemen points back to you.

This is suitable for Twitter-like use cases (although user privacy is a concern even in a broadcast social media context), but always has to be considered when building products on Nostr. For users, it's best to use a VPN and Tor in combination with Nostr if you're concerned about privacy. Even so, in the aggregate signed data can still be collected and used to understand both individual users and entire social clusters.

Dis-intermediating Data

With that in mind, signed data does help reduce the capture of user attention by dis-intermediating content delivery. The current business model of social media platforms is predicated on the attention users give the platform, which is maximized by designs which stimulate "engagement", the creation and consumption of digital content.

The old way of doing this was through centralized content production. A business would create content — for example, movies, magazines, or podcasts — and present it to users for their consumption. Of course, it was a lot easier to directly monetize this content because it was both high quality and protected by intellectual property laws.

On social media, content is not produced by the platform, but by users. This introduces a second side to engagement — users not only consume, but also produce content. This keeps them even more engaged, and provides even more information about them to the platform.

When content is signed, it can no longer be captured by the platform (even if it is still visible to the platform). The result is that platforms lose the ability to enforce their monopoly on user attention. As a result of signed data, user attention can be diverted to other platforms that host a copy of the data. Nostr takes this effect even further by decoupling data storage and user interaction — relays store notes, but clients mediate user interactions.

On Nostr, clients can be more aligned with users, since they can only capture user attention to the extent that their functionality is what's valuable to the user, not the data they have access to.

The ability users have on an open network to leave a platform without losing all their data or their social graph is called credible exit. This is the opposite of "vendor lock-in", which occurs when platforms make it difficult to leave them. The export features social platforms offer are nearly useless because they break all the links in your social graph. But if all your social data was signed and the social graph was open, it would be quite easy to leave.

Social media companies can still exist in a world of signed data, but they will have to offer a real value proposition to their users in order to retain them. This means that they'll be more likely to serve their users rather than extract as much value as possible from them.

Whether open source software wins out or for-profit companies start building on Nostr, signed data weakens platforms' hold on their users and realigns the interests of social media platforms with those of their users. And while I think there's still room for skepticism about the effects of social media in general on people and communities, removing lock-in fixes a lot of existing perverse incentives in the system.

Dealing with other people's signed events is always going to require some adversarial thinking. Events can be malformed, either from a buggy implementation or from someone trying to attack software applications. A p tag might have an invalid or missing value, or a relay hint that is trying to spy on users. Event content might include HTML injection attacks, or illegal content.

Distinguishing between when to discard an event and when to try to handle it anyway is difficult. The most important thing when handling events is to protect the user from attacks. Everything else should be handled in a fault-tolerant way, assuming incompetence rather than hostility.

It's important to note that Postel's "be liberal in what you accept" has two different possible interpretations here. Fault tolerance does not mean repairing data that contradicts established conventions. A common example of this is when clients publish events with bech32-encoded data in tags instead of hex-encoded data.

This is of course a difficult judgment call, since as mentioned above the vocabulary of nostr is idiomatic and evolving. The important thing is not to allow a minority, non-standard data format to hijack a well-established content type. This can go some way toward taming the chaos of a decentralized system, as well as protect against organized attacks that would attempt to "embrace, extend, extinguish" the protocol.

Here are a few examples of malformed data and how it should be handled:

• If an event's signature is invalid, discard it — there's no way to prove it isn't forged. There are of course exceptions to this, like with NIP 59 wrapped events, which have no signature, but are still authenticated cryptographically.
• If an event contains code intended to crash, denial-of-service, or hijack the client, the client should protect itself using conventional defensive programming techniques — by sanitizing HTML before displaying it, not using regular expressions with user provided data, and by handling errors using null checks and fallbacks.
• If an event contains an adversarial relay hint or suspicious media URL, clients should have a policy in place, informed by user preferences, which blocks connections to unknown or untrustworthy hosts. This can be accomplished through domain white/black listing, or through trust assessments based on proof-of-work or the reputation of the event author.
• If an event contains data that doesn't adhere to the relevant specification, for example missing start or end times on a calendar event, clients might choose to show a fallback value, e.g. "not specified", or an error indicator.

One reason to prefer fault-tolerant handling over aggressive validation is that strict validation reduces interoperability unnecessarily by converting minor, recoverable errors into catastrophic exceptions that ultimately result in a poor UX. Care has to be taken to balance user protection, user experience, and long-term stewardship of the protocol when making these decisions.

An additional wrinkle to this problem involves writing rather than reading events. A number of bugs have emerged where data published by one client will be unwittingly deleted by another client. Here are a few examples:

• Some clients at one point started adding JSON-encoded relay selections to the content field of kind 3 follower list events. Other clients would drop these selections when updating user follows.
• Some clients started adding muted words to kind 10000 mute lists, in addition to muted pubkeys. Clients that weren't expecting this ended up dropping muted words when updating muted pubkeys.
• Similarly, some clients began adding private mutes to kind 100000 mute lists by JSON-encoding the tags, encrypting the result, and placing it in the event's content field. This resulted in mutes being ignored or overwritten on update.

In every case, these bugs are a result of poor design — overloading a single kind for multiple purposes is always a recipe for disaster. But in cases where poor designs become conventional, it can be helpful to handle events in such a way that unanticipated data doesn't get dropped.

One way I've found to do this is to write parsers for an event in order to turn it into a standard data structure that can be used elsewhere in my application with the original event attached. Then, when saving a new version of the event, I update the event directly, keeping tags and content intact as much as possible before re-publishing. Here's an example:

readProfile = e => {...decodeJson(e.content), e}
writeProfile = ({e, ...p}) => {kind: 0, content: encodeJson(p), tags: e.tags}

You can see that the original event is passed through to writeProfile, allowing us to avoid dropping any tags that may have been included.

This adds a certain amount of additional effort to implementations, but is important for avoiding disruption of user experience.

Backwards Compatibility

Backwards incompatibility is one of the big problems of spec design. When breaking backwards compatibility, not only do you break other existing implementations, but in a system where events can't be migrated to the new format you also break all historical data, even if it was published by your own app.

At the same time, breaking backwards compatibility can free you up to improve the protocol in important ways. In many cases, it also won't have much of an impact. Supporting historical data is important for some applications, like archival services, but many applications have a strong recency bias such that they don't necessarily mind throwing away old data.

I think there's a balance to this. I don't think we can be backwards compatibility maximalists, but we also shouldn't be careless about throwing away old formats for no reason. There is a perverse impulse among software developers to refactor for dumb reasons which should absolutely be avoided.

When introducing new backwards-compatible functionality to Nostr, it's usually best to enhance existing data formats as long as it doesn't result in overloading a single kind with multiple use cases.

When breaking backwards compatibility however, it's best to create an entirely new kind and evangelize for migrating to it. This is much harder than modifying the behavior of existing event kinds, but it's far more polite. The downside is it breaks network effects - and can still result in a poor UX for clients that don't adopt the new format if it results in missing content.

Over time, Nostr protocol development has gotten increasingly conservative. As use cases proliferate, it becomes more difficult to get feedback on new data formats. As implementations proliferate, it becomes more difficult to advocate for breaking changes. Nostr specifications are not sacred, but their effectiveness relies almost entirely on interoperability. Maintaining compatibility requires conscientiousness, communication, and contributions to other projects - there's no better way to get people to listen to you than writing code to improve their implementation.

The Economics of Censorship

Nostr was originally designed to be "censorship-resistant". But this is only one part of a bigger vision. Nostr is designed to promote the digital freedoms of users — things like privacy, access to information, anonymity, freedom of association, and credible exit. Because of how cryptography was adopted (primarily by institutions) in the early days, the internet is no longer aligned with the interests of its users, but has largely been captured by "platforms".

As a result, we have a state of rent-seeking in which the users who contribute all the value that the internet has to offer are exploited, undermined, manipulated, deceived, and sold by the custodians of that value. This is a far cry from the humanitarian dream of the early internet. In the words of Tim Berners-Lee,

We create the Web, by designing computer protocols and software; this process is completely under our control. We choose what properties we want it to have and not have[...] The goal of the Web is to serve humanity. We build it now so that those who come to it later will be able to create things that we cannot ourselves imagine.

In the fifteen years since this was written, the internet has only become more predatory, and proprietary. Everywhere you go, you're being tracked — maybe for the purposes of understanding your behavior to serve you ads, maybe for reasons more convoluted, but always in the interest of the ones doing the tracking.

This isn't to say the internet isn't a net positive to those who use it. The internet is the single most successful technology in human history for promoting access to information, alternative community, and trade. But it is not what it should be. Reforming the internet requires active investment from those who stand the gain or lose the most from how it evolves — its users. No one can do this for us; we have to hold companies and platforms accountable ourselves. And in order to do that, we need to understand why censorship happens.

Censorship is not the product of mere arbitrary exercise of power, but of incentives. In authoritarian regimes, censorship is implemented to protect the regime's power. In business, censorship is implemented to protect the business' revenue.

Traditionally, products are sold directly to customers, who then get access to the good or service they've purchased. This business model can transfer to some extent to the realm of information-based products, particularly in a business-to-business setting, where information can be justified in terms of greater productivity.

But consumer software is an entirely different story. If your users are only there to improve their quality of life through access to entertainment or communication, and the marginal cost of production trends toward zero for digital goods, then the price will also trend toward zero because competitors can always offer the same product at a lower price.

Combine that with the imperative for platforms to aggressively acquire users, and you get a strong downward pressure on the upfront pricing of software products. "Network effect" is the idea that a network's value does not grow linearly with the number of users, but quadratically with the number of connections between users.

Building a large network requires significant resources in order to fund growth or stay ahead of competitors. These resources in turn must be provided by people with capital. As a result, businesses that rely on scaling network effects are typically funded by investors and oriented at a return on investment. This includes social media platforms and similar businesses, such as search engines, marketplaces, and entertainment platforms.

Capitalists want to see the businesses they've invested in make a profit. The best way to make a profit in a market, according to Peter Thiel's Zero to One, is to establish a monopoly. Because user retention depends on network effects, market saturation is always the goal, and switching costs must be high — companies must "capture" their users.

The art of creating this moat seems to have been perfected. Facebook has been around for a long time, as has Google. In contrast to early internet companies like MySpace, Yahoo, and AOL, these second-generation internet giants have learned how to simultaneously retain and monetize their users.

Very little of these companies' revenue comes from direct monetization. Instead, these big tech companies monetize their users by making them the product. Users have two things that platform owners can sell: their attention and their data. Platforms can only sell these things because they have access to them by virtue of being intermediaries. Platforms subsist by inserting themselves into private or public relationships in order to siphon off these intangible goods.

This works because the majority of internet users either don't believe their attention or data are worth anything ("why do I need privacy? I have nothing to hide"), or feel helpless to opt out of the attention economy. Seeing a billboard in your peripheral vision or having cameras record your license plate seem hardly comparable to paying a monthly fee for road use. Whether you care about being recorded or not, you're still going to use the roads.

It's true that data and attention are not particularly valuable on an individual basis. But they're massively powerful in the aggregate. "Big data" is not mere information, but the ability to predict patterns of behavior, which can then be used at scale to manipulate people — to buy, believe, and act.

Advertising is not just about informing willing buyers about products and services, it's about conditioning people to act in predictable ways. Social engineering (like all engineering) is progressive and totalizing — unless restrained. The cost of giving away our attention and data to these internet intermediaries is, ultimately, the loss of our free will.

This is particularly true because users are progressively getting less and worse service in exchange for their time and attention. Because big internet companies have access to economies of scale that individuals don't, they have the ability to asymmetrically impose complexity costs on users who fall outside of the streamlined "ideal customer profile" they've built their business on. Quoting from this blog post:

Bigger players like corporations and governments gained a big systemic advantage over individuals: they make the rules but they don’t bear the costs of the rules breaking. They increase the complexity of everyday life and instead of being punished for it, they are rewarded for it.

A Path Forward

This slide isn't inevitable. But it's also human nature — everyone wants something for nothing, and so internet users have voluntarily given up their free will to gain connectivity, entertainment, and efficiency. Simply pushing back is not enough — the problem is structural, and time has allowed its beneficiaries to entrench their position.

This problem needs more than a technical solution — politics, culture, community, and capital all have roles to play here. Nostr's role is to offer tools that individuals can use according to their own values, and for their own interests. Nostr makes it possible for internet users to preserve their own digital sovereignty and hold tech platforms accountable.

My personal hope for Nostr is that it will aid in the restoration and cultivation of human flourishing and agency in a world increasingly mediated by digital technology. Nostr does not solve every problem. The ones it does solve, it sometimes solves poorly. Nostr is not a panacea, it is a toolset that can be applied to our benefit or to our detriment, but which has certain intrinsic qualities that the internet desperately needs right now.

I want to emphasize that it is only through people that any of the internet's ills can be cured. Nostr is not a comprehensive system for restraining digital evil (such a thing would be itself totalitarian), but a way of allowing individual and community agency to be exercised in solving problems germane to individuals' and communities' own particular circumstances.

In this note Pablo is subtweeting something I said to him recently (although I'm sure he's heard it from other quarters as well):

nostr:nevent1qvzqqqqqqypzp75cf0tahv5z7plpdeaws7ex52nmnwgtwfr2g3m37r844evqrr6jqy2hwumn8ghj7un9d3shjtnyv9kh2uewd9hj7qghwaehxw309aex2mrp0yh8qunfd4skctnwv46z7qg6waehxw309ac8junpd45kgtnxd9shg6npvchxxmmd9uqzq0z48d4ttzzkupswnkyt5a2xfkhxl3hyavnxjujwn5k2k529aearwtecp4

There is a naive, curmudgeonly case for simply "not doing AI". I think the intuition is a good one, but the subject is obviously more complicated - not doing it, either on an individual or a collective level, is just not an option. I recently read Tools for Conviviality by Ivan Illich, which I think can help us here. For Illich, the best kind of tool is one which serves "politically interrelated individuals rather than managers".

This is obviously a core value for bitcoiners. And I think the talks given at the Oslo Freedom Forum this year present a compelling case for adoption of LLMs for the purposes of 1. using them for good, and 2. developing them further so that they don't get captured by corporations and governments. Illich calls both the telephone and print "almost ideally convivial". I would add the internet, cryptography, and LLMs to this list, because each one allows individuals to work cooperatively within communities to embody their values in their work.

But this is only half the story. Illich also points out how "the manipulative nature of institutions... have put these ideally convivial tools at the service of more [managerial dominance]."

Preventing the subversion and capture of our tools is not just a matter of who uses what, and for which ends. It also requires an awareness of the environment that the use of the tool (whether for virtuous or vicious ends) creates, which in turn forms the abilities, values, and desires of those who inhabit the environment.

The natural tendency of LLMs is to foster ignorance, dependence, and detachment from reality. This is not the fault of the tool itself, but that of humans' tendency to trade liberty for convenience. Nevertheless, the inherent values of a given tool naturally gives rise to an environment through use: the tool changes the world that the tool user lives in. This in turn indoctrinates the user into the internal logic of the tool, shaping their thinking, blinding them to the tool's influence, and neutering their ability to work in ways not endorsed by the structure of the tool-defined environment.

The result of this is that people are formed by their tools, becoming their slaves. We often talk about LLM misalignment, but the same is true of humans. Unreflective use of a tool creates people who are misaligned with their own interests. This is what I mean when I say that AI use is anti-human. I mean it in the same way that all unreflective tool use is anti-human. See Wendell Berry for an evaluation of industrial agriculture along the same lines.

What I'm not claiming is that a minority of high agency individuals can't use the technology for virtuous ends. In fact, I think that is an essential part of the solution. Tool use can be good. But tools that bring their users into dependence on complex industry and catechize their users into a particular system should be approached with extra caution. The plow was a convivial tool, and so were early tractors. Self-driving John Deere monstrosities are a straightforward extension of the earlier form of the technology, but are self-evidently an instrument of debt slavery, chemical dependency, industrial centralization, and degradation of the land. This over-extension of a given tool can occur regardless of the intentions of the user. As Illich says:

There is a form of malfunction in which growth does not yet tend toward the destruction of life, yet renders a tool antagonistic to its specific aims. Tools, in other words, have an optimal, a tolerable, and a negative range.

The initial form of a tool is almost always beneficial, because tools are made by humans for human ends. But as the scale of the tool grows, its logic gets more widely and forcibly applied. The solution to the anti-human tendencies of any technology is an understanding of scale. To prevent the overrun of the internal logic of a given tool and its creation of an environment hostile to human flourishing, we need to impose limits on scale.

Tools that require time periods or spaces or energies much beyond the order of corresponding natural scales are dysfunctional.

My problem with LLMs is:

• Not their imitation of human idioms, but their subversion of them and the resulting adoption of robotic idioms by humans
• Not the access they grant to information, but their ability to obscure accurate or relevant information
• Not their elimination of menial work, but its increase (Bullshit Jobs)
• Not their ability to take away jobs, but their ability to take away the meaning found in good work
• Not their ability to confer power to the user, but their ability to confer power to their owner which can be used to exploit the user
• Not their ability to solve problems mechanistically, but the extension of their mechanistic value system to human life
• Not their explicit promise of productivity, but the environment they implicitly create in which productivity depends on their use
• Not the conversations they are able to participate in, but the relationships they displace

All of these dysfunctions come from the over-application of the technology in evaluating and executing the fundamentally human task of living. AI work is the same kind of thing as an AI girlfriend, because work is not only for the creation of value (although that's an essential part of it), but also for the exercise of human agency in the world. In other words, tools must be tools, not masters. This is a problem of scale - when tool use is extended beyond its appropriate domain, it becomes what Illich calls a "radical monopoly" (the domination of a single paradigm over all of human life).

So the important question when dealing with any emergent technology becomes: how can we set limits such that the use of the technology is naturally confined to its appropriate scale?

Here are some considerations:

• Teach people how to use the technology well (e.g. cite sources when doing research, use context files instead of fighting the prompt, know when to ask questions rather than generate code)
• Create and use open source and self-hosted models and tools (MCP, stacks, tenex). Refuse to pay for closed or third-party hosted models and tools.
• Recognize the dependencies of the tool itself, for example GPU availability, and diversify the industrial sources to reduce fragility and dependence.
• Create models with built-in limits. The big companies have attempted this (resulting in Japanese Vikings), but the best-case effect is a top-down imposition of corporate values onto individuals. But the idea isn't inherently bad - a coding model that refuses to generate code in response to vague prompts, or which asks clarifying questions is an example. Or a home assistant that recognized childrens' voices and refuses to interact.
• Divert the productivity gains to human enrichment. Without mundane work to do, novice lawyers, coders, and accountants don't have an opportunity to hone their skills. But their learning could be subsidized by the bots in order to bring them up to a level that continues to be useful.
• Don't become a slave to the bots. Know when not to use it. Talk to real people. Write real code, poetry, novels, scripts. Do your own research. Learn by experience. Make your own stuff. Take a break from reviewing code to write some. Be independent, impossible to control. Don't underestimate the value to your soul of good work.
• Resist both monopoly and "radical monopoly". Both naturally collapse over time, but by cultivating an appreciation of the goodness of hand-crafted goods, non-synthetic entertainment, embodied relationship, and a balance between mobility and place, we can relegate new, threatening technologies to their correct role in society.

I think in all of this is implicit the idea of technological determinism, that productivity is power, and if you don't adapt you die. I reject this as an artifact of darwinism and materialism. The world is far more complex and full of grace than we think.

The idea that productivity creates wealth is, as we all know, bunk. GDP continues to go up, but ungrounded metrics don't reflect anything about the reality of human flourishing. We have to return to a qualitative understanding of life as whole, and contextualize quantitative tools and metrics within that framework.

Finally, don't believe the hype. Even if AI delivers everything it promises, conservatism in changing our ways of life will decelerate the rate of change society is subjected to and allow time for reflection and proper use of the tool. Curmudgeons are as valuable as technologists. There will be no jobspocalypse if there is sufficient political will to value human good over mere productivity. It's ok to pump the breaks.

An example of this is how for a while it was considered best practice to re-publish notes when replying to them. This practice emerged before the outbox model gained traction, and was a hacky way of attempting to ensure relays had the full context required for a given note. Over time though, pubkey hints emerged as a better way to ensure other clients could find required context.

Another one of these things is "relay-based groups", or as I prefer to call it "relays-as-groups" (RAG). Such a thing doesn't really exist - there's no spec for it (although some aspects of the concept are included in NIP 29), but at the same time there are two concrete implementations (Flotilla and Chachi) which leverage several different NIPs in order to create a cohesive system for groups on nostr.

This composability is one of the neat qualities of nostr. Not only would it be unhelpful to specify how different parts of the protocol should work together, it would be impossible because of the number of possible combinations possible just from applying a little bit of common sense to the NIPs repo. No one said it was ok to put t tags on a kind 0. But no one's stopping you! And the semantics are basically self-evident if you understand its component parts.

So, instead of writing a NIP that sets relay-based groups in stone, I'm writing this guide in order to document how I've combined different parts of the nostr protocol to create a compelling architecture for groups.

Relays

Relays already have a canonical identity, which is the relay's url. Events posted to a relay can be thought of as "posted to that group". This means that every relay is already a group. All nostr notes have already been posted to one or more groups.

One common objection to this structure is that identifying a group with a relay means that groups are dependent on the relay to continue hosting the group. In normal broadcast nostr (which forms organic permissionless groups based on user-centric social clustering), this is a very bad thing, because hosts are orthogonal to group identity. Communities are completely different. Communities actually need someone to enforce community boundaries, implement moderation, etc. Reliance on a host is a feature, not a bug (in contrast to NIP 29 groups, which tend to co-locate many groups on a single host, relays-as-groups tends to encourage one group, one host).

This doesn't mean that federation, mirrors, and migration can't be accomplished. In a sense, leaving this on the social layer is a good thing, because it adds friction to the dissolution/forking of a group. But the door is wide open to protocol additions to support those use cases for relay-based groups. One possible approach would be to follow this draft PR which specifies a "federation" event relays could publish on their own behalf.

Relay keys

This draft PR to NIP 11 specifies a self field which represents the relay's identity. Using this, relays can publish events on their own behalf. Currently, the pubkey field sort of does the same thing, but is overloaded as a contact field for the owner of the relay.

AUTH

Relays can control access using NIP 42 AUTH. There are any number of modes a relay can operate in:

1. No auth, fully public - anyone can read/write to the group.
2. Relays may enforce broad or granular access controls with AUTH.

Relays may deny EVENTs or REQs depending on user identity. Messages returned in AUTH, CLOSED, or OK messages should be human readable. It's crucial that clients show these error messages to users. Here's how Flotilla handles failed AUTH and denied event publishing:

LIMITS, PROBE, or some other reflection scheme could also be used in theory to help clients adapt their interface depending on user abilities and relay policy.

3. AUTH with implicit access controls.

In this mode, relays may exclude matching events from REQs if the user does not have permission to view them. This can be useful for multi-use relays that host hidden rooms. This mode should be used with caution, because it can result in confusion for the end user.

See Frith for a relay implementation that supports some of these auth policies.

Invite codes

If a user doesn't have access to a relay, they can request access using this draft NIP. This is true whether access has been explicitly or implicitly denied (although users will have to know that they should use an invite code to request access).

The above referenced NIP also contains a mechanism for users to request an invite code that they can share with other users.

The policy for these invite codes is entirely up to the relay. They may be single-use, multi-use, or require additional verification. Additional requirements can be communicated to the user in the OK message, for example directions to visit an external URL to register.

See Frith for a relay implementation that supports invite codes.

Content

Any kind of event can be published to a relay being treated as a group, unless rejected by the relay implementation. In particular, NIP 7D was added to support basic threads, and NIP C7 for chat messages.

Since which relay an event came from determines which group it was posted to, clients need to have a mechanism for keeping track of which relay they received an event from, and should not broadcast events to other relays (unless intending to cross-post the content).

Rooms

Rooms follow NIP 29. I wish NIP 29 wasn't called "relay based groups", which is very confusing when talking about "relays as groups". It's much better to think of them as sub-groups, or as Flotilla calls them, "rooms".

EDIT: Flotilla has migrated to exclusively use "managed rooms" — i.e., fully NIP 29 compliant rooms. Relays without NIP 29 support can still support chat, but all messages will be presented as sent to a single room. I've removed references to unmanaged rooms in what follows.

Rooms have two modes - managed and unmanaged. Managed rooms follow all the rules laid out in NIP 29 about metadata published by the relay and user membership. In either case, rooms are represented by a random room id, and are posted to by including the id in an event's h tag. This allows rooms to switch between managed and unmanaged modes without losing any content.

Managed room names come from kind 39000 room meta events, but unmanaged rooms don't have these. Instead, room names should come from members' NIP 51 kind 10009 membership lists. Tags on these lists should look like this: ["group", "groupid", "wss://group.example.com", "Cat lovers"]. If no name can be found for the room (i.e., there aren't any members), the room should be ignored by clients.

Rooms present a difficulty for publishing to the relay as a whole, since content with an h tag can't be excluded from requests. Currently, relay-wide posts are h-tagged with _ which works for "group" clients, but not more generally. I'm not sure how to solve this other than to ask relays to support negative filters. I have ideas on how to solve this in future iterations of relay-based groups, for example using virtual relays or just a better rooms spec.

Cross-posting

The simplest way to cross-post content from one group (or room) to another, is to quote the original note in whatever event kind is appropriate. For example, a blog post might be quoted in a kind 9 to be cross-posted to chat, or in a kind 11 to be cross-posted to a thread. kind 16 reposts can be used the same way if the reader's client renders reposts.

Posting the original event to multiple relays-as-groups is trivial, since all you have to do is send the event to the relay. Posting to multiple rooms simultaneously by appending multiple h tags is however not recommended, since group relays/clients are incentivised to protect themselves from spam by rejecting events with multiple h tags (similar to how events with multiple t tags are sometimes rejected).

Privacy

Currently, it's recommended to include a NIP 70 - tag on content posted to relays-as-groups to discourage replication of relay-specific content across the network.

Another slightly stronger approach would be for group relays to strip signatures in order to make events invalid (or at least deniable). For this approach to work, users would have to be able to signal that they trust relays to be honest. We could also use ZkSNARKS to validate signatures in bulk.

In any case, group posts should not be considered "private" in the same way E2EE groups might be. Relays-as-groups should be considered a good fit for low-stakes groups with many members (since trust deteriorates quickly as more people get involved).

Membership

There is currently no canonical member list published by relays (except for NIP 29 managed rooms). Instead, users keep track of their own relay and room memberships using kind 10009 lists. Relay-level memberships are represented by an r tag containing the relay url, and room-level memberships are represented using a group tag.

Users can choose to advertise their membership in a RAG by using unencrypted tags, or they may keep their membership private by using encrypted tags. Advertised memberships are useful for helping people find groups based on their social graph:

User memberships should not be trusted, since they can be published unilaterally by anyone, regardless of actual access, so it's better to think of them as "bookmarked groups" or "favorites". Possible improvements in this area would be the ability to provide proof of access:

• Relays could publish member lists (although this would sacrifice member privacy)
• Relays could support a new command that allows querying a particular member's access status
• Relays could provide a proof to the member that they could then choose to publish or not

Moderation

There are two parts to moderation: reporting and taking action based on these reports.

Reporting is already covered by NIP 56. Clients should be careful about encouraging users to post reports for illegal content under their own identity, since that can itself be illegal. Relays also should not serve reports to users, since that can be used to find rather than address objectionable content.

Reports are only one mechanism for flagging objectionable content. Relay operators and administrators can use whatever heuristics they like to identify and address objectionable content. This might be via automated policies that auto-ban based on reports from high-reputation people, a client that implements NIP 86 relay management API, or by some other admin interface.

There's currently no way for moderators of a given relay to be advertised, or for a moderator's client to know that the user is a moderator (so that they can enable UI elements for in-app moderation). This could be addressed via NIP 11, LIMITS, or some other mechanism in the future.

General best practices

In general, it's very important when developing a client to assume that the relay has no special support for any of the above features, instead treating all of this stuff as progressive enhancement.

For example, if a user enters an invite code, go ahead and send it to the relay using a kind 28934 event. If it's rejected, you know that it didn't work. But if it's accepted, you don't know that it worked - you only know that the relay allowed the user to publish that event. This is helpful, becaues it may imply that the user does indeed have access to the relay. But additional probing may be needed, and reliance on error messages down the road when something else fails unexpectedly is indispensable.

This paradigm may drive some engineers nuts, because it's basically equivalent to coding your clients to reverse-engineer relay support for every feature you want to use. But this is true of nostr as a whole - anyone can put whatever weird stuff in an event and sign it. Clients have to be extremely compliant with Postell's law - doing their absolute best to accept whatever weird data or behavior shows up and handle failure in any situation. Sure, it's annoying, but it's the cost of permissionless development. What it gets us is a completely open-ended protocol, in which anything can be built, and in which every solution is tested by the market.

But your experience has not been all you hoped. Before you've even had a chance to upload your AI-generated cyberpunk avatar or make up exploit codenames for your pseudonym's bio, you've been confronted with a new concept that has left you completely nonplussed.

It doesn't help that this new idea might be called by any number of strange names. You may have been asked to "paste your nsec", "generate a private key", "enter your seed words", "connect with a bunker", "sign in with extension", or even "generate entropy". Sorry about that.

All these terms are really referring to one concept under many different names: that of "cryptographic identity".

Now, you may have noticed that I just introduced yet another new term which explains exactly nothing. You're absolutely correct. And now I'm going to proceed to ignore your complaints and talk about something completely different. But bear with me, because the juice is worth the squeeze.

Identity

What is identity? There are many philosophical, political, or technical answers to this question, but for our purposes it's probably best to think of it this way:

Identity is the essence of a thing. Identity separates one thing from all others, and is itself indivisible.

This definition has three parts:

• Identity is "essential": a thing can change, but its identity cannot. I might re-paint my house, replace its components, sell it, or even burn it down, but its identity as something that can be referred to - "this house" - is durable, even outside the boundaries of its own physical existence.
• Identity is a unit: you can't break an identity into multiple parts. A thing might be composed of multiple parts, but that's only incidental to the identity of a thing, which is a concept, not a material thing.
• Identity is distinct: identity is what separates one thing from all others - the concept of an apple can't be mixed with that of an orange; the two ideas are distinct. In the same way, a single concrete apple is distinct in identity from another - even if the component parts of the apple decompose into compost used to grow more apples.

Identity is not a physical thing, but a metaphysical thing. Or, in simpler terms, identity is a "concept".

I (or someone more qualified) could at this point launch into a Scholastic tangent on what "is" is, but that is, fortunately, not necessary here. The kind of identities I want to focus on here are not our actual identities as people, but entirely fictional identities that we use to extend our agency into the digital world.

Think of it this way - your bank login does not represent you as a complete person. It only represents the access granted to you by the bank. This access is in fact an entirely new identity that has been associated with you, and is limited in what it's useful for.

Other examples of fictional identities include:

• The country you live in
• Your social media persona
• Your mortgage
• Geographical coordinates
• A moment in time
• A chess piece

Some of these identites are inert, for example points in space and time. Other identies have agency and so are able to act in the world - even as fictional concepts. In order to do this, they must "authenticate" themselves (which means "to prove they are real"), and act within a system of established rules.

For example, your D&D character exists only within the collective fiction of your D&D group, and can do anything the rules say. Its identity is authenticated simply by your claim as a member of the group that your character in fact exists. Similarly, a lawyer must prove they are a member of the Bar Association before they are allowed to practice law within that collective fiction.

"Cryptographic identity" is simply another way of authenticating a fictional identity within a given system. As we'll see, it has some interesting attributes that set it apart from things like a library card or your latitude and longitude. Before we get there though, let's look in more detail at how identities are authenticated.

Certificates

Merriam-Webster defines the verb "certify" as meaning "to attest authoritatively". A "certificate" is just a fancy way of saying "because I said so". Certificates are issued by a "certificate authority", someone who has the authority to "say so". Examples include your boss, your mom, or the Pope.

This method of authentication is how almost every institution authenticates the people who associate with it. Colleges issue student ID cards, governments issue passports, and websites allow you to "register an account".

In every case mentioned above, the "authority" creates a closed system in which a document (aka a "certificate") is issued which serves as a claim to a given identity. When someone wants to access some privileged service, location, or information, they present their certificate. The authority then validates it and grants or denies access. In the case of an international airport, the certificate is a little book printed with fancy inks. In the case of a login page, the certificate is a username and password combination.

This pattern for authentication is ubiquitous, and has some very important implications.

First of all, certified authentication implies that the issuer of the certificate has the right to exclusive control of any identity it issues. This identity can be revoked at any time, or its permissions may change. Your social credit score may drop arbitrarily, or money might disappear from your account. When dealing with certificate authorities, you have no inherent rights.

Second, certified authentication depends on the certificate authority continuing to exist. If you store your stuff at a storage facility but the company running it goes out of business, your stuff might disappear along with it.

Usually, authentication via certificate authority works pretty well, since an appeal can always be made to a higher authority (nature, God, the government, etc). Authorities also can't generally dictate their terms with impunity without losing their customers, alienating their constituents, or provoking revolt. But it's also true that certification by authority creates an incentive structure that frequently leads to abuse - arbitrary deplatforming is increasingly common, and the bigger the certificate authority, the less recourse the certificate holder (or "subject") has.

Certificates also put the issuer in a position to intermediate relationships that wouldn't otherwise be subject to their authority. This might take the form of selling user attention to advertisers, taking a cut of financial transactions, or selling surveillance data to third parties.

Proliferation of certificate authorities is not a solution to these problems. Websites and apps frequently often offer multiple "social sign-in" options, allowing their users to choose which certificate authority to appeal to. But this only piles more value into the social platform that issues the certificate - not only can Google shut down your email inbox, they can revoke your ability to log in to every website you used their identity provider to get into.

In every case, certificate issuance results in an asymmetrical power dynamic, where the issuer is able to exert significant control over the certificate holder, even in areas unrelated to the original pretext for the relationship between parties.

Self-Certification

But what if we could reverse this power dynamic? What if individuals could issue their own certificates and force institutions to accept them?

Ron Swanson's counterexample notwithstanding, there's a reason I can't simply write myself a parking permit and slip it under the windshield wiper. Questions about voluntary submission to legitimate authorities aside, the fact is that we don't have the power to act without impunity - just like any other certificate authority, we have to prove our claims either by the exercise of raw power or by appeal to a higher authority.

So the question becomes: which higher authority can we appeal to in order to issue our own certificates within a given system of identity?

The obvious answer here is to go straight to the top and ask God himself to back our claim to self-sovereignty. However, that's not how he normally works - there's a reason they call direct acts of God "miracles". In fact, Romans 13:1 explicitly says that "the authorities that exist have been appointed by God". God has structured the universe in such a way that we must appeal to the deputies he has put in place to govern various parts of the world.

Another tempting appeal might be to nature - i.e. the material world. This is the realm in which we most frequently have the experience of "self-authenticating" identities. For example, a gold coin can be authenticated by biting it or by burning it with acid. If it quacks like a duck, walks like a duck, and looks like a duck, then it probably is a duck.

In most cases however, the ability to authenticate using physical claims depends on physical access, and so appeals to physical reality have major limitations when it comes to the digital world. Captchas, selfies and other similar tricks are often used to bridge the physical world into the digital, but these are increasingly easy to forge, and hard to verify.

There are exceptions to this rule - an example of self-certification that makes its appeal to the physical world is that of a signature. Signatures are hard to forge - an incredible amount of data is encoded in physical signatures, from strength, to illnesses, to upbringing, to personality. These can even be scanned and used within the digital world as well. Even today, most contracts are sealed with some simulacrum of a physical signature. Of course, this custom is quickly becoming a mere historical curiosity, since the very act of digitizing a signature makes it trivially forgeable.

So: transcendent reality is too remote to subtantiate our claims, and the material world is too limited to work within the world of information. There is another aspect of reality remaining that we might appeal to: information itself.

Physical signatures authenticate physical identities by encoding unique physical data into an easily recognizable artifact. To transpose this idea to the realm of information, a "digital signature" might authenticate "digital identities" by encoding unique "digital data" into an easily recognizable artifact.

Unfortunately, in the digital world we have the additional challenge that the artifact itself can be copied, undermining any claim to legitimacy. We need something that can be easily verified and unforgeable.

Digital Signatures

In fact such a thing does exist, but calling it a "digital signature" obscures more than it reveals. We might just as well call the thing we're looking for a "digital fingerprint", or a "digital electroencephalogram". Just keep that in mind as we work our way towards defining the term - we are not looking for something looks like a physical signature, but for something that does the same thing as a physical signature, in that it allows us to issue ourselves a credential that must be accepted by others by encoding privileged information into a recognizable, unforgeable artifact.

With that, let's get into the weeds.

An important idea in computer science is that of a "function". A function is a sort of information machine that converts data from one form to another. One example is the idea of "incrementing" a number. If you increment 1, you get 2. If you increment 2, you get 3. Incrementing can be reversed, by creating a complementary function that instead subtracts 1 from a number.

A "one-way function" is a function that can't be reversed. A good example of a one-way function is integer rounding. If you round a number and get 5, what number did you begin with? It's impossible to know - 5.1, 4.81, 5.332794, in fact an infinite number of numbers can be rounded to the number 5. These numbers can also be infinitely long - for example rounding PI to the nearest integer results in the number 3.

A real-life example of a useful one-way function is sha256. This function is a member of a family of one-way functions called "hash functions". You can feed as much data as you like into sha256, and you will always get 256 bits of information out. Hash functions are especially useful because collisions between outputs are very rare - even if you change a single bit in a huge pile of data, you're almost certainly going to get a different output.

Taking this a step further, there is a whole family of cryptographic one-way "trapdoor" functions that act similarly to hash functions, but which maintain a specific mathematical relationship between the input and the output which allows the input/output pair to be used in a variety of useful applications. For example, in Elliptic Curve Cryptography, scalar multiplication on an elliptic curve is used to derive the output.

"Ok", you say, "that's all completely clear and lucidly explained" (thank you). "But what goes into the function?" You might expect that because of our analogy to physical signatures we would have to gather an incredible amount of digital information to cram into our cryptographic trapdoor function, mashing together bank statements, a record of our heartbeat, brain waves and cellular respiration. Well, we could do it that way (maybe), but there's actually a much simpler solution.

Let's play a quick game. What number am I thinking of? Wrong, it's 82,749,283,929,834. Good guess though.

The reason we use signatures to authenticate our identity in the physical world is not because they're backed by a lot of implicit physical information, but because they're hard to forge and easy to validate. Even so, there is a lot of variation in a single person's signature, even from one moment to the next.

Trapdoor functions solve the validation problem - it's trivially simple to compare one 256-bit number to another. And randomness solves the problem of forgeability.

Now, randomness (A.K.A. "entropy") is actually kind of hard to generate. Random numbers that don't have enough "noise" in them are known as "pseudo-random numbers", and are weirdly easy to guess. This is why Cloudflare uses a video stream of their giant wall of lava lamps to feed the random number generator that powers their CDN. For our purposes though, we can just imagine that our random numbers come from rolling a bunch of dice.

To recap, we can get a digital equivalent of a physical signature (or fingerprint, etc) by 1. coming up with a random number, and 2. feeding it into our chosen trapdoor function. The random number is called the "private" part. The output of the trapdoor function is called the "public" part. These two halves are often called "keys", hence the terms "public key" and "private key".

And now we come full circle - remember about 37 years ago when I introduced the term "cryptographic identity"? Well, we've finally arrived at the point where I explain what that actually is.

A "cryptographic identity" is identified by a public key, and authenticated by the ability to prove that you know the private key.

Notice that I didn't say "authenticated by the private key". If you had to reveal the private key in order to prove you know it, you could only authenticate a public key once without losing exclusive control of the key. But cryptographic identities can be authenticated any number of times because the certification is an algorithm that only someone who knows the private key can execute.

This is the super power that trapdoor functions have that hash functions don't. Within certain cryptosystems, it is possible to mix additional data with your private key to get yet another number in such a way that someone else who only knows the public key can prove that you know the private key.

For example, if my secret number is 12, and someone tells me the number 37, I can "combine" the two by adding them together and returning the number 49. This "proves" that my secret number is 12. Of course, addition is not a trapdoor function, so it's trivially easy to reverse, which is why cryptography is its own field of knowledge.

What's it for?

If I haven't completely lost you yet, you might be wondering why this matters. Who cares if I can prove that I made up a random number?

To answer this, let's consider a simple example: that of public social media posts.

Most social media platforms function by issuing credentials and verifying them based on their internal database. When you log in to your Twitter (ok, fine, X) account, you provide X with a phone number (or email) and password. X compares these records to the ones stored in the database when you created your account, and if they match they let you "log in" by issuing yet another credential, called a "session key".

Next, when you "say" something on X, you pass along your session key and your tweet to X's servers. They check that the session key is legit, and if it is they associate your tweet with your account's identity. Later, when someone wants to see the tweet, X vouches for the fact that you created it by saying "trust me" and displaying your name next to the tweet.

In other words, X creates and controls your identity, but they let you use it as long as you can prove that you know the secret that you agreed on when you registered (by giving it to them every time).

Now pretend that X gets bought by someone even more evil than Elon Musk (if such a thing can be imagined). The new owner now has the ability to control your identity, potentially making it say things that you didn't actually say. Someone could be completely banned from the platform, but their account could be made to continue saying whatever the owner of the platform wanted.

In reality, such a breach of trust would quickly result in a complete loss of credibility for the platform, which is why this kind of thing doesn't happen (at least, not that we know of).

But there are other ways of exploiting this system, most notably by censoring speech. As often happens, platforms are able to confiscate user identities, leaving the tenant no recourse except to appeal to the platform itself (or the government, but that doesn't seem to happen for some reason - probably due to some legalese in social platforms' terms of use). The user has to start completely from scratch, either on the same platform or another.

Now suppose that when you signed up for X instead of simply telling X your password you made up a random number and provided a cryptographic proof to X along with your public key. When you're ready to tweet (there's no need to issue a session key, or even to store your public key in their database) you would again prove your ownership of that key with a new piece of data. X could then publish that tweet or not, along with the same proof you provided that it really came from you.

What X can't do in this system is pretend you said something you didn't, because they don't know your private key.

X also wouldn't be able to deplatform you as effectively either. While they could choose to ban you from their website and refuse to serve your tweets, they don't control your identity. There's nothing they can do to prevent you from re-using it on another platform. Plus, if the system was set up in such a way that other users followed your key instead of an ID made up by X, you could switch platforms and keep your followers. In the same way, it would also be possible to keep a copy of all your tweets in your own database, since their authenticity is determined by your digital signature, not X's "because I say so".

This new power is not just limited to social media either. Here are some other examples of ways that self-issued cryptographic identites transform the power dynamic inherent in digital platforms:

• Banks sometimes freeze accounts or confiscate funds. If your money was stored in a system based on self-issued cryptographic keys rather than custodians, banks would not be able to keep you from accessing or moving your funds. This system exists, and it's called bitcoin.
• Identity theft happens when your identifying information is stolen and used to take out a loan in your name, and without your consent. The reason this is so common is because your credentials are not cryptographic - your name, address, and social security number can only be authenticated by being shared, and they are shared so often and with so many counterparties that they frequently end up in data breaches. If credit checks were authenticated by self-issued cryptographic keys, identity theft would cease to exist (unless your private key itself got stolen).
• Cryptographic keys allow credential issuers to protect their subjects' privacy better too. Instead of showing your ID (including your home address, birth date, height, weight, etc), the DMV could sign a message asserting that the holder of a given public key indeed over 21. The liquor store could then validate that claim, and your ownership of the named key, without knowing anything more about you. Zero-knowledge proofs take this a step further.

In each of these cases, the interests of the property owner, loan seeker, or customer are elevated over the interests of those who might seek to control their assets, exploit their hard work, or surveil their activity. Just as with personal privacy, freedom of speech, and Second Amendment rights the individual case is rarely decisive, but in the aggregate realigned incentives can tip the scale in favor of freedom.

Objections

Now, there are some drawbacks to digital signatures. Systems that rely on digital signatures are frequently less forgiving of errors than their custodial counterparts, and many of their strengths have corresponding weaknesses. Part of this is because people haven't yet developed an intuition for how to use cryptographic identities, and the tools for managing them are still being designed. Other aspects can be mitigated through judicious use of keys fit to the problems they are being used to solve.

Below I'll articulate some of these concerns, and explore ways in which they might be mitigated over time.

Key Storage

Keeping secrets is hard. "A lie can travel halfway around the world before the truth can get its boots on", and the same goes for gossip. Key storage has become increasingly important as more of our lives move online, to the extent that password managers have become almost a requirement for keeping track of our digital lives. But even with good password management, credentials frequently end up for sale on the dark web as a consequence of poorly secured infrastructure.

Apart from the fact that all of this is an argument for cryptographic identities (since keys are shared with far fewer parties), it's also true that the danger of losing a cryptographic key is severe, especially if that key is used in multiple places. Instead of hackers stealing your Facebook password, they might end up with access to all your other social media accounts too!

Keys should be treated with the utmost care. Using password managers is a good start, but very valuable keys should be stored even more securely - for example in a hardware signing device. This is a hassle, and something additional to learn, but is an indispensable part of taking advantage of the benefits associated with cryptographic identity.

There are ways to lessen the impact of lost or stolen secrets, however. Lots of different techniques exist for structuring key systems in such a way that keys can be protected, invalidated, or limited. Here are a few:

• Hierarchical Deterministic Keys allow for the creation of a single root key from which multiple child keys can be generated. These keys are hard to link to the parent, which provides additional privacy, but this link can also be proven when necessary. One limitation is that the identity system has to be designed with HD keys in mind.
• Key Rotation allows keys to become expendable. Additional credentials might be attached to a key, allowing the holder to prove they have the right to rotate the key. Social attestations can help with the process as well if the key is embedded in a web of trust.
• Remote Signing is a technique for storing a key on one device, but using it on another. This might take the form of signing using a hardware wallet and transferring an SD card to your computer for broadcasting, or using a mobile app like Amber to manage sessions with different applications.
• Key sharding takes this to another level by breaking a single key into multiple pieces and storing them separately. A coordinator can then be used to collaboratively sign messages without sharing key material. This dramatically reduces the ability of an attacker to steal a complete key.

Multi-Factor Authentication

One method for helping users secure their accounts that is becoming increasingly common is "multi-factor authentication". Instead of just providing your email and password, platforms send a one-time use code to your phone number or email, or use "time-based one time passwords" which are stored in a password manager or on a hardware device.

Again, MFA is a solution to a problem inherent in account-based authentication which would not be nearly so prevalent in a cryptographic identity system. Still, theft of keys does happen, and so MFA would be an important improvement - if not for an extra layer of authentication, then as a basis for key rotation.

In a sense, MFA is already being researched - key shards is one way of creating multiple credentials from a single key. However, this doesn't address the issue of key rotation, especially when an identity is tied to the public key that corresponds to a given private key. There are two possible solutions to this problem:

• Introduce a naming system. This would allow identities to use a durable name, assigning it to different keys over time. The downside is that this would require the introduction of either centralized naming authorities (back to the old model), or a blockchain in order to solve Zooko's trilemma.
• Establish a chain of keys. This would require a given key to name a successor key in advance and self-invalidate, or some other process like social recovery to invalidate an old key and assign the identity to a new one. This also would significantly increase the complexity of validating messages and associating them with a given identity.

Both solutions are workable, but introduce a lot of complexity that could cause more trouble than it's worth, depending on the identity system we're talking about.

Surveillance

One of the nice qualities that systems based on cryptographic identities have is that digitally signed data can be passed through any number of untrusted systems and emerge intact. This ability to resist tampering makes it possible to broadcast signed data more widely than would otherwise be the case in a system that relies on a custodian to authenticate information.

The downside of this is that more untrusted systems have access to data. And if information is broadcast publicly, anyone can get access to it.

This problem is compounded by re-use of cryptographic identities across multiple contexts. A benefit of self-issued credentials is that it becomes possible to bring everything attached to your identity with you, including social context and attached credentials. This is convenient and can be quite powerful, but it also means that more context is attached to your activity, making it easier to infer information about you for advertising or surveillance purposes. This is dangerously close to the dystopian ideal of a "Digital ID".

The best way to deal with this risk is to consider identity re-use an option to be used when desirable, but to default to creating a new key for every identity you create. This is no worse than the status quo, and it makes room for the ability to link identities when desired.

Another possible approach to this problem is to avoid broadcasting signed data when possible. This could be done by obscuring your cryptographic identity when data is served from a database, or by encrypting your signed data in order to selectively share it with named counterparties.

Still, this is a real risk, and should be kept in mind when designing and using systems based on cryptographic identity. If you'd like to read more about this, please see this blog post.

Making Keys Usable

You might be tempted to look at that list of trade-offs and get the sense that cryptographic identity is not for mere mortals. Key management is hard, and footguns abound - but there is a way forward. With nostr, some new things are happening in the world of key management that have never really happened before.

Plenty of work over the last 30 years has gone into making key management tractable, but none have really been widely adopted. The reason for this is simple: network effect.

Many of these older key systems only applied the thinnest veneer of humanity over keys. But an identity is much richer than a mere label. Having a real name, social connections, and a corpus of work to attach to a key creates a system of keys that humans care about.

By bootstrapping key management within a social context, nostr ensures that the payoff of key management is worth the learning curve. Not only is social engagement a strong incentive to get off the ground, people already on the network are eager to help you get past any roadblocks you might face.

So if I could offer an action item: give nostr a try today. Whether you're in it for the people and their values, or you just want to experiment with cryptographic identity, nostr is a great place to start. For a quick introduction and to securely generate keys, visit njump.me.

Thanks for taking the time to read this post. I hope it's been helpful, and I can't wait to see you on nostr!

I'll try not to complain too much in this post, since I'm grateful for the many years of Svelte 3/4 I've enjoyed. But I don't think I'll be choosing Svelte for any new projects going forward. I hope my reflections here will be useful to others as well.

If you're interested in reproductions for the issues I mention here, you can find them below.

• Can't save state to indexeddb
• Component unmount results in undefined variables in closures

The Need for Speed

To start with, let me just quickly acknowledge what the Svelte team is trying to do. It seems like most of the substantial changes in version 5 are built around "deep reactivity", which allows for more granular reactivity, leading to better performance. Performance is good, and the Svelte team has always excelled at reconciling performance with DX.

In previous versions of Svelte, the main way this was achieved was with the Svelte compiler. There were many ancillary techniques involved in improving performance, but having a framework compile step gave the Svelte team a lot of leeway for rearranging things under the hood without making developers learn new concepts. This is what made Svelte so original in the beginning.

At the same time, it resulted in an even more opaque framework than usual, making it harder for developers to debug more complex issues. To make matters worse, the compiler had bugs, resulting in errors which could only be fixed by blindly refactoring the problem component. This happened to me personally at least half a dozen times, and is what ultimately pushed me to migrate to Svelte 5.

Nevertheless, I always felt it was an acceptable trade-off for speed and productivity. Sure, sometimes I had to delete my project and port it to a fresh repository every so often, but the framework was truly a pleasure to use.

Svelte is not Javascript

Svelte 5 doubled down on this tradeoff — which makes sense, because it's what sets the framework apart. The difference this time is that the abstraction/performance tradeoff did not stay in compiler land, but intruded into runtime in two important ways:

• The use of proxies to support deep reactivity
• Implicit component lifecycle state

Both of these changes improved performance and made the API for developers look slicker. What's not to like? Unfortunately, both of these features are classic examples of a leaky abstraction, and ultimately make things more complex for developers, not less.

Proxies are not objects

The use of proxies seems to have allowed the Svelte team to squeeze a little more performance out of the framework, without asking developers to do any extra work. Threading state through multiple levels of components without provoking unnecessary re-renders in frameworks like React is an infamously difficult chore.

Svelte's compiler avoided some of the pitfalls associated with virtual DOM diffing solutions, but evidently there was still enough of a performance gain to be had to justify the introduction of proxies. The Svelte team also seems to argue that their introduction represents an improvement in developer experience:

we... can maximise both efficiency and ergonomics.

Here's the problem: Svelte 5 looks simpler, but actually introduces more abstractions.

Using proxies to monitor array methods (for example) is appealing because it allows developers to forget all the goofy heuristics involved with making sure state was reactive and just push to the array. I can't count how many times I've written value = value to trigger reactivity in svelte 4.

In Svelte 4, developers had to understand how the Svelte compiler worked. The compiler, being a leaky abstraction, forced its users to know that assignment was how you signaled reactivity. In svelte 5, developers can just "forget" about the compiler!

Except they can't. All the introduction of new abstractions really accomplishes is the introduction of more complex heuristics that developers have to keep in their heads in order to get the compiler to act the way they want it to.

In fact, this is why after years of using Svelte, I found myself using Svelte stores more and more often, and reactive declarations less. The reason being that Svelte stores are just javascript. Calling update on a store is simple, and being able to reference them with a $ was just a nice bonus — nothing to remember, and if I mess up the compiler yells at me.

Proxies introduce a similar problem to reactive declarations, which is that they look like one thing but act like another on the edges.

When I started using Svelte 5, everything worked great — until I tried to save a proxy to indexeddb, at which point I got a DataCloneError. To make matters worse, it's impossible to reliably tell if something is a Proxy without try/catching a structured clone, which is a performance-intensive operation.

This forces the developer to remember what is and what isn't a Proxy, calling $state.snapshot every time they pass a proxy to a context that doesn't expect or know about them. This obviates all the nice abstractions they gave us in the first place.

Components are not functions

The reason virtual DOM took off way back in 2013 was the ability to model your application as composed functions, each of which takes data and spits out HTML. Svelte retained this paradigm, using a compiler to sidestep the inefficiencies of virtual DOM and the complexities of lifecycle methods.

In Svelte 5, component lifecycles are back, react-hooks style.

In React, hooks are an abstraction that allows developers to avoid writing all the stateful code associated with component lifecycle methods. Modern React tutorials universally recommend using hooks instead, which rely on the framework invisibly synchronizing state with the render tree.

While this does result in cleaner code, it also requires developers to tread carefully to avoid breaking the assumptions surrounding hooks. Just try accessing state in a setTimeout and you'll see what I mean.

Svelte 4 had a few gotchas like this — for example, async code that interacts with a component's DOM elements has to keep track of whether the component is unmounted. This is pretty similar to the kind of pattern you'd see in old React components that relied on lifecycle methods.

It seems to me that Svelte 5 has gone the React 16 route by adding implicit state related to component lifecycles in order to coordinate state changes and effects.

For example, here is an excerpt from the documentation for $effect:

You can place $effect anywhere, not just at the top level of a component, as long as it is called during component initialization (or while a parent effect is active). It is then tied to the lifecycle of the component (or parent effect) and will therefore destroy itself when the component unmounts (or the parent effect is destroyed).

That's very complex! In order to use $effect... effectively (sorry), developers have to understand how state changes are tracked. The documentation for component lifecycles claims:

In Svelte 5, the component lifecycle consists of only two parts: Its creation and its destruction. Everything in-between — when certain state is updated — is not related to the component as a whole; only the parts that need to react to the state change are notified. This is because under the hood the smallest unit of change is actually not a component, it’s the (render) effects that the component sets up upon component initialization. Consequently, there’s no such thing as a “before update”/"after update” hook.

But then goes on to introduce the idea of tick in conjunction with $effect.pre. This section explains that "tick returns a promise that resolves once any pending state changes have been applied, or in the next microtask if there are none."

I'm sure there's some mental model that justifies this, but I don't think the claim that a component's lifecycle is only comprised of mount/unmount is really helpful when an addendum about state changes has to come right afterward.

The place where this really bit me, and which is the motivation for this blog post, is when state gets coupled to a component's lifecycle, even when the state is passed to another function that doesn't know anything about svelte.

In my application, I manage modal dialogs by storing the component I want to render alongside its props in a store and rendering it in the layout.svelte of my application. This store is also synchronized with browser history so that the back button works to close them. Sometimes, it's useful to pass a callback to one of these modals, binding caller-specific functionality to the child component:

const {value} = $props()
const callback = () => console.log(value)
const openModal = () => pushModal(MyModal, {callback})

This is a fundamental pattern in javascript. Passing a callback is just one of those things you do.

Unfortunately, if the above code lives in a modal dialog itself, the caller component gets unmounted before the callback gets called. In Svelte 4, this worked fine, but in Svelte 5 value gets updated to undefined when the component gets unmounted. Here's a minimal reproduction.

This is only one example, but it seems clear to me that any prop that is closed over by a callback function that lives longer than its component will be undefined when I want to use it — with no reassignment existing in lexical scope. It seems that the reason this happens is that the props "belong" to the parent component, and are accessed via getters so that the parent can revoke access when it unmounts.

I don't know why this is necessary, but I assume there's a good engineering reason for it. The problem is, this just isn't how javascript works. Svelte is essentially attempting to re-invent garbage collection around component lifecycles, which breaks the assumption every javascript developer has that variables don't simply disappear without an explicit reassignment. It should be safe to pass stuff around and let the garbage collector do its job.

Conclusion

Easy things are nice, but as Rich Hickey says, easy things are not always simple. And like Joel Spolsky, I don't like being surprised. Svelte has always been full of magic, but with the latest release I think the cognitive overhead of reciting incantations has finally outweighed the power it confers.

My point in this post is not to dunk on the Svelte team. I know lots of people like Svelte 5 (and react hooks). The point I'm trying to make is that there is a tradeoff between doing things on the user's behalf, and giving the user agency. Good software is built on understanding, not cleverness.

I also think this is an important lesson to remember as AI-assisted coding becomes increasingly popular. Don't choose tools that alienate you from your work. Choose tools that leverage the wisdom you've already accumulated, and which help you to cultivate a deeper understanding of the discipline.

Thank you to Rich Harris and team for many years of pleasant development. I hope that (if you read this) it's not so full of inaccuracies as to be unhelpful as user feedback.

For the uninitiated, DVM stands for "data vending machines". They're actually sort of hard to describe — in technical terms they act more like clients, since they simply read events from and publish events to relays. In most cases though, these events are part of a request/response flow initiated by users elsewhere on the network. In practice, DVMs are bots, but there's also nothing to prevent the work they do from being powered by human interaction. They're an amazingly flexible tool for building anything from custom feeds, to transcription services, to chatbots, to protocol gateways.

The hype cycle for DVMs seems to have reached escape velocity in a way few other things have - zaps being the possible exception. But what exactly DVMs are remains something of a mystery to many nostr developers - and how to build one may as well be written on clay tablets.

This blog post is designed to address that - below is a soup to nuts (no nutzaps though) guide to building a DVM flow, both from the client and the server side.

Here's what we'll be covering:

• Discovering DVM metadata
• Basic request/response flow
• Implementing a minimal example

Let's get started!

DVM Metadata

First of all, it's helpful to know how DVMs are reified on the nostr network. While not strictly necessary, this can be useful for discovering DVMs and presenting them to users, and for targeting specific DVMs we want a response from.

NIP 89 goes into this in more detail, but the basic idea is that anyone can create a kind 31990 "application handler" event and publish it to the network with their own (or a dedicated) public key. This handler was originally intended to advertise clients, but has been re-purposed for DVM listings as well.

Here's what the "Fluffy Frens" handler looks like:

{
    "content": "{\"name\": \"Fluffy Frens\", \"picture\": \"https://image.nostr.build/f609311532c470f663e129510a76c9a1912ae9bc4aaaf058e5ba21cfb512c88e.jpg\", \"about\": \"I show recent notes about animals\", \"lud16\": \"discovery_content_fluffy@nostrdvm.com\", \"supportsEncryption\": true, \"acceptsNutZaps\": false, \"personalized\": false, \"amount\": \"free\", \"nip90Params\": {\"max_results\": {\"required\": false, \"values\": [], \"description\": \"The number of maximum results to return (default currently 100)\"}}}",
    "created_at": 1738874694,
    "id": "0aa8d1f19cfe17e00ce55ca86fea487c83be39a1813601f56f869abdfa776b3c",
    "kind": 31990,
    "pubkey": "7b7373dd58554ff4c0d28b401b9eae114bd92e30d872ae843b9a217375d66f9d",
    "sig": "22403a7996147da607cf215994ab3b893176e5302a44a245e9c0d91214e4c56fae40d2239dce58ea724114591e8f95caed2ba1a231d09a6cd06c9f0980e1abd5",
    "tags": [
        ["k", "5300"],
        ["d", "198650843898570c"]
    ]
}

This event is rendered in various clients using the kind-0-style metadata contained in the content field, allowing users to browse DVMs and pick one for their use case. If a user likes using a particular DVM, they might publish a kind 31989 "application recommendation", which other users can use to find DVMs that are in use within their network.

Note the k tag in the handler event - this allows DVMs to advertise support only for specific job types. It's also important to note that even though the spec doesn't cover relay selection, most clients use the publisher's kind 10002 event to find out where the DVM listens for events.

If this looks messy to you, you're right. See this PR for a proposal to split DVMs out into their own handler kind, give them a dedicated pubkey along with dedicated metadata and relay selections, and clean up the data model a bit.

DVM Flow

Now that we know what a DVM looks like, we can start to address how they work. My explanation below will elide some of the detail involved in NIP 90 for simplicity, so I encourage you to read the complete spec.

The basic DVM flow can be a little (very) confusing to work with, because in essence it's a request/response paradigm, but it has some additional wrinkles.

First of all, the broker for the request isn't abstracted away as is usually the case with request/response flows. Regular HTTP requests involve all kinds of work in the background - from resolving domain names to traversing routers, VPNs, and ISP infrastructure. But developers don't generally have to care about all these intermediaries.

With DVMs, on the other hand, the essential complexity of relay selection can't simply be ignored. DVMs often advertise their own relay selections, which should be used rather than a hard-coded or randomly chosen relay to ensure messages are delivered. The benefit of this is that DVMs can avoid censorship, just as users can, by choosing relays that are willing to broker their activity. DVMs can even select multiple relays to broker requests, which means that clients might receive multiple copies of the same response.

Secondly, the DVM request/response model is far more fluid than is usually the case with request/response flows. There are a set of standard practices, but the flow is flexible enough to admit exceptions to these conventions for special use cases. Here are some examples:

• Normally, clients p-tag the DVM they wish to address. But if a client isn't picky about where a response comes from, they may choose to send an open request to the network and collect responses from multiple DVMs simultaneously.
• Normally, a client creates a request before collecting responses using a subscription with an e-tag filter matching the request event. But clients may choose to skip the request step entirely and collect responses from the network that have already been created. This can be useful for computationally intensive tasks or common queries, where a single result can be re-used multiple times.
• Sometimes, a DVM may respond with a kind 7000 job status event to let clients know they're working on the request. This is particularly useful for longer-running tasks, where feedback is useful for building a responsive UX.
• There are also some details in the spec regarding monetization, parameterization, error codes, encryption, etc.

Example DVM implementation

For the purposes of this blog post, I'll keep things simple by illustrating the most common kind of DVM flow: a kind 5300 content discovery request, addressed to a particular DVM. If you're interested in other use cases, please visit data-vending-machines.org for additional documented kinds.

The basic flow looks like this:

• The DVM starts by listening for kind 5300 job requests on some relays it has selected and advertised via NIP 89 (more on that later)
• A client creates a request event of kind 5300, p-tagged with the DVM's pubkey and sends it to the DVM's relay selections.
• The DVM receives the event and processes it, issuing optional kind 7000 job status events, and eventually issuing a kind 6300 job result event (job result event kinds are always 1000 greater than the request's kind).
• The client listens to the same relays for a response, and when it comes through does whatever it wants to with it.

Here's a swimlane diagram of that flow:

To avoid massive code samples, I'm going to implement our DVM entirely using nak (backed by the power of the human mind).

The first step is to start our DVM listening for requests that it wants to respond to. Nak's default pubkey is 79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798, so we'll only listen for requests sent to nak.

nak req -k 5300 -t p=79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798

This gives us the following filter:

["REQ","nak",{"kinds":[5300],"#p":["79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798"]}]

To open a subscription to nos.lol and stream job requests, add --stream wss://nos.lol to the previous request and leave it running.

Next, open a new terminal window for our "client" and create a job request. In this case, there's nothing we need to provide as input, but we'll include it just for illustration. It's also good practice to include an expiration tag so we're not asking relays to keep our ephemeral requests forever.

nak event -k 5300 -t p=79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798 -t expiration=$(( $(date +%s) + 30 )) -t input=hello

Here's what comes out:

{
  "kind": 5300,
  "id": "0e419d0b3c5d29f86d2132a38ca29cdfb81a246e1a649cb2fe1b9ed6144ebe30",
  "pubkey": "79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798",
  "created_at": 1739407684,
  "tags": [
    ["p", "79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798"],
    ["expiration", "1739407683"],
    ["input", "hello"]
  ],
  "content": "",
  "sig": "560807548a75779a7a68c0ea73c6f097583e2807f4bb286c39931e99a4e377c0a64af664fa90f43e01ddd1de2e9405acd4e268f1bf3bc66f0ed5a866ea093966"
}

Now go ahead and publish this event by adding nos.lol to the end of your nak command. If all goes well, you should see your event pop up in your "dvm" subscription. If so, great! That's half of the flow.

Next, we'll want our client to start listening for kind 6300 responses to the request. In your "client" terminal window, run:

nak req -k 6300 -t e=<your-eventid-here> --stream nos.lol

Note that if you only want to accept responses from the specified DVM (a good policy in general to avoid spam) you would include a p tag here. I've omitted it for brevity. Also notice the k tag specifies the request kind plus 1000 - this is just a convention for what kinds requests and responses use.

Now, according to data-vending-machines.org, kind 5300 responses are supposed to put a JSON-encoded list of e-tags in the content field of the response. Weird, but ok. Stop the subscription in your "dvm" terminal and respond to your "client" with a recommendation to read my first note:

nak event -k 6300 -t e=a65665a3a4ca2c0d7b7582f4f0d073cd1c83741c25a07e98d49a43e46d258caf -c '[["e","214f5898a7b75b7f95d9e990b706758ea525fe86db54c1a28a0f418c357f9b08","wss://nos.lol/"]]' nos.lol

Here's the response event we're sending:

{
  "kind": 6300,
  "id": "bb5f38920cbca15d3c79021f7d0051e82337254a84c56e0f4182578e4025232e",
  "pubkey": "79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798",
  "created_at": 1739408411,
  "tags": [
    ["e", "a65665a3a4ca2c0d7b7582f4f0d073cd1c83741c25a07e98d49a43e46d258caf"]
  ],
  "content": "[[\"e\",\"214f5898a7b75b7f95d9e990b706758ea525fe86db54c1a28a0f418c357f9b08\",\"wss://nos.lol/\"]]",
  "sig": "a0fe2c3419c5c54cf2a6d9a2a5726b2a5b766d3c9e55d55568140979354003aacb038e90bdead43becf5956faa54e3b60ff18c0ea4d8e7dfdf0c8dd97fb24ff9"
}

Notice the e tag targets our original request.

This should result in the job result event showing up in our "client" terminal. Success!

If something isn't working, I've also create a video of the full process with some commentary which you can find here.

Note that in practice, DVMs can be much more picky about the requests they will respond to, due to implementations failing to follow Postel's law. Hopefully that will improve over time. For now, here are a few resources that are useful when working with or developing DVMs:

• dvmdash
• data-vending-machines.org
• noogle
• nostrdvm

Conclusion

I started this post by hinting that DVMs might be as fundamental as relays are to making nostr work. But (apart from the fact that we'd end up with an acronym like DVMNOSTRZ+*, which would only exascerbate the pronounciation wars (if such a thing were possible)), that's not exactly true.

DVMs have emerged as a central paradigm in the nostr world because they're a generalization of a design pattern unique to nostr's architecture - but which exists in many other places, including NIP 46 signer flows and NIP 47 wallet connect. Each of these sub-protocols works by using relays as neutral brokers for requests in order to avoid coupling services to web addresses.

This approach has all kinds of neat benefits, not least of which is allowing service providers to host their software without having to accept incoming TCP connections. But it's really an emergent property of relays, which not only are useful for brokering communication between users (aka storing events), but also brokering communication between machines.

The possibilities of this architecture have only started to emerge, so be on the lookout for new applications, and don't be afraid to experiment - just please, don't serialize json inside json 🤦‍♂️

The result of the experiment was inconclusive, but as a side effect it produced a great case study in converting a NINO into a Real Nostr App.

Introducing Roz

Roz, a friendly notary for nostr events.

To use it, simply publish an event to relay.damus.io or nos.lol, and roz will make note of it. To find out when roz first saw a given event, just ask:

curl https://roz.coracle.social/notary/cb429632ae22557d677a11149b2d0ccd72a1cf66ac55da30e3534ed1a492765d

This will return a JSON payload with a seen key indicating when roz first saw the event. How (and whether) you use this is up to you!

De-NINO-fying roz

Roz is just a proof of concept, so don't rely on it being there forever. And anyway, roz is a NINO, since it provides value to nostr (potentially), but doesn't really do things in a nostr-native way. It also hard-codes its relays, and certainly doesn't use the outbox model or sign events. But that's ok, it's a proof of concept.

A much better way to do this would be to modify roz to properly leverage nostr's capabilities, namely:

• Use nostr-native data formats (i.e., draft a new kind)
• Use relays instead of proprietary servers for data storage
• Leverage nostr identities and signatures to decouple trust from storage, and allow trusted attestations to be discovered

Luckily, this is not hard at all. In fact, I've gone ahead and drafted a PR to the NIPs repo that adds timestamp annotations to NIP 03, as an alternative to OpenTimestamps. The trade-off is that while user attestations are far less reliable than OTS proofs, they're much easier to verify, and can reach a pretty high level of reliability by combining multiple attestation sources with other forms of reputation.

In other words, instead of going nuclear and embedding your attestations into The Time Chain, you can simply ask 5-10 relays or people you trust for their attestations for a given event.

This PR isn't terribly important on its own, but it does remove one small barrier between us and trusted key rotation events (or other types of event that require establishing a verifiable chain of causality).

In this post, I will be committing the "no true Scotsman" fallacy, in which I start with the most liberal definition I can come up with, and gradually refine it until all that is left is the purest, gleamingest, most imaginary and unattainable nostr app imaginable. As I write this, I wonder if anything built yet will actually qualify. In any case, here we go.

It uses nostr

The lowest bar for what a "nostr app" might be is an app ("application" - i.e. software, not necessarily a native app of any kind) that has some nostr-specific code in it, but which doesn't take any advantage of what makes nostr distinctive as a protocol.

Examples might include a scraper of some kind which fulfills its charter by fetching data from relays (regardless of whether it validates or retains signatures). Another might be a regular web 2.0 app which provides an option to "log in with nostr" by requesting and storing the user's public key.

In either case, the fact that nostr is involved is entirely neutral. A scraper can scrape html, pdfs, jsonl, whatever data source - nostr relays are just another target. Likewise, a user's key in this scenario is treated merely as an opaque identifier, with no appreciation for the super powers it brings along.

In most cases, this kind of app only exists as a marketing ploy, or less cynically, because it wants to get in on the hype of being a "nostr app", without the developer quite understanding what that means, or having the budget to execute properly on the claim.

It leverages nostr

Some of you might be wondering, "isn't 'leverage' a synonym for 'use'?" And you would be right, but for one connotative difference. It's possible to "use" something improperly, but by definition leverage gives you a mechanical advantage that you wouldn't otherwise have. This is the second category of "nostr app".

This kind of app gets some benefit out of the nostr protocol and network, but in an entirely selfish fashion. The intention of this kind of app is not to augment the nostr network, but to augment its own UX by borrowing some nifty thing from the protocol without really contributing anything back.

Some examples might include:

• Using nostr signers to encrypt or sign data, and then store that data on a proprietary server.
• Using nostr relays as a kind of low-code backend, but using proprietary event payloads.
• Using nostr event kinds to represent data (why), but not leveraging the trustlessness that buys you.

An application in this category might even communicate to its users via nostr DMs - but this doesn't make it a "nostr app" any more than a website that emails you hot deals on herbal supplements is an "email app". These apps are purely parasitic on the nostr ecosystem.

In the long-term, that's not necessarily a bad thing. Email's ubiquity is self-reinforcing. But in the short term, this kind of "nostr app" can actually do damage to nostr's reputation by over-promising and under-delivering.

It complements nostr

Next up, we have apps that get some benefit out of nostr as above, but give back by providing a unique value proposition to nostr users as nostr users. This is a bit of a fine distinction, but for me this category is for apps which focus on solving problems that nostr isn't good at solving, leaving the nostr integration in a secondary or supporting role.

One example of this kind of app was Mutiny (RIP), which not only allowed users to sign in with nostr, but also pulled those users' social graphs so that users could send money to people they knew and trusted. Mutiny was doing a great job of leveraging nostr, as well as providing value to users with nostr identities - but it was still primarily a bitcoin wallet, not a "nostr app" in the purest sense.

Other examples are things like Nostr Nests and Zap.stream, whose core value proposition is streaming video or audio content. Both make great use of nostr identities, data formats, and relays, but they're primarily streaming apps. A good litmus test for things like this is: if you got rid of nostr, would it be the same product (even if inferior in certain ways)?

A similar category is infrastructure providers that benefit nostr by their existence (and may in fact be targeted explicitly at nostr users), but do things in a centralized, old-web way; for example: media hosts, DNS registrars, hosting providers, and CDNs.

To be clear here, I'm not casting aspersions (I don't even know what those are, or where to buy them). All the apps mentioned above use nostr to great effect, and are a real benefit to nostr users. But they are not True Scotsmen.

It embodies nostr

Ok, here we go. This is the crème de la crème, the top du top, the meilleur du meilleur, the bee's knees. The purest, holiest, most chaste category of nostr app out there. The apps which are, indeed, nostr indigitate.

This category of nostr app (see, no quotes this time) can be defined by the converse of the previous category. If nostr was removed from this type of application, would it be impossible to create the same product?

To tease this apart a bit, apps that leverage the technical aspects of nostr are dependent on nostr the protocol, while apps that benefit nostr exclusively via network effect are integrated into nostr the network. An app that does both things is working in symbiosis with nostr as a whole.

An app that embraces both nostr's protocol and its network becomes an organic extension of every other nostr app out there, multiplying both its competitive moat and its contribution to the ecosystem:

• In contrast to apps that only borrow from nostr on the technical level but continue to operate in their own silos, an application integrated into the nostr network comes pre-packaged with existing users, and is able to provide more value to those users because of other nostr products. On nostr, it's a good thing to advertise your competitors.
• In contrast to apps that only market themselves to nostr users without building out a deep integration on the protocol level, a deeply integrated app becomes an asset to every other nostr app by becoming an organic extension of them through interoperability. This results in increased traffic to the app as other developers and users refer people to it instead of solving their problem on their own. This is the "micro-apps" utopia we've all been waiting for.

Credible exit doesn't matter if there aren't alternative services. Interoperability is pointless if other applications don't offer something your app doesn't. Marketing to nostr users doesn't matter if you don't augment their agency as nostr users.

If I had to choose a single NIP that represents the mindset behind this kind of app, it would be NIP 89 A.K.A. "Recommended Application Handlers", which states:

Nostr's discoverability and transparent event interaction is one of its most interesting/novel mechanics. This NIP provides a simple way for clients to discover applications that handle events of a specific kind to ensure smooth cross-client and cross-kind interactions.

These handlers are the glue that holds nostr apps together. A single event, signed by the developer of an application (or by the application's own account) tells anyone who wants to know 1. what event kinds the app supports, 2. how to link to the app (if it's a client), and (if the pubkey also publishes a kind 10002), 3. which relays the app prefers.

As a sidenote, NIP 89 is currently focused more on clients, leaving DVMs, relays, signers, etc somewhat out in the cold. Updating 89 to include tailored listings for each kind of supporting app would be a huge improvement to the protocol. This, plus a good front end for navigating these listings (sorry nostrapp.link, close but no cigar) would obviate the evil centralized websites that curate apps based on arbitrary criteria.

Examples of this kind of app obviously include many kind 1 clients, as well as clients that attempt to bring the benefits of the nostr protocol and network to new use cases - whether long form content, video, image posts, music, emojis, recipes, project management, or any other "content type".

To drill down into one example, let's think for a moment about forms. What's so great about a forms app that is built on nostr? Well,

• There is a spec for forms and responses, which means that...
• Multiple clients can implement the same data format, allowing for credible exit and user choice, even of...
• Other products not focused on forms, which can still view, respond to, or embed forms, and which can send their users via NIP 89 to a client that does...
• Cryptographically sign forms and responses, which means they are self-authenticating and can be sent to...
• Multiple relays, which reduces the amount of trust necessary to be confident results haven't been deliberately "lost".

Show me a forms product that does all of those things, and isn't built on nostr. You can't, because it doesn't exist. Meanwhile, there are plenty of image hosts with APIs, streaming services, and bitcoin wallets which have basically the same levels of censorship resistance, interoperability, and network effect as if they weren't built on nostr.

It supports nostr

Notice I haven't said anything about whether relays, signers, blossom servers, software libraries, DVMs, and the accumulated addenda of the nostr ecosystem are nostr apps. Well, they are (usually).

This is the category of nostr app that gets none of the credit for doing all of the work. There's no question that they qualify as beautiful nostrcorns, because their value propositions are entirely meaningless outside of the context of nostr. Who needs a signer if you don't have a cryptographic identity you need to protect? DVMs are literally impossible to use without relays. How are you going to find the blossom server that will serve a given hash if you don't know which servers the publishing user has selected to store their content?

In addition to being entirely contextualized by nostr architecture, this type of nostr app is valuable because it does things "the nostr way". By that I mean that they don't simply try to replicate existing internet functionality into a nostr context; instead, they create entirely new ways of putting the basic building blocks of the internet back together.

A great example of this is how Nostr Connect, Nostr Wallet Connect, and DVMs all use relays as brokers, which allows service providers to avoid having to accept incoming network connections. This opens up really interesting possibilities all on its own.

So while I might hesitate to call many of these things "apps", they are certainly "nostr".

Appendix: it smells like a NINO

So, let's say you've created an app, but when you show it to people they politely smile, nod, and call it a NINO (Nostr In Name Only). What's a hacker to do? Well, here's your handy-dandy guide on how to wash that NINO stench off and Become a Nostr.

You app might be a NINO if:

• There's no NIP for your data format (or you're abusing NIP 78, 32, etc by inventing a sub-protocol inside an existing event kind)
• There's a NIP, but no one knows about it because it's in a text file on your hard drive (or buried in your project's repository)
• Your NIP imposes an incompatible/centralized/legacy web paradigm onto nostr
• Your NIP relies on trusted third (or first) parties
• There's only one implementation of your NIP (yours)
• Your core value proposition doesn't depend on relays, events, or nostr identities
• One or more relay urls are hard-coded into the source code
• Your app depends on a specific relay implementation to work (ahem, relay29)
• You don't validate event signatures
• You don't publish events to relays you don't control
• You don't read events from relays you don't control
• You use legacy web services to solve problems, rather than nostr-native solutions
• You use nostr-native solutions, but you've hardcoded their pubkeys or URLs into your app
• You don't use NIP 89 to discover clients and services
• You haven't published a NIP 89 listing for your app
• You don't leverage your users' web of trust for filtering out spam
• You don't respect your users' mute lists
• You try to "own" your users' data

Now let me just re-iterate - it's ok to be a NINO. We need NINOs, because nostr can't (and shouldn't) tackle every problem. You just need to decide whether your app, as a NINO, is actually contributing to the nostr ecosystem, or whether you're just using buzzwords to whitewash a legacy web software product.

If you're in the former camp, great! If you're in the latter, what are you waiting for? Only you can fix your NINO problem. And there are lots of ways to do this, depending on your own unique situation:

• Drop nostr support if it's not doing anyone any good. If you want to build a normal company and make some money, that's perfectly fine.
• Build out your nostr integration - start taking advantage of webs of trust, self-authenticating data, event handlers, etc.
• Work around the problem. Think you need a special relay feature for your app to work? Guess again. Consider encryption, AUTH, DVMs, or better data formats.
• Think your idea is a good one? Talk to other devs or open a PR to the nips repo. No one can adopt your NIP if they don't know about it.
• Keep going. It can sometimes be hard to distinguish a research project from a NINO. New ideas have to be built out before they can be fully appreciated.
• Listen to advice. Nostr developers are friendly and happy to help. If you're not sure why you're getting traction, ask!

I sincerely hope this article is useful for all of you out there in NINO land. Maybe this made you feel better about not passing the totally optional nostr app purity test. Or maybe it gave you some actionable next steps towards making a great NINON (Nostr In Not Only Name) app. In either case, GM and PV.

This of course means that I broke compatibility with all the NIP 29 group clients out there, but I did it for good reasons. In this post I hope to explain those reasons, and speculate on how best to move forward with "relay-based" groups on nostr.

To give you some quick context, NIP 29 and my approach to groups (which I'll dub "relays-as-groups" for clarity) are very similar, with a fundamental difference. Both have chat, join requests, group metadata, membership, moderation, etc. However, the basic unit of a NIP 29 group is a random group id string, while the basic unit of a Flotilla group is a relay itself.

I believe this design difference emerged in part because of what we were attempting to build. NIP 29 groups tend to be more telegram-like, where groups function more like chat rooms, and users join each one individually. On flotilla, groups function more like Discord servers, or Slack workspaces, and users join an entire group of chat rooms at once.

"Relays as groups" has four major advantages over "groups hosted by relays".

Decentralization

First, using relays as groups supports decentralization better than hosting user-managed groups on relays.

(To be clear here, I'm not referring to client-managed groups, which is a whole different approach that we've experimented with in the past, both with NIP 72 communities and with NIP 87 encrypted groups. Both NIP 29 and relays-as-groups solve many of the consistency problems associated with attempting to have a linear, synchronous conversation across multiple relays. While both alternatives have a story for migrating or mirroring a group, both rely heavily on the host relay to not censor user messages.)

I know what you're thinking. How can reducing the number of relays improve decentralization? Relays were originally introduced in order to create redundancy and spread trust across many actors, creating censorship resistance. This model was difficult for many bitcoiners to wrap their heads around, because it's a very different kind of decentralization than what a blockchain delivers.

Communities are an essentially different use case from a broadcast network where content is delivered based on author or recipient relay selections. Every message to a community would have to be delivered to all members of the community. Sending each message to all members' personal inbox relays just doesn't make sense; there has to be some other inbox for the community to work off of.

Additionally, online communities almost always have moderators and admins. This is even true of very public, open types of communities, like subreddits. The only type of community that doesn't have moderators is one that emerges naturally from social clustering. And even in that case there is loose consensus about who is in and who isn't, based on the actions/follows/mutes of the participants, whether these clusters are huge or tiny. Socially-emergent groups are served well by chat applications or broadcast networks.

But the subset of online communities that do prefer to confer moderator status on certain members are essentially centralized. In other words, centralization and control is a feature, not a bug.

Now, that doesn't mean there don't need to be considerations around credible exit and removing/adding moderators over time. But the fact is that moderator-led communities are always under the oversight of the moderators at any given time, even if the identity of those moderators changes and their power is limited.

What this implies is that decentralization for moderator-led groups looks very different from decentralization for a broadcast network. There is nothing at all wrong with giving moderators full control over the group's communications (qua the group; DMs and public broadcast content between group members should happen outside the group's infrastructure, just as people also exist outside the communities they are a part of). What is important is that no one has control over groups that they aren't nominally the admin of.

In concrete terms, what this all means is that community moderators should self-host their infrastructure. This is the same principle as motivates self-custody and home servers, but applied to communities. If community leaders manage their own relays, this means that no hosting company or relay admin can de-platform their community. Centralization of network infrastructure in this case aligns with the trust structure of the group.

Applying this to our group dilemma, it's easy to see that NIP 29 groups are more vulnerable to censorship or data harvesting attacks by malicious relay admins, since many unrelated groups might live on a single relay. In contrast, if you treat relays as groups themselves, every group is forced to live on a separate relay, spreading risk across more hosting providers.

Now, this doesn't necessarily mean that many "relays" aren't "virtual relays" managed by the same hosting provider. So I'll admit that even "relay-based" groups don't completely solve this problem. But I think it will tend to nudge community organizers toward thinking about community infrastructure in a more self-sovereign (or community-sovereign) way.

Investment in Relays

While both NIP 29 and relays-as-groups rely heavily on relays to implement the features that support each specification, there's an important difference between the feature sets. In NIP 29, relay support is specific only to groups, and isn't applicable to other use cases. In contrast, every protocol feature added to support the "relays as groups" can be re-purposed for other types of relays.

Take join requests for example. NIP 29's kind 9021 events allow users to request access to a group, and that's all. Kind 28934 join requests on the other hand allow users to request access to relays. Which in the relays-as-groups model means group access, but it also means custom feed access, inbox relay access, maybe even blossom server access. In fact, kind 28934 was originally proposed at the beginning of this year in order to support a different version of hosted groups, but remains as relevant as it ever was despite iteration on groups.

The orthogonality of features added to relays to any specific use case will long-term result in simpler specs, and more interesting relay-based use cases being possible. Join requests are only one example. The same is true of 1984-based moderation, the proposed LIMITS command, AUTH, NIP 11 relay metadata, etc.

We already have web of trust relays, feed relays, archival relays, and many more. Being able to request access to closed versions of these is useful. Being able to signal federation between multiple instances of these, run by different people, is useful. And of course, relay metadata, reports, and LIMITS are self-evidently useful for normal relays, since they pre-date Flotilla.

I've always said that relays are some of the coolest and most under-appreciated parts of nostr. This doesn't mean that we should add every possible feature to them, but features related to data curation and access control fit really well with what relays are good for. For more on the role of relays and what features should be added to them, see my nostrasia talk Functional Relays.

Declarative vs Imperative

A common paradigm in programming is that of declarative vs imperative programming. Imperative programming focuses on "how" to achieve a given result, leaving "what" the code is doing to be inferred by the programmer. Declarative programming instead focuses on the "what", and allows some underlying implementation to solve the how. A good balance between these paradigms (and knowing when to use one over the other) allows programmers to work faster, make fewer mistakes, and produce less code.

Another way to look at this is that a specification should contain as much ambiguity as possible, but without compromising the system attributes the specification is supposed to guarantee. It can get complex when figuring out what attributes are core to the specification, since sometimes the "how" does actually matter a lot.

However, nostr in particular falls pretty far along the "declarative" end of this spectrum because of its decentralized nature. The only person who can say anything with any authority is the person who signs an event. This event is a "declaration", and any effects it has are necessarily up to the relays, clients, and people interpreting the event. However, what others do with an event is an expectation that must be taken into account by the publisher, forming a feedback loop. This dialectic is what creates stability in the protocol.

In more concrete terms, no one can "tell" anyone else what they have to do by publishing an event like you might in a traditional, centralized RPC-type system. Any event whose semantics are a "command" rather than a "fact" or "request" is broken unless the counter party is fully committed to carrying out the command. An example of a "command" scenario on nostr is NIP 46 remote signing, in which the bunker is the agent of the user making the request. If the bunker implementation fails to carry out a valid command initiated by the user, its interpretation of that event is objectively incorrect.

NIP 29 applies this same paradigm to relays, particularly in the area of moderation, membership edits, and group metadata. In other words, there are several "commands" which instruct the relay to do something.

This isn't necessarily a bad thing, but it does increase the number of things the interface between the client and relay have to agree on. A regular relay may accept an add-user request, but then do nothing with it, violating the contract it has implicitly accepted with the user. The solution to this is feature detection, which is a whole other API to be specified and implemented.

My ideal solution to this problem is to shift the semantics of events away from "commands" to "facts" - in other words, to make the interface more declarative.

In fact, we already have an interface for moderation that works like this. Many clients support kind 1984 "report" events. Users sending these reports have no expectations about how they will be used. They are a "fact", a declaration of opinion with certain semantics. Other actors in the network may choose whether or not to pay attention to these.

This same model is easily applied to communities. Without having to implement any feature detection (either for the relay's implementation, or for the user's role on that relay), anyone can simply send a "report". This goes into the black hole of the relay, and may subsequently be ignored, broadcasted, or acted on.

The really nice thing about this model is that because there is no expectation for "how" reports are to be interpreted, any approach to moderation can be used depending on relay policy or client implementation. In NIP 29, if you issue a delete-event, it either happens or it doesn't and if it doesn't, you have to explain the failure to the user somehow.

In the relays-as-groups model, e-tagging an event in a kind 1984 requires no user feedback, and therefore it can be interpreted however the relay prefer. This can result in insta-banning, manual review, thresholds based on number of reporters, a leaky-bucket social score algorithm, shadow banning, temporary banning, soft-moderation by allowing clients to request reports and respond to them by changing user interface elements, or anything else you can think of.

The reason I think this is important is that community moderation is a very hard problem, and baking certain semantics into the specification can result in the complete failure of the spec. NIP 72 should be considered an example of what not to do. Some NIP 72 communities have survived due to the dedication of the moderators, but many more have failed because of the rigid moderation model. We should try not to make the same mistake again.

Conclusion

Now, having said all that, I think there is actually a lot of value to NIP 29. What finally clicked for me this week after releasing Flotilla is that the two approaches are actually complementary to one another. One of the most common feature requests I've already heard for flotilla is to have more complete support for rooms, which are currently implemented as not much more than hashtags. Better rooms (i.e., "nested groups") would require: authentication, membership, moderation, and pretty much everything else that exists for the top-level group.

As much as I believe the relays-as-groups approach is superior to NIP 29 for top-level groups, it doesn't make any sense to try to "nest" relays to create sub-groups. Something like NIP 29 is needed in order to fully support rooms anyway, so I think the convergence of the two approaches is all but inevitable. In fact, fiatjaf has already merged a PR which will allow me to use the same event kinds in flotilla as exist already in NIP 29 clients.

There are just a few more changes that are necessary in order for me to fully adopt NIP 29 in Flotilla:

• NIP 29 feature detection
• Opaque ids for unmanaged groups prevent unmanaged groups from having human-readable names
• We need a mechanism for building membership lists without relay support
• Better handling for 9021 group join requests

I've opened PRs for each of these (linked above). Hopefully we can work through these issues and combine our powers to become the Captain Planet of group implementations.

Just to state the obvious, and to kick off this survey of nostr key management options, let me just state that asking users to paste their nsec into your app is a bad idea. However good your intentions, this opens your users up to all kinds of attack vectors, including clipboard hijacking attacks, exposing keys to insecure communication channels, exposing keys to many different apps, supply chain attacks, XSS attacks, and yes, bugs that cause your software to send keys to analytics or error reporting backends.

The era of nsec-pasting is over.

I've committed to embracing the pain and removing nsec login from Coracle, and I encourage other devs to do the same. The sooner we treat key management with the urgency it deserves, the sooner we can come up with a secure and convenient key management solution.

As an aside, ncryptsec is a great innovation for securely transporting keys, but it still doesn't protect against exposure to apps that need to use keys. It has its place though; in fact I'm of the opinion that nsec and seed words should be deprecated, and support for them should be removed. Giving friendly names and human-readable representations to data that is essentially private is a really bad idea (unless you're memorizing your key). But I digress.

Signer Comparisons

Let's go through a few existing options for key management, and compare their relative merits. I've tried to list them in the order they appeared on the scene, which also helps to clarify the logic of how signers have evolved. Throughout, I will be focusing on what kinds of user experience each approach unlocks for non-technical users, since my goal is to build products that work for regular people.

Extension Signers

The first signer application (that I know of) was nos2x, by fiatjaf. As I understand it, this was a proof-of-concept of how users might protect their keys without releasing custody of them. And it works really well! In fact, even though there have been many forks and imitators, I still use nos2x when using nostr on my desktop browser.

Extension signers offer a great user experience, along a narrow happy path. Setting up a browser extension is a relatively familiar process for normal users, and once it's done you don't really have to think about it. In theory, extensions can also include their own onboarding process and key backup strategies as well, allowing users to get started in a single place. Plus, there's very little latency involved in making calls to the signer extension.

This positive experience breaks down quickly though once a user wants to use a desktop or mobile application. When this happens, users have to start over essentially from scratch. Nothing they did to set up the extension helps them move to another signer application.

While it's technically possible to use extension signers on mobile via e.g. the Kiwi browser, this doesn't work for native apps or apps installed as PWAs. Instead, you either have to revert to pasting keys, or use some other solution.

One slight permutation of extension signers is browser signers, like Spring. Instead of adding a signer to your browser, Spring allows you to install a browser that holds your keys and allows you to use any nostr web application. But this has all the same basic limitations that extension signers do.

Hardware Signers

Hardware signers came around shortly after extension signers. I'm not going to spend much time talking about them here, because although they're about as far along the spectrum towards security as you can go, they're also not very convenient. Non-technical users aren't going to onboard by buying (or building) a device which they have to connect to their desktop via USB whenever they want to sign a message. Hardware signers have their place, but onboarding isn't it.

The only hardware signer I'm aware of (although I'm sure I've heard of others) is from LNBits, and is usually used via a browser extension like horse. This of course means that it has all the same limitations that browser extensions have, and then some (although mobile and desktop apps would likely be able to find a way to talk directly to the signer).

Hosted Signers

Remote signers (aka "bunkers") use the Nostr Connect protocol (also known as NIP 46) for remote signing.

Hosted signers in particular are one example of a NIP 46 remote signer, which lives on "somebody else's computer". Because they use a legacy web architecture, they can be built to be very familiar and convenient to users. It's trivial to build a hosted signer that offers email/password login along with 2FA, password resets, session revokation, the whole shebang. But they have one fatal flaw, which is that they are custodial. This means that not only do users have to relinquish exclusive control over their keys, but hosted signers also can become a target for hackers.

Desktop Signers

Several projects exist which allow users to run their own bunker, on their own hardware. These include nostr clients like Gossip, as well as command-line utilities like nak. This approach is mostly an improvement over extension signers, because it widens the scope of applications that can conveniently access the signer from those that run in the browser to those that run on the desktop computer the signer lives on. The downside is that they have to communicate via relays, which either introduces latency or requires an additional component to be running locally.

While it's technically possible to use desktop signers to log in on other computers or mobile apps, I don't think that's going to be very practical for most people. Mobile apps by definition are more portable than regular computers. Anyone who wants to access their nostr account on more than one device will have to either set up separate solutions, or go with another kind of remote signer. This isn't a huge obstacle for people highly invested in nostr, but it's a significant amount of friction for a new user.

Mobile Signers

Mobile signers solve the problem introduced by desktop signers of not always having access to your signer (or of your signer not having access to you, due to being powered down or disconnected from the internet). Mobile devices are generally more available than desktop devices, and also have better push notifications. This means that users can approve signer requests from any device as easily as tapping a notification.

Mobile signers on Android can also upgrade their UX by taking advantage of NIP 55 to avoid the round trip to relays, reducing latency and making it possible to sign things offline. Amber has been a pioneer in this area, and other projects like Nostrum and Alby's nostr-signer have been prototyped.

To date, there unfortunately haven't been any signer applications released for iOS, which leaves the mobile signer story incomplete. In my opinion, this is probably the most promising solution for end users, although it's currently under-developed.

Web Signers

One interesting alternative that combines the benefits of hosted, desktop, and mobile wallets is nsec.app. This is a web application frontend which keeps keys in the browser, so that they are never shared with a third party. Using web push notifications and a healthy sprinkle of black magic, nsec.app is able to respond to signer requests by opening itself in a browser window.

This works generally pretty well for desktop web applications, less well on android, still less well for android PWAs, and (to my understanding) not at all on iOS. Artur from nostr.band is working on these problems using a variety of approaches, one of which is embedding nsec.app in an iframe and communicating using postMessage.

This approach also makes it possible to sync keys between your phone and desktop, simulating a hosted UX by making them accessible from either location by signing in to nsec.app. This is done by encrypting user keys and storing them on the nsec.app server. In theory this should be secure, but it's something to consider.

I'm cautiously optimistic about this approach. If successful, it would enable a single brand to exist on every platform, which is important to reduce unnecessary configuration and cognitive overhead for users.

Multisig Signers

Another experimental approach is multi-sig. Promenade is a project by fiatjaf exploring this possibility. This would allow users to split their keys across different custodians and require all (or some majority of them) to approve an event signature before it would be valid.

The downsides of this are an increase in complexity (more moving parts for users to deal with) and latency (more parties to coordinate with to sign events). I'm also not clear on whether encryption is possible using multi-signature keys. If not, that would preclude not only existing direct messages (which will hopefully end up on MLS eventually anyway), but also things like private lists, mutes, and application settings. I think multi-signature signers are promising, but are definitely a long-term project.

Self-Hosted Signers

Coming nearly full circle, self-hosted signers are a special case of hosted signers, but, you know, self-hosted. These signers might live on a home server like a Start9 and be accessible for signer request approvals via tor, or they might live on a server run by the user (or an Uncle Jim). This would be an extremely convenient approach for anyone willing to deal with the complexities of hosting the infrastructure.

A good candidate for NIP 46 support might be AlbyHub, which is already one of the easiest self-hosted wallets to set up and use. Adding signer suppport to AlbyHub would allow users to have their wallet and nostr keys stored in the same place, and accessible anywhere either via the web interface or via AlbyGo.

Omniplatform Signers

This leads me to, finally, "omniplatform" signers. This isn't really a new architecture, but a combination of several. User choice is great, but nostr has a very tight complexity budget when onboarding new users. If a brand can manage to get new users set up with a very simple but sub-optimal solution, then grow them into a more complete integration into the nostr ecosystem, that would be a huge win.

I think Alby has a great shot at doing this, if it's something they want to prioritize. Bitwarden would also be a great candidate, since they already have apps on every platform, as well as a self-hosted option (Vaultwarden). If users could start with a mobile app, and incrementally set up a browser extension, self-hosted vault, and hardware signer as needed, that I think would be an ideal path.

Nostr Connect: broken, but promising

If you can't tell from the above comparison, I'm partial to NIP 46 as the best, most flexible way to build high-quality user experiences. Remote key management means a reduction in moving keys, hosting keys, and software installation and administration. If we can get users to the point where their keys live in only two places (their password manager and their signer), we'll be doing good.

There are however many ways to implement NIP 46. Implementing all of them in a single client (or signer) would be burdensome for developers, and introduce a lot of UI complexity to users. Here's a quick survey of flows that currently exist.

Signer -> Client

The simplest way to connect a client and a bunker is for a user to explicitly authorize the connection by copying a bunker:// URL from their signer application to their client. This allows the bunker to generate and validate a secret embedded in the URL without the client having to do anything other than pass it along in the initial connect request.

This is a great UX for people who know what they're doing, but isn't at all friendly to newcomers. Someone signing in for the first time isn't going to know what a bunker link is, and even if they do they're immediately confronted with the problem of picking a signer, setting it up, and finding out where in that app they can obtain a bunker link. This can be marginally smoothed out using things like protocol handlers and QR codes, but these won't apply in all (or even most) cases.

Client -> Signer

The reverse flow is similar. This relies on the user to explicitly authorize the connection by copying a nostrconnect:// url from the client into the signer app. In technical terms, this requires one fewer step, since in NIP 46 the connection is always initiated by the client. In this case, the pasting of the URL replaces the connect request. The client listens for a response containing a client-generated secret embedded in the nostrconnect:// url. This step isn't currently supported by all signer apps, some of which return an ack instead. This can result in session hijacking, where an attacker can intercept signing requests (although they can't do anything that would require the user's key, like decrypting messages).

While at first glance nostrconnect seems functionally identical to bunker links, the UX has the potential to be much better. The reason for this has to do with how people use which devices, and where a client or signer application is most likely to be run. This requires making some assumptions, but in my mind the most common scenario is that a user will want to host their signer on their phone, since that is the device that is most universally available for authorizations (apart from an always-online hosted signer on the open internet). In other words, users generally have their phones with them when they're using their computer, but often don't have a desktop available when using their phone. This idea is validated by (for example) the prevalence of SMS-based 2FA, which assumes the presence of a phone.

Assuming the signer is on the user's phone, QR-scan flows for client authorization make a lot more sense if the client is the one generating the link, since they can simply scan a code generated on another device with their camera, or copy/paste or use a protocol handler for a client on the same device. In contrast, when using a bunker link users might find themselves in the awkward position of having to copy a link from their phone to their desktop. Whether this is done via QR code or by sending yourself a link via DM/text/email, it's an awkward flow most people aren't really prepared for.

Auto-Connect

Some enhancements have been made to the bunker flow which allow clients to send an initial connect request without asking the user to copy links between apps. These allow clients to do away with opaque magic strings entirely and provide the idealized "just one click" flow. However, after trying to make this flow work over the course of a couple weeks, I've come to the opinion that the additional complexity involved in automating the flow just isn't worth it.

There are a few variants of this "auto-connect" flow:

• Signer NIP-05: Signers can register a NIP 05 address for a user's pubkey on their domain, allowing users to enter their address rather than their pubkey on login. Unfortunately, this address has no relation to their actual NIP 05 address, which can result in a lot of confusion.
• User NIP-05: To solve this problem, fiatjaf has proposed a new version which allows users to enter their own NIP 05 in at login instead of the one provided by the signer. The client would then look up the user's 10046 event and follow the signer pubkey listed there.
• Nostrconnect handler: Signers may publish a NIP 89 handler which includes a handler url that clients can send nostrconnect urls to. This isn't currently specified anywhere, but it is supported by nsec.app. This bypasses the NIP 05 address requirement entirely, allowing users to simply pick a signer and click a button.

Each of these flows have their own strengths and weaknesses, but all of them share a dependency on some external source of truth for routing a user to the correct bunker.

In the first case, this is done by remembering the NIP 05 address assigned by the signer, which relies on DNS and on users to not forget which address they're using.

In the second case, this is done by relying on the user having done a significant amount of configuration (setting up a NIP 05, adding it to their kind 0, and having published a 10046 event) which may or may not exist. This forces clients to gracefully degrade to some alternative login method anyway, and adds UX friction since users have to choose which interface will work for them.

The final method bypasses the need for users to remember their NIP 05 address, but it does require either the client or the user to select a trusted signer. If poorly implemented, this could result in users choosing an untrustworthy signer on signup (risking their keys), or the wrong signer on login resulting in a broken session.

For all these reasons, I've opted to go with the vanilla bunker/nostrconnect flow, which allows me to display a simple interface to users. Presenting a QR code without comment assumes that users know what to do with it, but the benefit is that it makes explicit the signer selection step which the auto-connect flows try to paper over. This is actually a good thing, because instead of using heuristics like addresses or lists of signers presented by a client to make the decision, users can choose based on which app they actually have installed, which is a richer mnemonic device.

Making NIP 46 Work

The bottom line here is that while NIP 46 is the best baseline for signer support, it doesn't currently work very well at all. There are a variety of reasons for this:

• The specification itself isn't clear, and is constantly changing. This leads to incompatibilities between apps and signers (or explosive complexity in trying to handle every case).
• Extensions to the basic bunker flow (both in terms of signer implementation and signer discovery) are worth researching, but each one creates another dimension of possible incompatibility. Signers will be incentivized to support every possible login flow, creating complexity for users and increasing attack surface area. Clients will have to implement fallbacks to their preferred signup flows, again resulting in UX complexity.
• Clients don't currently deal well with latency. In order for NIP 46 to work smoothly, clients will have to implement better loading, debouncing, optimistic updates, publish status, and "undo". There are downsides to this, but many of these features endu up being built by mature software products anyway, so supporting these patterns may actually improve rather than degrade UX.
• There's currently no easy and secure way for users to store keys in a single signer which they can access anywhere. This means that users have to set up multiple bunkers depending where they're sitting, or resort to alternative login methods like NIP 07 or 55. These are great upgrades, since they reduce latency and bandwidth use, but shouldn't be required for new users to learn.
• There's no unified experience across platforms. If a user signs up on their desktop, how do they safely transfer their keys to their Android signer app? If they're given seed words, how can they import them as an nsec? Consensus on best practices would be an improvement, but I think only a unified UX across platforms for a single signer can really solve this.
• As nice as it might be to bypass app stores and built-in push notifications, shunning traditional platforms drastically increases the friction for users. To my knowledge, no signer app currently exists in traditional app stores, or supports built-in push notifications. If we want nostr to be accessible to non-technical folks, we can't ask them to start by downloading Obtanium or zap.store and a UnifiedPush distributor for their platform.

As I mentioned above, I don't think NIP 46 will ever be the only solution for signers. But I do think it's a great baseline on which to build a kind of "progressive enhancement" approach. For example, clients should support at least nostrconnect/bunker links, and encourage users once they've logged in to upgrade to NIP 55 or NIP 07 signers. Signers should exist in the mainstream app store and use native push notifications, with an option to install elsewhere or opt-in to UnifiedPush.

The goal here is to balance user experience and security. The number one rule for this is to reduce attack vectors for obtaining user keys. This points to (ideally) a single non-custodial signer, easily accessible to the user, and a simple protocol for using that signer from any app. Progressive enhancement is fine, but we should always be able to fall back to this baseline.

In order to do this, users must be able to articulate "what" they are looking for, and clients must be able to articulate "how" to find that thing. This "how" is divided into two parts: building a request that will match the desired content (very easy), and selecting a relay that is able to serve that content to the user requesting it (very very hard).

Why guessing isn't good enough

As a concrete example, let's say the user wants to find everyone in their "network" who is using a particular topic. The process would look something like this:

1. The user clicks the "network" tab and types in the topic they want to browse. This is the "what".
2. The client then translates the term "network" to a list of public keys using whatever definition they prefer (Follows? WoT? Grapevine?), and builds a filter that might look something like this: [{"authors": pubkeys, "#t": ["mytopic"]}]. Any relay will happily accept, understand, and respond to that filter.
3. The client then has to decide which relays it should send that filter to. This is the ??? stage of the outbox model, which immediately precedes:
4. Profit

It may not be immediately obvious why selecting the correct relays might be difficult. Most people post to relay.damus.io, and most people read from relay.damus.io, so in most cases you should be good, right?

This approach to relay selection has historically worked "well enough", but it depends on a flawed definition of success. If you only want to find 90% of the content that matches your query, using the top 10 relays will suffice. But nostr is intended to be censorship-resistant. What if those 10 hubs have banned a particular public key? Nostr clients should (at least in theory) be 100% successful in retrieving requested content. Even if someone only posts to their self-hosted relay, you should be able to find their notes if their account is set up properly.

A naive solution to fixing the FOMO

A 90% hit rate results in a feeling of flakiness, even if users aren't completely aware of what isn't working. Feeds will be incomplete, quoted notes will be missing, replies will be orphaned, user profiles won't load. The natural response to the FOMO this creates is for users to "try harder" by adding more relays.

On the read side, this means clients open more connections, resulting in much higher data transfer requirements, with massively diminishing returns, since there's no reason to expect that a randomly chosen relay will have a substantially different data set.

One the publish side, this means that clients end up publishing more copies of their data to more relays. This approach has been automated in the past by services like Blastr, which don't store a copy of events published to the relay, but instead forward events to the top 300 relays in the network. This results in a two-orders-of-magnitude increase in storage required, and only makes the read side of the problem worse, since it reduces the uniqueness of the data set each relay stores. This in turn means that more duplicates are retrieved when querying relays.

Both halves of this approach are equivalent to guessing. On the read side, users are guessing which relays will have any arbitrary content they might ask for in the future. On the write side, users are guessing which relays other people might use to find their notes. It is a brute-force method for finding content.

Randomness results in centralization

In theory, random relay selection would result in a perfect distribution of content across all relays in the network. But in practice, this method of selection isn't random at all, but is strongly influenced by user bias in what constitutes a "good" relay. While some users may check nostr.watch for ping times, geographical proximity, or uptime, most will choose relays based on familiar names or other people's recommendations.

In either case, these biases are entirely orthogonal to achieving a higher content retrieval hit rate, except when bias in relay selection results in clustering — i.e., centralization. In other words, the kind of randomness exhibited by users when selecting relays actually results in pretty much everyone picking the same few relays. We see this same effect when people try to come up with passwords or seed phrases — human-provided randomness is anything but random.

Clustering improves the hit rate when requesting events (slightly), but it results in nearly as much centralization as if only a single relay was used — and a lot more duplicate events.

Something (anything) other than randomness

In early 2023, Mike Dilger introduced NIP 65 (now known as the "Outbox Model") with a problem statement in the spirit of the original description of nostr: "Nostr should scale better. People should be able to find what they want."

Historical note: NIP 65 was formerly known as the "Gossip Model", derived from the name of Mike's desktop nostr client, called "Gossip". This unfortunately created a lot of confusion, since gossip protocols work very differently from how nostr tends to work, hence the re-brand.

Before NIP 65, an informal standard existed in which kind 3 user contact lists also included a list of relays that clients could use as something similar to Mastodon's "home servers". This list included the option to only read or write from a given relay. Unfortunately, it wasn't really clear what the semantics of this relay list were, so different clients handled them differently (and many clients ignored them). Usually this amounted to user-provided static relay configurations, which resulted in the naive relay selection approach described above.

NIP 65 used a very similar format (a list of relay urls with optional "read" or "write" directives), but with a very important semantic difference: relays listed in a user's kind 10002 were intended to "advertise to others, not for configuring one's client." In other words, these relay selections were intended as a signal to other users that they should use certain relays when attempting to communicate with the author of the relay list.

I highly recommend reading the entire NIP, which is very short and easy to read. But the mechanics of the spec are very simple:

When seeking events from a user, Clients SHOULD use the WRITE relays of the user's kind:10002.

When seeking events about a user, where the user was tagged, Clients SHOULD use the READ relays of the user's kind:10002.

When broadcasting an event, Clients SHOULD:

• Broadcast the event to the WRITE relays of the author
• Broadcast the event to all READ relays of each tagged user

For the first time, we had a way to differentiate relays in terms of what content could be found where.

When looking for a note by a particular user, a client could now look up the author's write relays according to their kind 10002 event, and send its query there. The result is a much higher hit rate with much lower data transfer requirements, and fewer connections per query.

Making Outbox Work

There are of course some assumptions required to make this work.

First, the user must know which author they're looking for. This isn't always true when looking up a quote or parent note, but context and pubkey hints solve this difficulty in most cases.

The author must also publish a kind 10002 event. This may not always be the case, but clients should prompt users to set up their relay list correctly. This isn't really a flaw in the Outbox Model, just in implementations of it.

Additionally, the user's client must be able to find the author's kind 10002 event. This is the "bootstrapping" phase of the Outbox Model, during which the mechanisms the system provides for finding events aren't available. This requires us to fall back to randomly guessing which relays have the content we're looking for, which as we saw above doesn't work very well.

Other than guessing, there are a few different ways a client might find the relay selection event in question, each of which is applicable in different circumstances. In most cases, using one of a handful of indexer relays like purplepag.es or relay.nostr.band is a simple and efficient way to find user profiles and relay selections.

However, if an author's content has been aggressively purged from these indexers due to censorship, they obviously can't be relied upon. Even though the author in question hasn't been deplatformed from nostr itself (since he can always self-host a publicly accessible relay to store his content), he has been effectively shadow-banned.

To get around this, relay selections have to be communicated in some other way. Nostr has a few different mechanisms for this:

• If the author's NIP 05 address is known and properly configured (it may not be), clients can look up the author's NIP 05 endpoint to find some reasonable relay hints. Unfortunately, these are often neglected, and usually custodial, so they can run into the same problems.
• If the author's pubkey is found in another signed event found on nostr, relay hints can be a way to propagate relay selections through the network. This relies on implementations picking reliable relay hints which can be difficult, and hints do tend to become less reliable over time. However, this strategy is very effective in resisting censorship because it makes banning viral — if a relay wants to completely purge a particular pubkey from their database, they have to purge every event that references it, since events are tamper-proof.
• In extremis, relay recommendations can always be communicated out-of-band. This can be done using manual input, QR codes, DHTs, jsonl torrents full of kind 10002 events, or any other mechanism client developers choose to resort to.

Another, more technical assumption is that any given query can be fulfilled by few enough relays that a client can actually make all the connections needed, without running into resource limits. If you're trying to request content from 10,000 users across 1,000 relays, you're going to have a bad time. This was pointed out to me by Mazin of nostr.wine. He makes a good point, and it's definitely something to keep in mind. There are some mitigating factors though.

The first is that the current topology of the network probably won't persist forever. Because nostr is largely populated by self-hosting enthusiasts, the number of "tiny" relays is proportionally much higher than it will be if adoption picks up, even if the total number of relays grows. The trajectory is that nostr will drift toward fewer, larger relays, reducing the number of connections needed to fulfill any given query.

This is "centralizing", but it's important to understand that this isn't necessarily a bad thing. As long as there are more than one or two large hubs, there is user choice. And as long as it's possible to run a new relay, there is always an escape hatch. Nostr, like bitcoin, has no hard dependency on the biggest player in the network.

The other thing to consider is that there are lots of other techniques we can use to overcome the limits of the lowest-common denominator's limitations (mobile browser clients), including self hosted or third-party relay proxies. The trade-off here is that a little trust (aka centralization) can go a long way to reducing resource requirements needed to fulfill queries using the Outbox model.

If you're interested in more details on this topic, see this blog post.

That was a long digression, but there is one other thing that the Outbox model assumes to be the case. Even if the correct relays are found and connected to, they still may not return all desired content, either because they don't have it, or because they refuse to return it to the user requesting it.

This can happen if the publishing client isn't following the Outbox Model, if the author had migrated from one relay set to another without copying their notes over, or if the relay in question chose not to retain the author's content for some reason.

The first two issues can be fixed by improving implementations, but the question of policy is a little more interesting.

Relativistic relays

The Outbox Model is a mechanical process; it's only as useful as user relay selections are. In order for it to work, users have to be able to make intelligent relay selections.

Every relay has trade-offs, depending on its policy. 140.f7z.io would not be useful for long-form content, for example. Some relays might have a content retention policy that changes depending on whether you're a paying user. If you don't pay, you might find out too late that your content has been deleted from the relay.

So what makes a relay "good" for a particular use case? Well, it's complicated. Here are a few factors that go into that calculus:

• Is the relay in the same geographical as the user? Proximity reduces latency, but jurisdictional arbitrage might be desired. Users should probably have a variety of relays that fit different profiles.
• Will the relay ban the user? Do the operators have a history of good behavior? Is the relay focused on particular types of content? Is the relay's focus consistent with the user's goal in adding that relay to their list?
• What are the relay's retention policies? A user might want to set up an archival relay for her old content, or a multi-availability-zone relay so her notes are immediately accessible to the rest of the network.
• Does the relay require payment? Paid relays are more aligned with their users, but obviously come at a financial cost.
• Does the relay have policies for read-protecting content? If so, other users might not be able to find your posts published to that relay. On the other hand, some relays are configured to work as inboxes for direct messages, which can help preserve privacy.
• Does the relay request that users authenticate? Authentication can help manage spam, but it also allows relays to correlate content requests with users, reducing user privacy.
• Is the relay you use hosted by your client's developer? If so, you're in danger of getting banned from your client and your relay at the same time.
• Is the relay a hub? Using hubs can help smooth out rough areas in Outbox Model implementations, at the cost of centralization.
• Is the relay used by anyone else? One-off relays can be useful for archival purposes, but often won't be used by clients following the Outbox Model, depending on how they optimize requests.

There are lots of ways to approach the problem of helping users select relays, but it's an inherently complex problem which very few people will have the patience to properly address on their own. Relay selection is a multi-dimensional problem, and requires satisfying multiple constraints with a limited number of relay selections.

In the future, special-purpose clients might be used to help people build relay sets. Clients also might provide curated "relay kits" that users can choose and customize. Or, we might see an increase in hybrid solutions, like smarter relay proxies or client-local relays that synchronize using other protocols or platforms.

The Limitations of Outbox

Outbox is not a complete solution, not because of any of the caveats listed above, but because NIP 65 per se only addresses the question of how to index content by pubkey in a broadcast social media context. But there are many other scenarios for relay selection that Outbox does not solve:

• Community, chat, and group posts might be best posted to relays dedicated to that context.
• Direct messages shouldn't follow the same contours as public social media content.
• Topic-oriented relays, or relays serving a custom feed might be useful independent of who uses them.
• Relays focused on serving a particular kind of event, like music, long-form content, or relay selections, are useful independent of who reads from or writes to them.
• Certain clients might need to fulfill particular use cases by using relays that support certain protocol features, like search, count, or sync commands.
• Some events might not make sense to publish to relays, but should instead be shared only directly, out of band.

Some of these use cases might be solved by new specifications similar to Outbox that prescribe where certain data belongs — for example, NIP 17 requires users to publish a different relay list before they can receive direct messages, while NIP 72 places community relay recommendations directly into the group's metadata object. A reasonably complete list of different relay types can be found in this PR, very few of which have a canonical way to manage selections.

Other use cases might be supported more informally, either by relays advertising their own value proposition, or via third-party NIP 66 metadata. Still others might be supported by scoping the network down to only certain relays through explicit relay selection — this is how white-labeled Coracle instances work.

The basic idea here is that there are categories of events that don't have anything to do with where a particular person puts his or her "tweets". For every "what" on nostr, there should be a "how".

Keep nostr weird

Whatever additional systems we end up adopting for helping with relay selection, one thing is certain — people will continue to discover new, creative uses for relays, and we will always be playing catch up. This is one of the coolest things about nostr!

But it does mean that users will have to adapt their expectations to a network that partitions, re-configures, and evolves over time. Nostr is not a "worse" experience than legacy social media, but it is a version of social media that has itself been set free from the stagnant walled-garden model. Nostr is in many ways a living organism — we should be careful not to impose our expectations prematurely, leaving room to discover what this thing actually is, or can be.

If you enjoyed this post but want more take a look at the talk I gave at Nostrasia last year. I also wrote up a blog post at about the same time that addresses some of the same issues, but focuses more on privacy concerns around relays and nostr groups. Finally, I recently wrote this comment, which includes some details about challenges I've faced putting Outbox into Coracle.

A few weeks ago, Mike proposed the addition of a feature to nostr. The content of the proposal itself isn't important, but the resulting conversation illustrated something important about nostr development that I wanted to draw attention to.

If you're interested, you can find the issue here. The idea was basically a tag that disabled comments to a reply, for when you wanted to gracefully exit a conversation that had outlived its usefulness.

While I definitely sympathize with the experience of getting stuck in an unproductive argument and being unable to leave because you have to have the last word, I do think it's better to take responsibility for leaving the conversation, rather than make other people do it for you. You can either outlast your opponent, let them have the last word, or tell them "I don't want to talk about this any more, I'm not going to reply".

This is just my opinion, and it's really not important whether I'm right or not. What was interesting was how Vitor responded:

I am not sure if the NIP review process should consider "what's good for the user" in the discussion. That kind of nanny state thinking is what went wrong with regular social media in the first place.

Permaculture and Ethics

What it sounds like (although I have a hard time believing this is actually his position), is that Vitor is dismissing the relevance of an ethical framework in designing a protocol, preferring to stick to the mechanics of what is being suggested. As Vitor says, "Clients can do whatever they want, of course." This is true, but ignores the question of why a client developer might want to do any particular thing.

In a recent Thank God for Nostr episode, I interviewed Scott Mann of the Permaculture Podcast. When I asked him his opinion on how to manage decentralized protocol design and build effective consensus, here's what he had to say:

[You] don't need to frame it as a software development project, or even a protocol. I would look at it as a distributed community-based and community-supported project, whatever that is. Because permaculture is such a large umbrella, I like to go up to that 50,000 foot view and pull away from what the details are. Because the details are what we're going to build our solutions from.

And that's one of the things I didn't mention earlier — there's kind of a hierarchy within permaculture that goes, at the top are the ethics of permaculture: earth care, people care, fair share. Beneath that are the principles, then usually we have strategies and techniques.

But there's a dividing line between ethics and principles and the strategies and techniques, that we start at that top, and use the ethics to decide whether or not we're even going to launch a project.

He goes on to say:

I'm going to use the principles and see how can I apply those principles to my research and original design. To make sure that whatever I'm creating creates some kind of a surplus, to have a refinement process in place before I even launch, like what is that going to look like, even if I have to change it later, just having some of these building blocks in place.

And then once I've done that and have gone deep into my research into what this might look like, how I might launch it, that's where I would start going into strategies and going ok, how do I want to market this? How do I want to get this out to the people who are going to use it, how do I want to maintain this, how do I want to do distributed decision making, and then as I start to think about distributed decision making, looking at what is the form that I want to use for that?

Scott's thesis is that ethics and principles should be in the front of your mind both as you're considering a project, and as you continue to build it out. This not only makes sense as something to do if you want to succeed, it's categorically true. Action can't happen without agency, and it is your agency that informs what you choose to do, and how you plan to go about it.

The word "ethics" comes from the Greek ēthos, meaning "moral character". In other words, your ethic is who you are. Your values, hopes, preferences, faults all factor in to your actions.

This is not always clear, because in fiat-world, many people suspend their values in order to "get something done". If you want to protect your savings, you invest it in index funds, propping up the stock price of companies that hate you. If you want to make money, you go work at a job where you're berated quarterly for being racist. People think that they can exercise their own values in their private life, while actively undermining those values with the majority of their time and purchasing power.

But this is not how people with integrity act. And I think if you can say one thing about nostr — both its developers and the community at large — is that they have a very high level of integrity. In other words, their actions are clearly informed by their ēthos.

This is a very good thing. What is the point of building an entirely new internet if we're not going to impose our values upon it? What an absolute waste of effort.

A Nostr Manifesto

This is not to say that any one developer has the right to imposing his own vision on the protocol because of his own personal values and reasons for contributing, which is what I think Vitor was being cautious about. But there are lines I think we can draw as a community that can't really be crossed without excluding yourself from what I might call the "nostr group ethic".

So what are those boundaries? What is a nostrich? Here are some values I've observed to be generally shared among nostr developers and users. Not everyone would full agree with these (including myself), but I think they're a fair characterization of the community.

1. Free speech absolutism. No central entity should be able to globally censor any content. This comes with the trade-off of objectively evil content continuing to exist. This trade-off is acceptable, both because of the value of free speech, but also because evil will continue to exist regardless of attempts to suppress it.
2. Empower individuals over institutions. No centralized entity can be trusted to safeguard the interests of the individual. Institutional incentives are asymmetric and easily corrupted. Better to have many subjective views of the world, than a single, centrally managed view of the world.
3. Advertising-based business models should be viewed with great skepticism and caution. Advertising is a system of incentives that is central to the institutional corruption we see around us. Broadly, this includes paid ads, monetization of engagement, public/private partnerships, and "crypto".

There are others that are shared by many within nostr, although not as widely agreed upon. Two I can think of are:

• Economic activity should be voluntary. Software and content should be free, and creators should be amply rewarded via zaps or other value-for-value models.
• Social media should support real life community and relationships, not detract from them. We should all take time to touch grass.

I'm personally skeptical of the first of these, and strongly in support of the second. Much of my energy as a nostr developer has gone into attempting to subvert and reform traditional patterns of social media to not only support, but also resemble relationships that exist in the analog world God made, and placed us in.

This particular principle is the one at play in the conversation I linked to at the top of this post. My comments weren't an accusation that anyone was acting "unethically" in a universal sense, only that the proposed feature was incompatible with my vision for what nostr should be.

But of course, my vision is not shared by everyone, and the principle of "support real life" is clearly subservient to core ethic #2, which admits the value of a diverse set of opinions about the world. I have no right (or ability) to invalidate anyone else's core principles. But by the same token I'm free to express my own, and attempt to convince other people to share them. This is the basic value proposition of freedom of speech itself.

Ethical Cohesion

I would go further, and say that not only is it permissible to talk about ethical reasons for building one thing or another on nostr, it's essential. By having these conversations we fuse our individual ethics into a shared ethic. By calibrating our moral compasses to point in (roughly) the same direction, we also decrease the friction involved in getting something done.

I think this was a significant part of the idea behind Sovereign Engineering — get a bunch of people in a room together sharing meals and going on hikes, and the work will accelerate! This is also the way a church works. By meeting weekly together we strengthen our shared identity and build one another up through our activity. In fact, this is the basic definition of a community as Scott Mann puts it. In his words, a community can provide:

a series of connections, and a knowledge base, and a skillset that we can't fulfill as an individual, while having a social relationship with people in such a way that we can call on them for help.

So maybe, as I've said before, the real protocol is the friends we made along the way. Disagreement and discussion is a healthy thing for a community to have, and we should never stop asking "why?"

• Products with pubkeys: https://github.com/nostr-protocol/nips/pull/1225
• Shared-ownership events: https://github.com/nostr-protocol/nips/pull/1192 and https://github.com/nostr-protocol/nips/pull/1015
• Spreadsheets: https://github.com/nostr-protocol/nips/pull/1189
• Relational databases: https://github.com/nostr-protocol/nips/pull/1168
• Relay-specific notes: https://github.com/nostr-protocol/nips/pull/1146
• Editable notes: https://github.com/nostr-protocol/nips/pull/1090
• Restricted events: https://github.com/nostr-protocol/nips/pull/1083
• Relay-based access control: https://github.com/nostr-protocol/nips/pull/1030
• Protected events: https://github.com/nostr-protocol/nips/pull/1029
• Closed groups: https://github.com/nostr-protocol/nips/pull/875

Some of these I like, some I don't. But most of them go beyond adding new features to nostr (for example audio, video, speedruns, resumes, etc), and begin to change how nostr actually works.

Nostr can be an everything app, but I think that means something very specific. Nostr can represent data types from pretty much any domain, but it can't actually support all the semantics needed to build any arbitrary system.

I would suggest conservatism in what we build on nostr, but of course anyone can build whatever they want. But I do think it's possible to identify things that nostr is good at, and things it's bad at, and play to nostr's strengths.

Nostr's strengths:

• Being able to model any data type orthogonally to the rest
• Single-owner, self-authenticating, atomic data types
• Potential for robust content dispersal and retrieval if we can flesh out NIP 65 etc.

Nostr's weaknesses:

• Mutable state, non-atomic state
• Shared ownership, key delegation/rotation
• Privacy — metadata will always leak, lack of consistency makes key rotation harder
• Consistency — not everyone has the same view of the world
• Transactionality — nostr isn't good for updating multiple pieces of data in lockstep

It happens that the original use case of nostr — public broadcast social media — benefits greatly from nostr's strengths, and isn't bothered by any of nostr's weaknesses. Blob storage like what blossom is building also works well in this paradigm. A lot of the use cases @PABLOF7z​ has identified work beautifully well because of the single-owner public-read nature of nostr, which makes forks easy to model.

But things like access control, relational data, collaborative document creation, heavier datasets, or anything that requires a solution to the double-spend problem become very awkward (or impossible) to model on pure nostr. A simple example of this is lists. Not only is it common for a single user to mess up his follow lists because of a lack of consistency between clients or devices, but commonly requested features like shared ownership lists immediately result in a huge increase of complexity, either on the key management side or on the data structure side. Both of these problems are difficult to solve on nostr due to lack of consistency — keys can't always be reliably or safely shared, and linked data structures spanning multiple events by different authors can be hard to assemble reliably.

I think the danger here is that if we as a developer community fail to realize the limitations of nostr and try to adapt the protocol to fit every possible use case, on an ad-hoc basis, we're going to end up with a tragedy of the commons, where no developer can comprehend what must be done to get his work done, and all kinds of weird artifacts appear for end users that no one can explain.

Here are some suggestions I have for preventing this from happening. I realize no one is going to follow them. But maybe they can be helpful for avoidance of wasted time.

• Don't overload event kinds. Many people (including myself) have tried to extend kind 0 with attributes for forms, products, and groups, but that leads to madness. Instead, create a new metadata event signifying a different kind of agent.
• Don't model things as replaceable events if you can avoid it. This creates the problem of shared mutable state, which nostr doesn't have a good story for resolving. They also have a hard limit on how big they can be.
• Use different keys for different things. For domains where some kind of access control needs to be implemented, not tying everything to your main pubkey makes it possible to create and burn keys as needed. Incidentally, this can help users maintain better privacy. An example of this is private groups, which have a dedicated key separate from the group creator's own key.
• Event ownership should always be (is) single-key. If you need shared ownership, figure out a way to share keys. More work needs to go into key invalidation and rotation for this to really work.
• Explore the fork model — this is "my" version of the same thing, and coexists with rather than supersedes the original. This has potential not just for groups or wiki entries.
• Distinguish between different ways to use relays. Relays may be used as indexers (holding specific event types or supporting different features like search/count), repositories (holding many diverse events, to be used with filters), or curated feeds (to be used without filters, or with only a few filters).

These are just suggestions, and many of them may be wrong. Nostr development is hard, and getting harder. Keep it simple.

What's wrong with outbox?

It all started with a post from jack asking why more devs haven't implemented the outbox model. There are many answers to this question, not least having to do with there being two standards for user relay selections, and ongoing changes to NIP 65. But I don't want to talk about compatibility here.

nostr:nevent1qydhwumn8ghj7argv4nx7un9wd6zumn0wd68yvfwvdhk6tcprfmhxue69uhhq7tjv9kkjepwve5kzar2v9nzucm0d5hszymhwden5te0wfjkccte9enrw73wd9hj7qpq2uf488j3uy084kpsn594xcef9g9x3lplx4xnglf0xwghyw2n3tfqqnrm02

Mazin responded with some numbers which estimate how many connections the outbox model requires. Too many connections can become expensive for low-power clients like mobile phones, not to mention some privacy issues stemming from nosy relays.

nevent1qythwumn8ghj76twvfhhstnwdaehgu3wwa5kuef0qyv8wumn8ghj7cm9d3kxzu3wdehhxarj9emkjmn99uq3samnwvaz7tmrwfjkzarj9ehx7um5wgh8w6twv5hsqgrn7l6zj7ht6ruyk76vvvtkfs4xrhyzc3tm64l3eyfvd40y26sz0gshmunh

I have some minor disagreements with Mazin's numbers, but I basically agree with his point — a purist outbox model, where a large proportion of nostr users run their own relays results in a high number of connections to different relays. I brought this question up late last year in my interview with Mike Dilger and in a conversation with fiatjaf, who convinced me that in practice, this doesn't matter — enough people will use a handful of larger hubs that there will be a good amount of overlap in relay selections between most pubkeys.

To articulate this more clearly: the goal is not "personal web nodes", which is a pipe dream the Farcasters and BlueSkys (BlueSkies?) of the world aim at, but a more pragmatic mix between large hubs and smaller purpose-built relays. These small relays might be outlets for large publishers, small groups, or nerds who also run their own SMTP servers and lightning nodes.

The point of the outbox model is that these small nodes be possible to run, and discoverable from the rest of the network so that we can preserve the censorship-resistant qualities of nostr that brought us here in the first place.

Blast It!

It's no secret that I've long been a critic of Mutiny's blastr relay implementation. My main objection is that the blastr approach doesn't account for the hard limits involved in scaling smaller relays. If the goal is to cross-pollinate notes across all relays in the network, all relays will require the same size database, and contain all notes in the network. This works right now (sort of), but as the network grows, the relays running on a $5 VPS are going to have their disks fill up and will inevitably fall over.

nevent1qyvhwumn8ghj76r0v3kxymmy9ehx7um5wgcjucm0d5hszxnhwden5te0wpuhyctdd9jzuenfv96x5ctx9e3k7mf0qythwumn8ghj7un9d3shjtnwdaehgu3wvfskuep0qqs07jr9qx49h53nhw76u7c3up2s72k7le2zj94h5fugmcgtyde4j9qfrnwxj

Not only that, but the content breakdown on any given relay by default becomes an undifferentiated soup of "GM", chinese notes, bots, bitcoin memes, and porn. Blastr makes it impossible to run an interesting relay without implementing write policies.

Which is actually fine! Because that's always been true — servers that allow anonymous uploads always get abused. Tony is just helpfully pointing out to us that this is no less true of nostr relays. I only wish he could have waited a little longer before mounting his attack on the network, because lots of hobbyists are interested in running interesting relays, but the tools don't yet exist to protect those servers from unsolicited notes.

One other note on blastr — Tony at one point described blastr as a relay proxy. This is an interesting perspective, which puts things in a different light. More on proxies later.

Ban Jack?

Here's a thought experiment: how might we actually "ban blastr"? @Pablof7z suggested to me in a conversation that you could configure your relay to check every note that gets published to your relay against the big nostr hubs, and if it exists on any of them to simply delete it. Of course, that would result in your relay being basically empty, and the hubs having all of your content. That's game theory for you I guess.

Another approach that was floated was to encourage users to only publish to small relays. In theory, this would force clients to implement outbox so users could still see the content they were subscribed to. Fiatjaf even posted two identical notes, one to his personal relay, and one to a hub to see which would get more engagement. The note posted to the mainstream relay got 10x more replies and likes than the more obscure note.

nostr:nevent1qyd8wumn8ghj7urewfsk66ty9enxjct5dfskvtnrdakj7qgmwaehxw309aex2mrp0yh8wetnw3jhymnzw33jucm0d5hszymhwden5te0wp6hyurvv4cxzeewv4ej7qpqdc2drrmdmlkcyna5kkcv8yls4f8zaj82jjl00xrh2tmmhw3ejsmsmp945r

Of course, this is thwarted by blastr, since blastr not only replicates notes posted to it, it also actively crawls the network as well. So the next logical step in this train of thought would be for hubs to encourage people to use small relays by actively blocking high-profile accounts.

nostr:nevent1qydhwumn8ghj7argv4nx7un9wd6zumn0wd68yvfwvdhk6tcpzdmhxue69uhhyetvv9ujue3h0ghxjme0qyd8wumn8ghj7urewfsk66ty9enxjct5dfskvtnrdakj7qpqpjhnn69lej55kde9l64jgmdkx2ngy2yk87trgjuzdte2skkwwnhqv5esfq

This would of course never happen (Damus is one client that hasn't implemented NIP 65, and they also run the biggest relay), but it was a fun thought experiment. At any rate, the silliness of the suggestion didn't stop certain people from getting offended that we would "disrupt the free market" by "forcing" our opinions on everyone else. Oh well.

Death to Blastr

In reality, even though blastr makes it a little harder to adopt outbox in the short term, its days are numbered. Eventually, relay operators will start to feel the pain of unsolicted notes, and will either shut their relays down or look for tools that will help them curate the content they host.

From my perspective, these tools take two forms — read protection and write protection. This is something I alluded to in my talk at Nostrasia last November.

Write protection is straightforward — already many relays have access control lists based on active subscriptions, invite codes, or just static whitelists that determine who is allowed to post to a given relay, or what event authors are represented there. This approach effectively prevents blastr from using relays as free storage, which is a huge improvement.

Read protection is more tricky, because anything publicly readable will be scraped by blastr and replicated to unauthenticated-write relays across the network. In most cases, this is ok, but there are use cases for relays to exist that host a unique collection of notes oriented around some organizing principle. Unfortunately, with blastr in action (or any scraper that might exist), the only way to do this is to actively protect proprietary content. There are a few approaches that can work to make this happen:

• IP-based access control lists
• AUTH-based access control lists
• Stripping signatures when serving events
• Storing and serving encrypted content

Each of these approaches has its own set of trade-offs. But depending on use case, any of them or a combination of them could work to allow relay operators to carve out their own piece of the nostr-verse. In fact, this is a big part of what Coracle is about — the white-labeled version of the product confines certain notes to proprietary relays, with optional encrypted group support.

Enough of my polemic against blastr. Let's talk about how to make the outbox model actually work.

Hints are pointless

Right now, clients that implement the outbox model rely pretty heavily on relay hints to find related notes — whether user profiles, reply parents, or community definitions. The problem with hints is that they are prone to link rot. Many of the relays that were set up a year ago when nostr took off are no longer online, and yet they persist in user relay lists, and in relay hints. These hints can't be updated — they are set in stone. What this means is that a different mechanism has to be used to find the notes the hints were supposed to help locate.

Because of this, I've come around to the position that hints are basically pointless. They are fine as a stopgap, and might be appropriate for certain obscure and ill-defined use cases where relay urls are the most durable address type available. But they provide basically no value in supporting the long-term robustness of the network.

What are durable, however, are pubkeys. Pubkeys are available pretty much everywhere, except in event id hints — and there is a proposal in the works to add a pubkey to those too. The cool thing about pubkeys as hints is that once you have a pubkey, all you need to do is find that person's kind 10002 inbox/outbox selections, and you should be able to find any note they have published.

This goes with the caveat that when users change their relay selections, or rotate their key, they (or their relays) should be sure to copy their notes to the new relay/pubkey.

The question then is: how do I find a given pubkey's relay selections?

There are already several mechanisms that make this reasonably easy. First of all, NIP 65 explicitly recommends publishing relay selections to a wide range of relays. This is a place where the blastr approach is appropriate. As a result, relay selections are usually available on the most popular public relays. Then there are special purpose relays like purplepag.es, which actively seek out these notes and index them.

These indexes are not confined to relays either. It would be trivial to create a DVM that you could ask for a pubkey's relay selections, optionally for a fee. Alex Gleason's proxy tag could also be used to indicate indexes that exist outside the nostr network — whether that be torrents, DHT keys, or what have you.

The best part is that this doesn't negatively impact the decentralization of the network because in principle these indexes are stateless — in other words, they're easily derived from the state of the public part of the nostr network.

Just do it for me

Looping back to where we started — the complexity and technical challenges of implementing the outbox model — there is a simple solution that many people have experimented with in different ways that could solve both issues at once: proxies.

As I mentioned above, Tony thinks of blastr as a proxy, and he's right. More specifically, it's a write-proxy. This is only part of its functionality (it also acts as an independent agent which crawls the network. EDIT: apparently this is not true!), but it is an essential part of how people use it.

Another kind of proxy is a read proxy. There are several implementations of these, including my own multiplextr proxy, which is outbox-compatible (although it requires a wrapper protocol for use). The advantage of a proxy like this is that it can reduce the number of connections a client has to open, and the number of duplicate events it has to download.

Proxies can do all kinds of fancy things in the background too, like managing the outbox model on behalf of the client, building an index of everything the user would be likely to ask for in advance to speed up response times, and more.

One interesting possibility is that a NIP 46 signer could double as a proxy, reducing the number of round trips needed. And since a signer already has access to your private key, this kind of proxy would not result in an escalation in permissions necessary for the proxy to work.

It's simple

The number of cool and creative solutions to the content replication and indexing problem is huge, and certainly doesn't end with blastr. Just to summarize the next steps I'm excited to see (to be honest, I want to build them myself, but we all know how that goes):

• More clients supporting outbox
• Outbox implementations maturing (Coracle's still has some issues that need to be worked out)
• A shift from relying on relay hints to relying on pubkey hints + relay selection indexes of some kind
• Proxy/signer combos which can take on some of the heavy lifting for clients of delivering events to the right inboxes, and pulling events from the right outboxes

Let's get building!