Originally posted on the World Wide Web Foundation website.

________

Just hearing the names of some causes tells you all you need to know. You feel them in your gut instantly, and your mind follows quickly.

Democracy.

Human rights.

The environment.

There is however a cause of causes — a cause underpinning each of these and many others — that therefore requires just as much emotion, concern and action.

It’s not appreciated too widely, yet all ecologists get it.

It’s not celebrated broadly, yet historians of the printing press get it.

It’s too often neglected and may even be undermined by the free societies it promotes, yet sociologists get it.

It’s not readily embraced by the free markets it enables, yet the architects of both the Internet and the Web get it.

What is it? …

The deep cause of causes you care about deeply is decentralized and distributed networks.

And now you see the problem. It’s not a visceral subject. It’s difficult to imagine a protest march in its name. It doesn’t lend itself to emotive imagery or storytelling.

Here’s the title I started with here: “Decentralized and distributed networks — why they matter and why we must find a way to reverse their decline in the digital age.”

Excuse me while I yawn and click over to Facebook for something, anything, more interesting instead! Sympathize with writers who take it on. Pity those technologists attempting to secure the resources to defend it … a nut I hope to help crack here.

A global village

Decentralized and distributed networks are, quite simply, fundamental; everyone is affected in many ways by the degree of distribution — not least one’s personal agency, the development of our societal structures, and our impact on this planet’s living systems.

As we instrument the planet, as we build out a pervasive computing environment, as increasingly little ‘undigital’ remains, ensuring the decentralized and distributed nature of that digital infrastructure is nothing short of mission critical. We must resist the easy short-term temptations of centralization to avoid its longer-term miseries.

The 7th Global Drucker Forum looked at the technology revolution through the lens of humanity, and I was invited to help set the scene:

The ultimate information technology challenge is the care and maintenance of a digital infrastructure that can help us rise up to so-called super wicked problems, collectively. Given the growing appreciation of the nature of complexity and the complexity of nature, we know we’re in the domain of systems thinking and sustainability — the health and resilience of living systems including our planet, our societies, and our organisations.

Sustainability requires healthy, distributed networks, with both diversity and individual agency, to facilitate the emergence of collective intelligence. It is these qualities our digital technologies must enable and encourage.

Technologists understood the technological implications long before the Internet emerged, starting with Norbert Wiener’s insights into the dynamics and ethics of humans and large computer systems [1] and Marshall McLuhan’s anticipation of a shift from the centralized mechanical age to the decentralized electronic age, coining the term global village as shorthand for such a welcome outcome [2].

Paul Baran, a prominent engineer in the development of the Internet, subsequently explained the vista in 1964 with this now famous diagram [3]:

The Internet was designed as a distributed network. And the World Wide Web (that runs on top of the Internet) is in parts both decentralized and distributed. Why? Well, for each of these three types of network just ask yourself — how many dots would I need to erase to destroy the entire network?

The US Department of Defense played a major role in funding the development of the Internet, so you can see now why a distributed network was chosen — a distributed network can simply ‘route around’ any erased dots / destroyed facilities, carrying on doing what it was doing before [4].

The Web Foundation describes a key feature of the World Wide Web:

No permission is needed from a central authority to post anything on the Web, there is no central controlling node, and so no single point of failure … and no “kill switch”! This also implies freedom from indiscriminate censorship and surveillance.

Both distributed networks and, to a lesser degree, decentralized networks have a resilience, a vitality, a freedom, entirely missing from centralized networks and this simple observation has a profound implication — whenever you’re presented with an argument for technological centralization you must know you’re on the receiving end of commercial, political and / or naïve motivations however they may be dressed up differently. (One might just possibly label the essay here political, but only if one choses objectionably to confuse the corresponding mathematics for politics.)

Having said all that, centralization can simply emerge.

Network effects and centralization

Fifteen years ago you may have interacted with hundreds of websites and applications and services on your PC from one month to the next. Today [5], two thirds of our digital time is spent on our phones rather than a desktop or laptop, and the majority of that time is spent in apps rather than browsing the web. The majority of our attention is commanded by just half a dozen companies [6] (referred to here as ‘the half dozen’) most of which are working to develop a new layer above those apps, or more accurately above the purveyors of those apps, in a process that’s been called de-appification.

The half dozen have innovated, acquired, and aggregated, using a network effect in one market to build a massive customer base for another.

A network effect describes the situation in which the value of a product or service increases with the number of others using it. Such centralization doesn’t mean anyone is behaving badly, it’s just the network effect playing out. (Of course things in the digital world can have network effects without any one company owning it — the web itself being the classic example.)

Given that decentralization and distributed architecture is core to the very definitions of the Internet and the Web, to watch growing centralization is to witness their erosion.

I’m often criticized for such an assertion. I’m frequently challenged for example with references to the ease with which anyone can publish to the web. But here’s the snag. Before the arrival of social media, anyone could read the web but publishing content required some skills and modest resources. Now, social media makes publishers of us all, yet we increasingly publish to proprietary non-web media platforms that may choose to censor our content or hide it as they wish. Moreover, content that is published to the web is scooped up by proprietary media platforms to present to others in apps rather than in web browsers.

Centralization sucks

When the vast majority of your digital life is spread across just half a dozen companies, the power moves from you to them. They know who you are, where you are, how you got there, what you do, when you do it, and who you do it with. Your digital life is ruled by the technological architecture and policies they develop to make more money out of you, a form of commerce increasingly known as surveillance capitalism.

Invisible software algorithms (step-by-step sets of operations applied to our data) are developed to “monetize” you, influencing what you read and don’t read, what you watch and don’t watch, who you meet and don’t meet, what you do and don’t do.

What you think and don’t think.

A widely quoted contribution to a popular web forum in 2010 describes the situation bluntly:

If you are not paying for it, you’re not the customer; you’re the product being sold.

Ouch.

No matter who you are, if you’re online then it’s almost guaranteed that your personal power is now partially absorbed by corporates you’ve never met applying software you don’t understand to nudge your actions in ways you don’t appreciate. It might feel enjoyable and convenient, but that’s because it makes sense for such services to increase your enjoyment and convenience to their commercial advantage.

Is it any wonder for example that our politics are getting more divisive, more extreme, more angry, when opaque algorithms decide you need only be shown stuff that matches its understanding of your political outlook. We’re left with less opportunity to see, understand and debate other points of view like adults.

This is dangerous.

Not only is it tempting to just sit back and enjoy the experience (i.e. you haven’t read something like this), not only do we know that centralization on this scale isn’t likely to have a happy ending (it’s unnatural), but we’re passing up the opportunity to create a healthier, more robust, more adaptable system just at the very moment we really really need one.

I’m not saying that the current way forward is just a little less attractive than the decentralized, distributed and empowering alternative, but rather that the whole thing is of an entirely inferior quality. And it’s worse than that. The longer it goes on, the harder it might become to redecentralize.

(Re)decentralize!

I’m far from alone in recognizing the imperative to redecentralize — we had a conference on the matter in 2015 in London, and the Internet Archive hosted one in 2016 in San Francisco. And as you’d expect, there are different ideas as to how we might achieve our goal.

We want to nurture a distributed, sustainable living system. We want to secure personal agency. We want to encourage diverse participation. We want to build the conditions for collective intelligence to emerge. We want to blind commercial surveillance. We want to avoid an omniscient, Orwellian state. We want to enable new business models. We want to support network effects based on open protocols not proprietary software.

I’m going to describe my rationale here for progressing this cause of causes, and must in the process question the validity of some other ideas and assertions. We’re all striving for the same ends so such criticism can never be personal. Far from it. We’re a distributed network of individuals and organizations doing what distributed networks do best, riffing off one another to identify and hone a sustainable way forward.

Sir Tim Berners-Lee, inventor of the World Wide Web, describes our challenge here:

The web is already decentralized. The problem is the dominance of one search engine, one big social network, one Twitter for microblogging. We don’t have a technology problem, we have a social problem.

The blogger, journalist, and science fiction author Cory Doctorow addresses technologists:

The reason the web ceases to be decentralized is that there are a lot of short-term gains to centralizing things. The Web is centralized today because people like you make compromises.

A 2015 blog post by Brewster Kahle, Internet entrepreneur and activist and founder of the Internet Archive, describes how we might (re)architect the web as fully distributed. It’s the ideal introduction to the assembly of technologies that might be required, and served as the seed for the 2016 decentralization conference.

Together they’re onto something, obviously, but the problem is not singularly social, not one of geek culture, not merely technological.

This isn’t just a social problem. People are people, as the truism goes. Why for example would anyone wish to sever themselves from the comfort, convenience and connection of a centralized proprietary social network to join a decentralized alternative? They’d have to jump back on the proprietary social network just to let everyone know, which isn’t so much ironic as merely reflective of the network effect’s hold. Decentralization cannot be marketed. Marketing is based on the premise of benefit not burden.

This isn’t a geek culture problem. No profession hosts a majority of practitioners cognizant of the macro and/or philosophical ramifications of each and every daily action. Any aspiring solution here that relies upon the development and application of some moral framework is at risk of too slothful or simply inadequate adoption. It also fails to recognize emergent outcomes innate to this complex adaptive system that are by definition indeterminate in advance.

This isn’t just a technology problem. The challenge isn’t to curate and interweave the technologies required for decentralization but to secure their combined adoption in competition to today’s centralized and centralizing networks. Recall my comment above about marketing.

No. If we’re to rise up to our challenge, it will necessarily be social and technological and above all commercial. Our societies are capitalistic and profit is the primary driver of today’s digital technologies, networks, products and services.

For the immediate avoidance of any confusion, in advocating free open-source software here, the profit is derived because of a new architecture rather than with a new architecture. More on this below.

Concentrate

Centralization is a concentration of otherwise distributed entities, the most important of which must be people, the constituents of the three basic economic units: the firm, the household, and the government.

Commercial centralization of services relating to billions of people and their Internet access trends towards monopoly (only one seller or provider) which in turn encompasses the trend towards monopsony (only one buyer). Centralization of peopled networks then has a concentrating effect running down through the layers of the technology stack that’s more powerful than any concentrating effects that might run up through the layers. Consider that those half dozen companies are first and foremost dominant in the provision of consumer products and services. Concentration is then further compounded as vertical integration subsequently proves strategically and commercially tempting to these companies.

Such observation, indeed such logic, presents an inevitable conclusion. Any viable decentralizing construct must start with people. That doesn’t mean the provision of point solutions such as personal email clients or personal data stores — minority sports that merely abut rather than reverse the dominant centralizing dynamic — but with the very fabric of our interactions, encompassing all our senses and our sense-making. What could be more personal? What else should be more yours and less theirs?

The OS vendors and other consumer tech giants constituting the half dozen dominate interaction today. Centralizing forces are still playing out and this field will whittle down further, and if continuing relevance remains too challenging for one or more of these privileged behemoths, we must only conclude that our decentralization community cannot immediately respond by playing exactly the same game. Rather, our immediate response to the very fabric of our interactions will necessarily entail something that no one company can achieve alone, relying on the very qualities that no commercial centralizing service can easily inculcate let alone sustain — our diversity and distributedness.

Decentralization cannot be marketed. Yet if we know where the value lies for organizations and for individuals, we can design for decentralization to re-emerge in consequence.

Introducing the hi:project

Jef Raskin, a Human-Computer Interaction expert who conceived and started Apple’s Macintosh project, defines the interface as follows:

The way you accomplish tasks with a product — what you do and how it responds — that’s the interface.

The hi:project divides Raskin’s definition into three types of interface: the user interface (UI), the surveillance interface (SI), and the human interface (HI).

The way a machine or service helps you accomplish tasks with or through it, that’s the user interface.

What some refer to as the ‘personal assistant’, we call the surveillance interface.

The way a machine or service surveils, records, interprets and to some degree controls as much as it can about your life to help you accomplish tasks with or through it, for the profit of the service provider, that’s the surveillance interface.

And we define our vision for a free, distributed, open-source human interface:

The way your software helps you accomplish tasks with other software, that’s the human interface.

The distinction appears subtle at first but its manifestation is not. The UI and SI belong to the end- or intermediate machine or service, and ultimately to the provider of that machine or service. Your HI belongs to you. It supports many of the attributes scoped by attendees of the 22nd Internet Identity Workshop to describe sovereign technology.

As Web of Data developments decouple the app from the data, the hi:project decouples the interface from the app. The hi:project re-imagines the interface as a lightweight artefact that can be shared within an open, distributed ecosystem. Participants are able to freely copy, modify and share improved or customised components, distributed and discoverable across a named data network architecture. AI services may be integrated by choice, and the hi:project may prove to be the ‘Open Interaction’ partner to OpenAI.

Decentralization at this topmost level then has a ‘de-concentrating’ effect running down through the layers of the technology stack, expanding the viability of other decentralizing technologies operating at lower levels in the stack, such as all those listed by Kahle in his blog post. And in pure testament to the advantages of decentralization, the hi:project’s realisation doesn’t just help seed deeper redecentralization but also helps solve current problems relating to personal data and privacy, and promotes digital inclusion and accessibility, which in turn fuel its dissemination for commercial and social advantage and legal compliance.

My intention here has been to underline the criticality of decentralized and distributed networks, to rationalize the decentralization community’s approach to redecentralization, and to raise the hi:project to your attention. It’s not the place to explain the hi:project in greater detail — you can find out more at https://hi-project.org.

I will finish by elaborating on the all-important profit I noted earlier.

Profit

No-one profits from the centralization of the most powerful infrastructure mankind has ever conceived.

Following my opening comments, ecologists, historians, sociologists, computer scientists and engineers (of which I’m one) will struggle to maintain any hope that a centralized network can match a decentralized network’s capacity for helping us rise up co-operatively to the many problems we’re grappling with as a species, especially those worthy of the name super wicked. If anything, such centralization actually creates problems of its own.

More directly, any organization that is not part of the elite half dozen and that aspires to co-create value from our permissioned and contextualized personal data will incur the costs that the half dozen levy from their privileged mediating position.

And it’s not even a rosy future for the half dozen either.

The current trajectory leads us nowhere but regulated centralization — perhaps not regulation in every jurisdiction, but likely in many jurisdictions and differently from one to the next, contributing to the Balkanization of our digital infrastructure.

The outcome?… Deadlock. Stagnation. Caught between market failure and the denial of network effects, albeit proprietary ones. Massively centralized services staffed by people smart enough to get there and smart enough to realize we all took a wrong turn.

Alternatively, we can all co-operate now to realize the human interface.

Next

The hi:project is a collaborative, non-differentiating, open-source, engineering project dedicated to developing and disseminating the human interface (HI). Endorsed by the Web Science Trust, we’re currently building a community of interested parties, preparing to develop a technical proof of concept to demonstrate the potential of our vision. We’re in discussion with prospective participating companies in consumer tech, automotive, Internet of Things, healthcare and financial services.

If you would like to find out more — so that your company might profit and so we all might profit — please contact philip@hi-project.org.

---

With thanks to Dame Professor Wendy Hall, Professor Leslie Carr, David Ascher, Laura James, Steve Taylor, Adrian Gropper and Christina Bowen for their comments on draft versions.

[1] The Human Use of Human Beings, Norbert Wiener, 1950

[2] The Gutenberg Galaxy, Marshall McLuhan, 1962

[3] On Distributed Communications, Paul Baran, RAND, 1964

[4] http://www.rand.org/content/dam/rand/pubs/papers/2008/P1995.pdf

[5] US figures. Source: Comscore 2016. http://www.comscore.com/Insights/Presentations-and-Whitepapers/2016/2016-US-Cross-Platform-Future-in-Focus

[6] I exclude the special (centralizing) case of China here.