How file sharing got here

Napster kept the index on servers the company owned. Gnutella threw the index away and flooded the network with questions instead. eDonkey moved the index onto ordinary servers that never touched a byte of the file. BitTorrent stopped indexing anything and built a separate swarm for each file. Magnet links and distributed hash tables then removed the tracker that had coordinated that swarm. Read as engineering rather than as a court report, the whole era was one long search for a design with no switch on it.

What peer to peer file sharing meant, and what it did not

Peer to peer file sharing meant that the copy arrived from another person's computer instead of from a machine the service owned, and that is the whole definition. In a client and server arrangement, such as a shared drive at work or a document library in SharePoint, every copy passes through hardware one organization runs, so that organization can see the file, meter it and remove it. A Bluetooth transfer between two phones is direct, but it links two devices in one room and there is no network to search. Peer to peer file sharing software sat between the two: a program on an ordinary home machine that could find strangers, ask them for parts of a file and hand out parts in return. The glossary defines each of those words on its own.

The work always broke into the same three problems in the same order: find who has the file, connect to them, and move the data. Only the first problem, finding, was ever seriously redesigned, and it was redesigned again and again because finding was the part that ran on somebody's hardware and had somebody's name on it.

Napster put every search through one index

Napster put every search through one index because that was the simplest thing that could work, and in 1999 it worked. The program scanned the music files in a folder on the user's own machine, sent the list of file names to Napster's servers, and searched that combined list on request. The audio itself never went near the company: once a searcher picked a result, the two computers connected directly. The index was small, fast and complete, which is exactly why the design was never repeated.

The consequence was structural rather than moral. One company held a list of what was on the network, ran the hardware that held it, and could have filtered it. That combination, knowledge plus control, was what the courts fastened on, and a shutdown order against one index took the whole network with it in 2001. Every design that followed was built by people who had watched that happen.

Gnutella and Kazaa threw the index away

Gnutella threw the index away completely. Released in 2000 by developers at Nullsoft, it had no central list at all: a client asked the neighbors it knew, those neighbors asked theirs, and answers walked back along the path the question had taken. There was nothing to seize because there was nothing in the middle. It also drowned: a flooded question multiplies with every hop, so the cost of a search rose with the size of the crowd, and obscure files were nearly invisible.

FastTrack, the network behind Kazaa from 2001, split the difference. Well connected clients were promoted to supernodes and took on the indexing work for the clients near them, so searches went to a supernode instead of to everybody. That restored speed and kept the hierarchy replaceable, since a supernode that vanished was taken over by another user's machine. The company that wrote the software no longer operated the index, and that distinction later mattered a great deal in court.

eDonkey servers indexed the network without carrying the file

eDonkey servers indexed the network without ever carrying the file, which made each one cheap to run and cheap to lose. From 2000, an eDonkey server kept a list of which client held which file and stored no content at all, so a replacement could be stood up by anybody with a spare machine. Files were identified by a hash of their contents rather than by name, so two copies counted as the same file whatever anybody had called them. eMule, a free software client for the same network from 2002, later added Kad, a distributed table that found sources with no server involved at all. The eMule and eDonkey story is told in full on its own page.

BitTorrent built one swarm per file

BitTorrent built one swarm per file and refused to solve the search problem at all. The protocol, written by Bram Cohen and released between 2001 and 2003, describes only what happens once a person knows which file they want: a small metadata file lists the pieces the content is cut into and the hash of each, and a tracker keeps the list of who is working on that one file. There is no network wide index in the design, because there is no network: there is a separate temporary crowd for every file, and it dissolves when the last participant leaves.

The other half of the design was that giving was made compulsory in practice. A client hands out the pieces it has already verified while it is still collecting the ones it lacks, and it favors the peers that are sending it data fastest. A participant who has every piece and is now only giving is seeding. The more people wanted a file, the faster it moved, which reversed the behavior of every earlier network, where popularity meant congestion.

The 2005 client wave, and how little of it lasted

The 2005 client wave produced more programs than the protocol has ever needed, and almost none of them are still running. The protocol was open and simple enough that writing a client became a weekend project, and dozens appeared within a few years. Azureus, later renamed Vuze, put a heavy interface on top of the protocol, Transmission arrived in 2005 with the opposite intention and kept its window almost empty, and the same year brought a client written for very low memory use on Windows.

Among the programs that appeared around the protocol in that year was eXeem Lite, an advertising free build of a client that tried to fold a search index into the client itself so that no separate site would be needed. It was one of many such attempts, it never changed the protocol, and like most of that wave it stopped being maintained within about a year. Nothing of it survives in any current client, and the idea it chased arrived instead by another route.

Magnet links removed the last switchable part

Magnet links removed the last switchable part by making the tracker optional. From around 2005, clients added a distributed hash table, a shared lookup structure spread across the participating machines and keyed on the hash of the file's own metadata. Peer exchange arrived alongside it, letting connected peers tell each other about peers they had not met. With both in place, a client that knows a file's hash can assemble a swarm without asking a server at all. A magnet link is that hash written as a line of text, which is why a transfer started from one sits still for a few seconds at the beginning: the client is looking up who has the file before it can collect it.

What survived the era was the mechanism rather than the culture. The Internet Archive publishes torrents for items in its collections, and Linux and BSD distributions have used the protocol for years to move disc images without paying for the bandwidth. Syncthing took the peer discovery ideas in another direction, keeping folders on a person's own machines in step and never involving a shared index. The through line from 1999 onward is short and technical: each generation deleted the piece that had a name and an address, until nothing was left in the middle but the file's own hash.