This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these messages)
|
| Torrent files | |
|---|---|
| Filename extension |
.torrent |
| Internet media type |
application/x-bittorrent |
| Developed by | Bram Cohen |
| Initial release | July 2, 2001 |
| Standard | BEP-0003 (v1), BEP-0052 (v2) |
In the BitTorrent file distribution protocol, a torrent file or metainfo file is a computer file format that contains metadata about files and folders to be distributed, in addition to a list of network locations of trackers, which are computers that help participants in the system find each other and form efficient distribution groups called swarms. Torrent files use .torrent as the filename extension.
A torrent file acts as a table of contents (index), allowing a client to find other decentralized nodes interacting with the same torrent file. With the help of a torrent file, one can download small parts of the original file from nodes that have already downloaded it. These "peers" enable clients to download one or many files. A torrent file does not contain the content to be distributed; it only contains metadata information about those files, such as their names, folder structure, sizes, and cryptographic hash values for verifying file integrity. Torrent files also contain SHA cryptographic hashes of the file pieces.
The BitTorrent Project describes it as a free speech tool allowing users to share information without bearing high bandwidth costs. Instead of individual clients fetching files from a central server, BitTorrent eliminates the need for central content storage by allowing clients to download and upload thousands of chunks of the original file. Many free/freeware programs and operating systems, such as the various Linux distributions offer a torrent file as a P2P download option for users seeking the aforementioned benefits. Torrent files can store the metadata of any file format.
Traditional client-server architecture has an asymmetric flow of data: a server only sends data and a client only downloads the data. This puts an asymmetric bandwidth load on the server, especially as the number of concurrent clients increases. The BitTorrent protocol addresses this by decentralizing the distribution. Each client also acts as the server. Peers in BitTorrent swarms don't require a complete copy of the file to start contributing to the swarm; the client can start uploading as soon as it has a piece.
Each file to be distributed is divided into small information chunks called pieces. Downloading peers achieve high download speeds by requesting multiple pieces from different computers simultaneously in the swarm. Once obtained, these pieces are usually immediately made available for download by others in the swarm. In this way, the burden on the network is spread among the peers, rather than concentrating at a central distribution hub or cluster. As long as all the pieces are available, peers can come and go; no one peer needs to have all the chunks or to even stay connected to the swarm in order for distribution to continue among the remaining peers.
A torrent file is created to represent a file or folder to be shared. In order to get the list of peers who may be sharing pieces, the client connects to the trackers named in the torrent file. A similar result can also be achieved through the use of distributed hash tables (DHT). Then, the client connects directly to the peers in order to request pieces and otherwise participate in a swarm. The client may also report progress to trackers.
When the client has all the pieces, the torrent client assembles them into a complete form. They may also continue sharing the pieces, elevating their status to that of seeder rather than an ordinary peer.
A torrent file contains a list of files and integrity metadata about all the pieces, and optionally contains a large list of trackers.
A torrent file is a bencoded dictionary with the following keys (the keys in any bencoded dictionary are lexicographically ordered):
All strings must be UTF-8 encoded, except for pieces, which contains binary data.
A torrent is uniquely identified by an infohash, a SHA-1 hash calculated over the contents of the info dictionary in bencode form. Changes to other portions of the torrent do not affect the hash. This hash is used to identify the torrent to other peers via DHT and to the tracker. It is also used in magnet links.
The BitTorrent v2 protocol (BEP-0052) introduces a new definition of the torrent file. The basic structure is:
The new format uses SHA-256 in both the piece-hashing and the infohash, replacing the broken SHA-1 hash. The "btmh" magnet link would contain the full 32-byte hash, while communication with trackers and on the DHT uses the 20-byte truncated version to fit into the old message structure. It is possible to construct a torrent file with only updated new fields for a "v2" torrent, or with both the old and new fields for a "hybrid" format. However, as a torrent would have different infohashes in v1 and v2 networks, two swarms would form, requiring special handling by the client to merge the two.
A core feature of the new format is its application of merkle trees, allowing for 16KiB blocks of a piece to be individually verified and re-downloaded. Each file now always occupy whole piece sizes and have an independent merkle root hash, so that it's possible to find duplicate files across unrelated torrent files of any piece length. The file size is not reduced (assuming piece size stays the same; v2's tree structure allows larger pieces with fewer ill effects), but the info dictionary required for magnet links contains additional fields not present in the first version of the specification. (only in v2-only torrents).
A torrent file can also contain additional metadata defined in extensions to the BitTorrent specification. These extensions are known proposed to the community through the BitTorrent Enhancement Proposal (BEP) document.
These extensions have been approved and deployed in one or more implementations as well as having been proven useful through consistent and widespread use.
BEP-0005 extends BitTorrent to support distributed hash tables, specifically Mainline DHT.
A trackerless torrent dictionary does not have an announce key. Instead, a trackerless torrent has a nodes key:
{
# ...
'nodes': [["<host>", <port>], ["<host>", <port>], ...],
# ...
}
For example,
'nodes': [["127.0.0.1", 6881], ["your.router.node", 4804]],
The specification recommends that nodes "should be set to the K closest nodes in the torrent generating client's routing table. Alternatively, the key could be set to a known good node such as one operated by the person generating the torrent."
BEP-0012 extends BitTorrent to support multiple trackers.
A new key, announce-list, is placed in the top-most dictionary (i.e., with announce and info)
{
# ...
'announce-list': [['<tracker1-url>']['<tracker2-url>']],
# ...
}
BEP-0019 is one of two extensions allowing HTTP seeds to be used in BitTorrent.
In BEP-0019, a new key url-list, is placed in the top-most list. The client uses the links to assemble ordinary HTTP URLs – no server-side support is required. This feature is very commonly used by open source projects offering software downloads. Web seeds allow smart selection and simultaneous use of mirror sites, P2P or HTTP(S), by the client. Doing so reducing the load on the project's servers while maximizing download speed. MirrorBrain automatically generates torrents with web seeds.
BEP-0027 extends BitTorrent to support private torrents.
A new key, private, is placed in the info dictionary. This key's value is 1 if the torrent is private:
{
# ...
'info': {
# ...
'private': 1,
# ...
},
# ...
}
Private torrents are to be used with a private tracker. Such a tracker restricts access to torrents it tracks by checking the peer's IP, refusing to provide a peer list if the IP is unknown. The peer itself is usually registered to the tracker via a gated online community; the private tracker typically also keep statistics of data transfer for use in the community.
Decentralized methods like DHT, PeX, LSD are disabled to maintain the centralized control. A private torrent can be manually edited to remove the private flag, but doing so will change the info-hash (deterministically), forming a separate "swarm" of peers. On the other hand, changing the tracker list will not change the hash. The flag does not offer true privacy, instead operating as a gentlemen's agreement.
These extensions are under consideration for standardization. Most are already widely adopted as de facto standards.
BEP-0017 extends BitTorrent to support HTTP seeds, later more commonly termed "web seeds" to be inclusive of HTTPS.
In BEP-0017, a new key, httpseeds, is placed in the top-most list (i.e., with announce and info). This key's value is a list of web addresses where torrent data can be retrieved. Special server support is required. It remains at Draft status.
{
# ...
'httpseeds': ['http://www.site1.com/source1.php', 'http://www.site2.com/source2.php'],
# ...
}
BEP-0030 extends BitTorrent to support Merkle trees (originally implemented in Tribler). The purpose is to reduce the file size of torrent files, which reduces the burden on those that serve torrent files.
A torrent file using Merkle trees does not have a pieces key in the info list. Instead, such a torrent file has a root_hash key in the info list. This key's value is the root hash of the Merkle hash:
{
# ...
'info': {
# ...
'root hash': <binary SHA1 hash>,
# ...
},
# ...
}
BitTorrent v2 uses a different type of Merkle tree.
A de-bencoded torrent file (with piece length 256 KiB = 262,144 bytes) for a file debian-503-amd64-CD-1.iso (whose size is 678 301 696 bytes) might look like:
{
'announce': 'http://bttracker.debian.org:6969/announce',
'info':
{
'length': 678301696,
'name': 'debian-503-amd64-CD-1.iso',
'piece length': 262144,
'pieces': <binary SHA1 hashes>
}
}
A de-bencoded torrent file (with 'piece length' 256 KiB = 262144 B) for two files, 111.txt and 222.txt, might look like:
{
'announce': 'http://tracker.example.com/announce',
'info':
{
'files':
[
{'length': 111, 'path': ['111.txt']},
{'length': 222, 'path': ['222.txt']}
],
'name': 'directoryName',
'piece length': 262144,
'pieces': <binary SHA1 hashes>
}
}
A hybrid torrent contains both the v1 and the v2 fields, so the same torrent can be shared in v1 and v2 swarms. Both sets of fields must describe the same data in the same order. Version 2 requires every file to start at a piece boundary, but version 1 does not, so a hybrid torrent with multiple files inserts padding files into the v1 files list to make the two layouts match. Padding files are defined in BEP-0047: they are marked with the attribute p, are treated as all zeros when piece hashes are calculated, and do not need to be written to disk by clients that support them. The specification recommends placing them under a .pad directory, named after their length.
The following example describes the same two files as the previous example. Because 111.txt is shorter than one piece, a 262,033-byte padding file follows it so that 222.txt begins at the start of the second piece. Neither file is larger than one piece, so piece layers is empty; it is still required.
{
'announce': 'http://tracker.example.com/announce',
'info':
{
'file tree':
{
'111.txt': {'': {'length': 111, 'pieces root': <32-byte SHA-256 merkle root>}},
'222.txt': {'': {'length': 222, 'pieces root': <32-byte SHA-256 merkle root>}}
},
'files':
[
{'length': 111, 'path': ['111.txt']},
{'attr': 'p', 'length': 262033, 'path': ['.pad', '262033']},
{'length': 222, 'path': ['222.txt']}
],
'meta version': 2,
'name': 'directoryName',
'piece length': 262144,
'pieces': <binary SHA1 hashes>
},
'piece layers': {}
}
{{citation}}: CS1 maint: work parameter with ISBN (link)