Because the introduction of headers-first synchronization in Bitcoin Core 0.10, the blockchain information construction may be seen as having three parts:

• The block header tree: a tree information construction, with the genesis block as root, containing all headers for identified forks of the chain. This tree construction is at all times related, i.e., it’s not doable so as to add a header to the tree until its guardian is already within the tree as properly.
• The block information: for every entry within the block header tree, we might or might not have the corresponding block information. Blocks may be obtained (and requested for) out of order, so long as the corresponding header exists within the header tree. There may be gaps, so it’s not the case {that a} block can solely be saved if the guardian block can also be saved. In fact, a block can solely change into the lively chain tip if it is totally validated, which requires having seen all blocks earlier than it too. Because the introduction of block file pruning in Bitcoin Core 0.11, it is usually doable to delete blocks once more after being validated; on this case, the corresponding headers are nonetheless stored within the tree.
• The lively chaintip (a reference to an entry in our block header tree) with its corresponding UTXO set (or chainstate), representing the fully-validated block that we presently take into account lively (this was launched in Bitcoin 0.8).

Now to elucidate how synchronization occurs, there are three separate processes that govern these information constructions respectively:

1. Header synchronization: the method of requesting, receiving, and storing, headers {that a} peer has however we don’t.
2. Block synchronization: the method of requesting, receiving, and storing, the complete block information for headers we have already got, which friends have however we don’t.
3. Block activation: the complete validation of blocks we’ve, and ensuing altering of the lively chain tip.

All of them occur in parallel in follow, however it’s simpler to consider them as a sequence.

1. Headers synchronization

Requesting headers. Step one of synchronization is studying about headers which our friends have. That is primarily finished by sending getheaders to every peer after we join or settle for a connection (by sending a locator which signifies which headers we have already got), to which they’re going to reply with a headers message containing as much as 2000 new headers. If the complete 2000 headers are obtained, we’ll ship one other getheaders to ask for extra. This course of continues till we’ve all headers the peer has to supply. After we’re far behind, we solely ask one peer (the headers sync peer) to attenuate duplicating bandwidth utilization, however as soon as we get near the present timestamp, all friends are requested.

Non-connecting headers. If headers arrive that do not hook up with our present headers tree (i.e., the guardian of a obtained block is not in our tree but), we additionally ship a getheaders in response to first find out about all of the headers in between. By no means are headers processed after we do not know their dad and mom but. This doesn’t suggest they’re rejected (as in: they don’t seem to be marked completely invalid); they’re simply ignored (as in the event that they have been by no means obtained in any respect). If all goes properly, we’ll obtain them once more after their lacking guardian headers have arrived, at which level they’re going to be processed usually.

Direct headers announcement. Because the introduction of the direct headers announcement mechanism (negotiated utilizing the BIP130 sendheaders message) added in Bitcoin Core 0.12.0, friends can ship us headers messages on to announce a brand new lively chain tip (see beneath). In case this mechanism is just not used (both as a result of the peer believes we’re far behind, or as a result of the peer would not help BIP130), they’ll additionally ship us an inv message with only a block hash. This too will set off us to ship a getheaders in response to find out about any headers we miss, as much as and together with the hash that was simply inved to us.

Header validation. When headers arrive, they’re validated to the extent doable. This contains syntactic correctness, proof-of-work, problem changes, guidelines about timestamps (bigger than the median of the previous 11 blocks, no more than 2 hours sooner or later), and guidelines about model numbers (BIP34, BIP66, and BIP65 put necessities on the model quantity). If a header fails any of those, or descends from a block/header which is thought to be invalid, it’s ignored.

Header spam safety. Whereas block headers are tiny (81 bytes in a header message), we do want to guard towards storing tons of low-difficulty ones that an attacker may create. Proof-of-work inherently makes it very costly to supply good headers in mass, however an attacker can assemble a headers chain that forks off someplace early in historical past (e.g. simply after the genesis block) when the issue was low, with out ever amounting to a major quantity of labor. This might in the end result in reminiscence points for nodes if they might settle for these headers into their block header tree. Traditionally, checkpoints have been (amongst different issues) used to protect towards this, however since Bitcoin Core 24.0 one other, extra complete, mechanism is used: header pre-synchronization. In brief, it entails downloading the headers twice: a primary time to confirm the headers kind a series with a major quantity of labor (however not including them to the block header tree but), and a second time the place they’re redownloaded and in the event that they match what was despatched within the first section, they’re added to the block header tree. You may learn extra about this mechanism in this reply.

2. Block synchronization

Acceptable blocks. As soon as we all know concerning the headers, we are able to obtain blocks. Usually, blocks are solely requested (and accepted) if they’re a part of a headers chain whose tip has a minimum of as a lot cumulative work as the present lively chain tip. Blocks that aren’t on our headers tree are by no means accepted (these could be orphan blocks), nor are blocks on forks whose header chain would not have a aggressive quantity of cumulative work.

Request mechanism. Blocks are requested utilizing getdata messages, to which the peer responds with block messages (getdatas are additionally used for transaction fetching, however I’m ignoring that right here). To determine which friends these requests are despatched to, we maintain observe of the inv and headers bulletins friends have despatched us, so we’ve an thought of what the final block header we’ve in frequent with them is. We typically assume that friends who announce a block to us may truly present the corresponding block information (a minimum of once they set the NODE_NETWORK service flag, or NODE_NETWORK_LIMITED for sufficiently current blocks).

Scheduling of requests. The precise getdata requests are despatched for lacking blocks wherever alongside the trail from the genesis block to the headers tree tip with the best collected work (which may be distinct from the lively chain tip) amongst all block header tree entries which aren’t marked as completely invalid. The requests are unfold over all friends for which we all know they’ve the respective block, in a round-robin style, with a restrict of 16 requested-but-not-yet-received blocks per peer. We don’t request blocks that are greater than 1024 blocks forward of the present lively chain tip, to restrict how out-of-order blocks may be obtained (this issues for block file pruning). So long as a block request is excellent, the identical block is just not requested once more from one other peer to keep away from duplicating bandwidth utilization. If a peer stalls too lengthy in responding to a block request, it’s disconnected, and the request can be despatched elsewhere.

Unrequested blocks. It’s doable {that a} block is obtained with out it being requested. Bitcoin Core won’t ship unrequested blocks, however another P2P purchasers may. If this occurs, we are going to course of the header contained in the block as if it was obtained by way of a headers message, after which roughly if we’d need to request it after seeing the header, we additionally course of the block immediately. In any other case, it’s ignored (i.e., we act as if solely the header was obtained).

Compact block bulletins. In Bitcoin Core 0.13.0, help for BIP152 compact blocks was added. In compact blocks, blocks are despatched with out their full transaction information, however with quick (48-bit) salted hashes of the person transactions as a substitute. When receiving a compact block, the receiver tries to reconstruct the complete block utilizing transactions they’ve in reminiscence, after which requests any lacking ones. The small print are out of scope right here, however compact blocks features a function known as high-bandwidth mode. When enabled, we allow a restricted variety of friends to announce blocks to us immediately utilizing a cmpctblock message, skipping a roundtrip utilizing headers/inv and getdata which saves latency. On this case, we once more course of the header embedded within the compact block first, after which course of the compact block if the block appears like it is rather near our lively chain tip (max 2 blocks forward).

Block validation. When a block is processed, all relevant consensus guidelines are carried out, which incorporates syntactic validity, recomputing the transaction Merkle root, transaction finality, BIP34 (“top in coinbase”), segwit commitments, and the utmost block weight. If these checks fail, the block is ignored, or in some circumstances marked as completely invalid, relying on the kind of error:

• When the invalidity is because of a dedication failure (e.g. the Merkle root of the offered transactions don’t match the Merkle tree within the block header), the block is barely ignored. It can’t be marked invalid, as a result of we’ve not seen the block’s precise transactions on this case – and so they may nonetheless be legitimate.
• For different failures, the block header tree entry is marked as invalid.

Invalid blocks. When a block is completely invalid, the present block header tree entry is stored, however the block nor any of its descendants will ever be thought-about for downloading or activation anymore. If the checks succeed, the block information is saved on disk. Scripts, double-spending, and inflation can’t be verified but.

getblocks based mostly synchronization. Earlier than the introduction of headers-first synchronization a distinct mechanism was used. As an alternative of getheaders to provoke synchronization, the same getblocks message was despatched, to which the peer would reply with an inv containing as much as 500 block hashes, which might then set off getdatas and additional getblocks to proceed. Bitcoin Core not makes use of this mechanism since model 0.10, however it’s nonetheless supported for friends who need to synchronize this manner (i.e., Bitcoin Core will reply to getblocks messages, however won’t ship them).

3. Block activation

Lastly, when we’ve the blocks, we are able to determine what our lively chain tip needs to be, and totally validate the blocks concerned.

Lively chain tip choice. The rule is that at each time limit, we intention to make the lively chain tip be:

1. Amongst all blocks within the block header tree for which we’ve the complete information and full information for all its ancestors, and which aren’t (but) marked as completely invalid…
2. contemplating solely those with the best cumulative work…
3. and amongst these, choose the one for which the complete information was obtained first.

Transferring in the direction of the choice. If, at any level, the results of this choice is just not the identical as the present lively chain tip, we begin making progress in the direction of the choice. First the blocks which might be a part of the lively chain however not within the chosen chain are disconnected in reverse order, after which the blocks within the choice not within the lively chain are related in ahead order. If disconnections are concerned within the activation course of, it’s known as a reorganization.

Full block validation. Connecting entails performing full validation, of all consensus guidelines, together with script validity, subsidy checks, and checking for double-spends, which want entry to the UTXO set. We solely keep the UTXO set on the lively chain tip, so connecting entails eradicating from the UTXO set any inputs within the block, and including any outputs. Disconnecting does the reverse, eradicating any outputs the disconnected block added, and re-adding any inputs it spent. If validation fails right here, the block is marked as completely invalid, which can possible trigger the choice logic above to alter its thought about what the lively chain tip needs to be.

Asserting new chain ideas. Every time the activation course of stabilizes, friends are despatched an announcement of our new lively chain tip. If just a few blocks have been related, and the peer helps BIP130, a headers message is distributed with the headers of all related blocks. In any other case we revert to sending an inv of simply the brand new tip. In case the peer chosen us as a high-bandwidth compact blocks peer, the announcement is distributed earlier than full validation completes in circumstances permitted by BIP152 (once more as a way to scale back latency).