Hi Sjors, > In the context of BIP30 [0] Eric Voskuil brought up performance:
I didn't originate the point on performance. I said (in the BIP30 thread): > One reference states that “assume valid” speeds IBD, but of course it does so > by not validating. Which was in response to your OP of this thread, which provided that reference, specifically: "Assumed Valid Blocks: a feature designed to replace the secondary use of checkpoints for (optionally) speeding up Initial Block Download (IBD) by skipping validation of signatures in old blocks. This was deployed in Bitcoin Core 0.14" - David A. Harding This in turn references the following discussion: "Bitcoin 0.5.0 built upon those checkpoints to speed up IBD by skipping verification of signatures in blocks that were earlier in the block chain than the most recent checkpoint." https://bitcoincore.org/en/2017/03/08/release-0.14.0/#assumed-valid-blocks > > The top checkpoint is consensus for over 11 years and materially reduces > the validation cost of 295,000 blocks. > > I don't think performance should be a consideration when removing > checkpoints. To be perfectly clear, I am not arguing against this hard fork because it reduces IBD cost. I’m pointing out that one of the arguments *in favor* of removing checkpoints is that "assume valid" now serves this "secondary use". However, as I pointed out, assume valid is not consensus, it achieves this outcome by trusting, not validating. The existing checkpoints are consensus - they provide this advantage when fully validating. I'm not suggesting that checkpoints need to be added to improve performance. I'm pointing out that removing them hurts performance. So performance is obviously not a reason to accept such a hard fork. > Afaik checkpoints were not introduced as a performance feature, but rather as > a DoS vulnerability fix. It appears from the above discussion that there was more than one reason. However this isn't relevant. What matters is the consequences of removing them, not why they were introduced. > Even if they were, imo they shouldn't be used for performance enhancement, > as it creates a temptation to add more - although Eric clearly says: > > > I would never advocate for adding more > > But perhaps someone else would fork Bitcoin Core (or Libbitcoin) and start > with honest checkpoints, in order to gain adoption due its excellent > performance - especially when combined with a UTXO snapshot. And then > suddenly it has an "upgrade". > > But even if this wasn't a concern, This feels more like speculation than a reasonable concern. To date bitcoind has introduced two trust-based mechanisms (assume valid and assume utxo) both with the objective of avoiding validation to improve IBD time. As I understand it, BC ships with a very recent assume valid configuration by default. If you have this concern, it should be directed these current practices, not at ancient checkpoints. Removing the latter does not eliminate the former - which should be the current concern. I've recently seen declarations that the IBD problem is "solved" because of assume valid and/or assume utxo. That's not speculation. > the performance benefit of the latest > checkpoint from 2014 is just very small. That's because the early blocks don't > have that many signatures to check. The first 5 years of Bitcoin represent > only > 3% of all historical transactions. This assumes that signature validation is the major cost, which is not the case for bitcoind architectures. > On an M4 MacBook I did an IBD up to the last checkpoint at height 295,000. > This took 17.5 minutes. I have done this many times, on powerful machines, and it's generally at least an hour - even with checkpoints (and headers excluded). > I then ran it again with -assumevalid=0, i.e. checking all signatures, which > took > 18 minutes. The major cost for the bitcoind architecture is unspent output accumulation. There is only a modest distinction between full validation and assume valid across the entire chain. As a result this measurement doesn't reflect the true (necessary) cost differential. I just performed a 1-295,000 block libbitcoin sync with and without checkpoints (and no milestone/assume valid). The checkpointed time is 3.4 minutes and without is 8.3 minutes (on the same machine BC takes 69.3 minutes with checkpoints). So checkpoints validated at 41% of the time vs. without (vs. your 97%). Again, this is not the basis of my comments, just clarifying for the record. > These times are from connecting the first block until connecting block > 295,000. They exclude the 1.5 minutes spent on the header pre-sync > mechanism, which is required to operate safely with or without (new) > checkpoints. Same in both cases. > Libbitcoin would probably find a different ratio, as might a machine with > severe CPU limitations, but I suspect it will be equally negliable. And > without > new checkpoints the ratio keeps going down over time. Of course, the ratio is small and will keep decreasing. On this machine it's a difference of 4.9 minutes out of 4.4 hours for libbitcoin. The ratio is smaller for BC, as it takes 24.5 hours to sync to the same height even with assume valid. > That said, I've never been a fan of -assumevalid conceptually. I agree, not a fan. We provide milestones (same security model) so that people can rapidly reproduce a previously synced node from the network by specifying a milestone that they have previously validated. On my current low end ($350) benchmark on a 2.3 Gbps Internet, this takes under 40 minutes to block 850k, and checkpoints don't make a difference either way. It can be faster to sync from the p2p network than to copy the store, and of course doesn't require a local copy. So it's a useful feature, but not a substitute for validation. > Hopefully it can > be replaced by something like AssumeUTXO + SwiftSync [1] for the > background with full validation using block Undo data. I'm not so clear on why BC wants to give people the impression that a node is usable when its chain hasn't been validated. I don't see this as the "hopeful" outcome, but that's another topic I suppose. I also don't see the reason for all of this complexity to end up with a security model identical to assume valid (until it's actually validated). Why not just sync the strong blocks and validate them in the background without all of the fuss, extra bandwidth, and precomputations? In any case, I still do not see ANY justification for hard forking out the checkpoints, and there are of course legitimate reasons to avoid doing so. https://groups.google.com/g/bitcoindev/c/aqHRfa0UWFo Best, Eric -- You received this message because you are subscribed to the Google Groups "Bitcoin Development Mailing List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/bitcoindev/035701dbbba7%24807f23b0%24817d6b10%24%40voskuil.org.
