The BSV blockchain team are releasing 2 upgrades simultaneously.
These are:
- BSV blockchain Node software – Upgrade to v1.0.10 Release
- mAPI v1.4.0 Release
BSV blockchain Node software – Upgrade to v1.0.10 Release
Version 1.0.10 release is a recommended upgrade from version 1.0.9; This new version brings a number of improvements to both functionality and performance.
Content details listed below:
- Parallel Transaction Validation (PTV) scheduler improvements
- Modification to processing of User Agent strings
- Update default maxscriptsizepolicy, maxscriptnumlengthpolicy
- P2P Header Update
- Versioning
- Change Description
- sendrawtransactions – option to skip some policy checks
- New ZMQ Topics
- Original rawtx, hashtx, rawblock, hashblock topicstx notifications:
- New rawtx2, hashtx2, rawblock2, hashblock2 topicstx notifications:
- Configurable Timeouts
Parallel Transaction Validation (PTV) scheduler improvements
Performance improvements have been made to the processing of long complex transaction graphs where transactions arrive out of order.
Modification to processing of User Agent strings
Currently it is possible that a node may connect to BCH nodes (There is advice on how to avoid this on bitcoin-sv github). This does not cause any major issues but leads to wasted bandwidth and processing of invalid blocks. Invalid blocks generate error messages which pollute log files and make it difficult to see what is going on. The following steps have been taken to make it more likely that a node only connects to other BSV nodes.
- banclientua – contains the list of banned User Agent substrings (the node will ban any peer that returns one of these User Agent strings). The default list is set to: “abc”, “cash”, “bch”
- allowclientua – a newly introduced config option that contains a list of allowed User Agent substrings that overrides banclientua. The default list is empty.
Matching is case insensitive. Note that the default list is cleared if a single banclientua parameter is set in the bitcoin config file or command line.
Example config setting:
banclientua=XYZ
banclientua=ABC
allowclientua=its-not-abc
This will allow/disallow the following User Agent strings:
‘ThisAbcClient’ # banned, matches ABC
‘ThisBchClient’ # allowed because the default has been cleared, so connections to node that return “cash” or “bch” nodes
‘I-cant-believe-its-not-ABC’ # allowed, ABC matches but its-not-abc also matches and has precedence
Update default maxscriptsizepolicy, maxscriptnumlengthpolicy
The default values for maxscriptsizepolicy, maxscriptnumlengthpolicy config options have been updated.
- Default value for maxscriptsizepolicy = 500,000 (up from 10,000)
- Default value maxscriptnumlengthpolicy = 10,000 (down from 250000)
P2P Header Update
Every P2P message on the network has the same basic structure; a 24 byte header followed by some payload data. One of the fields within the header describes the length of that payload, and is currently encoded as a uint32_t. This therefore limits the maximum size of any message payload to 4GB.
In order to support block sizes greater than 4GB, a change has been made to the P2P message structure to overcome this limitation.
Versioning
As of this release the P2P protocol version number has been bumped from 70015 to 70016. Doing this allows a node to know in advance whether a connected peer will understand the new extended message format and therefore avoid sending such messages to that peer. Conforming nodes must not send messages in the extended format to peers with a version number lower than 70016, or they will be banned.
Change Description
In summary, the changes to the P2P message involve the use of special values of fields within the existing P2P header as flags that can be recognised by a peer that supports such changes to indicate that this is a message with a large payload. These special values also allow a peer that doesn’t understand them to reject such a message and fail cleanly if it were to come across one.
The existing P2P header contains a 12 byte message type field. We introduce a new message type in this field extmsg (short for extended message) that when seen will indicate to the receiver that following this message header are a series of new extended message header fields before the real payload begins.
The proposed full extended message format is shown below:
Field Size Name Data Type Description 4 magic uint32_t Existing network magic value. Unchanged here. 12 command char[12] Existing network message type identifier (NULL terminated). For new extended messages this would take the value extmsg. 4 length uint32_t Existing payload length field. Currently limited to a maximum payload size of 4GB. For new extended messages this will be set to the value 0xFFFFFFFF. The real
payload length will be read from the extended payload length field below.4 checksum uint32_t Checksum over the payload. For extended format messages this will be set to 0x00000000 and not checked by receivers. This is due to the long time required to
calculate and verify the checksum for very large data sets, and the limited utility of such a checksum.12 ext_command char[12] The extended message type identifier (NULL terminated). The real contained message type, for example block for a > 4GB block, or could also conceivably
be tx if we decide in future to support > 4GB transactions, or any other message type we need to be large.8 ext_length uint64_t The extended payload length. The real length of the following message payload. variable payload uint8_t[] The actual message payload.
sendrawtransactions – option to skip some policy checks
It is now possible to override policy checks per transaction or per whole batch (transaction specific overrides have precedence over batch specific overrides) when using sendrawtransactions RPC.
These parameters do not override consensus rules.
The following configuration parameters can be overridden: maxtxsizepolicy, datacarriersize, maxscriptsizepolicy, maxscriptnumlengthpolicy, maxstackmemoryusagepolicy, limitancestorcount, limitcpfpgroupmemberscount, acceptnonstdoutputs, datacarrier, dustrelayfee, maxstdtxvalidationduration, maxnonstdtxvalidationduration, minconsolidationfactor, maxconsolidationinputscriptsize, minconfconsolidationinput, acceptnonstdconsolidationinput, dustlimitfactor, skipscriptflags
Example1: Override policy for transactions individually
sendrawtransactions([{‘hex’: signed_tx, ‘dontcheckfee’: True, ‘config’: {“maxtxsizepolicy”: 10000000, “minconfconsolidationinput”: 200, “skipscriptflags”: [“CLEANSTACK”, “DERSIG”]}…])
Example2: Override policy for all transactions in the batch
sendrawtransactions([{‘hex’: signed_tx, ‘dontcheckfee’: True}, …], {“maxtxsizepolicy”: 10000000, “minconfconsolidationinput”: 200, “skipscriptflags”: [“CLEANSTACK”, “DERSIG”]})
New ZMQ Topics
New ZMQ topics have been added to the node. The differences between original and new ZMQ topics are shown below.
Original rawtx, hashtx, rawblock, hashblock topics
tx notifications:
- publish transaction when it is accepted to mempool
- publish transaction when block containing it is connected
- publish transaction when block containing it is disconnected
block notifications:
- publish block when it’s connected
- on reorg only publish the new tip
When a reorg happens, we get notified 3 times for the same transaction (from a disconnected block):
- on block disconnect
- when the transaction is accepted to mempool
- when the new block, the transaction is added to, is connected
New rawtx2, hashtx2, rawblock2, hashblock2 topics
tx notifications:
- publish transaction when it is first seen:
- accepted to mempool, or
- received in a block if we didn’t have it in mempool yet
- when we mine (connect) a block, we publish transactions that weren’t published before (usually only coinbase)
- when we receive a valid block, we publish all the transactions that weren’t published before (the ones in block and not in our mempool)
- we don’t publish the transaction when block containing it is disconnected
block notifications:
- we publish every connected block
When a reorg happens, we get notified once for each transaction and connected block:
- when transaction from a disconnected block is accepted to mempool, or
- when the new block from the longer chain that contains the transaction is connected
- we publish each connected block from the new, longer chain, not only the tip
- ! coinbases from the disconnected blocks are also not published !
Configurable Timeouts
New configuration options are available to address potential issues with download timeouts.
- blockdownloadtimeoutbasepercent (default=100% = 10 minutes)
- blockdownloadtimeoutbaseibdpercent (default=600% = 60 minutes)
- blockdownloadtimeoutbaseperpeerpercent (default=50% = 5 minutes)
Block Download Timeout Calculation
If the block is being downloaded as part of an initial block download (IBD), then the
base-timeout = blockdownloadtimeoutbaseibdpercent,
otherwise
base-timeout = lockdownloadtimeoutbasepercent.
The timeout is calculated as follows
Max. download timeout = 10 minutes x (base-timeout + no-of-peers x blockdownloadtimeoutbaseperpeerpercent)
For example, if there are a 3 connected peers, the default IBD timeout will be 10 minutes x (600 + 3 x 50)% = 10 minutes x (750%) = 75 minutes.
Binaries and source code can be downloaded here:
URL: https://download.bitcoinsv.io/bitcoinsv/
Should you have any support questions, please direct them via [email protected] or via telegram at https://t.me/bitcoinsvsupport
mAPI software – Upgrade to v1.4.0 Release
Version 1.4.0 release is a recommended upgrade from version 1.3.0 release; With this release of mAPI we are introducing miner Policy Quotes. These enable the merchant to obtain information on miner policies, including the IP address needed to build a DSNT transaction output (see below), and the ability to skip some policy checks when transactions are submitted to node, such as:
“maxtxsizepolicy”: 999999,
“datacarriersize”: 100000,
“maxscriptsizepolicy”: 100000,
“maxscriptnumlengthpolicy”: 100000,
“maxstackmemoryusagepolicy”: 100000,
“limitancestorcount”: 100000,
“limitcpfpgroupmemberscount”: 10,
“acceptnonstdoutputs”: true,
“datacarrier”: true,
“dustrelayfee”: 150,
“maxstdtxvalidationduration”: 99,
“maxnonstdtxvalidationduration”: 100,
“dustlimitfactor”: 10,
“skipscriptflags”: [“CLEANSTACK”, “DERSIG”]
Content details listed below:
- Supports policyQuote as well as feeQuote
- Provides DSNT information for transaction builder
- Administrator may configure accounts to specify Node options to skip some policy checks when transactions are submitted
- Made ready for blocks greater than 4 GB
- Performance Optimization
Specification: https://github.com/bitcoin-sv-specs/brfc-merchantapi
Miner Policy Quotes
This release of mAPI specification introduces support for obtaining policy quotes from miners.
These are a superset of the mAPI v1.3.0 fee quotes, which are maintained for backward compatibility.
Policies are configured by the mAPI Administrator for each account and include Node options to skip some policy checks when transactions are submitted.
Double-spend Processing
In addition to mAPI using ZMQ subscriptions for local node double-spend detection, any remote node in the BSV network that detects a double spend of a transaction which contains a DSNT output (a double spend notification record), will notify the specified DSNT servers (such as mAPI) about the detected double spend through a HTTP callback and handshake.
Please refer to https://github.com/bitcoin-sv-specs/protocol/blob/master/updates/double-spend-notifications.md for further details.
Binaries and source code can be downloaded here:
URL: https://github.com/bitcoin-sv/merchantapi-reference
Should you have any support questions, please direct them via [email protected] or via telegram at https://t.me/bitcoinsvsupport
Thank you for your continued support of BSV blockchain.
