Published on: 2014-01-27T02:05:31+00:00
Andreas Schildbach, a Bitcoin developer, encountered an issue with the protobuf format while experimenting with BIP70/71 (payment protocol) usage in face to face payments. Protobufs are not self-delimiting. This means if you're reading from an undelimited stream, you will read endlessly because you don't know how much to read. The current BIP70 implementations probably work because they're reading either from a file or from an HTTP resource which sets the Content-Length header. However, the Content-Length header is optional, and there are many kinds of streams that don't have this built-in delimiting mechanism.Andreas suggests using delimited I/O for PaymentRequest, Payment, and PaymentACK messages that write the size of the message as a varint before writing the data. Nonetheless, this leading varint is an incompatible change and would need to be added to the spec. Bluetooth, Wifi Direct, HTTP request/responses via broken proxies, smoke signals, basically anything that is a stream rather than a file can be affected by this problem. NFC NDEF and QR codes are not affected, so we can skip the delimiter for these mediums.Andreas Schildbach, an experienced developer in the Bitcoin industry, wrote an email to the Bitcoin development mailing list on January 26, 2014. In this email, he discussed his experimentation with BIP70/71 (payment protocol) usage in face-to-face payments. He highlighted an issue with the protobuf format and noted that Protobufs are not self-delimiting. This means that if you're reading from an undelimited stream, you will read endlessly because you don't know how much to read. The current BIP70 implementations probably work because they're reading either from a file or from an HTTP resource which sets the Content-Length header. However, the Content-Length header is optional, and also there are many kinds of streams that don't have this built-in delimiting mechanism.The Java protobuf API solves this by offering delimited I/O like payment.writeDelimitedTo(os).Schildbach suggested that it might be a good idea to apply this to all messages if any, but he specifically encountered this with PaymentMessage and PaymentACK. He further explained that if it turns out we need to retrofit messages with length prefixes, we would have to restart with new mime-types since the nature of prefixes being always at the start rules out simply incrementing a protocol version number.Mike Hearn, another member of the Bitcoin development community, responded to Schildbach's email. Hearn agreed with Schildbach's suggestion, saying that for "binding" the payment protocol to those transports, they should indeed use protobuf varint length prefixes. However, he noted that it's unnecessary for all cases and suggested skipping the delimiter for NFC NDEF and QR codes. Hearn also mentioned that unless Gavin feels it'd be better to be consistent everywhere and is willing to change the spec and code - as far as he knows though they're trying to ship 0.9rc1 soonish.The email thread concluded with a link to CenturyLink Cloud's services and the Bitcoin-development mailing list details.This email thread discusses the issue of using BIP70/71 (payment protocol) in face-to-face payments. The problem encountered is that protobufs are not self-delimiting, meaning that if one is reading from an undelimited stream, they may read endlessly because they don't know how much to read. Current BIP70 implementations work because they're either reading from a file or from an HTTP resource that sets the Content-Length header. However, the Content-Length header is optional and many types of streams don't have this built-in delimiting mechanism. The Java protobuf API solves this by offering delimited I/O, which writes the size of the message as a varint before writing the data. This leading varint is an incompatible change and would need to be added to the spec. The issue was encountered specifically with PaymentMessage and PaymentACK, but it might be a good idea to apply this to all messages if any. It's open for discussion.In a message to the Bitcoin-development mailing list dated 26th January, 2014, Andreas Schildbach reported an issue with the protobuf format when experimenting with BIP70/71 payment protocol usage in face-to-face payments. Protobufs are not self-delimiting, meaning that if reading from an undelimited stream, one would read endlessly because they don't know how much to read. The current BIP70 implementations probably work because they're reading either from a file or from an HTTP resource which sets the Content-Length header. However, the Content-Length header is optional, and also there are many kinds of streams that don't have this built-in delimiting mechanism.
Updated on: 2023-08-01T07:20:27.876420+00:00