DataStruct -- Data fragmentation over Lightning



Summary:

This mail proposes a spec for data fragmentation over custom records, allowing for transmission of data exceeding the maximum allowed size over a single HTLC. The purpose of this spec is to define a structure that describes fragmented data, allowing for transmission over separate HTLCs and assisting reassembly on the receiving end. Since these fragments are assumed to be transmitted over Lightning HTLCs, we want to use a compact encoding mechanism using protobuf. The `FragmentInfo` fields describe `fragset_id`, `total_size`, and `offset`. A sender intending to transmit the data to another node should split the bytes of data into fragments, generate an identifier for this data transmission, create a DataStruct instance, encode it to a byte array, and transmit it over the custom records of an HTLC. The receiving node can execute the steps for each received fragment in order to assemble the data. Interoperability could be achieved by different applications utilizing the same TLV as well as data encoding for transmission.


Updated on: 2023-06-01T19:06:15.668266+00:00