zkEVM DataStreamer
The first page is the header.
Header page size = 4096 bytes
Magic numbers
At the beginning of the file there are the following magic bytes (file signature): polygonDATSTREAM
u8 packetType = 1 // 1:Header
u32 headerLength = 29 // Total length of header entry
u64 streamType // 1:Sequencer
u64 TotalLength // Total bytes used in the file
u64 TotalEntries // Total number of data entries
Data page
From the second page starts the data pages.
Page size = 1 MB
Data entry format (FileEntry struct)
u8 packetType // 2:Data entry, 0:Padding
u32 Length // Total length of data entry (17 bytes + length(data))
u32 Type // 0xb0:Bookmark, 1:Event1, 2:Event2,...
u64 Number // Entry number (sequential starting with 0)
u8[] data
If an entry do not fits in the remaining page space we store this entry in the next page.
Diagram
TCP/IP Commands
All the commands available for stream clients returns a result.
Some commands like start may return more data.
Start
Syncs from entry number (fromEntryNumber
) and starts receiving data streaming.
u64 command = 1
u64 streamType // 1:Sequencer
u64 fromEntryNumber
StartBookmark
Syncs from bookmark (fromBookmark
) and starts receiving data streaming.
u64 command = 4
u64 streamType // 1:Sequencer
u32 bookmarkLength // Length of fromBookmark
u8[] fromBookmark
After either start command, the system start stream data from that entry number or bookmark.
If already started terminate connection.
Stop
u64 command = 2
u64 streamType // 1:Sequencer
Stop streaming data.
If not started terminate connection.
The result is sended just after it's stopped.
u64 command = 3
u64 streamType // 1:Sequencer
Returns the current status of the header.
If started, terminate the connection.
Result format (ResultEntry struct)
u8 packetType // 0xff:Result
u32 length // Total length of the entry
u32 errorNum // Error code (0:OK)
u8[] errorStr
Sequencer entries
Start L2 Block
Entry type = 1
Entry data:
u64 batchNum
u64 blockL2Num
u64 timestamp
u8[32] globalExitRoot
u8[20] coinbase
u16 forkId
L2 TX
Entry type = 2
Entry data:
u8 gasPricePercentage
u8 isValid // Intrinsic
u32 encodedTXLength
u8[] encodedTX
End L2 Block
Entry type = 3
Entry data:
u64 blockL2Num
u256 l2BlockHash
u256 stateRoot
API Interface
Send data
- StartAtomicOp()
- AddStreamBookmark(u8[] bookmark) -> returns u64 entryNumber
- AddStreamEntry(u32 entryType, u8[] data) -> returns u64 entryNumber
- CommitAtomicOp()
- RollbackAtomicOp()
Query data
- GetHeader() -> returns struct HeaderEntry
- GetEntry(u64 entryNumber) -> returns struct FileEntry
- GetBookmark(u8[] bookmark) -> returns u64 entryNumber
- GetFirstEventAfterBookmark(u8[] bookmark) -> returns struct FileEntry