Documentation ¶
Overview ¶
Package data supports schema-less serializaiton and deserialization of atproto data
Some restrictions from the data model include: - string sizes - array and object element counts - the "shape" of $bytes and $blob data objects - $type must contain a non-empty string
Details are specified at https://atproto.com/specs/data-model
This package includes types (CIDLink, Bytes, Blob) which are represent the corresponding atproto data model types. These implement JSON and CBOR marshaling in (with whyrusleeping/cbor-gen) the expected way.
Can parse generic atproto records (or other objects) in JSON or CBOR format in to map[string]interface{}, while validating atproto-specific constraints on data (eg, that cid-link objects have only a single field).
Has a helper for serializing generic data (map[string]interface{}) to CBOR, which handles converting JSON-style object types (like $link and $bytes) as needed. There is no "MarshalJSON" method; simply use the standard library's `encoding/json`.
Index ¶
- Constants
- func ExtractTypeCBOR(b []byte) (string, error)
- func ExtractTypeCBORReader(r io.Reader) (string, []byte, error)
- func ExtractTypeJSON(b []byte) (string, error)
- func MarshalCBOR(obj map[string]any) ([]byte, error)
- func UnmarshalCBOR(b []byte) (map[string]any, error)
- func UnmarshalJSON(b []byte) (map[string]any, error)
- func Validate(obj map[string]any) error
- type Blob
- type BlobSchema
- type Bytes
- type CIDLink
- func (ll CIDLink) CID() cid.Cid
- func (ll CIDLink) IsDefined() bool
- func (ll *CIDLink) MarshalCBOR(w io.Writer) error
- func (ll CIDLink) MarshalJSON() ([]byte, error)
- func (ll CIDLink) String() string
- func (ll *CIDLink) UnmarshalCBOR(r io.Reader) error
- func (ll *CIDLink) UnmarshalJSON(raw []byte) error
- type GenericRecord
- type JsonBytes
- type LegacyBlobSchema
Constants ¶
const ( // maximum size of any CBOR data, in any context, in atproto MAX_CBOR_SIZE = 5 * 1024 * 1024 // maximum serialized size of an individual atproto record, in CBOR format MAX_CBOR_RECORD_SIZE = 1 * 1024 * 1024 // maximum serialized size of an individual atproto record, in JSON format MAX_JSON_RECORD_SIZE = 2 * 1024 * 1024 // maximum serialized size of blocks (raw bytes) in an atproto repo stream event (NOT ENFORCED YET) MAX_STREAM_REPO_DIFF_SIZE = 4 * 1024 * 1024 // maximum size of a WebSocket frame in atproto event streams (NOT ENFORCED YET) MAX_STREAM_FRAME_SIZE = MAX_CBOR_SIZE // maximum size of any individual string inside an atproto record MAX_RECORD_STRING_LEN = MAX_CBOR_RECORD_SIZE // maximum size of any individual byte array (bytestring) inside an atproto record MAX_RECORD_BYTES_LEN = MAX_CBOR_RECORD_SIZE // limit on size of CID representation (NOT ENFORCED YET) MAX_CID_BYTES = 100 // limit on depth of nested containers (objects or arrays) for atproto data (NOT ENFORCED YET) MAX_CBOR_NESTED_LEVELS = 32 // maximum number of elements in an object or array in atproto data MAX_CBOR_CONTAINER_LEN = 128 * 1024 // largest integer which can be represented in a float64. integers in atproto "should" not be larger than this. (NOT ENFORCED) MAX_SAFE_INTEGER = 9007199254740991 // largest negative integer which can be represented in a float64. integers in atproto "should" not go below this. (NOT ENFORCED) MIN_SAFE_INTEGER = -9007199254740991 // maximum length of string (UTF-8 bytes) in an atproto object (map) MAX_OBJECT_KEY_LEN = 8192 )
Variables ¶
This section is empty.
Functions ¶
func ExtractTypeCBOR ¶
Parses the top-level $type field from generic atproto CBOR data
func ExtractTypeCBORReader ¶
Parses top-level $type field from generic atproto CBOR.
Returns that string field, and additional bytes (TODO: the parsed bytes, or remaining bytes?)
func ExtractTypeJSON ¶
Parses the top-level $type field from generic atproto JSON data
func MarshalCBOR ¶
Serializes generic atproto data (object) to DAG-CBOR bytes
Does not re-validate that data conforms to atproto data model, but does handle Blob, Bytes, and CIDLink as expected.
func UnmarshalCBOR ¶
Parses generic data (object) in CBOR (specifically, IPLD dag-cbor), validating against the atproto data model at the same time.
func UnmarshalJSON ¶
Parses generic data (object) in JSON, validating against the atproto data model at the same time.
The standard library's MarshalJSON can be used to invert this function.
Types ¶
type Blob ¶
Represents the "blob" type from the atproto data model.
This struct does not get marshaled/unmarshaled directly in to JSON or CBOR; see the BlobSchema and LegacyBlobSchema structs. This is the type that should be included in golang struct definitions.
When representing a "legacy" blob (no size field, string CID), size == -1.
func ExtractBlobs ¶
Recursively finds all the "blob" objects from generic atproto data (which has already been parsed).
Returns an array with all Blob instances; does not de-dupe.
func (Blob) MarshalJSON ¶
func (*Blob) UnmarshalJSON ¶
type BlobSchema ¶
type BlobSchema struct { LexiconTypeID string `json:"$type,const=blob" cborgen:"$type,const=blob"` Ref CIDLink `json:"ref" cborgen:"ref"` MimeType string `json:"mimeType" cborgen:"mimeType"` Size int64 `json:"size" cborgen:"size"` }
func (*BlobSchema) MarshalCBOR ¶
func (t *BlobSchema) MarshalCBOR(w io.Writer) error
func (*BlobSchema) UnmarshalCBOR ¶
func (t *BlobSchema) UnmarshalCBOR(r io.Reader) (err error)
type Bytes ¶
type Bytes []byte
Represents the "bytes" type from the atproto data model.
In JSON, marshals to an object with $bytes key and base64-encoded data.
In CBOR, marshals to a byte array.
func (Bytes) MarshalJSON ¶
func (*Bytes) UnmarshalJSON ¶
type CIDLink ¶
type CIDLink cid.Cid
Represents the "cid-link" type from the atproto data model.
Implementation is a simple wrapper around the github.com/ipfs/go-cid "cid.Cid" type.
func (CIDLink) CID ¶
func (ll CIDLink) CID() cid.Cid
Unwraps the inner cid.Cid type (github.com/ipfs/go-cid)
func (CIDLink) IsDefined ¶
Convenience helper, returns false if CID is "undefined" (golang zero value)
func (CIDLink) MarshalJSON ¶
func (CIDLink) String ¶
Returns string representation.
If the CID is "undefined", returns an empty string (note that this is different from how cid.Cid works).
func (*CIDLink) UnmarshalJSON ¶
type GenericRecord ¶
type GenericRecord struct {
Type string `json:"$type" cborgen:"$type"`
}
Helper type for extracting record $type from CBOR
func (*GenericRecord) MarshalCBOR ¶
func (t *GenericRecord) MarshalCBOR(w io.Writer) error
func (*GenericRecord) UnmarshalCBOR ¶
func (t *GenericRecord) UnmarshalCBOR(r io.Reader) (err error)
type LegacyBlobSchema ¶
type LegacyBlobSchema struct { Cid string `json:"cid" cborgen:"cid"` MimeType string `json:"mimeType" cborgen:"mimeType"` }
func (*LegacyBlobSchema) MarshalCBOR ¶
func (t *LegacyBlobSchema) MarshalCBOR(w io.Writer) error
func (*LegacyBlobSchema) UnmarshalCBOR ¶
func (t *LegacyBlobSchema) UnmarshalCBOR(r io.Reader) (err error)