ld

README

ld

General outline

Last updated: 2017-11-26 @ fe68394aeb57e8797c4ac6b68f2f870b929895d8

• parse.go contains the code related to the JSON parsing in ld's custom data structure. It allows for fast parsing before unmarshalling. Everything is organised around three types: value, ValueSlice and object. ValueSlice represents a possible JSON array, although it makes no real distinction between arrays and single values: in fact, when ParseValues encounters a string at the top level, it will simply create a ValueSlice, holding one value of kind string. Nested arrays are also flattened as a consequence.
  • value holds the Kind of the value, which may be null, true/false, number, string or an object. Number and strings are both treated as strings - and for them, the String field is set. Instead, for objects, the Object field is filled out, and it will contain the key-value pairings of the object.
• Parse basically runs the JSON input through ParseValues, and retrives the first object, then wraps it into a nice exported Object, which provides functions for marshalling/unmarshalling.
• context.go contains pretty much all of the code of the Context Processing Algorithms.

Documentation

Overview

Package ld provides functions to parse, unmarshal and marshal JSON-LD data structures, in an efficient manner and bringing static typing. It does not aim to be a JSON-LD 100% compliant processor; in fact, at the moment it's not even capable of any of the compaction, expansion or flattening algorithm, however it does correctly parse contexts and expand IRIs, so that they can be mapped to structs.

JSON parsing

Mostly for efficiency, ld does not use off-the-shelf JSON unmarshalling to do the initial decoding of data - instead, it parses the raw JSON tokens and places them in an ad-hoc data structure. One of its features is that it does not really make any difference between elements contained inside an array and elements that are not - for this reason it is sometimes more lenient than other JSON parsers. This should not bring any issues (on the contrary, the carelessness of arrays is a feature caused by JSON-LD's approach to arrays).

Index

• type ContextCacher
• type DocumentLoader
• type Object
  • func Parse(r io.Reader) (*Object, error)
  • func (o *Object) Marshal(v interface{}) ([]byte, error)
  • func (o *Object) ParseContext() error
  • func (o *Object) PeekType() string
  • func (o *Object) Unmarshal(v interface{}, structContext *Object) error
• type RolloverCache
  • func (t *RolloverCache) Get(id uint64) *Object
  • func (t *RolloverCache) Set(id uint64, o *Object)
• type ValueSlice
  • func ParseValues(d *json.Decoder) (ValueSlice, error)
  • func (v ValueSlice) MarshalJSON() ([]byte, error)
  • func (vs ValueSlice) String() string

Types

type ContextCacher

type ContextCacher interface {
	Get(id uint64) *Object
	Set(id uint64, v *Object)
}

ContextCacher is an interface for handling caching of Contexts. Caching allows for parsing to be much faster in many cases - though it will add the overhead of creating a binary representation of the context data and creating a hash of it. It has a relatively simple API which could be solved by a simple map[uint64]*Object with a RWMutex, though you can set up discarding elements after they've reached a certain age.

Implementations are expected to return through Get only objects stored using Set. If no object can be found with that ID, nil must be returned. Implementations may or may not be thread-safe depending on whether they wish to run ParseContext on multiple goroutines using the same ContextCacher.

You can probably find an implementation that suits your needs in RolloverCache, as it removes cached elements that have not been used for over a certain period of time.

type DocumentLoader

type DocumentLoader interface {
	LoadDocument(string) (ValueSlice, error)
}

DocumentLoader handles fetching remote documents. The first argument is the URI where the resource is located: data should then be parsed by using ParseValues, which will transform everything into this package's data structure for handling JSON data.

In places where there might be a DocumentLoader and there isn't any, the defaultDocumentLoader is used instead. defaultDocumentLoader will fetch documents through standard HTTP requests, using JSON-LD's Accept header. It will then parse the values and store them in the cache, in case they are requested in the future. Cache expires after 24 hours.

type Object

type Object struct {
	Loader DocumentLoader
	Cacher ContextCacher
	// contains filtered or unexported fields
}

Object represents a JSON-LD object that needs yet to be unmarshalled into a Go struct.

func Parse

func Parse(r io.Reader) (*Object, error)

Parse decodes JSON-LD data from r.

func (*Object) Marshal

func (o *Object) Marshal(v interface{}) ([]byte, error)

Marshal converts v into JSON-LD data.

func (*Object) ParseContext

func (o *Object) ParseContext() error

ParseContext creates a new context from the Object's @context JSON field.

func (*Object) PeekType

func (o *Object) PeekType() string

PeekType looks through the the attributes of the Object, to determine if any of them can be expanded to "@type". If one can, its value is returned.

func (*Object) Unmarshal

func (o *Object) Unmarshal(v interface{}, structContext *Object) error

Unmarshal places the object's data into v, which must be a pointer to a struct.

First of all, the object's context is parsed, using Object.DocumentLoader to fetch remote contexts. This allows us to expand all the object's property names. For instance, the following object's property names:

{
  "@context": "http://json-ld.org/contexts/person.jsonld",
  "name": "Manu Sporny",
  "homepage": "http://manu.sporny.org/",
  "image": "http://manu.sporny.org/images/manu.png"
}

Are expanded to this:

{
  "http://schema.org/name": "Manu Sporny",
  "http://schema.org/url": "http://manu.sporny.org/",
  "http://schema.org/image": "http://manu.sporny.org/images/manu.png"
}

Note the values are untouched: this is because we don't need to process the values just yet: they are processed on-demand, only if we need to place the value into the struct.

Done this, the struct's fields are then iterated through. If the field has a tag, in the form of `ld:"propertyName"`, then its value is expanded following structContext. If there is no field tag, then the field name, with the first letter lowercased, is used by default. This is because generally in JSON-LD type names AreCapitalized, whereas property names areNot.

Once the field's name is expanded, it is matched with the object's corresponding property, if any.

type RolloverCache

type RolloverCache struct {
	Every time.Duration
	// contains filtered or unexported fields
}

RolloverCache is a simple ContextCacher mechanism. It holds two maps, current and next. Next only contains only contexts requested or set over the current period, defined by Every and defaulting to 30 minutes. Current holds contexts requested in the current and previous period.

func (*RolloverCache) Get

func (t *RolloverCache) Get(id uint64) *Object

Get retrieves an object from the RolloverCache. Getting an item that already exists inside the "current" map will allow this item to be also placed inside the "next" map.

func (*RolloverCache) Set

func (t *RolloverCache) Set(id uint64, o *Object)

Set sets an object in the RolloverCache.

type ValueSlice

type ValueSlice []value

ValueSlice is a slice of value. value is mostly an internal struct of the ld package, and it is used for encoding and decoding JSON objects. You should only ever need to use this when making a custom DocumentLoader.

func ParseValues

func ParseValues(d *json.Decoder) (ValueSlice, error)

ParseValues takes a json.Decoder and creates a ValueSlice, which is a representation of JSON data that basically discards arrays and mostly resides on the stack. ParseValues is mostly an internal function used by the ld package, and if you're an user of this package you should only ever need to use this when you're making a custom DocumentLoader.

func (ValueSlice) MarshalJSON

func (v ValueSlice) MarshalJSON() ([]byte, error)

MarshalJSON implements json marshalling for ValueSlice.

func (ValueSlice) String

func (vs ValueSlice) String() string