Documentation
¶
Overview ¶
package index provides indexing functionality for CARv1 data payload represented as a mapping of CID to offset. This can then be used to implement random access over a CARv1.
Index can be written or read using the following static functions: index.WriteTo and index.ReadFrom.
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ErrNotFound = errors.New("not found")
ErrNotFound signals a record is not found in the index.
Functions ¶
func GetFirst ¶
GetFirst is a wrapper over Index.GetAll, returning the offset for the first matching indexed CID.
func WriteTo ¶
WriteTo writes the given idx into w. The written bytes include the index encoding. This can then be read back using index.ReadFrom
Example ¶
ExampleWriteTo unmarshalls an index from an indexed CARv2 file, and stores it as a separate file on disk.
package main
import (
"fmt"
"io"
"io/ioutil"
"os"
"reflect"
carv2 "github.com/ipld/go-car/v2"
"github.com/ipld/go-car/v2/index"
)
func main() {
// Open the CARv2 file
src := "../testdata/sample-wrapped-v2.car"
cr, err := carv2.OpenReader(src)
if err != nil {
panic(err)
}
defer func() {
if err := cr.Close(); err != nil {
panic(err)
}
}()
// Read and unmarshall index within CARv2 file.
idx, err := index.ReadFrom(cr.IndexReader())
if err != nil {
panic(err)
}
// Store the index alone onto destination file.
f, err := ioutil.TempFile(os.TempDir(), "example-index-*.carindex")
if err != nil {
panic(err)
}
defer func() {
if err := f.Close(); err != nil {
panic(err)
}
}()
err = index.WriteTo(idx, f)
if err != nil {
panic(err)
}
// Seek to the beginning of tile to read it back.
_, err = f.Seek(0, io.SeekStart)
if err != nil {
panic(err)
}
// Read and unmarshall the destination file as a separate index instance.
reReadIdx, err := index.ReadFrom(f)
if err != nil {
panic(err)
}
// Expect indices to be equal.
if reflect.DeepEqual(idx, reReadIdx) {
fmt.Printf("Saved index file matches the index embedded in CARv2 at %v.\n", src)
} else {
panic("expected to get the same index as the CARv2 file")
}
}
Output: Saved index file matches the index embedded in CARv2 at ../testdata/sample-wrapped-v2.car.
Types ¶
type Index ¶
type Index interface {
// Codec provides the multicodec code that the index implements.
//
// Note that this may return a reserved code if the index
// implementation is not defined in a spec.
Codec() multicodec.Code
// Marshal encodes the index in serial form.
Marshal(w io.Writer) error
// Unmarshal decodes the index from its serial form.
Unmarshal(r io.Reader) error
// Load inserts a number of records into the index.
Load([]Record) error
// Get looks up all blocks matching a given CID,
// calling a function for each one of their offsets.
//
// If the function returns false, GetAll stops.
//
// If no error occurred and the CID isn't indexed,
// meaning that no callbacks happen,
// ErrNotFound is returned.
GetAll(cid.Cid, func(uint64) bool) error
}
Index provides an interface for looking up byte offset of a given CID.
Note that each indexing mechanism is free to match CIDs however it sees fit. For example, multicodec.CarIndexSorted only indexes multihash digests, meaning that Get and GetAll will find matching blocks even if the CID's encoding multicodec differs. Other index implementations might index the entire CID, the entire multihash, or just part of a multihash's digest.
func ReadFrom ¶
ReadFrom reads index from r. The reader decodes the index by reading the first byte to interpret the encoding. Returns error if the encoding is not known.
Example ¶
ExampleReadFrom unmarshalls an index from an indexed CARv2 file, and for each root CID prints the offset at which its corresponding block starts relative to the wrapped CARv1 data payload.
package main
import (
"fmt"
carv2 "github.com/ipld/go-car/v2"
"github.com/ipld/go-car/v2/index"
)
func main() {
// Open the CARv2 file
cr, err := carv2.OpenReader("../testdata/sample-wrapped-v2.car")
if err != nil {
panic(err)
}
defer cr.Close()
// Get root CIDs in the CARv1 file.
roots, err := cr.Roots()
if err != nil {
panic(err)
}
// Read and unmarshall index within CARv2 file.
idx, err := index.ReadFrom(cr.IndexReader())
if err != nil {
panic(err)
}
// For each root CID print the offset relative to CARv1 data payload.
for _, r := range roots {
offset, err := index.GetFirst(idx, r)
if err != nil {
panic(err)
}
fmt.Printf("Frame with CID %v starts at offset %v relative to CARv1 data payload.\n", r, offset)
}
}
Output: Frame with CID bafy2bzaced4ueelaegfs5fqu4tzsh6ywbbpfk3cxppupmxfdhbpbhzawfw5oy starts at offset 61 relative to CARv1 data payload.
Source Files