converter

package
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 18, 2023 License: Apache-2.0 Imports: 40 Imported by: 10

Documentation

Index

Constants

View Source 
const (
	ManifestOSFeatureNydus   = "nydus.remoteimage.v1"
	MediaTypeNydusBlob       = "application/vnd.oci.image.layer.nydus.blob.v1"
	BootstrapFileNameInLayer = "image/image.boot"

	ManifestNydusCache = "containerd.io/snapshot/nydus-cache"

	LayerAnnotationFSVersion          = "containerd.io/snapshot/nydus-fs-version"
	LayerAnnotationNydusBlob          = "containerd.io/snapshot/nydus-blob"
	LayerAnnotationNydusBlobDigest    = "containerd.io/snapshot/nydus-blob-digest"
	LayerAnnotationNydusBlobSize      = "containerd.io/snapshot/nydus-blob-size"
	LayerAnnotationNydusBootstrap     = "containerd.io/snapshot/nydus-bootstrap"
	LayerAnnotationNydusSourceChainID = "containerd.io/snapshot/nydus-source-chainid"
	LayerAnnotationNydusEncryptedBlob = "containerd.io/snapshot/nydus-encrypted-blob"

	LayerAnnotationNydusReferenceBlobIDs = "containerd.io/snapshot/nydus-reference-blob-ids"

	LayerAnnotationUncompressed = "containerd.io/uncompressed"
)
View Source 
const EntryBlob = "image.blob"
View Source 
const EntryBlobMeta = "blob.meta"
View Source 
const EntryBlobMetaHeader = "blob.meta.header"
View Source 
const EntryBootstrap = "image.boot"
View Source 
const EntryTOC = "rafs.blob.toc"

Variables

View Source 
var (
	ErrNotFound = errors.New("data not found")
)

Functions

func ConvertHookFunc added in v0.3.0 

func ConvertHookFunc(opt MergeOption) converter.ConvertHookFunc

ConvertHookFunc returns a function which will be used as a callback called for each blob after conversion is done. The function only hooks the index conversion and the manifest conversion.

func DeryptNydusBootstrap added in v0.9.0 

func DeryptNydusBootstrap(ctx context.Context, cs content.Store, desc ocispec.Descriptor, decryptKeys []string, unwrapOnly bool) (ocispec.Descriptor, error)

Decrypt the Nydus boostrap layer. If unwrapOnly is set we will only try to decrypt the layer encryption key and return, the layer itself won't be decrypted actually.

func EncryptNydusBootstrap added in v0.9.0 

func EncryptNydusBootstrap(ctx context.Context, cs content.Store, desc ocispec.Descriptor, encryptRecipients []string) (ocispec.Descriptor, error)

Encrypt Nydus bootstrap layer

func IsNydusBlob added in v0.3.0 

func IsNydusBlob(desc ocispec.Descriptor) bool

IsNydusBlob returns true when the specified descriptor is nydus blob layer.

func IsNydusBlobAndExists added in v0.3.0 

func IsNydusBlobAndExists(ctx context.Context, cs content.Store, desc ocispec.Descriptor) bool

IsNydusBlobAndExists returns true when the specified digest of content exists in the content store and it's nydus blob format.

func IsNydusBootstrap added in v0.7.1 

func IsNydusBootstrap(desc ocispec.Descriptor) bool

IsNydusBootstrap returns true when the specified descriptor is nydus bootstrap layer.

func LayerConvertFunc added in v0.3.0 

func LayerConvertFunc(opt PackOption) converter.ConvertFunc

LayerConvertFunc returns a function which converts an OCI image layer to a nydus blob layer, and set the media type to "application/vnd.oci.image.layer.nydus.blob.v1".

func Merge 

func Merge(ctx context.Context, layers []Layer, dest io.Writer, opt MergeOption) ([]digest.Digest, error)

Merge multiple nydus bootstraps (from each layer of image) to a final bootstrap. And due to the possibility of enabling the `ChunkDictPath` option causes the data deduplication, it will return the actual blob digests referenced by the bootstrap.

func MergeLayers added in v0.3.0 

func MergeLayers(ctx context.Context, cs content.Store, descs []ocispec.Descriptor, opt MergeOption) (*ocispec.Descriptor, []ocispec.Descriptor, error)

MergeLayers merges a list of nydus blob layer into a nydus bootstrap layer. The media type of the nydus bootstrap layer is "application/vnd.oci.image.layer.v1.tar+gzip".

func Pack added in v0.3.0 

func Pack(ctx context.Context, dest io.Writer, opt PackOption) (io.WriteCloser, error)

Pack converts an OCI tar stream to nydus formatted stream with a tar-like structure that arranges the data as follows:

`data | tar_header | data | tar_header | [toc_entry | ... | toc_entry | tar_header]`

The caller should write OCI tar stream into the returned `io.WriteCloser`, then the Pack method will write the nydus formatted stream to `dest` provided by the caller.

Important: the caller must check `io.WriteCloser.Close() == nil` to ensure the conversion workflow is finished.

func Unpack added in v0.3.0 

func Unpack(ctx context.Context, ra content.ReaderAt, dest io.Writer, opt UnpackOption) error

Unpack converts a nydus blob layer to OCI formatted tar stream.

Types

type Backend added in v0.3.0 

type Backend interface {
	// Push pushes specified blob file to remote storage backend.
	Push(ctx context.Context, cs content.Store, desc ocispec.Descriptor) error
	// Check checks whether a blob exists in remote storage backend,
	// blob exists -> return (blobPath, nil)
	// blob not exists -> return ("", err)
	Check(blobDigest digest.Digest) (string, error)
	// Type returns backend type name.
	Type() string
}

Backend uploads blobs generated by nydus-image builder to a backend storage.

type Compressor added in v0.5.0 

type Compressor = uint32
const (
	CompressorNone     Compressor = 0x0000_0001
	CompressorZstd     Compressor = 0x0000_0002
	CompressorLz4Block Compressor = 0x0000_0004
	CompressorMask     Compressor = 0x0000_000f
)

type Layer 

type Layer struct {
	// Digest represents the hash of whole tar blob.
	Digest digest.Digest
	// Digest represents the original OCI tar(.gz) blob.
	OriginalDigest *digest.Digest
	// ReaderAt holds the reader of whole tar blob.
	ReaderAt content.ReaderAt
}

type MergeOption 

type MergeOption struct {
	// WorkDir is used as the work directory during layer merge.
	WorkDir string
	// BuilderPath holds the path of `nydus-image` binary tool.
	BuilderPath string
	// FsVersion specifies nydus RAFS format version, possible
	// values: `5`, `6` (EROFS-compatible), default is `6`.
	FsVersion string
	// ChunkDictPath holds the bootstrap path of chunk dict image.
	ChunkDictPath string
	// ParentBootstrapPath holds the bootstrap path of parent image.
	ParentBootstrapPath string
	// PrefetchPatterns holds file path pattern list want to prefetch.
	PrefetchPatterns string
	// WithTar puts bootstrap into a tar stream (no gzip).
	WithTar bool
	// OCI converts docker media types to OCI media types.
	OCI bool
	// OCIRef enables converting OCI tar(.gz) blob to nydus referenced blob.
	OCIRef bool
	// WithReferrer associates a reference to the original OCI manifest.
	// See the `subject` field description in
	// https://github.com/opencontainers/image-spec/blob/main/manifest.md#image-manifest-property-descriptions
	//
	// With this association, we can track all nydus images associated with
	// an OCI image. For example, in Harbor we can cascade to show nydus
	// images linked to an OCI image, deleting the OCI image can also delete
	// the corresponding nydus images. At runtime, nydus snapshotter can also
	// automatically upgrade an OCI image run to nydus image.
	WithReferrer bool
	// Backend uploads blobs generated by nydus-image builder to a backend storage.
	Backend Backend
	// Timeout cancels execution once exceed the specified time.
	Timeout *time.Duration
	// Recipients to encrypt bootstrap, do not encrypt if empty.
	EncryptRecipients []string
}

type PackOption added in v0.3.0 

type PackOption struct {
	// WorkDir is used as the work directory during layer pack.
	WorkDir string
	// BuilderPath holds the path of `nydus-image` binary tool.
	BuilderPath string
	// FsVersion specifies nydus RAFS format version, possible
	// values: `5`, `6` (EROFS-compatible), default is `6`.
	FsVersion string
	// ChunkDictPath holds the bootstrap path of chunk dict image.
	ChunkDictPath string
	// PrefetchPatterns holds file path pattern list want to prefetch.
	PrefetchPatterns string
	// Compressor specifies nydus blob compression algorithm.
	Compressor string
	// OCIRef enables converting OCI tar(.gz) blob to nydus referenced blob.
	OCIRef bool
	// AlignedChunk aligns uncompressed data chunks to 4K, only for RAFS V5.
	AlignedChunk bool
	// ChunkSize sets the size of data chunks, must be power of two and between 0x1000-0x1000000.
	ChunkSize string
	// BacthSize sets the size of batch data chunks, must be power of two and between 0x1000-0x1000000 or zero.
	BatchSize string
	// Backend uploads blobs generated by nydus-image builder to a backend storage.
	Backend Backend
	// Timeout cancels execution once exceed the specified time.
	Timeout *time.Duration
	// Whether the generated Nydus blobs should be encrypted.
	Encrypt bool
	// contains filtered or unexported fields
}

type TOCEntry added in v0.5.0 

type TOCEntry struct {
	// Feature flags of entry
	Flags     uint32
	Reserved1 uint32
	// Name of entry data
	Name [16]byte
	// Sha256 of uncompressed entry data
	UncompressedDigest [32]byte
	// Offset of compressed entry data
	CompressedOffset uint64
	// Size of compressed entry data
	CompressedSize uint64
	// Size of uncompressed entry data
	UncompressedSize uint64
	Reserved2        [44]byte
}

func UnpackEntry added in v0.5.0 

func UnpackEntry(ra content.ReaderAt, targetName string, target io.Writer) (*TOCEntry, error)

Unpack the file from nydus formatted tar stream. The nydus formatted tar stream is a tar-like structure that arranges the data as follows:

`data | tar_header | ... | data | tar_header | [toc_entry | ... | toc_entry | tar_header]`

func (*TOCEntry) GetCompressedOffset added in v0.5.0 

func (entry *TOCEntry) GetCompressedOffset() uint64

func (*TOCEntry) GetCompressedSize added in v0.5.0 

func (entry *TOCEntry) GetCompressedSize() uint64

func (*TOCEntry) GetCompressor added in v0.5.0 

func (entry *TOCEntry) GetCompressor() (Compressor, error)

func (*TOCEntry) GetName added in v0.5.0 

func (entry *TOCEntry) GetName() string

func (*TOCEntry) GetUncompressedDigest added in v0.5.0 

func (entry *TOCEntry) GetUncompressedDigest() string

func (*TOCEntry) GetUncompressedSize added in v0.5.0 

func (entry *TOCEntry) GetUncompressedSize() uint64

type UnpackOption added in v0.3.0 

type UnpackOption struct {
	// WorkDir is used as the work directory during layer unpack.
	WorkDir string
	// BuilderPath holds the path of `nydus-image` binary tool.
	BuilderPath string
	// Timeout cancels execution once exceed the specified time.
	Timeout *time.Duration
	// Stream enables streaming mode, which doesn't unpack the blob data to disk,
	// but setup a http server to serve the blob data.
	Stream bool
}

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.