imageblk

package
v0.8.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 18, 2016 License: BSD-3-Clause Imports: 21 Imported by: 0

Documentation

Overview 

	Package imageblk implements DVID support for image blocks of various formats (uint8, uint16, rgba8).
    For label data, use labelblk.

Index

Constants

View Source 
const (
	Version = "0.2"
	RepoURL = "github.com/janelia-flyem/dvid/datatype/imageblk"
)
View Source 
const (
	IngestBlockEvent = "BLOCK_INGEST"
	MutateBlockEvent = "BLOCK_MUTATE"
	DeleteBlockEvent = "BLOCK_DELETE"
)

Events for imageblk

View Source 
const HelpMessage = `` /* 13684-byte string literal not displayed */
View Source 
const KVWriteSize = 500

KVWriteSize is the # of key-value pairs we will write as one atomic batch write.

Variables

View Source 
var (
	// DefaultBlockSize specifies the default size for each block of this data type.
	DefaultBlockSize int32 = 32

	DefaultRes float32 = 8

	DefaultUnits = "nanometers"
)

Functions

func DecodeTKey 

func DecodeTKey(tk storage.TKey) (*dvid.IndexZYX, error)

DecodeTKey returns a spatial index from a image block key. TODO: Extend this when necessary to allow any form of spatial indexing like CZYX.

func NewTKey 

func NewTKey(idx dvid.Index) storage.TKey

NewTKey returns a type-specific key component for an image block. TKey = s

func NewTKeyByCoord 

func NewTKeyByCoord(izyx dvid.IZYXString) storage.TKey

NewTKeyByCoord returns a TKey for a block coord in string format.

Types

type ArbSlice 

type ArbSlice struct {
	// contains filtered or unexported fields
}

ArbSlice is a 2d rectangle that can be positioned arbitrarily in 3D.

func (ArbSlice) String 

func (s ArbSlice) String() string

type Block 

type Block struct {
	Index *dvid.IndexZYX
	Data  []byte
}

Block encodes a 3d block coordinate and block data. It is the unit of delta for a IngestBlockEvent.

type Data 

type Data struct {
	*datastore.Data
	Properties
}

Data embeds the datastore's Data and extends it with voxel-specific properties.

func (*Data) AllocateBlock added in v0.8.0 

func (d *Data) AllocateBlock() []byte

func (*Data) BackgroundBlock 

func (d *Data) BackgroundBlock() []byte

BackgroundBlock returns a block buffer that has been preinitialized to the background value.

func (*Data) BlankImage 

func (d *Data) BlankImage(dstW, dstH int32) (*dvid.Image, error)

BlankImage initializes a blank image of appropriate size and depth for the current data values. Returns an error if the geometry is not 2d.

func (*Data) BlockSize 

func (d *Data) BlockSize() dvid.Point

func (*Data) CopyPropertiesFrom added in v0.8.2 

func (d *Data) CopyPropertiesFrom(src datastore.DataService, fs storage.FilterSpec) error

CopyPropertiesFrom copies the data instance-specific properties from a given data instance into the receiver's properties.

func (*Data) DoRPC 

func (d *Data) DoRPC(req datastore.Request, reply *datastore.Response) error

DoRPC acts as a switchboard for RPC commands.

func (*Data) Equals 

func (d *Data) Equals(d2 *Data) bool

func (*Data) Extents 

func (d *Data) Extents() *dvid.Extents

func (*Data) ForegroundROI 

func (d *Data) ForegroundROI(req datastore.Request, reply *datastore.Response) error

ForegroundROI creates a new ROI by determining all non-background blocks.

func (*Data) GetArbitraryImage 

func (d *Data) GetArbitraryImage(ctx storage.Context, tlStr, trStr, blStr, resStr string) (*dvid.Image, error)

func (*Data) GetBlocks 

func (d *Data) GetBlocks(v dvid.VersionID, start dvid.ChunkPoint3d, span int32) ([]byte, error)

GetBlocks returns a slice of bytes corresponding to all the blocks along a span in X

func (*Data) GetImage 

func (d *Data) GetImage(v dvid.VersionID, vox *Voxels, roiname dvid.InstanceName) (*dvid.Image, error)

GetImage retrieves a 2d image from a version node given a geometry of voxels.

func (*Data) GetVolume 

func (d *Data) GetVolume(v dvid.VersionID, vox *Voxels, roiname dvid.InstanceName) ([]byte, error)

GetVolume retrieves a n-d volume from a version node given a geometry of voxels.

func (*Data) GetVoxels 

func (d *Data) GetVoxels(v dvid.VersionID, vox *Voxels, roiname dvid.InstanceName) error

GetVoxels copies voxels from the storage engine to Voxels, a requested subvolume or 2d image.

func (*Data) GobDecode 

func (d *Data) GobDecode(b []byte) error

func (*Data) GobEncode 

func (d *Data) GobEncode() ([]byte, error)

func (*Data) Help 

func (d *Data) Help() string

func (*Data) IngestVoxels added in v0.8.0 

func (d *Data) IngestVoxels(v dvid.VersionID, vox *Voxels, roiname dvid.InstanceName) error

IngestVoxels ingests voxels from a subvolume into the storage engine. The subvolume must be aligned to blocks of the data instance, which simplifies the routine since we are simply replacing a value instead of modifying values (GET + PUT).

func (*Data) LoadImages 

func (d *Data) LoadImages(v dvid.VersionID, offset dvid.Point, filenames []string) error

LoadImages bulk loads images using different techniques if it is a multidimensional file like HDF5 or a sequence of PNG/JPG/TIF images.

func (*Data) MarshalJSON 

func (d *Data) MarshalJSON() ([]byte, error)

func (*Data) ModifyConfig 

func (d *Data) ModifyConfig(config dvid.Config) error

func (*Data) MutateVoxels added in v0.8.0 

func (d *Data) MutateVoxels(v dvid.VersionID, vox *Voxels, roiname dvid.InstanceName) error

MutateVoxels mutates voxels from a subvolume into the storage engine. This differs from the IngestVoxels function in firing off a MutateBlockEvent instead of an IngestBlockEvent, which tells subscribers that a previous value has changed instead of a completely new key/value being inserted. There will be some decreased performance due to cleanup of prior denormalizations compared to IngestVoxels.

func (*Data) NewArbSlice 

func (d *Data) NewArbSlice(topLeft, topRight, bottomLeft dvid.Vector3d, res float64) (*ArbSlice, error)

NewArbSlice returns an image with arbitrary 3D orientation. The 3d points are in real world space definited by resolution, e.g., nanometer space.

func (*Data) NewArbSliceFromStrings 

func (d *Data) NewArbSliceFromStrings(tlStr, trStr, blStr, resStr, sep string) (*ArbSlice, error)

NewArbSliceFromStrings returns an image with arbitrary 3D orientation given string parameters. The 3d points are in real world space definited by resolution, e.g., nanometer space.

func (*Data) NewFilter added in v0.8.2 

func (d *Data) NewFilter(fs storage.FilterSpec) (storage.Filter, error)

NewFilter returns a Filter for use with a push of key-value pairs.

func (*Data) NewVoxels 

func (d *Data) NewVoxels(geom dvid.Geometry, img interface{}) (*Voxels, error)

NewVoxels returns Voxels with given geometry and optional image data. If img is passed in, the function will initialize the Voxels with data from the image. Otherwise, it will allocate a zero buffer of appropriate size.

func (*Data) PushData added in v0.8.2 

func (d *Data) PushData(p *datastore.PushSession) error

PushData does an imageblk-specific push using optional ROI filters.

func (*Data) PutBlocks 

func (d *Data) PutBlocks(v dvid.VersionID, start dvid.ChunkPoint3d, span int, data io.ReadCloser, mutate bool) error

PutBlocks stores blocks of data in a span along X

func (*Data) PutChunk 

func (d *Data) PutChunk(chunk *storage.Chunk, putbuffer storage.RequestBuffer) error

PutChunk puts a chunk of data as part of a mapped operation. Only some multiple of the # of CPU cores can be used for chunk handling before it waits for chunk processing to abate via the buffered server.HandlerToken channel.

func (*Data) PutLocal 

func (d *Data) PutLocal(request datastore.Request, reply *datastore.Response) error

PutLocal adds image data to a version node, altering underlying blocks if the image intersects the block.

The image filename glob MUST BE absolute file paths that are visible to the server. This function is meant for mass ingestion of large data files, and it is inappropriate to read gigabytes of data just to send it over the network to a local DVID.

func (*Data) PutVoxels 

func (d *Data) PutVoxels(v dvid.VersionID, vox *Voxels, roiname dvid.InstanceName, mutate bool) error

PutVoxels persists voxels from a subvolume into the storage engine. The subvolume must be aligned to blocks of the data instance, which simplifies the routine if the PUT is a mutation (signals MutateBlockEvent) instead of ingestion.

func (*Data) ReadChunk 

func (d *Data) ReadChunk(chunk *storage.Chunk) error

ReadChunk reads a chunk of data as part of a mapped operation. Only some multiple of the # of CPU cores can be used for chunk handling before it waits for chunk processing to abate via the buffered server.HandlerToken channel.

func (*Data) Resolution 

func (d *Data) Resolution() dvid.Resolution

func (*Data) ServeHTTP 

func (d *Data) ServeHTTP(uuid dvid.UUID, ctx *datastore.VersionedCtx, w http.ResponseWriter, r *http.Request)

ServeHTTP handles all incoming HTTP requests for this data.

func (*Data) SetExtents added in v0.8.3 

func (d *Data) SetExtents(uuid dvid.UUID, jsonBytes []byte) error

SetExtents loads JSON data giving MinPoint and MaxPoint.

func (*Data) String 

func (d *Data) String() string

type Filter added in v0.8.2 

type Filter struct {
	*Data
	// contains filtered or unexported fields
}

func (Filter) Check added in v0.8.2 

func (f Filter) Check(tkv *storage.TKeyValue) (skip bool, err error)

type KeyFunc 

type KeyFunc func(dvid.Point3d) []byte

type MutatedBlock added in v0.8.0 

type MutatedBlock struct {
	Index *dvid.IndexZYX
	Prev  []byte
	Data  []byte
}

MutatedBlock encodes a 3d block coordinate and previous and updated block data. It is the unit of delta for a MutateBlockEvent.

type PopulateFunc 

type PopulateFunc func([]byte) ([]byte, error)

type Properties 

type Properties struct {
	// Values describes the data type/label for each value within a voxel.
	Values dvid.DataValues

	// Interpolable is true if voxels can be interpolated when resizing.
	Interpolable bool

	// Block size for this repo
	BlockSize dvid.Point

	dvid.Resolution

	dvid.Extents // The only mutable property.  TODO -- move to in-memory store.

	// Background value for data
	Background uint8
}

Properties are additional properties for image block data instances beyond those in standard datastore.Data. These will be persisted to metadata storage.

func (*Properties) NdDataMetadata 

func (p *Properties) NdDataMetadata() (string, error)

NdDataSchema returns the metadata in JSON for this Data

type ROI 

type ROI struct {
	Iter *roi.Iterator
	// contains filtered or unexported fields
}

ROI encapsulates a request-specific ROI check with a given scaling for voxels outside the ROI.

func GetROI added in v0.8.0 

func GetROI(v dvid.VersionID, roiname dvid.InstanceName, bnd dvid.Bounder) (*ROI, error)

GetROI returns an imageblk.ROI that can iterate over the provided Voxels unless roiname is empty, which prompts a nil ROI returned.

type Type 

type Type struct {
	datastore.Type
	// contains filtered or unexported fields
}

Type embeds the datastore's Type to create a unique type with voxel functions. Refinements of general voxel types can be implemented by embedding this type, choosing appropriate # of values and bytes/value, overriding functions as needed, and calling datastore.Register(). Note that these fields are invariant for all instances of this type. Fields that can change depending on the type of data (e.g., resolution) should be in the Data type.

func NewType 

func NewType(values dvid.DataValues, interpolable bool) Type

NewType returns a pointer to a new imageblk Type with default values set.

func (*Type) Help 

func (dtype *Type) Help() string

func (*Type) NewData 

func (dtype *Type) NewData(uuid dvid.UUID, id dvid.InstanceID, name dvid.InstanceName, c dvid.Config) (*Data, error)

NewData returns a pointer to a new Voxels with default values.

func (*Type) NewDataService 

func (dtype *Type) NewDataService(uuid dvid.UUID, id dvid.InstanceID, name dvid.InstanceName, c dvid.Config) (datastore.DataService, error)

NewDataService returns a pointer to a new Voxels with default values.

type ValueCache 

type ValueCache struct {
	sync.Mutex
	// contains filtered or unexported fields
}

ValueCache is a concurrency-friendly cache

func NewValueCache 

func NewValueCache(size int) *ValueCache

func (*ValueCache) Clear 

func (vc *ValueCache) Clear()

Clear clears the cache.

func (ValueCache) Get 

func (vc ValueCache) Get(key []byte, pf PopulateFunc) ([]byte, bool, error)

Get returns the cached value of a key. On a miss, it uses the passed PopulateFunc to retrieve the key and stores it in the cache. If nil is passed for the PopulateFunc, the function just returns a "false" with no value.

type Voxels 

type Voxels struct {
	dvid.Geometry
	// contains filtered or unexported fields
}

Voxels represents subvolumes or slices and implements the ExtData interface.

func NewVoxels 

func NewVoxels(geom dvid.Geometry, values dvid.DataValues, data []byte, stride int32) *Voxels

func (*Voxels) BytesPerVoxel 

func (v *Voxels) BytesPerVoxel() int32

func (*Voxels) ComputeTransform 

func (v *Voxels) ComputeTransform(block *storage.TKeyValue, blockSize dvid.Point) (blockBeg, dataBeg, dataEnd dvid.Point, err error)

ComputeTransform determines the block coordinate and beginning + ending voxel points for the data corresponding to the given Block.

func (*Voxels) Data 

func (v *Voxels) Data() []byte

func (*Voxels) DownRes 

func (v *Voxels) DownRes(magnification dvid.Point) error

DownRes downsamples 2d Voxels data by averaging where the down-magnification are integers. If the source image size in Voxels is not an integral multiple of the reduction factor, the edge voxels on the right and bottom side are truncated. This function modifies the Voxels data. An error is returned if a non-2d Voxels receiver is used.

func (*Voxels) GetImage2d 

func (v *Voxels) GetImage2d() (*dvid.Image, error)

GetImage2d returns a 2d image suitable for use external to DVID. TODO -- Create more comprehensive handling of endianness and encoding of multibytes/voxel data into appropriate images.

func (*Voxels) IndexIterator 

func (v *Voxels) IndexIterator(chunkSize dvid.Point) (dvid.IndexIterator, error)

IndexIterator returns an iterator that can move across the voxel geometry, generating indices or index spans.

func (*Voxels) Interpolable 

func (v *Voxels) Interpolable() bool

func (*Voxels) NewChunkIndex 

func (v *Voxels) NewChunkIndex() dvid.ChunkIndexer

func (*Voxels) ReadBlock 

func (v *Voxels) ReadBlock(block *storage.TKeyValue, blockSize dvid.Point, attenuation uint8) error

ReadBlock reads the possibly intersecting block data into the receiver Voxels.

func (*Voxels) SetData 

func (v *Voxels) SetData(data []byte)

func (*Voxels) SetGeometry 

func (v *Voxels) SetGeometry(geom dvid.Geometry)

func (*Voxels) SetStride 

func (v *Voxels) SetStride(stride int32)

func (*Voxels) SetValues 

func (v *Voxels) SetValues(values dvid.DataValues)

func (*Voxels) Stride 

func (v *Voxels) Stride() int32

func (*Voxels) String 

func (v *Voxels) String() string

func (*Voxels) Values 

func (v *Voxels) Values() dvid.DataValues

func (*Voxels) WriteBlock 

func (v *Voxels) WriteBlock(block *storage.TKeyValue, blockSize dvid.Point) error

WriteBlock writes a subvolume or 2d image into a possibly intersecting block.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.