Documentation
¶
Overview ¶
Package slicer provides raw data slicing into NeoFS objects.
Index ¶
- Variables
- func Put(ctx context.Context, ow ObjectWriter, header object.Object, signer user.Signer, ...) (oid.ID, error)
- type NetworkedClient
- type ObjectWriter
- type Options
- func (x *Options) BearerToken() *bearer.Token
- func (x *Options) CalculateHomomorphicChecksum()
- func (x *Options) CopiesNumber() uint32
- func (x *Options) CurrentNeoFSEpoch() uint64
- func (x *Options) IsHomomorphicChecksumEnabled() bool
- func (x *Options) ObjectPayloadLimit() uint64
- func (x *Options) PayloadBuffer() []byte
- func (x *Options) Session() *session.Object
- func (x *Options) SetBearerToken(bearerToken bearer.Token)
- func (x *Options) SetCopiesNumber(copiesNumber uint32)
- func (x *Options) SetCurrentNeoFSEpoch(e uint64)
- func (x *Options) SetObjectPayloadLimit(l uint64)
- func (x *Options) SetPayloadBuffer(payloadBuffer []byte)
- func (x *Options) SetPayloadSize(size uint64)
- func (x *Options) SetSession(sess session.Object)
- type PayloadWriter
- type Slicer
Constants ¶
This section is empty.
Variables ¶
var ( // ErrIncompleteHeader indicates some fields are missing in header. ErrIncompleteHeader = errors.New("incomplete header") )
Functions ¶
func Put ¶
func Put(ctx context.Context, ow ObjectWriter, header object.Object, signer user.Signer, data io.Reader, opts Options) (oid.ID, error)
Put works similar to Slicer.Put, but allows flexible configuration of object header. The method accepts Options for adjusting max object size, epoch, session token, etc.
Types ¶
type NetworkedClient ¶
type NetworkedClient interface {
ObjectWriter
NetworkInfo(ctx context.Context, prm client.PrmNetworkInfo) (netmap.NetworkInfo, error)
}
NetworkedClient represents a virtual object recorder with possibility to get actual netmap.NetworkInfo data.
type ObjectWriter ¶
type ObjectWriter interface {
// ObjectPutInit initializes and returns a stream of writable data associated
// with the object according to its header. Provided header includes at least
// container, owner and object ID fields. Header length is limited to
// [object.MaxHeaderLen].
//
// Signer is required and must not be nil. The operation is executed on behalf of
// the account corresponding to the specified Signer, which is taken into account, in particular, for access control.
ObjectPutInit(ctx context.Context, hdr object.Object, signer user.Signer, prm client.PrmObjectPutInit) (client.ObjectWriter, error)
}
ObjectWriter represents a virtual object recorder.
type Options ¶
type Options struct {
// contains filtered or unexported fields
}
Options groups Slicer options.
func (*Options) BearerToken ¶
BearerToken returns bearer token.
func (*Options) CalculateHomomorphicChecksum ¶
func (x *Options) CalculateHomomorphicChecksum()
CalculateHomomorphicChecksum makes Slicer to calculate and set homomorphic checksum of the processed objects.
func (*Options) CopiesNumber ¶
CopiesNumber returns the number of object copies.
func (*Options) CurrentNeoFSEpoch ¶
CurrentNeoFSEpoch returns epoch.
func (*Options) IsHomomorphicChecksumEnabled ¶
IsHomomorphicChecksumEnabled indicates homomorphic checksum calculation status.
func (*Options) ObjectPayloadLimit ¶
ObjectPayloadLimit returns required max object size.
func (*Options) PayloadBuffer ¶
PayloadBuffer returns chunk which are using to object uploading.
func (*Options) SetBearerToken ¶
SetBearerToken allows to attach signed Extended ACL rules to the request.
func (*Options) SetCopiesNumber ¶
SetCopiesNumber sets the minimal number of copies (out of the number specified by container placement policy) for the object PUT operation to succeed. This means that object operation will return with successful status even before container placement policy is completely satisfied.
func (*Options) SetCurrentNeoFSEpoch ¶
SetCurrentNeoFSEpoch sets current NeoFS epoch.
func (*Options) SetObjectPayloadLimit ¶
SetObjectPayloadLimit specifies data size limit for produced physically stored objects.
func (*Options) SetPayloadBuffer ¶
SetPayloadBuffer sets pre-allocated payloadBuffer to be used to object uploading. For better performance payloadBuffer length should be MaxObjectSize from NeoFS.
func (*Options) SetPayloadSize ¶
SetPayloadSize allows to specify object's payload size known in advance. If set, reading functions will read at least size bytes while writing functions will expect exactly size bytes.
If the size is known, the option is recommended as it improves the performance of the application using the Slicer.
func (*Options) SetSession ¶
SetSession sets session object.
type PayloadWriter ¶
type PayloadWriter struct {
// contains filtered or unexported fields
}
PayloadWriter is a single-object payload stream provided by Slicer.
func InitPut ¶
func InitPut(ctx context.Context, ow ObjectWriter, header object.Object, signer user.Signer, opts Options) (*PayloadWriter, error)
InitPut works similar to slicer.Put, but provides ObjectWriter allowing the caller to write data himself.
func (*PayloadWriter) Close ¶
func (x *PayloadWriter) Close() error
Close finalizes object with written payload data, saves the object and closes the stream. Reference to the stored object can be obtained by ID method.
func (*PayloadWriter) ID ¶
func (x *PayloadWriter) ID() oid.ID
ID returns unique identifier of the stored object representing its reference in the configured container.
ID MUST NOT be called before successful Close (undefined behavior otherwise).
type Slicer ¶
type Slicer struct {
// contains filtered or unexported fields
}
Slicer converts input raw data streams into NeoFS objects. Working Slicer must be constructed via New.
func New ¶
func New(ctx context.Context, nw NetworkedClient, signer user.Signer, cnr cid.ID, owner user.ID, sessionToken *session.Object) (*Slicer, error)
New constructs Slicer which writes sliced ready-to-go objects owned by particular user into the specified container using provided ObjectWriter. All objects are signed using provided neofscrypto.Signer.
If ObjectWriter returns data streams which provide io.Closer, they are closed in Slicer.Slice after the payload of any object has been written. In this case, Slicer.Slice fails immediately on Close error.
NetworkedClient parameter allows to extract all required network-depended information for default Slicer behavior tuning.
If payload size limit is specified via Options.SetObjectPayloadLimit, outgoing objects has payload not bigger than the limit. NeoFS stores the corresponding value in the network configuration. Ignore this option if you don't (want to) have access to it. By default, single object is limited by 1MB. Slicer uses this value to enforce the maximum object payload size limit described in the NeoFS Specification. If the total amount of data exceeds the specified limit, Slicer applies the slicing algorithm described within the same specification. The outcome will be a group of "small" objects containing a chunk of data, as well as an auxiliary linking object. All derived objects are written to the parameterized ObjectWriter. If the amount of data is within the limit, one object is produced. Note that Slicer can write multiple objects, but returns the root object ID only.
Parameter sessionToken may be nil, if no session is used.
func (*Slicer) InitPut ¶
InitPut works similar to Slicer.Put but provides PayloadWriter allowing the caller to write data himself.
func (*Slicer) Put ¶
Put creates new NeoFS object from the input data stream, associates the object with the configured container and writes the object via underlying ObjectWriter. After a successful write, Put returns an oid.ID which is a unique reference to the object in the container. Put sets all required calculated fields like payload length, checksum, etc.
Put allows you to specify object.Attribute parameters to be written to the resulting object's metadata. Keys SHOULD NOT start with system-reserved '__NEOFS__' prefix.
See New for details.
Source Files