metainfo

package
v1.117.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 11, 2024 License: AGPL-3.0 Imports: 51 Imported by: 1

Documentation

Index

Constants

View Source 
const (
	// BoltPointerBucket is the string representing the bucket used for `PointerEntries` in BoltDB.
	BoltPointerBucket = "pointers"
)
View Source 
const MaxUserAgentLength = 500

MaxUserAgentLength is the maximum allowable length of the User Agent.

Variables

View Source 
var (

	// Error general metainfo error.
	Error = errs.Class("metainfo")
	// ErrNodeAlreadyExists pointer already has a piece for a node err.
	ErrNodeAlreadyExists = errs.Class("metainfo: node already exists")
	// ErrBucketNotEmpty is returned when bucket is required to be empty for an operation.
	ErrBucketNotEmpty = errs.Class("bucket not empty")
)

Functions

func EncodeSegmentID added in v1.17.1 

func EncodeSegmentID(ctx context.Context, segmentID *internalpb.SegmentID) (_ []byte, err error)

EncodeSegmentID encodes segment ID into bytes for signing.

func EncodeStreamID added in v1.17.1 

func EncodeStreamID(ctx context.Context, streamID *internalpb.StreamID) (_ []byte, err error)

EncodeStreamID encodes stream ID into bytes for signing.

func GetNewSuccessTracker added in v1.106.1 

func GetNewSuccessTracker(kind string) (func() SuccessTracker, bool)

GetNewSuccessTracker returns a function that creates a new SuccessTracker based on the kind. The bool return value is false if the kind is unknown.

func SignSegmentID added in v1.17.1 

func SignSegmentID(ctx context.Context, signer signing.Signer, unsigned *internalpb.SegmentID) (_ *internalpb.SegmentID, err error)

SignSegmentID signs the segment ID using the specified signer. Signer is a satellite.

func SignStreamID added in v1.17.1 

func SignStreamID(ctx context.Context, signer signing.Signer, unsigned *internalpb.StreamID) (_ *internalpb.StreamID, err error)

SignStreamID signs the stream ID using the specified signer. Signer is a satellite.

func TrimUserAgent added in v1.43.1 

func TrimUserAgent(userAgent []byte) ([]byte, error)

TrimUserAgent returns userAgentBytes that consist of only the product portion of the user agent, and is bounded by the maxUserAgentLength.

func VerifySegmentID added in v1.17.1 

func VerifySegmentID(ctx context.Context, satellite signing.Signee, signed *internalpb.SegmentID) (err error)

VerifySegmentID verifies that the signature inside segment ID belongs to the satellite.

func VerifyStreamID added in v1.17.1 

func VerifyStreamID(ctx context.Context, satellite signing.Signer, signed *internalpb.StreamID) (err error)

VerifyStreamID verifies that the signature inside stream ID belongs to the satellite.

Types

type APIKeys 

type APIKeys interface {
	GetByHead(ctx context.Context, head []byte) (*console.APIKeyInfo, error)
}

APIKeys is api keys store methods used by endpoint.

architecture: Database

type Config added in v0.11.0 

type Config struct {
	DatabaseURL          string      `help:"the database connection string to use" default:"postgres://"`
	MinRemoteSegmentSize memory.Size `default:"1240" testDefault:"0" help:"minimum remote segment size"` // TODO: fix tests to work with 1024
	MaxInlineSegmentSize memory.Size `default:"4KiB" help:"maximum inline segment size"`
	// we have such default value because max value for ObjectKey is 1024(1 Kib) but EncryptedObjectKey
	// has encryption overhead 16 bytes. So overall size is 1024 + 16 * 16.
	MaxEncryptedObjectKeyLength  int                 `default:"4000" help:"maximum encrypted object key length"`
	MaxSegmentSize               memory.Size         `default:"64MiB" help:"maximum segment size"`
	MaxMetadataSize              memory.Size         `default:"2KiB" help:"maximum segment metadata size"`
	MaxCommitInterval            time.Duration       `default:"48h" testDefault:"1h" help:"maximum time allowed to pass between creating and committing a segment"`
	MinPartSize                  memory.Size         `default:"5MiB" testDefault:"0" help:"minimum allowed part size (last part has no minimum size limit)"`
	MaxNumberOfParts             int                 `default:"10000" help:"maximum number of parts object can contain"`
	Overlay                      bool                `default:"true" help:"toggle flag if overlay is enabled"`
	RS                           RSConfig            `` /* 132-byte string literal not displayed */
	RateLimiter                  RateLimiterConfig   `help:"rate limiter configuration"`
	UploadLimiter                ObjectLimiterConfig `help:"object upload limiter configuration"`
	DownloadLimiter              ObjectLimiterConfig `help:"object download limiter configuration"`
	ProjectLimits                ProjectLimitConfig  `help:"project limit configuration"`
	SuccessTrackerKind           string              `default:"percent" help:"success tracker kind, bitshift or percent"`
	SuccessTrackerTickDuration   time.Duration       `default:"10m" help:"how often to bump the generation in the node success tracker"`
	SuccessTrackerTrustedUplinks []string            `help:"list of trusted uplinks for success tracker"`

	// TODO remove this flag when server-side copy implementation will be finished
	ServerSideCopy         bool `help:"enable code for server-side copy, deprecated. please leave this to true." default:"true"`
	ServerSideCopyDisabled bool `` /* 192-byte string literal not displayed */
	UseListObjectsIterator bool `help:"switch to iterator based implementation." default:"false"`

	NodeAliasCacheFullRefresh bool `help:"node alias cache does a full refresh when a value is missing" default:"false"`

	UseBucketLevelObjectVersioning bool `help:"enable the use of bucket level object versioning" default:"false"`
	// flag to simplify testing by enabling bucket level versioning feature only for specific projects
	UseBucketLevelObjectVersioningProjects []string `help:"list of projects which will have UseBucketLevelObjectVersioning feature flag enabled" default:"" hidden:"true"`

	ObjectLockEnabled bool `help:"enable the use of bucket-level Object Lock" default:"false"`

	UserInfoValidation UserInfoValidationConfig `help:"Config for user info validation"`

	// TODO remove when we benchmarking are done and decision is made.
	TestListingQuery                bool      `default:"false" help:"test the new query for non-recursive listing"`
	TestOptimizedInlineObjectUpload bool      `default:"false" devDefault:"true" help:"enables optimization for uploading objects with single inline segment"`
	TestingPrecommitDeleteMode      int       `` /* 126-byte string literal not displayed */
	TestingSpannerProjects          UUIDsFlag `default:""  help:"list of project IDs for which Spanner metabase DB is enabled" hidden:"true"`
}

Config is a configuration struct that is everything you need to start a metainfo.

func (Config) Metabase added in v1.67.1 

func (c Config) Metabase(applicationName string) metabase.Config

Metabase constructs Metabase configuration based on Metainfo configuration with specific application name.

type DeleteCommittedObject added in v1.113.1 

type DeleteCommittedObject struct {
	metabase.ObjectLocation
	Version          []byte
	BypassGovernance bool
}

DeleteCommittedObject contains arguments necessary for deleting a committed version of an object via the (*Endpoint).DeleteCommittedObject method.

type Endpoint 

type Endpoint struct {
	pb.DRPCMetainfoUnimplementedServer
	// contains filtered or unexported fields
}

Endpoint metainfo endpoint.

architecture: Endpoint

func NewEndpoint 

func NewEndpoint(log *zap.Logger, buckets *buckets.Service, metabaseDB *metabase.DB,
	orders *orders.Service, cache *overlay.Service, attributions attribution.DB, peerIdentities overlay.PeerIdentities,
	apiKeys APIKeys, projectUsage *accounting.Service, projects console.Projects, projectMembers console.ProjectMembers, users console.Users,
	satellite signing.Signer, revocations revocation.DB, successTrackers *SuccessTrackers, config Config, placement nodeselection.PlacementDefinitions) (*Endpoint, error)

NewEndpoint creates new metainfo endpoint instance.

func TestingNewAPIKeysEndpoint added in v1.106.1 

func TestingNewAPIKeysEndpoint(log *zap.Logger, apiKeys APIKeys) *Endpoint

TestingNewAPIKeysEndpoint returns an endpoint suitable for testing api keys behaviour.

func (*Endpoint) Batch added in v0.17.0 

func (endpoint *Endpoint) Batch(ctx context.Context, req *pb.BatchRequest) (resp *pb.BatchResponse, err error)

Batch handle requests sent in batch.

func (*Endpoint) BeginCopyObject added in v1.50.1 

func (endpoint *Endpoint) BeginCopyObject(ctx context.Context, req *pb.ObjectBeginCopyRequest) (resp *pb.ObjectBeginCopyResponse, err error)

BeginCopyObject begins copying object to different key.

func (*Endpoint) BeginDeleteObject added in v0.16.0 

func (endpoint *Endpoint) BeginDeleteObject(ctx context.Context, req *pb.ObjectBeginDeleteRequest) (resp *pb.ObjectBeginDeleteResponse, err error)

BeginDeleteObject begins object deletion process.

func (*Endpoint) BeginMoveObject added in v1.39.4 

func (endpoint *Endpoint) BeginMoveObject(ctx context.Context, req *pb.ObjectBeginMoveRequest) (resp *pb.ObjectBeginMoveResponse, err error)

BeginMoveObject begins moving object to different key.

func (*Endpoint) BeginObject added in v0.16.0 

func (endpoint *Endpoint) BeginObject(ctx context.Context, req *pb.ObjectBeginRequest) (resp *pb.ObjectBeginResponse, err error)

BeginObject begins object.

func (*Endpoint) BeginSegment added in v0.16.0 

func (endpoint *Endpoint) BeginSegment(ctx context.Context, req *pb.SegmentBeginRequest) (resp *pb.SegmentBeginResponse, err error)

BeginSegment begins segment uploading.

func (*Endpoint) Close 

func (endpoint *Endpoint) Close() error

Close closes resources.

func (*Endpoint) CommitInlineObject added in v1.110.1 

func (endpoint *Endpoint) CommitInlineObject(ctx context.Context, beginObjectReq *pb.ObjectBeginRequest, makeInlineSegReq *pb.SegmentMakeInlineRequest, commitObjectReq *pb.ObjectCommitRequest) (
	_ *pb.ObjectBeginResponse,
	_ *pb.SegmentMakeInlineResponse,
	_ *pb.ObjectCommitResponse, err error,
)

CommitInlineObject commits a full inline object.

func (*Endpoint) CommitObject added in v0.16.0 

func (endpoint *Endpoint) CommitObject(ctx context.Context, req *pb.ObjectCommitRequest) (resp *pb.ObjectCommitResponse, err error)

CommitObject commits an object when all its segments have already been committed.

func (*Endpoint) CommitSegment 

func (endpoint *Endpoint) CommitSegment(ctx context.Context, req *pb.SegmentCommitRequest) (resp *pb.SegmentCommitResponse, err error)

CommitSegment commits segment after uploading.

func (*Endpoint) CompressedBatch added in v1.101.1 

func (endpoint *Endpoint) CompressedBatch(ctx context.Context, req *pb.CompressedBatchRequest) (resp *pb.CompressedBatchResponse, err error)

CompressedBatch handles requests sent in batch that are compressed.

func (*Endpoint) ConvertMetabaseErr added in v1.106.1 

func (endpoint *Endpoint) ConvertMetabaseErr(err error) error

ConvertMetabaseErr converts domain errors from metabase to appropriate rpc statuses errors.

func (*Endpoint) CountBuckets added in v1.9.1 

func (endpoint *Endpoint) CountBuckets(ctx context.Context, projectID uuid.UUID) (count int, err error)

CountBuckets returns the number of buckets a project currently has. TODO: add this to the uplink client side.

func (*Endpoint) CreateBucket added in v0.15.0 

func (endpoint *Endpoint) CreateBucket(ctx context.Context, req *pb.BucketCreateRequest) (resp *pb.BucketCreateResponse, err error)

CreateBucket creates a new bucket.

func (*Endpoint) DeleteBucket added in v0.15.0 

func (endpoint *Endpoint) DeleteBucket(ctx context.Context, req *pb.BucketDeleteRequest) (resp *pb.BucketDeleteResponse, err error)

DeleteBucket deletes a bucket.

func (*Endpoint) DeleteCommittedObject added in v1.26.2 

func (endpoint *Endpoint) DeleteCommittedObject(ctx context.Context, opts DeleteCommittedObject) (deletedObjects []*pb.Object, err error)

DeleteCommittedObject deletes all the pieces of the storage nodes that belongs to the specified object.

NOTE: this method is exported for being able to individually test it without having import cycles.

func (*Endpoint) DeletePart added in v1.36.1 

func (endpoint *Endpoint) DeletePart(ctx context.Context, req *pb.PartDeleteRequest) (resp *pb.PartDeleteResponse, err error)

DeletePart is a no-op.

It was used to perform the deletion of a single part from satellite db and from storage nodes. We made this method noop because now we can overwrite segments for pending objects. It's returning no error to avoid failures with uplinks that still are using this method.

func (*Endpoint) DeletePendingObject added in v1.26.2 

func (endpoint *Endpoint) DeletePendingObject(ctx context.Context, stream metabase.ObjectStream) (deletedObjects []*pb.Object, err error)

DeletePendingObject deletes all the pieces of the storage nodes that belongs to the specified pending object.

NOTE: this method is exported for being able to individually test it without having import cycles.

func (*Endpoint) DownloadObject added in v1.27.0 

func (endpoint *Endpoint) DownloadObject(ctx context.Context, req *pb.ObjectDownloadRequest) (resp *pb.ObjectDownloadResponse, err error)

DownloadObject gets object information, creates a download for segments and lists the object segments.

func (*Endpoint) DownloadSegment 

func (endpoint *Endpoint) DownloadSegment(ctx context.Context, req *pb.SegmentDownloadRequest) (resp *pb.SegmentDownloadResponse, err error)

DownloadSegment returns data necessary to download segment.

func (*Endpoint) FinishCopyObject added in v1.50.1 

func (endpoint *Endpoint) FinishCopyObject(ctx context.Context, req *pb.ObjectFinishCopyRequest) (resp *pb.ObjectFinishCopyResponse, err error)

FinishCopyObject accepts new encryption keys for object copy and updates the corresponding object ObjectKey and segments EncryptedKey. It optionally sets retention mode and period on the new object.

func (*Endpoint) FinishMoveObject added in v1.40.3 

func (endpoint *Endpoint) FinishMoveObject(ctx context.Context, req *pb.ObjectFinishMoveRequest) (resp *pb.ObjectFinishMoveResponse, err error)

FinishMoveObject accepts new encryption keys for moved object and updates the corresponding object ObjectKey and segments EncryptedKey. It optionally sets retention mode and period on the new object.

func (*Endpoint) GetBucket added in v0.15.0 

func (endpoint *Endpoint) GetBucket(ctx context.Context, req *pb.BucketGetRequest) (resp *pb.BucketGetResponse, err error)

GetBucket returns a bucket.

func (*Endpoint) GetBucketLocation added in v1.87.1 

func (endpoint *Endpoint) GetBucketLocation(ctx context.Context, req *pb.GetBucketLocationRequest) (resp *pb.GetBucketLocationResponse, err error)

GetBucketLocation responds with the location that the bucket's placement is annotated with (if any) and any error encountered.

func (*Endpoint) GetBucketObjectLockConfiguration added in v1.107.1 

func (endpoint *Endpoint) GetBucketObjectLockConfiguration(ctx context.Context, req *pb.GetBucketObjectLockConfigurationRequest) (resp *pb.GetBucketObjectLockConfigurationResponse, err error)

GetBucketObjectLockConfiguration returns a bucket's Object Lock configuration.

func (*Endpoint) GetBucketVersioning added in v1.93.1 

func (endpoint *Endpoint) GetBucketVersioning(ctx context.Context, req *pb.GetBucketVersioningRequest) (resp *pb.GetBucketVersioningResponse, err error)

GetBucketVersioning responds with the versioning state of the bucket and any error encountered.

func (*Endpoint) GetObject added in v0.16.0 

func (endpoint *Endpoint) GetObject(ctx context.Context, req *pb.ObjectGetRequest) (resp *pb.ObjectGetResponse, err error)

GetObject gets single object metadata.

func (*Endpoint) GetObjectIPs added in v1.11.1 

func (endpoint *Endpoint) GetObjectIPs(ctx context.Context, req *pb.ObjectGetIPsRequest) (resp *pb.ObjectGetIPsResponse, err error)

GetObjectIPs returns the IP addresses of the nodes holding the pieces for the provided object. This is useful for knowing the locations of the pieces.

func (*Endpoint) GetObjectLegalHold added in v1.113.1 

func (endpoint *Endpoint) GetObjectLegalHold(ctx context.Context, req *pb.GetObjectLegalHoldRequest) (_ *pb.GetObjectLegalHoldResponse, err error)

GetObjectLegalHold returns an object's Object Lock legal hold configuration.

func (*Endpoint) GetObjectRetention added in v1.110.1 

func (endpoint *Endpoint) GetObjectRetention(ctx context.Context, req *pb.GetObjectRetentionRequest) (_ *pb.GetObjectRetentionResponse, err error)

GetObjectRetention returns an object's Object Lock retention configuration.

func (*Endpoint) ListBuckets added in v0.15.0 

func (endpoint *Endpoint) ListBuckets(ctx context.Context, req *pb.BucketListRequest) (resp *pb.BucketListResponse, err error)

ListBuckets returns buckets in a project where the bucket name matches the request cursor.

func (*Endpoint) ListObjects added in v0.16.0 

func (endpoint *Endpoint) ListObjects(ctx context.Context, req *pb.ObjectListRequest) (resp *pb.ObjectListResponse, err error)

ListObjects list objects according to specific parameters.

func (*Endpoint) ListPendingObjectStreams added in v1.22.1 

func (endpoint *Endpoint) ListPendingObjectStreams(ctx context.Context, req *pb.ObjectListPendingStreamsRequest) (resp *pb.ObjectListPendingStreamsResponse, err error)

ListPendingObjectStreams list pending objects according to specific parameters.

func (*Endpoint) ListSegments 

func (endpoint *Endpoint) ListSegments(ctx context.Context, req *pb.SegmentListRequest) (resp *pb.SegmentListResponse, err error)

ListSegments list object segments.

func (*Endpoint) MakeInlineSegment added in v0.16.0 

func (endpoint *Endpoint) MakeInlineSegment(ctx context.Context, req *pb.SegmentMakeInlineRequest) (resp *pb.SegmentMakeInlineResponse, err error)

MakeInlineSegment makes inline segment on satellite.

func (*Endpoint) ProjectInfo added in v0.14.0 

func (endpoint *Endpoint) ProjectInfo(ctx context.Context, req *pb.ProjectInfoRequest) (_ *pb.ProjectInfoResponse, err error)

ProjectInfo returns allowed ProjectInfo for the provided API key.

func (*Endpoint) RetryBeginSegmentPieces added in v1.72.2 

func (endpoint *Endpoint) RetryBeginSegmentPieces(ctx context.Context, req *pb.RetryBeginSegmentPiecesRequest) (resp *pb.RetryBeginSegmentPiecesResponse, err error)

RetryBeginSegmentPieces replaces put order limits for failed piece uploads.

func (*Endpoint) RevokeAPIKey added in v1.7.1 

func (endpoint *Endpoint) RevokeAPIKey(ctx context.Context, req *pb.RevokeAPIKeyRequest) (resp *pb.RevokeAPIKeyResponse, err error)

RevokeAPIKey handles requests to revoke an api key.

func (*Endpoint) Run added in v1.105.2 

func (endpoint *Endpoint) Run(ctx context.Context) error

Run manages the internal dependencies of the endpoint such as the success tracker.

func (*Endpoint) SetBucketObjectLockConfiguration added in v1.116.3 

func (endpoint *Endpoint) SetBucketObjectLockConfiguration(ctx context.Context, req *pb.SetBucketObjectLockConfigurationRequest) (resp *pb.SetBucketObjectLockConfigurationResponse, err error)

SetBucketObjectLockConfiguration updates a bucket's Object Lock configuration.

func (*Endpoint) SetBucketVersioning added in v1.93.1 

func (endpoint *Endpoint) SetBucketVersioning(ctx context.Context, req *pb.SetBucketVersioningRequest) (resp *pb.SetBucketVersioningResponse, err error)

SetBucketVersioning attempts to enable or disable versioning for a bucket and responds with any error encountered.

func (*Endpoint) SetObjectLegalHold added in v1.113.1 

func (endpoint *Endpoint) SetObjectLegalHold(ctx context.Context, req *pb.SetObjectLegalHoldRequest) (_ *pb.SetObjectLegalHoldResponse, err error)

SetObjectLegalHold sets an object's Object Lock legal hold configuration.

func (*Endpoint) SetObjectRetention added in v1.110.1 

func (endpoint *Endpoint) SetObjectRetention(ctx context.Context, req *pb.SetObjectRetentionRequest) (_ *pb.SetObjectRetentionResponse, err error)

SetObjectRetention sets an object's Object Lock retention configuration.

func (*Endpoint) TestSetObjectLockEnabled added in v1.113.1 

func (endpoint *Endpoint) TestSetObjectLockEnabled(enabled bool)

TestSetObjectLockEnabled sets whether bucket-level Object Lock functionality should be globally enabled. Used for testing.

func (*Endpoint) TestSetUseBucketLevelVersioning added in v1.110.1 

func (endpoint *Endpoint) TestSetUseBucketLevelVersioning(enabled bool)

TestSetUseBucketLevelVersioning sets whether bucket-level Object Versioning functionality should be globally enabled. Used for testing.

func (*Endpoint) TestSetUseBucketLevelVersioningByProjectID added in v1.110.1 

func (endpoint *Endpoint) TestSetUseBucketLevelVersioningByProjectID(projectID uuid.UUID, enabled bool)

TestSetUseBucketLevelVersioningByProjectID sets whether bucket-level Object Versioning functionality should be enabled for a specific project. Used for testing.

func (*Endpoint) TestingSetRSConfig added in v1.114.4 

func (endpoint *Endpoint) TestingSetRSConfig(rs RSConfig)

TestingSetRSConfig set endpoint RS config for testing.

func (*Endpoint) TestingSetRateLimiterTime added in v1.116.3 

func (endpoint *Endpoint) TestingSetRateLimiterTime(time func() time.Time)

TestingSetRateLimiterTime sets the time function used by the rate limiter.

func (*Endpoint) UpdateObjectMetadata added in v1.34.1 

func (endpoint *Endpoint) UpdateObjectMetadata(ctx context.Context, req *pb.ObjectUpdateMetadataRequest) (resp *pb.ObjectUpdateMetadataResponse, err error)

UpdateObjectMetadata replaces object metadata.

func (*Endpoint) ValidateAuthAny added in v1.110.1 

func (endpoint *Endpoint) ValidateAuthAny(ctx context.Context, header *pb.RequestHeader, rateLimitKind console.LimitKind, permissions ...VerifyPermission) (_ *console.APIKeyInfo, err error)

ValidateAuthAny validates things like API keys, rate limit and user permissions. At least one required (not optional) permission must be permitted. If not, an error is returned, and optional actions aren't checked. It always returns valid RPC errors.

func (*Endpoint) ValidateAuthN added in v1.106.1 

func (endpoint *Endpoint) ValidateAuthN(ctx context.Context, header *pb.RequestHeader, rateLimitKind console.LimitKind, permissions ...VerifyPermission) (_ *console.APIKeyInfo, err error)

ValidateAuthN validates things like API keys, rate limit and user permissions for each permission from permissions. It returns an error for the first required (not optional) permission that the check fails for. There must be at least one required (not optional) permission. In case all permissions are optional, it will return an error. It always returns valid RPC errors.

type ExtendedConfig added in v1.87.1 

type ExtendedConfig struct {
	Config
	// contains filtered or unexported fields
}

ExtendedConfig extended config keeps additional helper fields and methods around Config.

func NewExtendedConfig added in v1.87.1 

func NewExtendedConfig(config Config) (_ ExtendedConfig, err error)

NewExtendedConfig creates new instance of extended config.

func (ExtendedConfig) ObjectLockEnabledByProject added in v1.113.1 

func (ec ExtendedConfig) ObjectLockEnabledByProject(project *console.Project) bool

ObjectLockEnabledByProject checks if bucket-level Object Lock functionality should be enabled for a specific project.

func (ExtendedConfig) UseBucketLevelObjectVersioningByProject added in v1.93.1 

func (ec ExtendedConfig) UseBucketLevelObjectVersioningByProject(project *console.Project) bool

UseBucketLevelObjectVersioningByProject checks if UseBucketLevelObjectVersioning should be enabled for specific project.

type ObjectLimiterConfig added in v1.116.7 

type ObjectLimiterConfig struct {
	Enabled           bool          `help:"whether rate limiting is enabled." releaseDefault:"true" devDefault:"true"`
	SingleObjectLimit time.Duration `help:"how often we can interact with a single object (the same location) per API instance" default:"10ms" devDefault:"1ms"`

	CacheCapacity int `help:"number of object locations to cache." releaseDefault:"10000" devDefault:"10" testDefault:"100"`
}

ObjectLimiterConfig is a configuration struct for endpoint limiting per object.

type ProjectLimitConfig added in v1.9.1 

type ProjectLimitConfig struct {
	MaxBuckets int `help:"max bucket count for a project." default:"100" testDefault:"10"`
}

ProjectLimitConfig is a configuration struct for default project limits.

type RSConfig added in v0.14.0 

type RSConfig struct {
	ErasureShareSize memory.Size
	Min              int
	Repair           int
	Success          int
	Total            int
}

RSConfig is a configuration struct that keeps details about default redundancy strategy information.

Can be used as a flag.

func (*RSConfig) Override added in v1.107.1 

func (rs *RSConfig) Override(o nodeselection.ECParameters) *RSConfig

Override creates a new RSConfig instance, all non-zero parameters of o will be used to override current values.

func (*RSConfig) RedundancyStrategy added in v1.53.1 

func (rs *RSConfig) RedundancyStrategy() (eestream.RedundancyStrategy, error)

RedundancyStrategy creates eestream.RedundancyStrategy from config values.

func (*RSConfig) Set added in v1.17.1 

func (rs *RSConfig) Set(s string) error

Set sets the value from a string in the format satellite/satellitedb/overlaycache.gok/m/o/n-size (min/repair/optimal/total-erasuresharesize).

func (*RSConfig) String added in v1.17.1 

func (rs *RSConfig) String() string

String is required for pflag.Value.

func (RSConfig) Type added in v1.17.1 

func (RSConfig) Type() string

Type implements pflag.Value.

type RateLimiterConfig added in v0.31.0 

type RateLimiterConfig struct {
	Enabled         bool          `help:"whether rate limiting is enabled." releaseDefault:"true" devDefault:"true"`
	Rate            float64       `help:"request rate per project per second." releaseDefault:"100" devDefault:"100" testDefault:"1000"`
	CacheCapacity   int           `help:"number of projects to cache." releaseDefault:"10000" devDefault:"10" testDefault:"100"`
	CacheExpiration time.Duration `help:"how long to cache the projects limiter." releaseDefault:"10m" devDefault:"10s"`
}

RateLimiterConfig is a configuration struct for endpoint rate limiting.

type SuccessTracker added in v1.105.3 

type SuccessTracker interface {
	// Increment tells the SuccessTracker if a node was recently successful or
	// not.
	Increment(node storj.NodeID, success bool)

	// Get returns a value that represents how successful a node is expected to
	// be. It can return NaN to indicate that it has no information about the
	// node.
	Get(node *nodeselection.SelectedNode) float64

	// BumpGeneration should be called periodically to clear out stale
	// information.
	BumpGeneration()

	monkit.StatSource
}

SuccessTracker describes a type that is told about successes of nodes and can be queried for an aggregate value that represents how successful a node is expected to be.

func NewBigBitshiftSuccessTracker added in v1.108.1 

func NewBigBitshiftSuccessTracker(length int) SuccessTracker

NewBigBitshiftSuccessTracker creates a new BigBitshiftSuccessTracker.

type SuccessTrackers added in v1.105.3 

type SuccessTrackers struct {
	// contains filtered or unexported fields
}

SuccessTrackers manages global and uplink level trackers.

func NewSuccessTrackers added in v1.105.3 

func NewSuccessTrackers(approvedUplinks []storj.NodeID, newTracker func() SuccessTracker) *SuccessTrackers

NewSuccessTrackers creates a new success tracker.

func (*SuccessTrackers) BumpGeneration added in v1.105.3 

func (t *SuccessTrackers) BumpGeneration()

BumpGeneration will bump all the managed trackers.

func (*SuccessTrackers) Get added in v1.105.3 

func (t *SuccessTrackers) Get(uplink storj.NodeID) func(node *nodeselection.SelectedNode) float64

Get returns a function that can be used to get an estimate of how good a node is for a given uplink.

func (*SuccessTrackers) GetTracker added in v1.105.3 

func (t *SuccessTrackers) GetTracker(uplink storj.NodeID) SuccessTracker

GetTracker returns the tracker for the specific uplink. Returns with the global tracker, if uplink is not whitelisted.

func (*SuccessTrackers) Stats added in v1.115.5 

func (t *SuccessTrackers) Stats(cb func(monkit.SeriesKey, string, float64))

Stats reports monkit statistics for all of the trackers.

type UUIDsFlag added in v1.112.2 

type UUIDsFlag map[uuid.UUID]struct{}

UUIDsFlag is a configuration struct that keeps info about project IDs

Can be used as a flag.

func (*UUIDsFlag) Set added in v1.112.2 

func (m *UUIDsFlag) Set(s string) error

Set is required for pflag.Value.

func (UUIDsFlag) String added in v1.112.2 

func (m UUIDsFlag) String() string

String is required for pflag.Value.

func (UUIDsFlag) Type added in v1.112.2 

func (m UUIDsFlag) Type() string

Type is required for pflag.Value.

type UserInfoValidationConfig added in v1.109.1 

type UserInfoValidationConfig struct {
	Enabled         bool          `help:"whether validation is enabled for user account info" default:"false"`
	CacheExpiration time.Duration `help:"user info cache expiration" default:"5m"`
	CacheCapacity   int           `help:"user info cache capacity" default:"10000"`
}

UserInfoValidationConfig is a configuration struct for user info validation.

type VerifyPermission added in v1.106.1 

type VerifyPermission struct {
	Action          macaroon.Action
	ActionPermitted *bool
	Optional        bool
}

VerifyPermission specifies one permission that is required by an endpoint.

Source Files

View all Source files

Directories

Path Synopsis
Package expireddeletion contains the functions needed to run expired segment deletion
 Package expireddeletion contains the functions needed to run expired segment deletion
Package pointerverification implements verification of pointers.
 Package pointerverification implements verification of pointers.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.