kbfsblock

package
v3.0.0+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 5, 2019 License: BSD-3-Clause, BSD-3-Clause, BSD-3-Clause Imports: 18 Imported by: 0

README

kbfsblock

Types and functions to work with KBFS blocks. The package is named kbfsblock rather than block to avoid clashes with existing variable names.

All errors returned by this package are wrapped with pkg/errors, and so need errors.Cause() to unwrap them.

Documentation

Index

Constants

View Source 
const (
	// ServerTokenServer is the expected server type for bserver authentication.
	ServerTokenServer = "kbfs_block"
	// ServerTokenExpireIn is the TTL to use when constructing an authentication token.
	ServerTokenExpireIn = 24 * 60 * 60 // 24 hours
)
View Source 
const (
	// StatusCodeServerError is the error code for a generic block server error.
	StatusCodeServerError = 2700
	// StatusCodeServerErrorBadRequest is the error code for a generic client error.
	StatusCodeServerErrorBadRequest = 2701
	// StatusCodeServerErrorUnauthorized is the error code for when the session has not been validated
	StatusCodeServerErrorUnauthorized = 2702
	// StatusCodeServerErrorOverQuota is the error code for when the user has exceeded his quota
	StatusCodeServerErrorOverQuota = 2703
	// StatusCodeServerErrorBlockNonExistent is the error code for when bserver cannot find a block
	StatusCodeServerErrorBlockNonExistent = 2704
	// StatusCodeServerErrorBlockArchived is the error code for a block has been archived
	StatusCodeServerErrorBlockArchived = 2705
	// StatusCodeServerErrorNoPermission is the error code for when there's no permission
	StatusCodeServerErrorNoPermission = 2706
	// StatusCodeServerErrorBlockDeleted is the error code for a block has been deleted
	StatusCodeServerErrorBlockDeleted = 2707
	// StatusCodeServerErrorNonceNonExistent is the error code when a nonce cannot be found
	StatusCodeServerErrorNonceNonExistent = 2708
	// StatusCodeServerErrorMaxRefExceeded is the error code to indicate there are too many refs to a block
	StatusCodeServerErrorMaxRefExceeded = 2709
	// StatusCodeServerErrorThrottle is the error code to indicate the client should initiate backoff.
	StatusCodeServerErrorThrottle = 2799
)
View Source 
const (
	// MaxIDStringLength is the maximum length of the string
	// representation of a ID.
	MaxIDStringLength = kbfshash.MaxHashStringLength
)

Variables

View Source 
var ZeroID = ID{}

ZeroID is a zero-valued ID.

View Source 
var ZeroRefNonce = RefNonce([8]byte{0, 0, 0, 0, 0, 0, 0, 0})

ZeroRefNonce is a special BlockRefNonce used for the initial reference to a block.

Functions

func BatchDowngradeReferences 

func BatchDowngradeReferences(ctx context.Context, log logger.Logger,
	tlfID tlf.ID, contexts ContextMap, archive bool,
	server keybase1.BlockInterface) (
	doneRefs map[ID]map[RefNonce]int, finalError error)

BatchDowngradeReferences archives or deletes a batch of references, handling all batching and throttles.

func GetLiveCounts 

func GetLiveCounts(doneRefs map[ID]map[RefNonce]int) map[ID]int

GetLiveCounts computes the maximum live count for each ID over its RefNonces.

func GetReferenceCount 

func GetReferenceCount(
	ctx context.Context, tlfID tlf.ID, contexts ContextMap,
	refStatus keybase1.BlockStatus, server keybase1.BlockInterface) (
	liveCounts map[ID]int, err error)

GetReferenceCount returns the number of live references (at least as "live" as `refStatus`) for each block ID.

func IsThrottleError 

func IsThrottleError(err error) bool

IsThrottleError returns whether or not the given error signals throttling.

func MakeAddReferenceArg 

func MakeAddReferenceArg(tlfID tlf.ID, id ID, context Context) keybase1.AddReferenceArg

MakeAddReferenceArg builds a keybase1.AddReferenceArg from the given params.

func MakeGetBlockArg 

func MakeGetBlockArg(tlfID tlf.ID, id ID, context Context) keybase1.GetBlockArg

MakeGetBlockArg builds a keybase1.GetBlockArg from the given params.

func MakePutBlockAgainArg 

func MakePutBlockAgainArg(tlfID tlf.ID, id ID,
	bContext Context, buf []byte, serverHalf kbfscrypto.BlockCryptKeyServerHalf) keybase1.PutBlockAgainArg

MakePutBlockAgainArg builds a keybase1.PutBlockAgainArg from the given params.

func MakePutBlockArg 

func MakePutBlockArg(tlfID tlf.ID, id ID,
	bContext Context, buf []byte,
	serverHalf kbfscrypto.BlockCryptKeyServerHalf) keybase1.PutBlockArg

MakePutBlockArg builds a keybase1.PutBlockArg from the given params.

func ParseGetBlockRes 

func ParseGetBlockRes(res keybase1.GetBlockRes, resErr error) (
	buf []byte, serverHalf kbfscrypto.BlockCryptKeyServerHalf, err error)

ParseGetBlockRes parses the given keybase1.GetBlockRes into its components.

func VerifyID 

func VerifyID(encodedEncryptedData []byte, id ID) error

VerifyID verifies that the given block ID is the permanent block ID for the given encoded and encrypted data.

Types

type Context 

type Context struct {
	// Creator is the UID that was first charged for the initial
	// reference to this block.
	Creator keybase1.UserOrTeamID `codec:"c"`
	// Writer is the UID that should be charged for this reference to
	// the block.  If empty, it defaults to Creator.
	Writer keybase1.UserOrTeamID `codec:"w,omitempty" json:",omitempty"`
	// When RefNonce is all 0s, this is the initial reference to a
	// particular block.  Using a constant refnonce for the initial
	// reference allows the server to identify and optimize for the
	// common case where there is only one reference for a block.  Two
	// initial references cannot happen simultaneously, because the
	// encrypted block contents (and thus the block ID) will be
	// randomized by the server-side block crypt key half.  All
	// subsequent references to the same block must have a random
	// RefNonce (it can't be a monotonically increasing number because
	// that would require coordination among clients).
	RefNonce RefNonce `codec:"r,omitempty"`
	// BlockType indicates the type of the block (data
	// vs. metadata). This is used, for example, when deciding how the
	// block affects quotas.
	BlockType keybase1.BlockType `codec:"b,omitempty"`
}

Context contains all the information used by the server to identify blocks (other than the ID).

NOTE: Don't add or modify anything in this struct without considering how old clients will handle them.

func MakeContext 

func MakeContext(
	creator keybase1.UserOrTeamID, writer keybase1.UserOrTeamID, nonce RefNonce,
	bType keybase1.BlockType) Context

MakeContext makes a context with the given creator, writer, and nonce, where the writer is not necessarily equal to the creator, and the nonce is usually non-zero.

func MakeFirstContext 

func MakeFirstContext(
	creator keybase1.UserOrTeamID, bType keybase1.BlockType) Context

MakeFirstContext makes the initial context for a block with the given creator.

func (Context) GetBlockType 

func (c Context) GetBlockType() keybase1.BlockType

GetBlockType returns the block type of the associated block.

func (Context) GetCreator 

func (c Context) GetCreator() keybase1.UserOrTeamID

GetCreator returns the creator of the associated block.

func (Context) GetRefNonce 

func (c Context) GetRefNonce() RefNonce

GetRefNonce returns the ref nonce of the associated block.

func (Context) GetWriter 

func (c Context) GetWriter() keybase1.UserOrTeamID

GetWriter returns the writer of the associated block.

func (Context) IsFirstRef 

func (c Context) IsFirstRef() bool

IsFirstRef returns whether or not p represents the first reference to the corresponding ID.

func (*Context) SetWriter 

func (c *Context) SetWriter(newWriter keybase1.UserOrTeamID)

SetWriter sets the Writer field, if necessary.

func (Context) String 

func (c Context) String() string

type ContextMap 

type ContextMap map[ID][]Context

ContextMap is a map from a block ID to a list of its contexts.

type ID 

type ID struct {
	// contains filtered or unexported fields
}

ID is the (usually content-based) ID for a data block.

func FakeID 

func FakeID(b byte) ID

FakeID returns an ID derived from the given byte, suitable for testing.

func FakeIDAdd 

func FakeIDAdd(id ID, b byte) ID

FakeIDAdd returns an ID derived from the given ID and the given byte, suitable for testing.

func FakeIDMul 

func FakeIDMul(id ID, b byte) ID

FakeIDMul returns an ID derived from the given ID and given byte using *, suitable for testing.

TODO: Fix the test that breaks when this is replaced with FakeIDAdd.

func IDFromBytes 

func IDFromBytes(idBytes []byte) (ID, error)

IDFromBytes creates a ID from the given bytes. If the returned error is nil, the returned ID is valid.

func IDFromString 

func IDFromString(idStr string) (ID, error)

IDFromString creates a ID from the given string. If the returned error is nil, the returned ID is valid.

func MakePermanentID 

func MakePermanentID(
	encodedEncryptedData []byte, encryptionVer kbfscrypto.EncryptionVer) (
	ID, error)

MakePermanentID computes the permanent ID of a block given its encoded and encrypted contents.

func MakeRandomIDInRange 

func MakeRandomIDInRange(start, end float64, mathRandForTest bool) (ID, error)

MakeRandomIDInRange generates a random block ID using a CSPRNG, distributing the random variable over the interval [start, end), where the full range is [0, MaxUint64). This corresponds to a normalized representation of the range [kbfshash.RawDefaultHash{}, kbfshash.MaxDefaultHash).

func MakeTemporaryID 

func MakeTemporaryID() (ID, error)

MakeTemporaryID generates a temporary block ID using a CSPRNG. This is used for indirect blocks before they're committed to the server.

func (ID) Bytes 

func (id ID) Bytes() []byte

Bytes returns the bytes of the block ID.

func (*ID) HashType 

func (id *ID) HashType() kbfshash.HashType

HashType returns the type used for this ID.

func (ID) IsValid 

func (id ID) IsValid() bool

IsValid returns whether the block ID is valid. A zero block ID is considered invalid.

func (ID) MarshalBinary 

func (id ID) MarshalBinary() (data []byte, err error)

MarshalBinary implements the encoding.BinaryMarshaler interface for ID. Returns an error if the ID is invalid and not the zero ID.

func (ID) MarshalText 

func (id ID) MarshalText() ([]byte, error)

MarshalText implements the encoding.TextMarshaler interface for ID.

func (ID) String 

func (id ID) String() string

func (*ID) UnmarshalBinary 

func (id *ID) UnmarshalBinary(data []byte) error

UnmarshalBinary implements the encoding.BinaryUnmarshaler interface for ID. Returns an error if the given byte array is non-empty and the ID is invalid.

func (*ID) UnmarshalText 

func (id *ID) UnmarshalText(buf []byte) error

UnmarshalText implements the encoding.TextUnmarshaler interface for ID.

type QuotaInfo 

type QuotaInfo struct {
	Folders  map[string]*UsageStat
	Total    *UsageStat
	Limit    int64
	GitLimit int64
}

QuotaInfo contains a user's quota usage information

func NewQuotaInfo 

func NewQuotaInfo() *QuotaInfo

NewQuotaInfo returns a newly constructed QuotaInfo.

func ParseGetQuotaInfoRes 

func ParseGetQuotaInfoRes(codec kbfscodec.Codec, res []byte, resErr error) (
	info *QuotaInfo, err error)

ParseGetQuotaInfoRes parses the given quota result into a *QuotaInfo.

func QuotaInfoDecode 

func QuotaInfoDecode(b []byte, codec kbfscodec.Codec) (
	*QuotaInfo, error)

QuotaInfoDecode decodes b into a QuotaInfo

func (*QuotaInfo) Accum 

func (u *QuotaInfo) Accum(another *QuotaInfo, accumF func(int64, int64) int64)

Accum combines changes to the existing QuotaInfo object using accumulation function accumF.

func (*QuotaInfo) AccumOne 

func (u *QuotaInfo) AccumOne(change int, folder string, usage UsageType)

AccumOne combines one quota charge to the existing QuotaInfo

func (*QuotaInfo) ToBytes 

func (u *QuotaInfo) ToBytes(codec kbfscodec.Codec) ([]byte, error)

ToBytes marshals this QuotaInfo

type RefNonce 

type RefNonce [8]byte

RefNonce is a 64-bit unique sequence of bytes for identifying this reference of a block ID from other references to the same (duplicated) block.

func MakeRefNonce 

func MakeRefNonce() (RefNonce, error)

MakeRefNonce generates a non-zero block reference nonce using a CSPRNG. This is used for distinguishing different references to the same ID.

func (RefNonce) String 

func (nonce RefNonce) String() string

type ServerError 

type ServerError struct {
	Msg string
}

ServerError is a generic bserver-side error.

func (ServerError) Error 

func (e ServerError) Error() string

Error implements the Error interface for ServerError.

func (ServerError) ToStatus 

func (e ServerError) ToStatus() (s keybase1.Status)

ToStatus implements the ExportableError interface for ServerError.

type ServerErrorBadRequest 

type ServerErrorBadRequest struct {
	Msg string
}

ServerErrorBadRequest is a generic client-side error.

func (ServerErrorBadRequest) Error 

func (e ServerErrorBadRequest) Error() string

Error implements the Error interface for ServerError.

func (ServerErrorBadRequest) ToStatus 

func (e ServerErrorBadRequest) ToStatus() (s keybase1.Status)

ToStatus implements the ExportableError interface for ServerError.

type ServerErrorBlockArchived 

type ServerErrorBlockArchived struct {
	Msg string
}

ServerErrorBlockArchived is an exportable error from bserver

func (ServerErrorBlockArchived) Error 

func (e ServerErrorBlockArchived) Error() string

Error implements the Error interface for ServerErrorBlockArchived.

func (ServerErrorBlockArchived) ToStatus 

func (e ServerErrorBlockArchived) ToStatus() (s keybase1.Status)

ToStatus implements the ExportableError interface for ServerErrorBlockArchived

type ServerErrorBlockDeleted 

type ServerErrorBlockDeleted struct {
	Msg string
}

ServerErrorBlockDeleted is an exportable error from bserver

func (ServerErrorBlockDeleted) Error 

func (e ServerErrorBlockDeleted) Error() string

Error implements the Error interface for ServerErrorBlockDeleted

func (ServerErrorBlockDeleted) ToStatus 

func (e ServerErrorBlockDeleted) ToStatus() (s keybase1.Status)

ToStatus implements the ExportableError interface for ServerErrorBlockDeleted

type ServerErrorBlockNonExistent 

type ServerErrorBlockNonExistent struct {
	Msg string
}

ServerErrorBlockNonExistent is an exportable error from bserver

func (ServerErrorBlockNonExistent) Error 

func (e ServerErrorBlockNonExistent) Error() string

Error implements the Error interface for ServerErrorBlockNonExistent.

func (ServerErrorBlockNonExistent) ToStatus 

func (e ServerErrorBlockNonExistent) ToStatus() (s keybase1.Status)

ToStatus implements the ExportableError interface for ServerErrorBlockNonExistent

type ServerErrorMaxRefExceeded 

type ServerErrorMaxRefExceeded struct {
	Msg string
}

ServerErrorMaxRefExceeded is an exportable error from bserver

func (ServerErrorMaxRefExceeded) Error 

func (e ServerErrorMaxRefExceeded) Error() string

Error implements the Error interface for ServerErrorMaxRefExceeded

func (ServerErrorMaxRefExceeded) ToStatus 

func (e ServerErrorMaxRefExceeded) ToStatus() (s keybase1.Status)

ToStatus implements the ExportableError interface for ServerErrorMaxRefExceeded

type ServerErrorNoPermission 

type ServerErrorNoPermission struct {
	Msg string
}

ServerErrorNoPermission is an exportable error from bserver

func (ServerErrorNoPermission) Error 

func (e ServerErrorNoPermission) Error() string

Error implements the Error interface for ServerErrorNoPermission.

func (ServerErrorNoPermission) ToStatus 

func (e ServerErrorNoPermission) ToStatus() (s keybase1.Status)

ToStatus implements the ExportableError interface for ServerErrorBlockArchived

type ServerErrorNonceNonExistent 

type ServerErrorNonceNonExistent struct {
	Msg string
}

ServerErrorNonceNonExistent is an exportable error from bserver

func (ServerErrorNonceNonExistent) Error 

func (e ServerErrorNonceNonExistent) Error() string

Error implements the Error interface for ServerErrornonceNonExistent.

func (ServerErrorNonceNonExistent) ToStatus 

func (e ServerErrorNonceNonExistent) ToStatus() (s keybase1.Status)

ToStatus implements the ExportableError interface for ServerErrorNonceNonExistent

type ServerErrorOverQuota 

type ServerErrorOverQuota struct {
	Msg string
	// Usage indicates the current usage
	Usage int64
	// Limit indicates the current quota limit
	Limit int64
	// Throttled indicates if request has not been completed due to server throttle
	Throttled bool
}

ServerErrorOverQuota is returned when a user is over quota.

func (ServerErrorOverQuota) Error 

func (e ServerErrorOverQuota) Error() string

Error implements the Error interface for ServerErrorOverQuota.

func (ServerErrorOverQuota) ToStatus 

func (e ServerErrorOverQuota) ToStatus() (s keybase1.Status)

ToStatus implements the ExportableError interface for ServerErrorOverQuota.

type ServerErrorThrottle 

type ServerErrorThrottle struct {
	Msg string
}

ServerErrorThrottle is returned when the server wants the client to backoff.

func (ServerErrorThrottle) Error 

func (e ServerErrorThrottle) Error() string

Error implements the Error interface for ServerErrorThrottle.

func (ServerErrorThrottle) ToStatus 

func (e ServerErrorThrottle) ToStatus() (s keybase1.Status)

ToStatus implements the ExportableError interface for ServerErrorThrottle.

type ServerErrorUnauthorized 

type ServerErrorUnauthorized struct {
	Msg string
}

ServerErrorUnauthorized is a generic client-side error.

func (ServerErrorUnauthorized) Error 

func (e ServerErrorUnauthorized) Error() string

Error implements the Error interface for ServerErrorUnauthorized.

func (ServerErrorUnauthorized) ToStatus 

func (e ServerErrorUnauthorized) ToStatus() (s keybase1.Status)

ToStatus implements the ExportableError interface for ServerErrorUnauthorized.

type ServerErrorUnwrapper 

type ServerErrorUnwrapper struct{}

ServerErrorUnwrapper unwraps errors from a remote block server.

func (ServerErrorUnwrapper) MakeArg 

func (eu ServerErrorUnwrapper) MakeArg() interface{}

MakeArg implements rpc.ErrorUnwrapper.

func (ServerErrorUnwrapper) UnwrapError 

func (eu ServerErrorUnwrapper) UnwrapError(arg interface{}) (appError error, dispatchError error)

UnwrapError implements rpc.ErrorUnwrapper.

type UsageStat 

type UsageStat struct {
	Bytes  map[UsageType]int64
	Blocks map[UsageType]int64
	// Mtime is in unix nanoseconds
	Mtime int64
}

UsageStat tracks the amount of bytes/blocks used, broken down by usage types

func NewUsageStat 

func NewUsageStat() *UsageStat

NewUsageStat creates a new UsageStat

func (*UsageStat) Accum 

func (u *UsageStat) Accum(another *UsageStat, accumF func(int64, int64) int64)

Accum combines changes to the existing QuotaInfo object using accumulation function accumF.

func (*UsageStat) AccumOne 

func (u *UsageStat) AccumOne(change int, usage UsageType)

AccumOne records the usage of one block, whose size is denoted by change A positive change means the block is newly added, negative means the block is deleted. If archive is true, it means the block is archived.

func (*UsageStat) NonZero 

func (u *UsageStat) NonZero() bool

NonZero checks whether UsageStat has accumulated any usage info

type UsageType 

type UsageType int

UsageType indicates the type of usage that quota manager is keeping stats of 

const (
	// UsageWrite indicates a data block is written (written blocks include archived blocks)
	UsageWrite UsageType = iota
	// UsageArchive indicates an existing (data) block is archived
	UsageArchive
	// UsageRead indicates a block is read
	UsageRead
	// UsageMDWrite indicates a MD block is written
	UsageMDWrite
	// UsageGitWrite indicates a git block is written
	UsageGitWrite
	// UsageGitArchive indicates an existing git block is archived
	UsageGitArchive
	// NumUsage indicates the number of usage types
	NumUsage
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.