gitaly

Documentation

Index

• Variables
• func OpenDatabase(databasePath string) (*badger.DB, error)
• type CustomHooksUpdate
• type DefaultBranchUpdate
• type InvalidReferenceFormatError
  • func (err InvalidReferenceFormatError) Error() string
• type LogIndex
  • func (index LogIndex) String() string
• type PartitionManager
  • func NewPartitionManager(db *badger.DB, storages []config.Storage, localRepoFactory localrepo.Factory, ...) *PartitionManager
  • func (pm *PartitionManager) Begin(ctx context.Context, repo repo.GitRepo) (*Transaction, error)
  • func (pm *PartitionManager) Stop()
• type ReferenceToBeDeletedError
  • func (err ReferenceToBeDeletedError) Error() string
• type ReferenceUpdate
• type ReferenceUpdates
• type ReferenceVerificationError
  • func (err ReferenceVerificationError) Error() string
• type Snapshot
• type Transaction
  • func (txn *Transaction) Commit(ctx context.Context) (returnedErr error)
  • func (txn *Transaction) QuarantineDirectory() (string, error)
  • func (txn *Transaction) Rollback() error
  • func (txn *Transaction) SetCustomHooks(hooksTAR []byte)
  • func (txn *Transaction) SetDefaultBranch(new git.ReferenceName)
  • func (txn *Transaction) SkipVerificationFailures()
  • func (txn *Transaction) Snapshot() Snapshot
  • func (txn *Transaction) UpdateReferences(updates ReferenceUpdates)
• type TransactionManager
  • func NewTransactionManager(db *badger.DB, storagePath, relativePath, stagingDir string, ...) *TransactionManager
  • func (mgr *TransactionManager) Begin(ctx context.Context) (*Transaction, error)
  • func (mgr *TransactionManager) Run() (returnedErr error)
  • func (mgr *TransactionManager) Stop()

Variables

var ErrPartitionManagerStopped = errors.New("partition manager stopped")

ErrPartitionManagerStopped is returned when the PartitionManager stops processing transactions.

var ErrTransactionProcessingStopped = errors.New("transaction processing stopped")

ErrTransactionProcessingStopped is returned when the TransactionManager stops processing transactions.

Functions

func OpenDatabase

func OpenDatabase(databasePath string) (*badger.DB, error)

OpenDatabase opens a new database handle to a database at the given path.

Types

type CustomHooksUpdate

type CustomHooksUpdate struct {
	// CustomHooksTAR contains the custom hooks as a TAR. The TAR contains a `custom_hooks`
	// directory which contains the hooks. Setting the update with nil `custom_hooks_tar` clears
	// the hooks from the repository.
	CustomHooksTAR []byte
}

CustomHooksUpdate models an update to the custom hooks.

type DefaultBranchUpdate

type DefaultBranchUpdate struct {
	// Reference is the reference to update the default branch to.
	Reference git.ReferenceName
}

DefaultBranchUpdate provides the information to update the default branch of the repo.

type InvalidReferenceFormatError

type InvalidReferenceFormatError struct {
	// ReferenceName is the reference with invalid format.
	ReferenceName git.ReferenceName
}

InvalidReferenceFormatError is returned when a reference name was invalid.

func (InvalidReferenceFormatError) Error

func (err InvalidReferenceFormatError) Error() string

Error returns the formatted error string.

type LogIndex

type LogIndex uint64

LogIndex points to a specific position in a repository's write-ahead log.

func (LogIndex) String

func (index LogIndex) String() string

String returns a string representation of the LogIndex.

type PartitionManager

type PartitionManager struct {
	// contains filtered or unexported fields
}

PartitionManager is responsible for managing the lifecycle of each TransactionManager.

func NewPartitionManager

func NewPartitionManager(db *badger.DB, storages []config.Storage, localRepoFactory localrepo.Factory, logger logrus.FieldLogger, stagingDir string) *PartitionManager

NewPartitionManager returns a new PartitionManager.

func (*PartitionManager) Begin

func (pm *PartitionManager) Begin(ctx context.Context, repo repo.GitRepo) (*Transaction, error)

Begin gets the TransactionManager for the specified repository and starts a Transaction. If a TransactionManager is not already running, a new one is created and used. The partition tracks the number of pending transactions and this counter gets incremented when Begin is invoked.

func (*PartitionManager) Stop

func (pm *PartitionManager) Stop()

Stop stops transaction processing for all running transaction managers and waits for shutdown completion.

type ReferenceToBeDeletedError

type ReferenceToBeDeletedError struct {
	// ReferenceName is the name of the reference that is scheduled to be deleted.
	ReferenceName git.ReferenceName
}

ReferenceToBeDeletedError is returned when the reference used is scheduled to be deleted.

func (ReferenceToBeDeletedError) Error

func (err ReferenceToBeDeletedError) Error() string

Error returns the formatted error string.

type ReferenceUpdate

type ReferenceUpdate struct {
	// OldOID is the old OID the reference is expected to point to prior to updating it.
	// If the reference does not point to the old value, the reference verification fails.
	OldOID git.ObjectID
	// NewOID is the new desired OID to point the reference to.
	NewOID git.ObjectID
}

ReferenceUpdate describes the state of a reference's old and new tip in an update.

type ReferenceUpdates

type ReferenceUpdates map[git.ReferenceName]ReferenceUpdate

ReferenceUpdates contains references to update. Reference name is used as the key and the value is the expected old tip and the desired new tip.

type ReferenceVerificationError

type ReferenceVerificationError struct {
	// ReferenceName is the name of the reference that failed verification.
	ReferenceName git.ReferenceName
	// ExpectedOID is the OID the reference was expected to point to.
	ExpectedOID git.ObjectID
	// ActualOID is the OID the reference actually pointed to.
	ActualOID git.ObjectID
}

ReferenceVerificationError is returned when a reference's old OID did not match the expected.

func (ReferenceVerificationError) Error

func (err ReferenceVerificationError) Error() string

Error returns the formatted error string.

type Snapshot

type Snapshot struct {
	// ReadIndex is the index of the log entry this Transaction is reading the data at.
	ReadIndex LogIndex
	// HookIndex is index of the hooks on the disk that are included in this Transactions's snapshot
	// and were the latest on the read index.
	HookIndex LogIndex
}

Snapshot contains the read snapshot details of a Transaction.

type Transaction

type Transaction struct {
	// contains filtered or unexported fields
}

Transaction is a unit-of-work that contains reference changes to perform on the repository.

func (*Transaction) Commit

func (txn *Transaction) Commit(ctx context.Context) (returnedErr error)

Commit performs the changes. If no error is returned, the transaction was successful and the changes have been performed. If an error was returned, the transaction may or may not be persisted.

func (*Transaction) QuarantineDirectory

func (txn *Transaction) QuarantineDirectory() (string, error)

QuarantineDirectory returns an absolute path to the transaction's quarantine directory. The quarantine directory is a Git object directory where the new objects introduced in the transaction must be written. The quarantined objects needed by the updated reference tips will be included in the transaction.

func (*Transaction) Rollback

func (txn *Transaction) Rollback() error

Rollback releases resources associated with the transaction without performing any changes.

func (*Transaction) SetCustomHooks

func (txn *Transaction) SetCustomHooks(hooksTAR []byte)

SetCustomHooks sets the custom hooks as part of the transaction. If SetCustomHooks is called multiple times, only the changes from the latest invocation take place. The hooks are extracted as is and are not validated. Setting a nil hooksTAR removes the hooks from the repository.

func (*Transaction) SetDefaultBranch

func (txn *Transaction) SetDefaultBranch(new git.ReferenceName)

SetDefaultBranch sets the default branch as part of the transaction. If SetDefaultBranch is called multiple times, only the changes from the latest invocation take place. The reference is validated to exist.

func (*Transaction) SkipVerificationFailures

func (txn *Transaction) SkipVerificationFailures()

SkipVerificationFailures configures the transaction to skip reference updates that fail verification. If a reference update fails verification with this set, the update is dropped from the transaction but other successful reference updates will be made. By default, the entire transaction is aborted if a reference fails verification.

The default behavior models `git push --atomic`. Toggling this option models the behavior without the `--atomic` flag.

func (*Transaction) Snapshot

func (txn *Transaction) Snapshot() Snapshot

Snapshot returns the details of the Transaction's read snapshot.

func (*Transaction) UpdateReferences

func (txn *Transaction) UpdateReferences(updates ReferenceUpdates)

UpdateReferences updates the given references as part of the transaction. If UpdateReferences is called multiple times, only the changes from the latest invocation take place.

type TransactionManager

type TransactionManager struct {
	// contains filtered or unexported fields
}

TransactionManager is responsible for transaction management of a single repository. Each repository has a single TransactionManager; it is the repository's single-writer. It accepts writes one at a time from the admissionQueue. Each admitted write is processed in three steps:

1. The references being updated are verified by ensuring the expected old tips match what the references actually point to prior to update. The entire transaction is by default aborted if a single reference fails the verification step. The reference verification behavior can be controlled on a per-transaction level by setting: - The reference verification failures can be ignored instead of aborting the entire transaction. If done, the references that failed verification are dropped from the transaction but the updates that passed verification are still performed. - The reference verification may also be skipped if the write is force updating references. If done, the current state of the references is ignored and they are directly updated to point to the new tips.
2. The transaction is appended to the write-ahead log. Once the write has been logged, it is effectively committed and will be applied to the repository even after restarting.
3. The transaction is applied from the write-ahead log to the repository by actually performing the reference changes.

The goroutine that issued the transaction is waiting for the result while these steps are being performed. As there is no transaction control for readers yet, the issuer is only notified of a successful write after the write has been applied to the repository.

TransactionManager recovers transactions after interruptions by applying the write-ahead logged transactions to the repository on start up.

TransactionManager maintains the write-ahead log in a key-value store. It maintains the following key spaces: - `repository/<repository_id:string>/log/index/applied`

• This key stores the index of the log entry that has been applied to the repository. This allows for determining how far a repository is in processing the log and which log entries need to be applied after starting up. Repository starts from log index 0 if there are no log entries recorded to have been applied.

- `repository/<repository_id:string>/log/entry/<log_index:uint64>`

• These keys hold the actual write-ahead log entries. A repository's first log entry starts at index 1 and the log index keeps monotonically increasing from there on without gaps. The write-ahead log entries are processed in ascending order.

The values in the database are marshaled protocol buffer messages. Numbers in the keys are encoded as big endian to preserve the sort order of the numbers also in lexicographical order.

func NewTransactionManager

func NewTransactionManager(db *badger.DB, storagePath, relativePath, stagingDir string, repositoryFactory localrepo.StorageScopedFactory, transactionFinalizer func()) *TransactionManager

NewTransactionManager returns a new TransactionManager for the given repository.

func (*TransactionManager) Begin

func (mgr *TransactionManager) Begin(ctx context.Context) (*Transaction, error)

Begin opens a new transaction. The caller must call either Commit or Rollback to release the resources tied to the transaction. The returned Transaction is not safe for concurrent use.

The returned Transaction's read snapshot includes all writes that were committed prior to the Begin call. Begin blocks until the committed writes have been applied to the repository.

func (*TransactionManager) Run

func (mgr *TransactionManager) Run() (returnedErr error)

Run starts the transaction processing. On start up Run loads the indexes of the last appended and applied log entries from the database. It will then apply any transactions that have been logged but not applied to the repository. Once the recovery is completed, Run starts processing new transactions by verifying the references, logging the transaction and finally applying it to the repository. The transactions are acknowledged once they've been applied to the repository.

Run keeps running until Stop is called or it encounters a fatal error. All transactions will error with ErrTransactionProcessingStopped when Run returns.

func (*TransactionManager) Stop

func (mgr *TransactionManager) Stop()

Stop stops the transaction processing causing Run to return.