intents

Documentation

Overview

Package intents provides utilities for performing dump/restore operations.

Index

• type BySize
  • func (s BySize) Len() int
  • func (s BySize) Less(i, j int) bool
  • func (s BySize) Swap(i, j int)
• type BySizeAndView
  • func (s BySizeAndView) Len() int
  • func (s BySizeAndView) Less(i, j int) bool
  • func (s BySizeAndView) Swap(i, j int)
• type DBHeap
  • func (dbh DBHeap) Len() int
  • func (dbh DBHeap) Less(i, j int) bool
  • func (dbh *DBHeap) Pop() interface{}
  • func (dbh *DBHeap) Push(x interface{})
  • func (dbh DBHeap) Swap(i, j int)
• type DestinationConflictError
  • func (e DestinationConflictError) Error() string
• type FileNeedsIOBuffer
• type Intent
  • func (it *Intent) DataCollection() string
  • func (it *Intent) DataNamespace() string
  • func (it *Intent) HasSimpleCollation() bool
  • func (it *Intent) IsAuthVersion() bool
  • func (it *Intent) IsOplog() bool
  • func (it *Intent) IsRoles() bool
  • func (it *Intent) IsSpecialCollection() bool
  • func (it *Intent) IsSystemIndexes() bool
  • func (it *Intent) IsSystemProfile() bool
  • func (it *Intent) IsTimeseries() bool
  • func (it *Intent) IsUsers() bool
  • func (it *Intent) IsView() bool
  • func (it *Intent) MergeIntent(newIt *Intent)
  • func (it *Intent) Namespace() string
• type IntentPrioritizer
• type Manager
  • func NewIntentManager() *Manager
  • func (mgr *Manager) AuthVersion() *Intent
  • func (mgr *Manager) Finalize(pType PriorityType)
  • func (mgr *Manager) Finish(intent *Intent)
  • func (mgr *Manager) GetDestinationConflicts() (errs []DestinationConflictError)
  • func (mgr *Manager) GetOplogConflict() bool
  • func (mgr *Manager) HasConfigDBIntent() bool
  • func (mgr *Manager) IntentForNamespace(ns string) *Intent
  • func (mgr *Manager) Intents() []*Intent
  • func (mgr *Manager) NormalIntents() []*Intent
  • func (mgr *Manager) Oplog() *Intent
  • func (mgr *Manager) Peek() *Intent
  • func (mgr *Manager) Pop() *Intent
  • func (mgr *Manager) Put(intent *Intent)
  • func (mgr *Manager) PutOplogIntent(intent *Intent, managerKey string)
  • func (mgr *Manager) PutWithNamespace(ns string, intent *Intent)
  • func (mgr *Manager) Roles() *Intent
  • func (mgr *Manager) SetSmartPickOplog(smartPick bool)
  • func (mgr *Manager) SystemIndexDBs() []string
  • func (mgr *Manager) SystemIndexes(dbName string) *Intent
  • func (mgr *Manager) UsePrioritizer(prioritizer IntentPrioritizer)
  • func (mgr *Manager) Users() *Intent
• type PriorityType

Types

type BySize

type BySize []*Intent

For sorting intents from largest to smallest size

func (BySize) Len

func (s BySize) Len() int

func (BySize) Less

func (s BySize) Less(i, j int) bool

func (BySize) Swap

func (s BySize) Swap(i, j int)

type BySizeAndView

type BySizeAndView []*Intent

BySizeAndView attaches the methods for sort.Interface for sorting intents from largest to smallest size, taking into account if it's a view or not.

func (BySizeAndView) Len

func (s BySizeAndView) Len() int

func (BySizeAndView) Less

func (s BySizeAndView) Less(i, j int) bool

func (BySizeAndView) Swap

func (s BySizeAndView) Swap(i, j int)

type DBHeap

type DBHeap []*dbCounter

Returns the largest collection of the databases with the least active restores. Implements the container/heap interface. None of its methods are meant to be called directly.

func (DBHeap) Len

func (dbh DBHeap) Len() int

func (DBHeap) Less

func (dbh DBHeap) Less(i, j int) bool

func (*DBHeap) Pop

func (dbh *DBHeap) Pop() interface{}

func (*DBHeap) Push

func (dbh *DBHeap) Push(x interface{})

func (DBHeap) Swap

func (dbh DBHeap) Swap(i, j int)

type DestinationConflictError

type DestinationConflictError struct {
	Src, Dst string
}

DestinationConflictError occurs when multiple namespaces map to the same destination.

func (DestinationConflictError) Error

func (e DestinationConflictError) Error() string

type FileNeedsIOBuffer

type FileNeedsIOBuffer interface {
	TakeIOBuffer([]byte)
	ReleaseIOBuffer()
}

FileNeedsIOBuffer is an interface that denotes that a struct needs an IO buffer that is managed by an outside control. This interface is used to both hand off a buffer to a struct and signal that it should release its buffer. Added to reduce memory usage as outlined in TOOLS-1088.

type Intent

type Intent struct {
	// Destination namespace info
	DB string
	C  string

	// File locations as absolute paths
	BSONFile     file
	BSONSize     int64
	MetadataFile file

	// Indicates where the intent will be read from or written to
	Location         string
	MetadataLocation string

	// Collection options
	Options bson.M

	// UUID (for MongoDB 3.6+) as a big-endian hex string
	UUID string

	// File/collection size, for some prioritizer implementations.
	// Units don't matter as long as they are consistent for a given use case.
	Size int64

	// Either view or timeseries. Empty string "" is a regular collection.
	Type string
}

mongorestore first scans the directory to generate a list of all files to restore and what they map to. TODO comments

func (*Intent) DataCollection

func (it *Intent) DataCollection() string

func (*Intent) DataNamespace

func (it *Intent) DataNamespace() string

func (*Intent) HasSimpleCollation

func (it *Intent) HasSimpleCollation() bool

HasSimpleCollation returns true if the collection does not have a collation specified or if the collation locale is "simple"

func (*Intent) IsAuthVersion

func (it *Intent) IsAuthVersion() bool

func (*Intent) IsOplog

func (it *Intent) IsOplog() bool

func (*Intent) IsRoles

func (it *Intent) IsRoles() bool

func (*Intent) IsSpecialCollection

func (it *Intent) IsSpecialCollection() bool

func (*Intent) IsSystemIndexes

func (it *Intent) IsSystemIndexes() bool

func (*Intent) IsSystemProfile

func (it *Intent) IsSystemProfile() bool

func (*Intent) IsTimeseries

func (it *Intent) IsTimeseries() bool

func (*Intent) IsUsers

func (it *Intent) IsUsers() bool

func (*Intent) IsView

func (it *Intent) IsView() bool

func (*Intent) MergeIntent

func (it *Intent) MergeIntent(newIt *Intent)

func (*Intent) Namespace

func (it *Intent) Namespace() string

type IntentPrioritizer

type IntentPrioritizer interface {
	Get() *Intent
	Finish(*Intent)
}

IntentPrioritizer encapsulates the logic of scheduling intents for restoration. It can know about which intents are in the process of being restored through the "Finish" hook.

Oplog entries and auth entries are not handled by the prioritizer, as these are special cases handled by the regular mongorestore code.

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

func NewIntentManager

func NewIntentManager() *Manager

func (*Manager) AuthVersion

func (mgr *Manager) AuthVersion() *Intent

AuthVersion returns the intent of the version collection to restore, a special case

func (*Manager) Finalize

func (mgr *Manager) Finalize(pType PriorityType)

Finalize processes the intents for prioritization. Currently only two kinds of prioritizers are supported. No more "Put" operations may be done after finalize is called.

func (*Manager) Finish

func (mgr *Manager) Finish(intent *Intent)

Finish tells the prioritizer that mongorestore is done restoring the given collection intent.

func (*Manager) GetDestinationConflicts

func (mgr *Manager) GetDestinationConflicts() (errs []DestinationConflictError)

func (*Manager) GetOplogConflict

func (mgr *Manager) GetOplogConflict() bool

func (*Manager) HasConfigDBIntent

func (mgr *Manager) HasConfigDBIntent() bool

HasConfigDBIntent returns a bool indicating if any of the intents refer to the "config" database. This can be used to check for possible unwanted conflicts before restoring to a sharded system.

func (*Manager) IntentForNamespace

func (mgr *Manager) IntentForNamespace(ns string) *Intent

func (*Manager) Intents

func (mgr *Manager) Intents() []*Intent

Intents returns a slice containing all of the intents in the manager. Intents is not thread safe

func (*Manager) NormalIntents

func (mgr *Manager) NormalIntents() []*Intent

NormalIntents returns a slice containing all of the normal intents in the manager. NormalIntents is not thread safe.

func (*Manager) Oplog

func (mgr *Manager) Oplog() *Intent

Oplog returns the intent representing the oplog, which isn't stored with the other intents, because it is dumped and restored in a very different way from other collections.

func (*Manager) Peek

func (mgr *Manager) Peek() *Intent

Peek returns a copy of a stored intent from the manager without removing the intent. This method is useful for edge cases that need to look ahead at what collections are in the manager before they are scheduled.

NOTE: There are no guarantees that peek will return a usable intent after Finalize() is called.

func (*Manager) Pop

func (mgr *Manager) Pop() *Intent

Pop returns the next available intent from the manager. If the manager is empty, it returns nil. Pop is thread safe.

func (*Manager) Put

func (mgr *Manager) Put(intent *Intent)

Put inserts an intent into the manager with the same source namespace as its destinations.

func (*Manager) PutOplogIntent

func (mgr *Manager) PutOplogIntent(intent *Intent, managerKey string)

PutOplogIntent takes an intent for an oplog and stores it in the intent manager with the provided key. If the manager has smartPickOplog enabled, then it uses a priority system to determine which oplog intent to maintain as the actual oplog.

func (*Manager) PutWithNamespace

func (mgr *Manager) PutWithNamespace(ns string, intent *Intent)

PutWithNamespace inserts an intent into the manager with the source set to the provided namespace. Intents for the same collection are merged together, so that BSON and metadata files for the same collection are returned in the same intent.

func (*Manager) Roles

func (mgr *Manager) Roles() *Intent

Roles returns the intent of the user roles collection to restore, a special case

func (*Manager) SetSmartPickOplog

func (mgr *Manager) SetSmartPickOplog(smartPick bool)

func (*Manager) SystemIndexDBs

func (mgr *Manager) SystemIndexDBs() []string

SystemIndexes returns the databases for which there are system.indexes

func (*Manager) SystemIndexes

func (mgr *Manager) SystemIndexes(dbName string) *Intent

SystemIndexes returns the system.indexes bson for a database

func (*Manager) UsePrioritizer

func (mgr *Manager) UsePrioritizer(prioritizer IntentPrioritizer)

func (*Manager) Users

func (mgr *Manager) Users() *Intent

Users returns the intent of the users collection to restore, a special case

type PriorityType

type PriorityType int

const (
	Legacy PriorityType = iota
	LongestTaskFirst
	MultiDatabaseLTF
)