state

package
v2.51.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 15, 2024 License: Apache-2.0 Imports: 22 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ConvertRFC3339ToTime 

func ConvertRFC3339ToTime(timeToConvert string) (time.Time, error)

func ConvertTimeToEpochMilliseconds 

func ConvertTimeToEpochMilliseconds(timeToConvert time.Time) string

func ConvertTimeToRFC3339 

func ConvertTimeToRFC3339(timeToConvert time.Time) string

func GetJfrogTransferRepoSnapshotDir added in v2.27.0 

func GetJfrogTransferRepoSnapshotDir(repoKey string) (string, error)

func GetJfrogTransferRepoSubDir added in v2.27.0 

func GetJfrogTransferRepoSubDir(repoKey, subDirName string) (string, error)

func GetOldTransferDirectoryStructureError added in v2.31.0 

func GetOldTransferDirectoryStructureError() error

func GetRepoSnapshotFilePath added in v2.27.0 

func GetRepoSnapshotFilePath(repoKey string) (string, error)

func GetRepoStateFilepath added in v2.27.0 

func GetRepoStateFilepath(repoKey string, snapshot bool) (string, error)

func GetRepositoryTransferDir added in v2.25.0 

func GetRepositoryTransferDir(repoKey string) (string, error)

func SecondsToLiteralTime added in v2.36.1 

func SecondsToLiteralTime(secondsToConvert int64, prefix string) string

SecondsToLiteralTime converts a number of seconds to an easy-to-read string. Prefix is not taken into account if the time is less than a minute.

func SetAutoSaveState added in v2.27.0 

func SetAutoSaveState() (cleanUp func())

Set the state's save-interval to 0 so every action will be persisted and data can be asserted.

func UpdateChunkInState added in v2.24.0 

func UpdateChunkInState(stateManager *TransferStateManager, chunk *api.ChunkStatus) (err error)

func VerifyTransferRunStatusVersion added in v2.31.0 

func VerifyTransferRunStatusVersion() error

Types

type ActionOnStateFunc 

type ActionOnStateFunc func(state *TransferState) error

type ActionOnStatusFunc 

type ActionOnStatusFunc func(transferRunStatus *TransferRunStatus) error

type AlreadyLockedError 

type AlreadyLockedError struct{}

func (*AlreadyLockedError) Error 

func (m *AlreadyLockedError) Error() string

type DiffDetails 

type DiffDetails struct {
	// The start and end time of a complete transferring Files Diff phase
	FilesDiffRunTime PhaseDetails `json:"files_diff,omitempty"`
	// The start and end time of the last handled range
	HandledRange PhaseDetails `json:"handled_range,omitempty"`
	// If false, start the Diff phase from the start time of the full transfer
	Completed bool `json:"completed,omitempty"`
}

type PhaseDetails 

type PhaseDetails struct {
	Started string `json:"started,omitempty"`
	Ended   string `json:"ended,omitempty"`
}

type ProgressState 

type ProgressState struct {
	TotalSizeBytes       int64 `json:"total_size_bytes,omitempty"`
	TransferredSizeBytes int64 `json:"transferred_size_bytes,omitempty"`
	ProgressStateUnits
}

type ProgressStateUnits added in v2.24.0 

type ProgressStateUnits struct {
	TotalUnits       int64 `json:"total_units,omitempty"`
	TransferredUnits int64 `json:"transferred_units,omitempty"`
}

type RepoTransferSnapshot added in v2.27.0 

type RepoTransferSnapshot struct {
	// contains filtered or unexported fields
}

RepoTransferSnapshot handles saving and loading the repository's transfer snapshot. A repository transfer snapshot stores the progress of transferring a repository, as explained in RepoSnapshotManager. In case a transfer is interrupted, the transfer can continue from it's saved snapshot instead of starting from scratch.

type Repository 

type Repository struct {
	Phase1Info   ProgressState `json:"phase1_info,omitempty"`
	Phase2Info   ProgressState `json:"phase2_info,omitempty"`
	Phase3Info   ProgressState `json:"phase3_info,omitempty"`
	Name         string        `json:"name,omitempty"`
	FullTransfer PhaseDetails  `json:"full_transfer,omitempty"`
	Diffs        []DiffDetails `json:"diffs,omitempty"`
}

type SnapshotActionFunc added in v2.27.0 

type SnapshotActionFunc func(rts *RepoTransferSnapshot) error

type StaleChunk added in v2.36.1 

type StaleChunk struct {
	ChunkID string   `json:"chunk_id,omitempty"`
	Files   []string `json:"files,omitempty"`
	Sent    int64    `json:"sent,omitempty"`
}

type StaleChunks added in v2.36.1 

type StaleChunks struct {
	NodeID string       `json:"node_id,omitempty"`
	Chunks []StaleChunk `json:"stale_node_chunks,omitempty"`
}

This structure contains a collection of chunks that have been undergoing processing for over 30 minutes

type TimeEstimationManager added in v2.24.0 

type TimeEstimationManager struct {
	// Speeds of the last done chunks, in bytes/ms
	LastSpeeds []float64 `json:"last_speeds,omitempty"`
	// Sum of the speeds in LastSpeeds. The speeds are in bytes/ms.
	LastSpeedsSum float64 `json:"last_speeds_sum,omitempty"`
	// The last calculated sum of speeds, in bytes/ms
	SpeedsAverage float64 `json:"speeds_average,omitempty"`
	// Total transferred bytes since the beginning of the current transfer execution
	CurrentTotalTransferredBytes uint64 `json:"current_total_transferred_bytes,omitempty"`
	// contains filtered or unexported fields
}

func (*TimeEstimationManager) AddChunkStatus added in v2.24.0 

func (tem *TimeEstimationManager) AddChunkStatus(chunkStatus api.ChunkStatus, durationMillis int64)

func (*TimeEstimationManager) GetEstimatedRemainingTimeString added in v2.24.0 

func (tem *TimeEstimationManager) GetEstimatedRemainingTimeString() string

GetEstimatedRemainingTimeString gets the estimated remaining time as an easy-to-read string. Return "Not available yet" in the following cases: 1. 5 minutes not passed since the beginning of the transfer 2. No files transferred 3. The transfer speed is less than 1 byte per second

func (*TimeEstimationManager) GetSpeedString added in v2.24.0 

func (tem *TimeEstimationManager) GetSpeedString() string

GetSpeedString gets the transfer speed as an easy-to-read string.

type TransferRunStatus 

type TransferRunStatus struct {

	// This variable holds the total/transferred number of repositories (not their files).
	OverallTransfer   ProgressState      `json:"overall_transfer,omitempty"`
	TotalRepositories ProgressStateUnits `json:"total_repositories,omitempty"`
	OverallBiFiles    ProgressStateUnits `json:"overall_bi_files,omitempty"`
	// Version of the TransferRunStatus file.
	Version        int    `json:"version,omitempty"`
	CurrentRepoKey string `json:"current_repo,omitempty"`
	// True if currently transferring a build info repository.
	BuildInfoRepo         bool   `json:"build_info_repo,omitempty"`
	CurrentRepoPhase      int    `json:"current_repo_phase,omitempty"`
	WorkingThreads        int    `json:"working_threads,omitempty"`
	VisitedFolders        uint64 `json:"visited_folders,omitempty"`
	DelayedFiles          uint64 `json:"delayed_files,omitempty"`
	TransferFailures      uint64 `json:"transfer_failures,omitempty"`
	TimeEstimationManager `json:"time_estimation,omitempty"`
	StaleChunks           []StaleChunks `json:"stale_chunks,omitempty"`
	// contains filtered or unexported fields
}

This struct holds the run status of the current transfer. It is saved to a file in JFrog CLI's home, but gets reset every time the transfer begins. This state is used to allow showing the current run status by the 'jf rt transfer-files --status' command. It is also used for the time estimation and more.

type TransferState 

type TransferState struct {

	// The Version of the state.json file of a repository.
	Version     int        `json:"state_version,omitempty"`
	CurrentRepo Repository `json:"repository,omitempty"`
	// contains filtered or unexported fields
}

This struct holds the state of the current repository being transferred from the source Artifactory instance. It is saved to a file located in a directory named with the sha of the repository, under the transfer directory. The transfer-files command uses this state to determine which phases should be executed for the repository, as well as other decisions related to the process execution.

func LoadTransferState added in v2.25.0 

func LoadTransferState(repoKey string, snapshot bool) (transferState TransferState, exists bool, err error)

func (*TransferState) Action added in v2.29.8 

func (ts *TransferState) Action(action ActionOnStateFunc) error

func (*TransferState) IsRepoTransferred added in v2.27.0 

func (ts *TransferState) IsRepoTransferred() (isTransferred bool, err error)

type TransferStateManager 

type TransferStateManager struct {
	TransferState
	TransferRunStatus
	// contains filtered or unexported fields
}

func InitStateTest added in v2.24.0 

func InitStateTest(t *testing.T) (stateManager *TransferStateManager, cleanUp func())

func NewTransferStateManager 

func NewTransferStateManager(loadRunStatus bool) (*TransferStateManager, error)

func (*TransferStateManager) AddNewDiffToState 

func (ts *TransferStateManager) AddNewDiffToState(startTime time.Time) error

Adds new diff details to the repo's diff array in state. Marks files handling as started, and sets the handling range.

func (*TransferStateManager) ChangeDelayedFilesCountBy added in v2.45.2 

func (ts *TransferStateManager) ChangeDelayedFilesCountBy(count uint64, increase bool) error

func (*TransferStateManager) ChangeTransferFailureCountBy added in v2.23.2 

func (ts *TransferStateManager) ChangeTransferFailureCountBy(count uint64, increase bool) error

func (*TransferStateManager) DisableRepoTransferSnapshot added in v2.27.0 

func (ts *TransferStateManager) DisableRepoTransferSnapshot()

func (*TransferStateManager) GetDiffHandlingRange 

func (ts *TransferStateManager) GetDiffHandlingRange() (start, end time.Time, err error)

func (*TransferStateManager) GetDirectorySnapshotNodeWithLru added in v2.27.0 

func (ts *TransferStateManager) GetDirectorySnapshotNodeWithLru(relativePath string) (node *reposnapshot.Node, err error)

func (*TransferStateManager) GetReposTransferredSizeBytes 

func (ts *TransferStateManager) GetReposTransferredSizeBytes(repoKeys ...string) (transferredSizeBytes int64, err error)

func (*TransferStateManager) GetRunningTimeString added in v2.47.0 

func (ts *TransferStateManager) GetRunningTimeString() (runningTime string)

func (*TransferStateManager) GetStaleChunks added in v2.36.1 

func (ts *TransferStateManager) GetStaleChunks() (staleChunks []StaleChunks, err error)

func (*TransferStateManager) GetStartTimestamp added in v2.47.0 

func (ts *TransferStateManager) GetStartTimestamp() time.Time

func (*TransferStateManager) GetStorageAndFilesRepoPointers added in v2.25.0 

func (ts *TransferStateManager) GetStorageAndFilesRepoPointers(phase int) (totalFailedStorage, totalUploadedFailedStorage, totalFailedFiles, totalUploadedFailedFiles *int64, err error)

Returns pointers to TotalStorage, TotalFiles, TransferredFiles and TransferredStorage from progressState of a specific Repository.

func (*TransferStateManager) GetTransferredSizeBytes 

func (ts *TransferStateManager) GetTransferredSizeBytes() (transferredSizeBytes int64, err error)

func (*TransferStateManager) GetWorkingThreads 

func (ts *TransferStateManager) GetWorkingThreads() (workingThreads int, err error)

func (*TransferStateManager) IncRepositoriesTransferred 

func (ts *TransferStateManager) IncRepositoriesTransferred() error

func (*TransferStateManager) IncTotalSizeAndFilesPhase2 added in v2.25.0 

func (ts *TransferStateManager) IncTotalSizeAndFilesPhase2(filesNumber, totalSize int64) error

func (*TransferStateManager) IncTransferredSizeAndFilesPhase1 added in v2.28.0 

func (ts *TransferStateManager) IncTransferredSizeAndFilesPhase1(chunkTotalFiles, chunkTotalSizeInBytes int64) error

Increasing Transferred Diff files (modified files) and SizeByBytes value in suitable repository progress state

func (*TransferStateManager) IncTransferredSizeAndFilesPhase2 added in v2.25.0 

func (ts *TransferStateManager) IncTransferredSizeAndFilesPhase2(chunkTotalFiles, chunkTotalSizeInBytes int64) error

func (*TransferStateManager) IncTransferredSizeAndFilesPhase3 added in v2.25.0 

func (ts *TransferStateManager) IncTransferredSizeAndFilesPhase3(chunkTotalFiles, chunkTotalSizeInBytes int64) error

Increase transferred storage and files in phase 3

func (*TransferStateManager) IncVisitedFolders added in v2.45.2 

func (ts *TransferStateManager) IncVisitedFolders() error

func (*TransferStateManager) InitStartTimestamp added in v2.47.0 

func (ts *TransferStateManager) InitStartTimestamp() (running bool, err error)

func (*TransferStateManager) IsRepoTransferSnapshotEnabled added in v2.27.0 

func (ts *TransferStateManager) IsRepoTransferSnapshotEnabled() bool

func (*TransferStateManager) LookUpNode added in v2.27.0 

func (ts *TransferStateManager) LookUpNode(relativePath string) (requestedNode *reposnapshot.Node, err error)

func (*TransferStateManager) Running added in v2.47.0 

func (ts *TransferStateManager) Running() (running bool, err error)

func (*TransferStateManager) SaveStateAndSnapshots added in v2.27.0 

func (ts *TransferStateManager) SaveStateAndSnapshots() error

func (*TransferStateManager) SetFilesDiffHandlingCompleted 

func (ts *TransferStateManager) SetFilesDiffHandlingCompleted() error

func (*TransferStateManager) SetRepoFullTransferCompleted 

func (ts *TransferStateManager) SetRepoFullTransferCompleted() error

func (*TransferStateManager) SetRepoFullTransferStarted 

func (ts *TransferStateManager) SetRepoFullTransferStarted(startTime time.Time) error

func (*TransferStateManager) SetRepoPhase 

func (ts *TransferStateManager) SetRepoPhase(phaseId int) error

func (*TransferStateManager) SetRepoState 

func (ts *TransferStateManager) SetRepoState(repoKey string, totalSizeBytes, totalFiles int64, buildInfoRepo, reset bool) error

Set the repository state. repoKey - Repository key totalSizeBytes - Repository size in bytes totalFiles - Total files in the repository buildInfoRepo - True if build info repository reset - Delete the repository's previous transfer info

func (*TransferStateManager) SetStaleChunks added in v2.36.1 

func (ts *TransferStateManager) SetStaleChunks(staleChunks []StaleChunks) error

func (*TransferStateManager) SetStartTimestamp added in v2.47.0 

func (ts *TransferStateManager) SetStartTimestamp(startTimestamp time.Time)

func (*TransferStateManager) SetTotalSizeAndFilesPhase3 added in v2.25.0 

func (ts *TransferStateManager) SetTotalSizeAndFilesPhase3(filesNumber, totalSize int64) error

Set relevant information of files and storage we need to transfer in phase3

func (*TransferStateManager) SetWorkingThreads 

func (ts *TransferStateManager) SetWorkingThreads(workingThreads int) error

func (*TransferStateManager) TryLockTransferStateManager 

func (ts *TransferStateManager) TryLockTransferStateManager() error

Try to lock the transfer state manager. If file-transfer is already running, return "Already locked" error.

func (*TransferStateManager) UnlockTransferStateManager 

func (ts *TransferStateManager) UnlockTransferStateManager() error

func (*TransferStateManager) WasSnapshotLoaded added in v2.27.0 

func (ts *TransferStateManager) WasSnapshotLoaded() bool

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.