utils

Documentation

Overview

Package utils contains common helper functions used across multiple other packages. Utils should not import any Spanner migration tool packages.

Package utils contains common helper functions used across multiple other packages. Utils should not import any Spanner migration tool packages.

Index

• func CheckEqualSets(a, b []string) bool
• func Close(f *os.File)
• func CompareSchema(sessionFileConv, actualSpannerConv *internal.Conv) error
• func ConcatDirectoryPath(basePath, subPath string) string
• func DownloadFromGCS(bucketName, filePath, tmpFile string) (*os.File, error)
• func FindInPrimaryKey(id string, primaryKeys []ddl.IndexKey) bool
• func GenerateHashStr() string
• func GenerateName(prefix string) (string, error)
• func GetBanner(now time.Time, db string) string
• func GetClient(ctx context.Context, db string) (*sp.Client, error)
• func GetDataflowTemplatePath() string
• func GetFileSize(f *os.File) (int64, error)
• func GetLegacyModeSupportedDrivers() []string
• func GetValidDrivers() []string
• func IsLegacyModeSupportedDriver(driver string) bool
• func IsValidDriver(driver string) bool
• func NewDatabaseAdminClient(ctx context.Context) (*database.DatabaseAdminClient, error)
• func NewInstanceAdminClient(ctx context.Context) (*instance.InstanceAdminClient, error)
• func NewSpannerClient(ctx context.Context, db string) (*sp.Client, error)
• func ParseGCSFilePath(filePath string) (*url.URL, error)
• func PrintPermissionsWarning(driver string, out *os.File)
• func PrintSeekError(driver string, err error, out *os.File)
• func ReadSpannerSchema(ctx context.Context, conv *internal.Conv, client *sp.Client) error
• func SetDataflowTemplatePath(path string)
• func SetupLogFile() (*os.File, error)
• func SumMapValues(m map[string]int64) int64
• func TargetDbToDialect(targetDb string) string
• type GetUtilInfoImpl
  • func (gui *GetUtilInfoImpl) GetDatabaseName(driver string, now time.Time) (string, error)
  • func (gui *GetUtilInfoImpl) GetInstance(ctx context.Context, project string, out *os.File) (string, error)
  • func (gui *GetUtilInfoImpl) GetPassword() string
  • func (gui *GetUtilInfoImpl) GetProject() (string, error)
• type GetUtilInfoInterface
• type IOStreams
  • func NewIOStreams(driver string, dumpFile string) IOStreams
• type ManifestTable
  • func PreloadGCSFiles(tables []ManifestTable) ([]ManifestTable, error)

Functions

func CheckEqualSets

func CheckEqualSets(a, b []string) bool

CheckEqualSets checks if the set of values in a and b are equal.

func Close

func Close(f *os.File)

Close closes file.

func CompareSchema

func CompareSchema(sessionFileConv, actualSpannerConv *internal.Conv) error

CompareSchema compares the spanner schema of two conv objects and returns specific error if they don't match

func ConcatDirectoryPath

func ConcatDirectoryPath(basePath, subPath string) string

func DownloadFromGCS

func DownloadFromGCS(bucketName, filePath, tmpFile string) (*os.File, error)

DownloadFromGCS returns the dump file that is downloaded from GCS.

func FindInPrimaryKey

func FindInPrimaryKey(id string, primaryKeys []ddl.IndexKey) bool

func GenerateHashStr

func GenerateHashStr() string

func GenerateName

func GenerateName(prefix string) (string, error)

func GetBanner

func GetBanner(now time.Time, db string) string

GetBanner prints banner message after command line process is finished.

func GetClient

func GetClient(ctx context.Context, db string) (*sp.Client, error)

GetClient returns a new Spanner client. It uses the background context.

func GetDataflowTemplatePath

func GetDataflowTemplatePath() string

func GetFileSize

func GetFileSize(f *os.File) (int64, error)

func GetLegacyModeSupportedDrivers

func GetLegacyModeSupportedDrivers() []string

func GetValidDrivers

func GetValidDrivers() []string

func IsLegacyModeSupportedDriver

func IsLegacyModeSupportedDriver(driver string) bool

func IsValidDriver

func IsValidDriver(driver string) bool

func NewDatabaseAdminClient

func NewDatabaseAdminClient(ctx context.Context) (*database.DatabaseAdminClient, error)

NewDatabaseAdminClient returns a new db-admin client. It respects SPANNER_API_ENDPOINT.

func NewInstanceAdminClient

func NewInstanceAdminClient(ctx context.Context) (*instance.InstanceAdminClient, error)

NewInstanceAdminClient returns a new instance-admin client. It respects SPANNER_API_ENDPOINT.

func NewSpannerClient

func NewSpannerClient(ctx context.Context, db string) (*sp.Client, error)

NewSpannerClient returns a new Spanner client. It respects SPANNER_API_ENDPOINT.

func ParseGCSFilePath

func ParseGCSFilePath(filePath string) (*url.URL, error)

func PrintPermissionsWarning

func PrintPermissionsWarning(driver string, out *os.File)

PrintPermissionsWarning prints permission warning.

func PrintSeekError

func PrintSeekError(driver string, err error, out *os.File)

func ReadSpannerSchema

func ReadSpannerSchema(ctx context.Context, conv *internal.Conv, client *sp.Client) error

ReadSpannerSchema fills conv by querying Spanner infoschema treating Spanner as both the source and dest.

func SetDataflowTemplatePath

func SetDataflowTemplatePath(path string)

func SetupLogFile

func SetupLogFile() (*os.File, error)

SetupLogFile configures the file used for logs. By default we just drop logs on the floor. To enable them (e.g. to debug Cloud Spanner client library issues), set logfile to a non-empty filename. Note: this tool itself doesn't generate logs, but some of the libraries it uses do. If we don't set the log file, we see a number of unhelpful and unactionable logs spamming stdout, which is annoying and confusing.

func SumMapValues

func SumMapValues(m map[string]int64) int64

func TargetDbToDialect

func TargetDbToDialect(targetDb string) string

Types

type GetUtilInfoImpl

type GetUtilInfoImpl struct{}

func (*GetUtilInfoImpl) GetDatabaseName

func (gui *GetUtilInfoImpl) GetDatabaseName(driver string, now time.Time) (string, error)

GetDatabaseName generates database name with driver_date prefix.

func (*GetUtilInfoImpl) GetInstance

func (gui *GetUtilInfoImpl) GetInstance(ctx context.Context, project string, out *os.File) (string, error)

GetInstance returns the Spanner instance we should use for creating DBs. If the user specified instance (via flag 'instance') then use that. Otherwise try to deduce the instance using gcloud.

func (*GetUtilInfoImpl) GetPassword

func (gui *GetUtilInfoImpl) GetPassword() string

func (*GetUtilInfoImpl) GetProject

func (gui *GetUtilInfoImpl) GetProject() (string, error)

GetProject returns the cloud project we should use by default to create resources. Use environment variable GCLOUD_PROJECT if it is set. Otherwise, use the default project returned from gcloud.

type GetUtilInfoInterface

type GetUtilInfoInterface interface {
	GetProject() (string, error)
	GetInstance(ctx context.Context, project string, out *os.File) (string, error)
	GetPassword() string
	GetDatabaseName(driver string, now time.Time) (string, error)
}

Interface to fetch spanner details

type IOStreams

type IOStreams struct {
	In, SeekableIn, Out *os.File
	BytesRead           int64
}

IOStreams is a struct that contains the file descriptor for dumpFile.

func NewIOStreams

func NewIOStreams(driver string, dumpFile string) IOStreams

NewIOStreams returns a new IOStreams struct such that input stream is set to open file descriptor for dumpFile if driver is PGDUMP or MYSQLDUMP. Input stream defaults to stdin. Output stream is always set to stdout.

type ManifestTable

type ManifestTable struct {
	Table_name    string   `json:"table_name"`
	File_patterns []string `json:"file_patterns"`
}

Spanner migration tool accepts a manifest file in the form of a json which unmarshalls into the ManifestTables struct.

func PreloadGCSFiles

func PreloadGCSFiles(tables []ManifestTable) ([]ManifestTable, error)

PreloadGCSFiles downloads gcs files to tmp and updates the file paths in manifest with the local path.