datasets

Documentation

Overview

Package datasets provides functionality for managing datasets.

Index

• Variables
• func AddTable(table string) error
• func CompressClickhouse(ctx *cli.Context) error
• func CompressPostgres(ctx *cli.Context) error
• func CompressSources(ctx *cli.Context) error
• func DecompressClickhouse(ctx *cli.Context) error
• func DecompressPostgres(ctx *cli.Context) error
• func DecompressSources(ctx *cli.Context) error
• func Download(ctx *cli.Context) error
• func ExportClickhouse(ctx *cli.Context) error
• func ExportPostgres(ctx *cli.Context) error
• func GetAllowedTables() []string
• func ImportClickhouse(ctx *cli.Context) error
• func ImportPostgres(ctx *cli.Context) error
• func RemoveTable(table string) error
• func TableExists(table string) bool
• func UploadClickhouse(ctx *cli.Context) error
• func UploadPostgres(ctx *cli.Context) error
• func UploadSources(ctx *cli.Context) error
• type ExportMapping

Variables

var ErrTableAlreadyExists = errors.New("table already exists")

ErrTableAlreadyExists indicates that the table already exists in the databaseTables slice.

var ErrTableNotFound = errors.New("table not found")

ErrTableNotFound indicates that the table does not exist in the databaseTables slice.

Functions

func AddTable

func AddTable(table string) error

AddTable adds a new table to the databaseTables slice if it doesn't already exist.

func CompressClickhouse

func CompressClickhouse(ctx *cli.Context) error

CompressClickhouse compresses exported ClickHouse database files using 7zip.

func CompressPostgres

func CompressPostgres(ctx *cli.Context) error

CompressPostgres compresses exported Postgresql database files using 7zip.

func CompressSources

func CompressSources(ctx *cli.Context) error

CompressSources compresses smart contract source code files using 7zip. It takes the context from the CLI, retrieves the output path and network name from the context, and constructs the source and destination paths. It then compresses the files located in the source path to a .7z file in the destination path.

Parameters:

• ctx: The CLI context that contains the command-line arguments and flags.

Returns:

• error: An error if one occurs during the execution of the function.

func DecompressClickhouse

func DecompressClickhouse(ctx *cli.Context) error

DecompressClickhouse decompresses exported ClickHouse database files using 7zip.

func DecompressPostgres

func DecompressPostgres(ctx *cli.Context) error

DecompressPostgres decompresses exported Postgresql database files using 7zip.

func DecompressSources

func DecompressSources(ctx *cli.Context) error

DecompressSources decompresses smart contract source code files using 7zip. It takes the context from the CLI, retrieves the path and network name from the context, and constructs the source and destination paths. It then decompresses the .7z file located in the source path to the destination directory.

Parameters:

• ctx: The CLI context that contains the command-line arguments and flags.

Returns:

• error: An error if one occurs during the execution of the function.

func Download

func Download(ctx *cli.Context) error

Download downloads exported archive datasets from Cloudflare R2.

func ExportClickhouse

func ExportClickhouse(ctx *cli.Context) error

ExportClickhouse exports tables from ClickHouse database to native ClickHouse format files.

func ExportPostgres

func ExportPostgres(ctx *cli.Context) error

ExportPostgres exports Postgresql tables into appropriate files.

func GetAllowedTables

func GetAllowedTables() []string

GetAllowedTables returns the list of allowed database tables.

func ImportClickhouse

func ImportClickhouse(ctx *cli.Context) error

ImportClickhouse imports tables from native ClickHouse format files into ClickHouse database.

func ImportPostgres

func ImportPostgres(ctx *cli.Context) error

ImportPostgres imports tables from SQL files into a Postgresql database.

func RemoveTable

func RemoveTable(table string) error

RemoveTable removes a table from the databaseTables slice if it exists.

func TableExists

func TableExists(table string) bool

TableExists checks if a table exists in the databaseTables slice.

func UploadClickhouse

func UploadClickhouse(ctx *cli.Context) error

UploadClickhouse uploads exported ClickHouse database files to Cloudflare R2.

func UploadPostgres

func UploadPostgres(ctx *cli.Context) error

UploadPostgres uploads exported Postgresql database files to Cloudflare R2 or Amazon S3.

func UploadSources

func UploadSources(ctx *cli.Context) error

UploadSources uploads compressed source code files to Cloudflare R2 or Amazon S3. It takes the context from the CLI, retrieves the path from the context, and uploads the compressed source `.7z` files to the specified cloud storage.

Parameters:

• ctx: The CLI context that contains the command-line arguments and flags.

Returns:

• error: An error if one occurs during the execution of the function.

Types

type ExportMapping

type ExportMapping struct {
	Tables map[string][]string `json:"tables"`
}