storageapi

package
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 20, 2022 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package storageapi contains request definitions for the Storage API. The definitions are not complete and can be extended as needed. Requests can be sent by any HTTP client that implements the client.Sender interface. It is necessary to set API host and "X-StorageApi-Token" header in the HTTP client, see the ClientWithHost and the ClientWithHostAndToken functions.

Index

Constants

View Source 
const (
	VariablesComponentID     = ComponentID(`keboola.variables`)
	SchedulerComponentID     = ComponentID("keboola.scheduler")
	SharedCodeComponentID    = ComponentID("keboola.shared-code")
	OrchestratorComponentID  = ComponentID("keboola.orchestrator")
	TransformationType       = "transformation"
	DeprecatedFlag           = `deprecated`
	ExcludeFromNewListFlag   = `excludeFromNewList`
	ComponentTypeCodePattern = `code-pattern`
	ComponentTypeProcessor   = `processor`
)

Component constants

View Source 
const TimeFormat = "2006-01-02T15:04:05-0700"

TimeFormat used in Storage API

Variables

This section is empty.

Functions

func AppendBranchMetadataRequest 

func AppendBranchMetadataRequest(key BranchKey, metadata Metadata) client.APIRequest[client.NoResult]

AppendBranchMetadataRequest https://keboola.docs.apiary.io/#reference/metadata/development-branch-metadata/create-or-update

func AppendConfigMetadataRequest 

func AppendConfigMetadataRequest(key ConfigKey, metadata Metadata) client.APIRequest[client.NoResult]

AppendConfigMetadataRequest https://keboola.docs.apiary.io/#reference/metadata/components-configurations-metadata/create-or-update

func AppendMetadataRequest 

func AppendMetadataRequest(key any, metadata map[string]string) client.APIRequest[client.NoResult]

AppendMetadataRequest creates request to append object metadata according its type.

func CleanProjectRequest 

func CleanProjectRequest() client.APIRequest[*Branch]

CleanProjectRequest cleans the whole project, the default branch is reset to the default state and other branches are deleted. Useful for E2E tests. Result is default branch.

func ClientWithHost 

func ClientWithHost(c client.Client, apiHost string) client.Client

ClientWithHost returns HTTP client with api host set.

func ClientWithHostAndToken 

func ClientWithHostAndToken(c client.Client, apiHost, apiToken string) client.Client

ClientWithHostAndToken returns HTTP client with api host and token set.

func ClientWithToken 

func ClientWithToken(c client.Client, apiToken string) client.Client

ClientWithToken returns HTTP client with api token set.

func CreatEventRequest 

func CreatEventRequest(event *Event) client.APIRequest[*Event]

CreatEventRequest https://keboola.docs.apiary.io/#reference/events/events/create-event

func CreateBranchAsyncRequest 

func CreateBranchAsyncRequest(branch *Branch) client.APIRequest[*Job]

CreateBranchAsyncRequest https://keboola.docs.apiary.io/#reference/development-branches/branches/create-branch

func CreateBranchRequest 

func CreateBranchRequest(branch *Branch) client.APIRequest[*Branch]

CreateBranchRequest https://keboola.docs.apiary.io/#reference/development-branches/branches/create-branch

func CreateConfigRequest 

func CreateConfigRequest(config *ConfigWithRows) client.APIRequest[*ConfigWithRows]

CreateConfigRequest https://keboola.docs.apiary.io/#reference/components-and-configurations/component-configurations/create-development-branch-configuration

func CreateConfigRowRequest 

func CreateConfigRowRequest(row *ConfigRow) client.APIRequest[*ConfigRow]

CreateConfigRowRequest https://kebooldocs.apiary.io/#reference/components-and-configurations/create-or-list-configuration-rows/create-development-branch-configuration-row

func CreateRequest 

func CreateRequest(object Object) client.APIRequest[Object]

CreateRequest creates request to create object according its type.

func DeleteBranchAsyncRequest 

func DeleteBranchAsyncRequest(key BranchKey) client.APIRequest[*Job]

DeleteBranchAsyncRequest https://keboola.docs.apiary.io/#reference/development-branches/branch-manipulation/delete-branch

func DeleteBranchMetadataRequest 

func DeleteBranchMetadataRequest(branch BranchKey, metaID string) client.APIRequest[client.NoResult]

DeleteBranchMetadataRequest https://keboola.docs.apiary.io/#reference/metadata/development-branch-metadata/delete

func DeleteBranchRequest 

func DeleteBranchRequest(key BranchKey) client.APIRequest[client.NoResult]

DeleteBranchRequest https://keboola.docs.apiary.io/#reference/development-branches/branch-manipulation/delete-branch

func DeleteConfigRequest 

func DeleteConfigRequest(config ConfigKey) client.APIRequest[client.NoResult]

DeleteConfigRequest https://keboola.docs.apiary.io/#reference/components-and-configurations/manage-configurations/delete-configuration

func DeleteConfigRowRequest 

func DeleteConfigRowRequest(key ConfigRowKey) client.APIRequest[client.NoResult]

DeleteConfigRowRequest https://kebooldocs.apiary.io/#reference/components-and-configurations/manage-configuration-rows/update-row

func DeleteConfigsInBranchRequest 

func DeleteConfigsInBranchRequest(branch BranchKey) client.APIRequest[client.NoResult]

DeleteConfigsInBranchRequest lists all configs in branch and deletes them all.

func DeleteRequest 

func DeleteRequest(key any) client.APIRequest[client.NoResult]

DeleteRequest creates request to delete object according its type.

func GenerateIDRequest 

func GenerateIDRequest() client.APIRequest[*Ticket]

GenerateIDRequest https://keboola.docs.apiary.io/#reference/tickets/generate-unique-id/generate-new-id

func GetBranchRequest 

func GetBranchRequest(key BranchKey) client.APIRequest[*Branch]

GetBranchRequest https://keboola.docs.apiary.io/#reference/development-branches/branch-manipulation/branch-detail

func GetConfigRequest 

func GetConfigRequest(key ConfigKey) client.APIRequest[*Config]

GetConfigRequest https://keboola.docs.apiary.io/#reference/components-and-configurations/manage-configurations/development-branch-configuration-detail

func GetConfigRowRequest 

func GetConfigRowRequest(key ConfigRowKey) client.APIRequest[*ConfigRow]

GetConfigRowRequest https://kebooldocs.apiary.io/#reference/components-and-configurations/manage-configuration-rows/row-detail

func GetDefaultBranchRequest 

func GetDefaultBranchRequest() client.APIRequest[*Branch]

GetDefaultBranchRequest lists all branches and returns the default branch.

func GetJobRequest 

func GetJobRequest(key JobKey) client.APIRequest[*Job]

GetJobRequest https://keboola.docs.apiary.io/#reference/jobs/manage-jobs/job-detail

func IndexComponentsRequest 

func IndexComponentsRequest() client.APIRequest[*IndexComponents]

IndexComponentsRequest returns index of Storage API with components definitions.

func IndexRequest 

func IndexRequest() client.APIRequest[*Index]

IndexRequest returns index of Storage API without components definitions.

func ListBranchMetadataRequest 

func ListBranchMetadataRequest(key BranchKey) client.APIRequest[*MetadataDetails]

ListBranchMetadataRequest https://keboola.docs.apiary.io/#reference/metadata/development-branch-metadata/list

func ListBranchesRequest 

func ListBranchesRequest() client.APIRequest[*[]*Branch]

ListBranchesRequest https://keboola.docs.apiary.io/#reference/development-branches/branches/list-branches

func ListConfigMetadataRequest 

func ListConfigMetadataRequest(branchID BranchID) client.APIRequest[*ConfigsMetadata]

ListConfigMetadataRequest https://keboola.docs.apiary.io/#reference/search/search-components-configurations/search-component-configurations

func ListConfigsAndRowsFrom 

func ListConfigsAndRowsFrom(branch BranchKey) client.APIRequest[*[]*ComponentWithConfigs]

ListConfigsAndRowsFrom https://keboola.docs.apiary.io/#reference/components-and-configurations/get-components/get-components

func SortComponents 

func SortComponents(components Components)

SortComponents by name, keboola vendor will be first.

func UpdateBranchRequest 

func UpdateBranchRequest(branch *Branch, changedFields []string) client.APIRequest[*Branch]

UpdateBranchRequest https://keboola.docs.apiary.io/#reference/development-branches/branches/update-branch

func UpdateConfigRequest 

func UpdateConfigRequest(config *Config, changedFields []string) client.APIRequest[*Config]

UpdateConfigRequest https://keboola.docs.apiary.io/#reference/components-and-configurations/manage-configurations/update-development-branch-configuration

func UpdateConfigRowRequest 

func UpdateConfigRowRequest(row *ConfigRow, changedFields []string) client.APIRequest[*ConfigRow]

UpdateConfigRowRequest https://kebooldocs.apiary.io/#reference/components-and-configurations/manage-configuration-rows/update-row-for-development-branch

func UpdateRequest 

func UpdateRequest(object Object, changedFields []string) client.APIRequest[Object]

UpdateRequest creates request to update object according its type.

func VerifyTokenRequest 

func VerifyTokenRequest(token string) client.APIRequest[*Token]

VerifyTokenRequest https://keboola.docs.apiary.io/#reference/tokens-and-permissions/token-verification/token-verification

Types

type Branch 

type Branch struct {
	BranchKey
	Name        string `json:"name"`
	Description string `json:"description"`
	Created     Time   `json:"created" readonly:"true"`
	IsDefault   bool   `json:"isDefault" readonly:"true"`
}

Branch https://keboola.docs.apiary.io/#reference/development-branches/branches/list-branches

type BranchID 

type BranchID int

BranchID is an ID of a development branch in Storage API.

func (BranchID) String 

func (id BranchID) String() string

type BranchKey 

type BranchKey struct {
	ID BranchID `json:"id" writeoptional:"true"`
}

BranchKey is a unique identifier of a branch.

func (BranchKey) ObjectId 

func (k BranchKey) ObjectId() any

type Component 

type Component struct {
	ComponentKey
	Type           string                 `json:"type"`
	Name           string                 `json:"name"`
	Flags          []string               `json:"flags,omitempty"`
	Schema         json.RawMessage        `json:"configurationSchema,omitempty"`
	SchemaRow      json.RawMessage        `json:"configurationRowSchema,omitempty"`
	EmptyConfig    *orderedmap.OrderedMap `json:"emptyConfiguration,omitempty"`
	EmptyConfigRow *orderedmap.OrderedMap `json:"emptyConfigurationRow,omitempty"`
	Data           ComponentData          `json:"data"`
}

Component https://keboola.docs.apiary.io/#reference/components-and-configurations/get-development-branch-components/get-development-branch-components

func (*Component) IsCodePattern 

func (c *Component) IsCodePattern() bool

IsCodePattern returns true, if component is IsCodePattern.

func (*Component) IsDeprecated 

func (c *Component) IsDeprecated() bool

IsDeprecated returns true, if component is deprecated.

func (*Component) IsExcludedFromNewList 

func (c *Component) IsExcludedFromNewList() bool

IsExcludedFromNewList returns true, if component should be excluded from list of available new components.

func (*Component) IsOrchestrator 

func (c *Component) IsOrchestrator() bool

IsOrchestrator returns true, if component is orchestration.

func (*Component) IsProcessor 

func (c *Component) IsProcessor() bool

IsProcessor returns true, if component is processor.

func (*Component) IsScheduler 

func (c *Component) IsScheduler() bool

IsScheduler returns true, if component is scheduler.

func (*Component) IsSharedCode 

func (c *Component) IsSharedCode() bool

IsSharedCode returns true, if component is shared code.

func (*Component) IsTransformation 

func (c *Component) IsTransformation() bool

IsTransformation returns true, if component is transformation.

func (*Component) IsVariables 

func (c *Component) IsVariables() bool

IsVariables returns true, if component has "variables" typ.

type ComponentData 

type ComponentData struct {
	DefaultBucket      bool   `json:"default_bucket"`       //nolint: tagliatelle
	DefaultBucketStage string `json:"default_bucket_stage"` //nolint: tagliatelle
}

ComponentData https://keboola.docs.apiary.io/#reference/components-and-configurations/get-development-branch-components/get-development-branch-components

type ComponentID 

type ComponentID string

ComponentID is id of a Keboola component.

func (ComponentID) String 

func (v ComponentID) String() string

func (ComponentID) WithoutVendor 

func (v ComponentID) WithoutVendor() string

type ComponentKey 

type ComponentKey struct {
	ID ComponentID `json:"id"`
}

ComponentKey is a unique identifier of a component.

type ComponentWithConfigs 

type ComponentWithConfigs struct {
	BranchID BranchID `json:"branchId"`
	Component
	Configs []*ConfigWithRows `json:"configurations"`
}

ComponentWithConfigs is result of ListConfigsAndRowsFrom request.

type Components 

type Components []*Component

Components slice.

func (Components) NewComponentList 

func (v Components) NewComponentList() Components

NewComponentList returns only the components that should be included in list of available new components.

func (Components) ToMap 

func (v Components) ToMap() ComponentsMap

ToMap converts Components slice to ComponentsMap.

type ComponentsMap 

type ComponentsMap struct {
	// contains filtered or unexported fields
}

ComponentsMap is immutable map of components, see Components.ToMap.

func (ComponentsMap) ForEach 

func (m ComponentsMap) ForEach(fn func(component *Component))

func (ComponentsMap) Get 

func (m ComponentsMap) Get(id ComponentID) (*Component, bool)

type Config 

type Config struct {
	ConfigKey
	Name              string                 `json:"name"`
	Description       string                 `json:"description"`
	ChangeDescription string                 `json:"changeDescription"`
	IsDeleted         bool                   `json:"isDeleted" readonly:"true"`
	Created           Time                   `json:"created" readonly:"true"`
	Version           int                    `json:"version" readonly:"true"`
	State             *orderedmap.OrderedMap `json:"state" readonly:"true"`
	IsDisabled        bool                   `json:"isDisabled"`
	Content           *orderedmap.OrderedMap `json:"configuration"`
}

Config https://keboola.docs.apiary.io/#reference/components-and-configurations/component-configurations/list-configurations

type ConfigID 

type ConfigID string

ConfigID represents an ID of a configuration in Storage API.

func (ConfigID) String 

func (v ConfigID) String() string

type ConfigKey 

type ConfigKey struct {
	BranchID    BranchID    `json:"branchId"`
	ComponentID ComponentID `json:"componentId"`
	ID          ConfigID    `json:"id" writeas:"configurationId" writeoptional:"true"`
}

ConfigKey is a unique identifier of a configuration.

func (ConfigKey) ObjectId 

func (k ConfigKey) ObjectId() any

type ConfigMetadataItem 

type ConfigMetadataItem struct {
	BranchID    BranchID
	ComponentID ComponentID     `json:"idComponent"`
	ConfigID    ConfigID        `json:"configurationId"`
	Metadata    MetadataDetails `json:"metadata"`
}

ConfigMetadataItem is one item of configuration metadata.

type ConfigRow 

type ConfigRow struct {
	ConfigRowKey
	Name              string                 `json:"name"`
	Description       string                 `json:"description"`
	ChangeDescription string                 `json:"changeDescription"`
	IsDisabled        bool                   `json:"isDisabled"`
	Created           Time                   `json:"created" readonly:"true"`
	Version           int                    `json:"version" readonly:"true"`
	State             *orderedmap.OrderedMap `json:"state" readonly:"true"`
	Content           *orderedmap.OrderedMap `json:"configuration"`
}

ConfigRow https://keboola.docs.apiary.io/#reference/components-and-configurations/component-configurations/list-configurations

type ConfigRowKey 

type ConfigRowKey struct {
	BranchID    BranchID    `json:"-"`
	ComponentID ComponentID `json:"-"`
	ConfigID    ConfigID    `json:"-"`
	ID          RowID       `json:"id" writeas:"rowId" writeoptional:"true"`
}

ConfigRowKey is a unique identifier of ConfigRow.

func (ConfigRowKey) ObjectId 

func (k ConfigRowKey) ObjectId() any

type ConfigWithRows 

type ConfigWithRows struct {
	*Config
	Rows []*ConfigRow `json:"rows"`
}

ConfigWithRows is a configuration with its configuration rows.

func (*ConfigWithRows) SortRows 

func (c *ConfigWithRows) SortRows()

SortRows by name.

type ConfigsMetadata 

type ConfigsMetadata []*ConfigMetadataItem

ConfigsMetadata slice.

func (ConfigsMetadata) ToMap 

func (v ConfigsMetadata) ToMap() map[ConfigKey]Metadata

ToMap converts slice to map.

type DurationSeconds 

type DurationSeconds time.Duration

DurationSeconds is time.Duration encoded/decoded as number of seconds.

func (DurationSeconds) MarshalJSON 

func (d DurationSeconds) MarshalJSON() ([]byte, error)

MarshalJSON implements JSON encoding.

func (DurationSeconds) String 

func (d DurationSeconds) String() string

func (*DurationSeconds) UnmarshalJSON 

func (d *DurationSeconds) UnmarshalJSON(data []byte) (err error)

UnmarshalJSON implements JSON decoding.

type Error 

type Error struct {
	Message     string `json:"error"`
	ErrCode     string `json:"code"`
	ExceptionID string `json:"exceptionId"`
	// contains filtered or unexported fields
}

Error represents the structure of Storage API error.

func (Error) Error 

func (e Error) Error() string

func (Error) ErrorExceptionID 

func (e Error) ErrorExceptionID() string

ErrorExceptionID returns exception ID to find details in logs.

func (Error) ErrorName 

func (e Error) ErrorName() string

ErrorName returns a human-readable name of the error.

func (Error) ErrorUserMessage 

func (e Error) ErrorUserMessage() string

ErrorUserMessage returns error message for end user.

func (*Error) SetRequest 

func (e *Error) SetRequest(request *http.Request)

SetRequest method allows injection of HTTP request to the error, it implements client.errorWithRequest.

func (*Error) SetResponse 

func (e *Error) SetResponse(response *http.Response)

SetResponse method allows injection of HTTP response to the error, it implements client.errorWithResponse.

func (Error) StatusCode 

func (e Error) StatusCode() int

StatusCode returns HTTP status code.

type Event 

type Event struct {
	ID          EventID         `json:"id" readonly:"true"`
	ComponentID ComponentID     `json:"component"`
	Message     string          `json:"message"`
	Type        string          `json:"type"`
	Duration    DurationSeconds `json:"duration"`
	Params      JSONString      `json:"params"`
	Results     JSONString      `json:"results"`
}

Event https://keboola.docs.apiary.io/#reference/events/events/create-event

type EventID 

type EventID string

EventID represents an ID of an event in Storage API.

func (EventID) String 

func (v EventID) String() string

type Features 

type Features []string

func (Features) ToMap 

func (v Features) ToMap() FeaturesMap

ToMap converts Features slice to FeaturesMap.

type FeaturesMap 

type FeaturesMap struct {
	// contains filtered or unexported fields
}

FeaturesMap is immutable map of features, see Features.ToMap.

func (FeaturesMap) Has 

func (m FeaturesMap) Has(feature string) bool

Has returns true if project has the feature enabled.

type Index 

type Index struct {
	Services Services `json:"services"`
	Features Features `json:"features"`
}

Index of Storage API.

func (Index) AllServices 

func (i Index) AllServices() ServicesMap

AllServices converts services slice to map.

func (Index) ServiceURLByID 

func (i Index) ServiceURLByID(serviceID ServiceID) (ServiceURL, bool)

ServiceURLByID return service URL by service ID.

type IndexComponents 

type IndexComponents struct {
	Index
	Components Components `json:"components"`
}

IndexComponents is the Index of Storage API with components included.

type JSONString 

type JSONString map[string]any

JSONString is Json encoded as string, see CreatEventRequest.

func (*JSONString) MarshalJSON 

func (v *JSONString) MarshalJSON() ([]byte, error)

MarshalJSON implements JSON encoding.

func (JSONString) String 

func (v JSONString) String() string

func (*JSONString) UnmarshalJSON 

func (v *JSONString) UnmarshalJSON(data []byte) (err error)

UnmarshalJSON implements JSON decoding.

type Job 

type Job struct {
	JobKey
	Status          string         `json:"status"`
	URL             string         `json:"url"`
	OperationName   string         `json:"operationName"`
	OperationParams map[string]any `json:"operationParams"`
	Results         map[string]any `json:"results"`
	CreateTime      Time           `json:"createdTime"`
	StartTime       *Time          `json:"startTime"`
	EndTime         *Time          `json:"endTime"`
}

Job is a storage job.

type JobID 

type JobID int

JobID is an ID of a storage job.

func (JobID) String 

func (id JobID) String() string

type JobKey 

type JobKey struct {
	ID JobID `json:"id"`
}

JobKey is a unique identifier of Job.

type Metadata 

type Metadata map[string]string

Metadata - object metadata.

type MetadataDetail 

type MetadataDetail struct {
	ID        string `json:"id"`
	Key       string `json:"key"`
	Value     string `json:"value"`
	Timestamp string `json:"timestamp"`
}

MetadataDetail - metadata with details (id, timestamp).

type MetadataDetails 

type MetadataDetails []MetadataDetail

MetadataDetails - metadata with details (id, timestamp).

func (MetadataDetails) ToMap 

func (v MetadataDetails) ToMap() Metadata

ToMap converts slice to map.

type Object 

type Object interface {
	ObjectId() any
}

type RowID 

type RowID string

RowID is id of configuration row.

func (RowID) String 

func (v RowID) String() string

type Service 

type Service struct {
	ID  ServiceID  `json:"id"`
	URL ServiceURL `json:"url"`
}

Service is a Keboola service, for example Encryption API.

type ServiceID 

type ServiceID string

ServiceID is an ID of a Keboola service, for example "encryption".

func (ServiceID) String 

func (u ServiceID) String() string

type ServiceURL 

type ServiceURL string

ServiceURL is an url of a Keboola service, for example "https://encryption.keboola.com".

func (ServiceURL) String 

func (u ServiceURL) String() string

type Services 

type Services []*Service

Services slice.

func (Services) ToMap 

func (v Services) ToMap() ServicesMap

ToMap converts Services slice to ServicesMap.

type ServicesMap 

type ServicesMap struct {
	// contains filtered or unexported fields
}

ServicesMap is immutable map of services, see Services.ToMap.

func (ServicesMap) URLByID 

func (m ServicesMap) URLByID(serviceID ServiceID) (ServiceURL, bool)

URLByID return service URL by service ID.

type Ticket 

type Ticket struct {
	ID string `json:"id"`
}

Ticket https://keboola.docs.apiary.io/#reference/tickets/generate-unique-id/generate-new-id

type TicketProvider 

type TicketProvider struct {
	// contains filtered or unexported fields
}

TicketProvider generates new IDs and GUARANTEES that the IDs will be returned with the same order as the Request method was called.

func NewTicketProvider 

func NewTicketProvider(ctx context.Context, sender client.Sender) *TicketProvider

NewTicketProvider creates TicketProvider.

func (*TicketProvider) Request 

func (t *TicketProvider) Request(onSuccess func(ticket *Ticket))

Request queues ID generation requests and registers callback.

func (*TicketProvider) Resolve 

func (t *TicketProvider) Resolve() error

Resolve sends all ID generation requests and then calls all callbacks.

type Time 

type Time time.Time

Time is encoded/decoded in TimeFormat used in Storage API.

func (Time) MarshalJSON 

func (t Time) MarshalJSON() ([]byte, error)

MarshalJSON implements JSON encoding.

func (Time) String 

func (t Time) String() string

func (*Time) UnmarshalJSON 

func (t *Time) UnmarshalJSON(data []byte) (err error)

UnmarshalJSON implements JSON decoding.

type Token 

type Token struct {
	ID       string     `json:"id"`
	Token    string     `json:"token"`
	IsMaster bool       `json:"isMasterToken"`
	Owner    TokenOwner `json:"owner"`
}

Token https://keboola.docs.apiary.io/#reference/tokens-and-permissions/token-verification/token-verification

func (*Token) ProjectID 

func (t *Token) ProjectID() int

ProjectID returns ID of project to which the token belongs.

func (*Token) ProjectName 

func (t *Token) ProjectName() string

ProjectName returns name of project to which the token belongs.

type TokenOwner 

type TokenOwner struct {
	ID   int    `json:"id"`
	Name string `json:"name"`
}

TokenOwner - owner of Token.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.