api

package
v1.3.24 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 27, 2024 License: MIT Imports: 27 Imported by: 6

Documentation

Overview

Package api provides native Go-based API/SDK over HTTP(S).

  • Copyright (c) 2024, NVIDIA CORPORATION. All rights reserved.

Package api provides native Go-based API/SDK over HTTP(S).

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package api provides native Go-based API/SDK over HTTP(S).

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package api provides native Go-based API/SDK over HTTP(S).

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package api provides native Go-based API/SDK over HTTP(S).

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package api provides native Go-based API/SDK over HTTP(S).

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package api provides native Go-based API/SDK over HTTP(S).

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package api provides native Go-based API/SDK over HTTP(S).

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package api provides native Go-based API/SDK over HTTP(S).

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package api provides native Go-based API/SDK over HTTP(S).

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package api provides native Go-based API/SDK over HTTP(S).

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package api provides native Go-based API/SDK over HTTP(S).

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package api provides native Go-based API/SDK over HTTP(S).

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package api provides native Go-based API/SDK over HTTP(S).

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package api provides native Go-based API/SDK over HTTP(S).

  • Copyright (c) 2024, NVIDIA CORPORATION. All rights reserved.

Package api provides native Go-based API/SDK over HTTP(S).

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AbortDownload 

func AbortDownload(bp BaseParams, id string) error

func AbortDsort added in v1.3.21 

func AbortDsort(bp BaseParams, managerUUID string) error

func AbortXaction 

func AbortXaction(bp BaseParams, args *xact.ArgsMsg) (err error)

Abort ("stop") xactions

func AppendObject 

func AppendObject(args *AppendArgs) (string, error)

func ArchiveMultiObj added in v1.3.18 

func ArchiveMultiObj(bp BaseParams, bckFrom cmn.Bck, msg *cmn.ArchiveBckMsg) (string, error)

Archive multiple objects from the specified source bucket. The option to append multiple objects to an existing archive is also supported. The source and the destination buckets are defined as `bckFrom` and `bckTo`, respectively (not necessarily distinct) For supported archiving formats, see `archive.FileExtensions`. See also: api.PutApndArch

func AttachMountpath 

func AttachMountpath(bp BaseParams, node *meta.Snode, mountpath string, label ...ios.Label) error

func AttachRemoteAIS 

func AttachRemoteAIS(bp BaseParams, alias, u string) error

func BlobDownload added in v1.3.22 

func BlobDownload(bp BaseParams, bck cmn.Bck, objName string, msg *apc.BlobMsg) (xid string, err error)

func CopyBucket 

func CopyBucket(bp BaseParams, bckFrom, bckTo cmn.Bck, msg *apc.CopyBckMsg, fltPresence ...int) (xid string, err error)

CopyBucket copies existing `bckFrom` bucket to the destination `bckTo` thus, effectively, creating a copy of the `bckFrom`.

  • AIS will create `bckTo` on the fly but only if the destination bucket does not exist and _is_ provided by AIStore; 3rd party backend destination must exist - otherwise the copy operation won't be successful.
  • There are no limitations on copying buckets across Backend providers: you can copy AIS bucket to (or from) AWS bucket, and the latter to Google or Azure bucket, etc.
  • Copying multiple buckets to the same destination bucket is also permitted.

`fltPresence` applies exclusively to remote `bckFrom` and is ignored if the source is ais:// The value is enum { apc.FltExists, apc.FltPresent, ... } - for complete enum, see api/apc/query.go Namely: * apc.FltExists - copy all objects, including those that are not (present) in AIS * apc.FltPresent - copy the current `bckFrom` content in the cluster (default) * apc.FltExistsOutside - copy only those remote objects that are not (present) in AIS

msg.Prefix, if specified, applies always and regardless.

Returns xaction ID if successful, an error otherwise. See also closely related api.ETLBucket

func CopyMultiObj 

func CopyMultiObj(bp BaseParams, bckFrom cmn.Bck, msg *cmn.TCOMsg, fltPresence ...int) (xid string, err error)

func CreateBucket 

func CreateBucket(bp BaseParams, bck cmn.Bck, props *cmn.BpropsToSet, dontHeadRemote ...bool) error

CreateBucket sends request to create an AIS bucket with the given name and, optionally, specific non-default properties (via cmn.BpropsToSet).

See also:

  • github.com/NVIDIA/aistore/blob/main/docs/bucket.md#default-bucket-properties
  • cmn.BpropsToSet (cmn/api.go)

Bucket properties can be also changed at any time via SetBucketProps (above).

func DecommissionCluster 

func DecommissionCluster(bp BaseParams, rmUserData bool) error

DecommissionCluster permanently decommissions entire cluster

func DecommissionNode 

func DecommissionNode(bp BaseParams, actValue *apc.ActValRmNode) (xid string, err error)

func DeleteMultiObj added in v1.3.22 

func DeleteMultiObj(bp BaseParams, bck cmn.Bck, objNames []string, template string) (string, error)

func DeleteObject 

func DeleteObject(bp BaseParams, bck cmn.Bck, objName string) error

func DestroyBucket 

func DestroyBucket(bp BaseParams, bck cmn.Bck) error

DestroyBucket sends request to remove an AIS bucket with the given name.

func DetachMountpath 

func DetachMountpath(bp BaseParams, node *meta.Snode, mountpath string, dontResilver bool) error

func DetachRemoteAIS 

func DetachRemoteAIS(bp BaseParams, alias string) error

func DisableBackend added in v1.3.24 

func DisableBackend(bp BaseParams, provider string) error

func DisableMountpath 

func DisableMountpath(bp BaseParams, node *meta.Snode, mountpath string, dontResilver bool) error

func DisableRebalance added in v1.3.24 

func DisableRebalance(bp BaseParams) error

func DoWithRetry 

func DoWithRetry(client *http.Client, cb newRequestCB, reqArgs *cmn.HreqArgs) (resp *http.Response, err error)

func DownloadBackend 

func DownloadBackend(bp BaseParams, descr string, bck cmn.Bck, prefix, suffix string, ivals ...time.Duration) (string, error)

func DownloadGetList 

func DownloadGetList(bp BaseParams, regex string, onlyActive bool) (dlList dload.JobInfos, err error)

func DownloadMulti 

func DownloadMulti(bp BaseParams, description string, bck cmn.Bck, msg any, intervals ...time.Duration) (string, error)

func DownloadRange 

func DownloadRange(bp BaseParams, description string, bck cmn.Bck, template string, intervals ...time.Duration) (string, error)

func DownloadSingle 

func DownloadSingle(bp BaseParams, description string,
	bck cmn.Bck, objName, link string, intervals ...time.Duration) (string, error)

func DownloadStatus 

func DownloadStatus(bp BaseParams, id string, onlyActive bool) (dlStatus *dload.StatusResp, err error)

func DownloadWithParam 

func DownloadWithParam(bp BaseParams, dlt dload.Type, body any) (id string, err error)

func ECEncodeBucket 

func ECEncodeBucket(bp BaseParams, bck cmn.Bck, data, parity int) (xid string, err error)

Erasure-code entire `bck` bucket at a given `data`:`parity` redundancy. The operation requires at least (`data + `parity` + 1) storage targets in the cluster. Returns xaction ID if successful, an error otherwise.

func ETLBucket 

func ETLBucket(bp BaseParams, bckFrom, bckTo cmn.Bck, msg *apc.TCBMsg, fltPresence ...int) (xid string, err error)

Transform src bucket => dst bucket, i.e.: - visit all (matching) source objects; for each object: - read it, transform using the specified (ID-ed) ETL, and write the result to dst bucket

`fltPresence` applies exclusively to remote `bckFrom` (is ignored otherwise) and is one of: { apc.FltExists, apc.FltPresent, ... } - for complete enum, see api/apc/query.go Namely: * apc.FltExists - copy all objects, including those that are not (present) in AIS * apc.FltPresent - copy the current `bckFrom` content in the cluster (default) * apc.FltExistsOutside - copy only those remote objects that are not (present) in AIS

msg.Prefix, if specified, applies always and regardless.

Returns xaction ID if successful, an error otherwise. See also: api.CopyBucket

func ETLDelete 

func ETLDelete(bp BaseParams, etlName string) (err error)

func ETLGetInitMsg 

func ETLGetInitMsg(params BaseParams, etlName string) (etl.InitMsg, error)

func ETLHealth 

func ETLHealth(params BaseParams, etlName string) (healths etl.HealthByTarget, err error)

func ETLInit 

func ETLInit(bp BaseParams, msg etl.InitMsg) (xid string, err error)

Initiate custom ETL workload by executing one of the documented `etl.InitMsg` message types. The API call results in deploying multiple ETL containers (K8s pods): one container per storage target. Returns xaction ID if successful, an error otherwise.

func ETLList 

func ETLList(bp BaseParams) (list []etl.Info, err error)

func ETLLogs 

func ETLLogs(bp BaseParams, etlName string, targetID ...string) (logs etl.LogsByTarget, err error)

func ETLMetrics added in v1.3.16 

func ETLMetrics(params BaseParams, etlName string) (healths etl.CPUMemByTarget, err error)

func ETLMultiObj 

func ETLMultiObj(bp BaseParams, bckFrom cmn.Bck, msg *cmn.TCOMsg, fltPresence ...int) (xid string, err error)

func ETLObject 

func ETLObject(bp BaseParams, etlName string, bck cmn.Bck, objName string, w io.Writer) (err error)

TODO: add ETL-specific query param and change the examples/docs (!4455)

func ETLStart 

func ETLStart(bp BaseParams, etlName string) (err error)

func ETLStop 

func ETLStop(bp BaseParams, etlName string) (err error)

func EnableBackend added in v1.3.24 

func EnableBackend(bp BaseParams, provider string) error

func EnableMountpath 

func EnableMountpath(bp BaseParams, node *meta.Snode, mountpath string) error

func EnableRebalance added in v1.3.24 

func EnableRebalance(bp BaseParams) error

func EvictMultiObj added in v1.3.22 

func EvictMultiObj(bp BaseParams, bck cmn.Bck, objNames []string, template string) (string, error)

func EvictObject 

func EvictObject(bp BaseParams, bck cmn.Bck, objName string) error

func EvictRemoteBucket 

func EvictRemoteBucket(bp BaseParams, bck cmn.Bck, keepMD bool) error

EvictRemoteBucket sends request to evict an entire remote bucket from the AIStore - keepMD: evict objects but keep bucket metadata

func FlushObject 

func FlushObject(args *FlushArgs) error

FlushObject must be called after all the appends (via `api.AppendObject`). To "flush", it uses the handle returned by `api.AppendObject`. This call will create a fully operational and accessible object.

func FreeRp 

func FreeRp(reqParams *ReqParams)

func FshcMountpath added in v1.3.24 

func FshcMountpath(bp BaseParams, node *meta.Snode, mountpath string) error

func GetAllRunningXactions added in v1.3.16 

func GetAllRunningXactions(bp BaseParams, kindOrName string) (out []string, err error)

returns a slice of canonical xaction names, as in: `xact.Cname()` e.g.: put-copies[D-ViE6HEL_j] list[H96Y7bhR2s] copy-bck[matRQMRes] put-copies[pOibtHExY] TODO: return idle xactions separately

func GetAllXactionStatus added in v1.3.16 

func GetAllXactionStatus(bp BaseParams, args *xact.ArgsMsg) (matching nl.StatusVec, err error)

same as above, except that it returns _all_ matching xactions

func GetAnyStats added in v1.3.24 

func GetAnyStats(bp BaseParams, sid, what string) (out []byte, err error)

func GetBMD 

func GetBMD(bp BaseParams) (bmd *meta.BMD, err error)

get bucket metadata (BMD) from a BaseParams-referenced node

func GetBucketInfo 

func GetBucketInfo(bp BaseParams, bck cmn.Bck, args *BinfoArgs) (string, *cmn.Bprops, *cmn.BsummResult, error)

Bucket information - a runtime addendum to `BucketProps`. In addition to `cmn.Bprops` properties (which are user configurable), bucket runtime info: - includes usage, capacity, other statistics - is obtained via GetBucketInfo() API - and delivered via apc.HdrBucketInfo header (compare with GetBucketSummary) The API uses http.MethodHead and can be considered an extension of HeadBucket (above)

func GetBucketSummary 

func GetBucketSummary(bp BaseParams, qbck cmn.QueryBcks, msg *apc.BsummCtrlMsg, args BsummArgs) (xid string,
	res cmn.AllBsummResults, err error)

GetBucketSummary returns bucket capacity ulitization percentages, sizes (total and min/max/average), and the numbers of objects, both _in_ the cluster and remote GetBucketSummary supports a single specified bucket or multiple buckets, as per `cmn.QueryBcks` query. (e.g., GetBucketSummary with an empty bucket query will return "summary" info for all buckets)

func GetClusterConfig 

func GetClusterConfig(bp BaseParams) (*cmn.ClusterConfig, error)

GetClusterConfig returns cluster-wide configuration (compare with `api.GetDaemonConfig`)

func GetClusterMap 

func GetClusterMap(bp BaseParams) (smap *meta.Smap, err error)

get cluster map from a BaseParams-referenced node

func GetClusterStats 

func GetClusterStats(bp BaseParams) (res stats.Cluster, err error)

func GetClusterSysInfo 

func GetClusterSysInfo(bp BaseParams) (info apc.ClusterSysInfo, err error)

GetClusterSysInfo retrieves cluster's system information

func GetConfiguredBackends added in v1.3.24 

func GetConfiguredBackends(bp BaseParams) (out []string, err error)

(see also enable/disable backend below)

func GetDaemonConfig 

func GetDaemonConfig(bp BaseParams, node *meta.Snode) (config *cmn.Config, err error)

GetDaemonConfig returns the configuration of a specific daemon in a cluster. (compare with `api.GetClusterConfig`)

func GetDaemonLog 

func GetDaemonLog(bp BaseParams, node *meta.Snode, args GetLogInput) (int64, error)

Returns log of a specific node in a cluster.

func GetDaemonStats 

func GetDaemonStats(bp BaseParams, node *meta.Snode) (ds *stats.Node, err error)

NOTE: direct call used only in tests (remove?)

func GetMetricNames added in v1.3.16 

func GetMetricNames(bp BaseParams, node *meta.Snode) (kvs cos.StrKVs, err error)

names _and_ kinds, i.e. (name, kind) pairs

func GetMountpaths 

func GetMountpaths(bp BaseParams, node *meta.Snode) (mpl *apc.MountpathList, err error)

GetMountpaths given the direct public URL of the target, returns the target's mountpaths or error.

func GetNodeClusterMap 

func GetNodeClusterMap(bp BaseParams, sid string) (smap *meta.Smap, err error)

GetNodeClusterMap retrieves cluster map from the specified node.

func GetNodeMeta added in v1.3.19 

func GetNodeMeta(bp BaseParams, sid, what string) (out any, err error)

- get (smap, bmd, config) *cluster-level* metadata from the spec-ed node - compare with GetClusterMap, GetNodeClusterMap, GetClusterConfig et al. - TODO: etl meta

func GetObjectReader 

func GetObjectReader(bp BaseParams, bck cmn.Bck, objName string, args *GetArgs) (r io.ReadCloser, size int64, err error)

Returns reader of the requested object. It does not read body bytes, nor validates a checksum. Caller is responsible for closing the reader.

func GetObjectS3 

func GetObjectS3(bp BaseParams, bck cmn.Bck, objectName string, args ...GetArgs) (int64, error)

s3/<bucket-name>/<object-name>

func GetOneXactionStatus added in v1.3.16 

func GetOneXactionStatus(bp BaseParams, args *xact.ArgsMsg) (status *nl.Status, err error)

GetOneXactionStatus queries one of the IC (proxy) members for status of the `args`-identified xaction. NOTE: - is used internally by the WaitForXactionIC() helper function (to wait on xaction) - returns a single matching xaction or none; - when the `args` filter "covers" multiple xactions the returned status corresponds to any matching xaction that's currently running, or - if nothing's running - the one that's finished most recently, if exists

func GetProxyReadiness 

func GetProxyReadiness(bp BaseParams) error

to be used by external watchdogs (Kubernetes, etc.) (compare with api.Health below)

func GetRemoteAIS 

func GetRemoteAIS(bp BaseParams) (remais meta.RemAisVec, err error)

func GetStatsAndStatus added in v1.3.16 

func GetStatsAndStatus(bp BaseParams, node *meta.Snode) (ds *stats.NodeStatus, err error)

returns both node's stats (as above) and extended status

func GetStatsAndStatusV322 added in v1.3.23 

func GetStatsAndStatusV322(bp BaseParams, node *meta.Snode) (ds *stats.NodeStatusV322, err error)

func GetWhatRawQuery 

func GetWhatRawQuery(getWhat, getProps string) string

func GetX509Info added in v1.3.24 

func GetX509Info(bp BaseParams, nodeID ...string) (info cos.StrKVs, err error)

func HTTPStatus 

func HTTPStatus(err error) int

HTTPStatus returns HTTP status or (-1) for non-HTTP error.

func HeadBucket 

func HeadBucket(bp BaseParams, bck cmn.Bck, dontAddRemote bool) (p *cmn.Bprops, err error)

HEAD(bucket): apc.HdrBucketProps => cmn.Bprops{} and apc.HdrBucketInfo => BucketInfo{}

Converts the string type fields returned from the HEAD request to their corresponding counterparts in the cmn.Bprops struct.

By default, AIStore adds remote buckets to the cluster metadata on the fly. Remote bucket that was never accessed before just "shows up" when user performs HEAD, PUT, GET, SET-PROPS, and a variety of other operations. This is done only once (and after confirming the bucket's existence and accessibility) and doesn't require any action from the user. Use `dontAddRemote` to override the default behavior: as the name implies, setting `dontAddRemote = true` prevents AIS from adding remote bucket to the cluster's metadata.

func HeadObject 

func HeadObject(bp BaseParams, bck cmn.Bck, objName string, args HeadArgs) (*cmn.ObjectProps, error)

func Health 

func Health(bp BaseParams, readyToRebalance ...bool) error

func HealthUptime 

func HealthUptime(bp BaseParams, readyToRebalance ...bool) (string, string, error)

func JoinCluster 

func JoinCluster(bp BaseParams, nodeInfo *meta.Snode) (rebID, sid string, err error)

JoinCluster add a node to a cluster.

func ListBuckets 

func ListBuckets(bp BaseParams, qbck cmn.QueryBcks, fltPresence int) (cmn.Bcks, error)

ListBuckets returns buckets for provided query, where - `fltPresence` is one of { apc.FltExists, apc.FltPresent, ... } - see api/apc/query.go - ListBuckets utilizes `cmn.QueryBcks` - control structure that's practically identical to `cmn.Bck`, except for the fact that some or all its fields can be empty (to facilitate the corresponding query). See also: QueryBuckets, ListObjects

func ListDsort added in v1.3.21 

func ListDsort(bp BaseParams, regex string, onlyActive bool) (jobInfos []*dsort.JobInfo, err error)

func ListObjects 

func ListObjects(bp BaseParams, bck cmn.Bck, lsmsg *apc.LsoMsg, args ListArgs) (*cmn.LsoRes, error)

ListObjects returns a list of objects in a bucket - a slice of structures in the `cmn.LsoRes` that look like `cmn.LsoEnt`.

The `numObjects` argument is the maximum number of objects to be returned (where 0 (zero) means returning all objects in the bucket).

This API supports numerous options and flags. In particular, `apc.LsoMsg` structure supports "opening" objects formatted as one of the supported archival types and include contents of archived directories in generated result sets.

In addition, `lsmsg` (`apc.LsoMsg`) provides options (flags) to optimize the request's latency, to list anonymous public-access Cloud buckets, and more. Further details at `api/apc/lsmsg.go` source.

AIS supports listing buckets that have millions of objects. For large and very large buckets, it is strongly recommended to use the `ListObjectsPage` API - effectively, an iterator returning _next_ listed page along with associated _continuation token_.

See also: - docs/cli/* for CLI usage examples - `apc.LsoMsg` - `api.ListObjectsPage`

func ListObjectsInvalidateCache 

func ListObjectsInvalidateCache(bp BaseParams, bck cmn.Bck) error

TODO: obsolete this function after introducing mechanism to detect remote bucket changes.

func ListObjectsPage 

func ListObjectsPage(bp BaseParams, bck cmn.Bck, lsmsg *apc.LsoMsg, args ListArgs) (*cmn.LsoRes, error)

ListObjectsPage returns the first page of bucket objects. On success the function updates `lsmsg.ContinuationToken` which client then can reuse to fetch the next page. See also: - docs/cli/* for CLI usage examples - `apc.LsoMsg` - `api.ListObjects`

func LoadX509Cert added in v1.3.24 

func LoadX509Cert(bp BaseParams, nodeID ...string) error

func MakeNCopies 

func MakeNCopies(bp BaseParams, bck cmn.Bck, copies int) (xid string, err error)

MakeNCopies starts an extended action (xaction) to bring a given bucket to a certain redundancy level (num copies). Returns xaction ID if successful, an error otherwise.

func MetricsDsort added in v1.3.21 

func MetricsDsort(bp BaseParams, managerUUID string) (metrics map[string]*dsort.JobInfo, err error)

func Prefetch added in v1.3.22 

func Prefetch(bp BaseParams, bck cmn.Bck, msg apc.PrefetchMsg) (string, error)

func PrefetchObject added in v1.3.22 

func PrefetchObject(bp BaseParams, bck cmn.Bck, objName string) (string, error)

func Promote 

func Promote(bp BaseParams, bck cmn.Bck, args *apc.PromoteArgs) (xid string, err error)

func PutApndArch added in v1.3.18 

func PutApndArch(args *PutApndArchArgs) (err error)

func QueryBuckets 

func QueryBuckets(bp BaseParams, qbck cmn.QueryBcks, fltPresence int) (bool, error)

QueryBuckets is a little convenience helper. It returns true if the selection contains at least one bucket that satisfies the (qbck) criteria. - `fltPresence` - as per QparamFltPresence enum (see api/apc/query.go)

func QueryXactionSnaps 

func QueryXactionSnaps(bp BaseParams, args *xact.ArgsMsg) (xs xact.MultiSnap, err error)

QueryXactionSnaps gets all xaction snaps based on the specified selection. NOTE: args.Kind can be either xaction kind or name - here and elsewhere

func RemoveDownload 

func RemoveDownload(bp BaseParams, id string) error

func RemoveDsort added in v1.3.21 

func RemoveDsort(bp BaseParams, managerUUID string) error

func RemoveNodeUnsafe added in v1.3.18 

func RemoveNodeUnsafe(bp BaseParams, sid string) error

Remove node node from the cluster immediately. - NOTE: potential data loss, advanced usage only! - NOTE: the node remains running (compare w/ shutdown) and can be re-joined at a later time (see api.JoinCluster).

func RenameBucket 

func RenameBucket(bp BaseParams, bckFrom, bckTo cmn.Bck) (xid string, err error)

RenameBucket renames bckFrom as bckTo. Returns xaction ID if successful, an error otherwise.

func RenameObject 

func RenameObject(bp BaseParams, bck cmn.Bck, oldName, newName string) error

func RescanMountpath added in v1.3.24 

func RescanMountpath(bp BaseParams, node *meta.Snode, mountpath string, dontResilver bool) error

func ResetBucketProps 

func ResetBucketProps(bp BaseParams, bck cmn.Bck) (string, error)

ResetBucketProps resets the properties of a bucket to the global configuration.

func ResetClusterConfig 

func ResetClusterConfig(bp BaseParams) error

all nodes: reset configuration to cluster defaults

func ResetClusterStats added in v1.3.16 

func ResetClusterStats(bp BaseParams, errorsOnly bool) (err error)

func ResetDaemonConfig 

func ResetDaemonConfig(bp BaseParams, nodeID string) error

reset node's configuration to cluster defaults

func ResetDaemonStats added in v1.3.16 

func ResetDaemonStats(bp BaseParams, node *meta.Snode, errorsOnly bool) error

func RotateClusterLogs added in v1.3.22 

func RotateClusterLogs(bp BaseParams) error

func RotateLogs added in v1.3.22 

func RotateLogs(bp BaseParams, nodeID string) error

func SetAuxHeaders 

func SetAuxHeaders(r *http.Request, bp *BaseParams)

func SetBucketProps 

func SetBucketProps(bp BaseParams, bck cmn.Bck, props *cmn.BpropsToSet) (string, error)

SetBucketProps sets the properties of a bucket. Validation of the properties passed in is performed by AIStore Proxy.

func SetClusterConfig 

func SetClusterConfig(bp BaseParams, nvs cos.StrKVs, transient bool) error

SetClusterConfig given key-value pairs of cluster configuration parameters, sets the cluster-wide configuration accordingly. Setting cluster-wide configuration requires sending the request to a proxy.

func SetClusterConfigUsingMsg 

func SetClusterConfigUsingMsg(bp BaseParams, configToUpdate *cmn.ConfigToSet, transient bool) error

SetClusterConfigUsingMsg sets the cluster-wide configuration using the `cmn.ConfigToSet` parameter provided.

func SetDaemonConfig 

func SetDaemonConfig(bp BaseParams, nodeID string, nvs cos.StrKVs, transient ...bool) error

SetDaemonConfig, given key value pairs, sets the configuration accordingly for a specific node.

func SetObjectCustomProps 

func SetObjectCustomProps(bp BaseParams, bck cmn.Bck, objName string, custom cos.StrKVs, setNew bool) error

func SetPrimaryProxy 

func SetPrimaryProxy(bp BaseParams, newPrimaryID string, force bool) error

SetPrimaryProxy given a daemonID sets that corresponding proxy as the primary proxy of the cluster.

func ShutdownCluster 

func ShutdownCluster(bp BaseParams) error

ShutdownCluster shuts down the whole cluster

func ShutdownNode 

func ShutdownNode(bp BaseParams, actValue *apc.ActValRmNode) (id string, err error)

ShutdownNode shuts down a specific node

func StartDsort added in v1.3.21 

func StartDsort(bp BaseParams, rs *dsort.RequestSpec) (id string, err error)

func StartMaintenance 

func StartMaintenance(bp BaseParams, actValue *apc.ActValRmNode) (xid string, err error)

func StartXaction 

func StartXaction(bp BaseParams, args *xact.ArgsMsg, extra string) (xid string, err error)

Start xaction

func StopMaintenance 

func StopMaintenance(bp BaseParams, actValue *apc.ActValRmNode) (xid string, err error)

func WaitForXactionIC 

func WaitForXactionIC(bp BaseParams, args *xact.ArgsMsg) (status *nl.Status, err error)

WaitForXactionIC waits for a given xaction to complete. Use it only for global xactions (those that execute on all targets and report their status to IC, e.g. rebalance).

func WaitForXactionIdle 

func WaitForXactionIdle(bp BaseParams, args *xact.ArgsMsg) (err error)

WaitForXactionIdle waits for a given on-demand xaction to be idle.

func WaitForXactionNode 

func WaitForXactionNode(bp BaseParams, args *xact.ArgsMsg, fn func(xact.MultiSnap) (bool, bool)) error

WaitForXactionNode waits for a given xaction to complete. Use for xactions that do _not_ report their status to IC members, namely: - xact.IdlesBeforeFinishing() - x-resilver (as it usually runs on a single node)

Types

type AppendArgs 

type AppendArgs struct {
	Reader     cos.ReadOpenCloser
	BaseParams BaseParams
	Bck        cmn.Bck
	Object     string
	Handle     string
	Size       int64
}

APPEND(object)

type BaseParams 

type BaseParams struct {
	Client *http.Client
	URL    string
	Method string
	Token  string
	UA     string
}

type BinfoArgs added in v1.3.21 

type BinfoArgs struct {
	BsummArgs
	UUID          string
	Prefix        string
	FltPresence   int
	Summarize     bool
	WithRemote    bool
	DontAddRemote bool
}

type BsummArgs added in v1.3.21 

type BsummArgs struct {
	Callback  BsummCB
	CallAfter time.Duration
	DontWait  bool
}

type BsummCB added in v1.3.21 

type BsummCB func(*cmn.AllBsummResults, bool)

type FlushArgs 

type FlushArgs struct {
	Cksum      *cos.Cksum
	BaseParams BaseParams
	Bck        cmn.Bck
	Object     string
	Handle     string
}

APPEND, Archive, Promote (object)

type GetArgs added in v1.3.16 

type GetArgs struct {
	// If not specified (or same: if `nil`), Writer defaults to `io.Discard`
	// (in other words, with no writer the object that is being read will be discarded)
	Writer io.Writer

	// Currently, this optional Query field can (optionally) carry:
	// - `apc.QparamETLName`: named ETL to transform the object (i.e., perform "inline transformation")
	// - `apc.QparamOrigURL`: GET from a vanilla http(s) location (`ht://` bucket with the corresponding `OrigURLBck`)
	// - `apc.QparamSilent`: do not log errors
	// - `apc.QparamLatestVer`: get latest version from the associated Cloud bucket; see also: `ValidateWarmGet`
	// - and also a group of parameters used to read aistore-supported serialized archives ("shards"),
	//   namely:
	//   - `apc.QparamArchpath`
	//   - `apc.QparamArchmime`
	//   - `apc.QparamArchregx`
	//   - `apc.QparamArchmode`
	// - TODO: add `apc.QparamValidateCksum`
	Query url.Values

	// The field is used to facilitate a) range read, and b) blob download
	// E.g. range:
	// * Header.Set(cos.HdrRange, fmt.Sprintf("bytes=%d-%d", fromOffset, toOffset))
	//   For range formatting, see https://www.rfc-editor.org/rfc/rfc7233#section-2.1
	// E.g. blob download:
	// * Header.Set(apc.HdrBlobDownload, "true")
	Header http.Header
}

GET(object)

type GetLogInput 

type GetLogInput struct {
	Writer   io.Writer
	Severity string // one of: {cmn.LogInfo, ...}
	Offset   int64
	All      bool
}

type HeadArgs added in v1.3.24 

type HeadArgs struct {
	FltPresence   int  // `apc.QparamFltPresence`  - in-cluster vs remote; for enumerated values, see api/apc/query
	Silent        bool // `apc.QparamSilent`       - when true, do not log (not-found) error
	LatestVer     bool // `apc.QparamLatestVer`    - check (with remote backend) whether in-cluster version is the latest
	ValidateCksum bool // `apc.QparamValidateCksum`- validate (ie., recompute and check) in-cluster object's checksums
}

optional

type ListArgs added in v1.3.18 

type ListArgs struct {
	Callback  LsoCB
	CallAfter time.Duration
	Header    http.Header // to optimize listing very large buckets, e.g.: Header.Set(apc.HdrInventory, "true")
	Limit     int64
}

additional and optional list-objects args (compare with: GetArgs, PutArgs)

type LsoCB added in v1.3.21 

type LsoCB func(*LsoCounter)

type LsoCounter added in v1.3.21 

type LsoCounter struct {
	// contains filtered or unexported fields
}

func (*LsoCounter) Count added in v1.3.21 

func (ctx *LsoCounter) Count() int

func (*LsoCounter) Elapsed added in v1.3.21 

func (ctx *LsoCounter) Elapsed() time.Duration

func (*LsoCounter) IsFinished added in v1.3.21 

func (ctx *LsoCounter) IsFinished() bool

type ObjAttrs added in v1.3.16 

type ObjAttrs struct {
	// contains filtered or unexported fields
}

`ObjAttrs` represents object attributes and can be further used to retrieve the object's size, checksum, version, and other metadata.

Note that while `GetObject()` and related GET APIs return `ObjAttrs`, `HeadObject()` API returns `cmn.ObjectProps` - a superset.

func GetObject 

func GetObject(bp BaseParams, bck cmn.Bck, objName string, args *GetArgs) (oah ObjAttrs, err error)

func GetObjectWithValidation 

func GetObjectWithValidation(bp BaseParams, bck cmn.Bck, objName string, args *GetArgs) (oah ObjAttrs, err error)

Same as above with checksum validation. Returns `cmn.ErrInvalidCksum` when the expected and actual checksum values are different.

func PutObject 

func PutObject(args *PutArgs) (oah ObjAttrs, err error)

func (*ObjAttrs) Attrs added in v1.3.16 

func (oah *ObjAttrs) Attrs() (out cmn.ObjAttrs)

func (*ObjAttrs) RespHeader added in v1.3.16 

func (oah *ObjAttrs) RespHeader() http.Header

e.g. usage: range read response

func (*ObjAttrs) Size added in v1.3.16 

func (oah *ObjAttrs) Size() int64

type PutApndArchArgs added in v1.3.18 

type PutApndArchArgs struct {
	ArchPath string // filename _in_ archive
	Mime     string // user-specified mime type (NOTE: takes precedence if defined)
	Flags    int64  // apc.ArchAppend and apc.ArchAppendIfExist (the former requires destination shard to exist)
	PutArgs
}

Archive files and directories

type PutArgs added in v1.3.16 

type PutArgs struct {
	Reader cos.ReadOpenCloser

	// optional; if provided:
	// - if object exists: load the object's metadata, compare checksums - skip writing if equal
	// - otherwise, compare the two checksums upon writing (aka, "end-to-end protection")
	Cksum *cos.Cksum

	BaseParams BaseParams

	Bck     cmn.Bck
	ObjName string

	Size uint64 // optional

	// Skip loading existing object's metadata in order to
	// compare its Checksum and update its existing Version (if exists);
	// can be used to reduce PUT latency when:
	// - we massively write a new content into a bucket, and/or
	// - we simply don't care.
	SkipVC bool
}

PUT(object)

type ReqParams 

type ReqParams struct {
	Query      url.Values
	Header     http.Header
	BaseParams BaseParams
	Path       string

	// Authentication
	User     string
	Password string

	// amsg, lsmsg etc.
	Body []byte
	// contains filtered or unexported fields
}

ReqParams is used in constructing client-side API requests to aistore. Stores Query and Headers for providing arguments that are not used commonly in API requests 

See also: cmn.HreqArgs

func AllocRp 

func AllocRp() *ReqParams

func (*ReqParams) DoReqAny added in v1.3.16 

func (reqParams *ReqParams) DoReqAny(out any) (int, error)

Makes request via do(), decodes `resp.Body` into the `out` structure, closes the former, and returns the entire wrapped response (as well as `out`)

Returns an error if the response status >= 400.

func (*ReqParams) DoRequest 

func (reqParams *ReqParams) DoRequest() error

uses do() to make the request; if successful, checks, drains, and closes the response body

Source Files

View all Source files

Directories

Path Synopsis
Package apc: API control messages and constants
 Package apc: API control messages and constants
Package authn provides AuthN API over HTTP(S)
 Package authn provides AuthN API over HTTP(S)
Package env contains environment variables
 Package env contains environment variables

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.