operation

package
v1.5.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 16, 2023 License: MIT Imports: 38 Imported by: 1

Documentation

Index

Constants

View Source 
const (
	QINIU_ENV               = "QINIU"
	QINIU_MULTI_CLUSTER_ENV = "QINIU_MULTI_CLUSTER"
)
View Source 
const DefaultPathPrefix = ""
View Source 
const FileError = -3
View Source 
const HostError = -2
View Source 
const PathError = -1
View Source 
const QINIU_DISABLE_CONFIG_HOT_RELOADING_ENV = "QINIU_DISABLE_CONFIG_HOT_RELOADING"

Variables

View Source 
var (
	MaxContinuousFailureTimes    = 5
	MaxContinuousFailureDuration = 1 * time.Minute
	MaxFindHostsPrecent          = 50
)
View Source 
var (
	ErrUndefinedConfig                        = errors.New("undefined config")
	ErrCannotTransferBetweenDifferentClusters = errors.New("cannot transfer between different clusters")
)

Functions

func FileServer 

func FileServer(root FileSystem) http.Handler

FileServer returns a handler that serves HTTP http.Requests with the contents of the file system rooted at root.

To use the operating system's file system implementation, use http.Dir: 

http.Handle("/", http.FileServer(http.Dir("/tmp")))

As a special case, the returned file server redirects any http.Request ending in "/index.html" to the same path, without the final "index.html".

func ServeContent 

func ServeContent(w http.ResponseWriter, req *http.Request, name string, modtime time.Time, content io.ReadSeeker)

ServeContent replies to the http.Request using the content in the provided ReadSeeker. The main benefit of ServeContent over io.Copy is that it handles Range http.Requests properly, sets the MIME type, and handles If-Match, If-Unmodified-Since, If-None-Match, If-Modified-Since, and If-Range http.Requests.

If the response's Content-Type header is not set, ServeContent first tries to deduce the type from name's file extension and, if that fails, falls back to reading the first block of the content and passing it to DetectContentType. The name is otherwise unused; in particular it can be empty and is never sent in the response.

If modtime is not the zero time or Unix epoch, ServeContent includes it in a Last-Modified header in the response. If the http.Request includes an If-Modified-Since header, ServeContent uses modtime to decide whether the content needs to be sent at all.

The content's Seek method must work: ServeContent uses a seek to the end of the content to determine its size.

If the caller has set w's ETag header formatted per RFC 7232, section 2.3, ServeContent uses it to handle http.Requests using If-Match, If-None-Match, or If-Range.

Note that *os.File implements the io.ReadSeeker interface.

func ServeFile 

func ServeFile(w http.ResponseWriter, r *http.Request, name string)

ServeFile replies to the http.Request with the contents of the named file or directory.

If the provided file or directory name is a relative path, it is interpreted relative to the current directory and may ascend to parent directories. If the provided name is constructed from user input, it should be sanitized before calling ServeFile.

As a precaution, ServeFile will reject http.Requests where r.URL.Path contains a ".." path element; this protects against callers who might unsafely use filepath.Join on r.URL.Path without sanitizing it and then use that filepath.Join result as the name argument.

As another special case, ServeFile redirects any http.Request where r.URL.Path ends in "/index.html" to the same path, without the final "index.html". To avoid such redirects either modify the path or use ServeContent.

Outside of those two special cases, ServeFile does not use r.URL.Path for selecting the file or directory to serve; only the file or directory provided in the name argument is used.

func SetCacheDirectoryAndLoad 

func SetCacheDirectoryAndLoad(path string) error

设置查询结果缓存目录

func SetLogger 

func SetLogger(logger kodocli.Ilog)

SetLogger 设置全局 Logger

func StartServer 

func StartServer(cfg *Config) (*http.Server, error)

func StartSimulateErrorServer deprecated 

func StartSimulateErrorServer(_ *Config)

Deprecated:

func WithCurrentConfigurable 

func WithCurrentConfigurable(f func(configurable Configurable))

Types

type ApiServer 

type ApiServer struct {
	// contains filtered or unexported fields
}

func NewApiServer 

func NewApiServer(c *Config) *ApiServer

NewApiServer 根据配置创建 API Server

func NewApiServerV2 

func NewApiServerV2() *ApiServer

NewApiServerV2 根据环境变量创建 API Server

func (*ApiServer) GetLogicalAvailableSizes 

func (svr *ApiServer) GetLogicalAvailableSizes() (map[string]uint64, error)

type Config 

type Config struct {
	UpHosts          []string `json:"up_hosts" toml:"up_hosts"`
	RsHosts          []string `json:"rs_hosts" toml:"rs_hosts"`
	RsfHosts         []string `json:"rsf_hosts" toml:"rsf_hosts"`
	ApiServerHosts   []string `json:"api_server_hosts" toml:"api_server_hosts"`
	Bucket           string   `json:"bucket" toml:"bucket"`
	Ak               string   `json:"ak" toml:"ak"`
	Sk               string   `json:"sk" toml:"sk"`
	PartSize         int64    `json:"part" toml:"part"`
	Addr             string   `json:"addr" toml:"addr"`
	Delete           bool     `json:"delete" toml:"delete"`
	UpConcurrency    int      `json:"up_concurrency" toml:"up_concurrency"`
	BatchConcurrency int      `json:"batch_concurrency" toml:"batch_concurrency"`
	BatchSize        int      `json:"batch_size" toml:"batch_size"`

	DownPath string `json:"down_path" toml:"down_path"`
	Sim      bool   `json:"sim" toml:"sim"`

	IoHosts []string `json:"io_hosts" toml:"io_hosts"`
	UcHosts []string `json:"uc_hosts" toml:"uc_hosts"`

	RecycleBin string `json:"recycle_bin" toml:"recycle_bin"`
	// contains filtered or unexported fields
}

Config 配置文件

func Load 

func Load(file string) (*Config, error)

Load 加载配置文件

type Configurable 

type Configurable interface {
	// contains filtered or unexported methods
}

type CopyKeyInput 

type CopyKeyInput FromToKey

type CopyKeysError 

type CopyKeysError FromToKeyError

type DeleteKeysError 

type DeleteKeysError SingleKeyError

type Dir 

type Dir string

A Dir implements FileSystem using the native file system restricted to a specific directory tree.

While the FileSystem.Open method takes '/'-separated paths, a Dir's string value is a filename on the native file system, not a URL, so it is separated by filepath.Separator, which isn't necessarily '/'.

Note that Dir could expose sensitive files and directories. Dir will follow symlinks pointing out of the directory tree, which can be especially dangerous if serving from a directory in which users are able to create arbitrary symlinks. Dir will also allow access to files and directories starting with a period, which could expose sensitive directories like .git or sensitive files like .htpasswd. To exclude files with a leading period, remove the files/directories from the server or create a custom FileSystem implementation.

An empty Dir is treated as ".".

func (Dir) Open 

func (d Dir) Open(name string) (File, error)

Open implements FileSystem using os.Open, opening files for reading rooted and relative to the directory d.

type Downloader 

type Downloader struct {
	// contains filtered or unexported fields
}

Downloader 下载器

func NewDownloader 

func NewDownloader(c *Config) *Downloader

NewDownloader 根据配置创建下载器

func NewDownloaderV2 

func NewDownloaderV2() *Downloader

NewDownloaderV2 根据环境变量创建下载器

func (*Downloader) DownloadBytes 

func (d *Downloader) DownloadBytes(key string) (data []byte, err error)

DownloadBytes 下载指定对象到内存中

func (*Downloader) DownloadCheck 

func (d *Downloader) DownloadCheck(key string) (l int64, err error)

DownloadCheck 检查文件

func (*Downloader) DownloadCheckList 

func (d *Downloader) DownloadCheckList(ctx context.Context, keys []string) ([]*FileStat, error)

DownloadCheckList 检查多个文件

func (*Downloader) DownloadFile 

func (d *Downloader) DownloadFile(key, path string) (f *os.File, err error)

DownloadFile 下载指定对象到文件里

func (*Downloader) DownloadRangeBytes 

func (d *Downloader) DownloadRangeBytes(key string, offset, size int64) (l int64, data []byte, err error)

DownloadRangeBytes 下载指定对象的指定范围到内存中

func (*Downloader) DownloadRangeReader 

func (d *Downloader) DownloadRangeReader(key string, offset, size int64) (l int64, reader io.ReadCloser, err error)

DownloadRangeReader 下载指定对象的指定范围为Reader

func (*Downloader) DownloadRaw 

func (d *Downloader) DownloadRaw(key string, headers http.Header) (*http.Response, error)

DownloadRaw 使用给定的 HTTP Header 请求下载接口，并直接获得 http.Response 响应

type File 

type File interface {
	io.Closer
	io.Reader
	io.Seeker
	Readdir(count int) ([]os.FileInfo, error)
	Stat() (os.FileInfo, error)
}

A File is returned by a FileSystem's Open method and can be served by the FileServer implementation.

The methods should behave the same as those on an *os.File.

type FileStat 

type FileStat struct {
	Name string `json:"name"`
	Size int64  `json:"size"`
	// contains filtered or unexported fields
}

FileStat 文件元信息

type FileSystem 

type FileSystem interface {
	Open(name string) (File, error)
}

A FileSystem implements access to a collection of named files. The elements in a file path are separated by slash ('/', U+002F) characters, regardless of host operating system convention.

type FromToKey 

type FromToKey struct {
	FromKey string
	ToKey   string
}

type FromToKeyError 

type FromToKeyError struct {
	Error   string
	Code    int
	FromKey string
	ToKey   string
}

type Lister 

type Lister struct {
	// contains filtered or unexported fields
}

Lister 列举器

func NewLister 

func NewLister(c *Config) *Lister

NewLister 根据配置创建列举器

func NewListerV2 

func NewListerV2() *Lister

NewListerV2 根据环境变量创建列举器

func (*Lister) Copy 

func (l *Lister) Copy(fromKey, toKey string) error

Copy 复制对象到当前存储空间的指定对象中

func (*Lister) CopyDirectory 

func (l *Lister) CopyDirectory(srcDir, destDir string) (copyErrors []CopyKeysError, err error)

CopyDirectory 目录级别的Copy操作

func (*Lister) CopyKeys 

func (l *Lister) CopyKeys(input []CopyKeyInput) ([]*CopyKeysError, error)

CopyKeys 批量复制

func (*Lister) Delete 

func (l *Lister) Delete(key string) error

Delete 删除指定对象，如果配置了回收站，该 API 将会将文件移动到回收站中，而不做实际的删除

func (*Lister) DeleteDirectory 

func (l *Lister) DeleteDirectory(dir string) (deleteErrors []DeleteKeysError, err error)

DeleteDirectory 目录级别的Delete操作

func (*Lister) DeleteKeys 

func (l *Lister) DeleteKeys(keys []string) ([]*DeleteKeysError, error)

DeleteKeys 删除多个对象，如果配置了回收站，该 API 将会将文件移动到回收站中，而不做实际的删除

func (*Lister) ForceDelete 

func (l *Lister) ForceDelete(key string) error

ForceDelete 强制删除指定对象，无论是否配置回收站，该 API 都会直接删除文件

func (*Lister) ForceDeleteDirectory 

func (l *Lister) ForceDeleteDirectory(dir string) (deleteErrors []DeleteKeysError, err error)

ForceDeleteDirectory 目录级别的强制Delete操作

func (*Lister) ForceDeleteKeys 

func (l *Lister) ForceDeleteKeys(keys []string) ([]*DeleteKeysError, error)

ForceDeleteKeys 强制删除多个对象，无论是否配置回收站，该 API 都会直接删除文件

func (*Lister) ListPrefix 

func (l *Lister) ListPrefix(prefix string) []string

ListPrefix 根据前缀列举存储空间

func (*Lister) ListStat 

func (l *Lister) ListStat(keys []string) []*FileStat

ListStat 获取指定对象列表的元信息

func (*Lister) MoveDirectoryTo 

func (l *Lister) MoveDirectoryTo(srcDir, destDir, toBucket string) (moveErrors []MoveKeysError, err error)

MoveDirectoryTo 目录级别的Move操作

func (*Lister) MoveKeys 

func (l *Lister) MoveKeys(input []MoveKeyInput) ([]*MoveKeysError, error)

MoveKeys 批量移动

func (*Lister) MoveTo 

func (l *Lister) MoveTo(fromKey, toBucket, toKey string) error

MoveTo 移动对象到指定存储空间的指定对象中

func (*Lister) Rename 

func (l *Lister) Rename(fromKey, toKey string) error

Rename 重命名对象

func (*Lister) RenameDirectory 

func (l *Lister) RenameDirectory(srcDir, destDir string) (renameErrors []RenameKeysError, err error)

RenameDirectory 目录级别的Rename操作

func (*Lister) RenameKeys 

func (l *Lister) RenameKeys(input []RenameKeyInput) ([]*RenameKeysError, error)

RenameKeys 批量重命名

type MoveKeyInput 

type MoveKeyInput struct {
	FromToKey
	ToBucket string
}

type MoveKeysError 

type MoveKeysError struct {
	FromToKeyError
	ToBucket string
}

type MultiClustersConfig 

type MultiClustersConfig struct {
	// contains filtered or unexported fields
}

MultiClustersConfig 多集群配置文件

func LoadMultiClusterConfigs 

func LoadMultiClusterConfigs(file string) (*MultiClustersConfig, error)

LoadMultiClusterConfigs 加载多集群配置文件

func (*MultiClustersConfig) SetConfigSelectCallback 

func (config *MultiClustersConfig) SetConfigSelectCallback(f func(configs map[string]*Config, key string) (*Config, bool))

SetConfigSelectCallback 设置一个配置选取策略回调

type Queryer 

type Queryer struct {
	// contains filtered or unexported fields
}

Queryer 域名查询器

func NewQueryer 

func NewQueryer(c *Config) *Queryer

NewQueryer 根据配置创建域名查询器

func (*Queryer) QueryApiServerHosts 

func (queryer *Queryer) QueryApiServerHosts(https bool) (urls []string)

QueryApiServerHosts 查询 APISERVER 服务器 URL

func (*Queryer) QueryIoHosts 

func (queryer *Queryer) QueryIoHosts(https bool) (urls []string)

QueryIoHosts 查询 IO 服务器 URL

func (*Queryer) QueryRsHosts 

func (queryer *Queryer) QueryRsHosts(https bool) (urls []string)

QueryRsHosts 查询 RS 服务器 URL

func (*Queryer) QueryRsfHosts 

func (queryer *Queryer) QueryRsfHosts(https bool) (urls []string)

QueryRsfHosts 查询 RSF 服务器 URL

func (*Queryer) QueryUpHosts 

func (queryer *Queryer) QueryUpHosts(https bool) (urls []string)

QueryUpHosts 查询 UP 服务器 URL

type RenameKeyInput 

type RenameKeyInput FromToKey

type RenameKeysError 

type RenameKeysError FromToKeyError

type Req 

type Req struct {
	Path   string `json:"path"`
	Key    string `json:"key"`
	Delete *bool  `json:"del"`
}

type SingleClusterConfig 

type SingleClusterConfig = Config

SingleClusterConfig 单集群配置文件

type SingleKeyError 

type SingleKeyError struct {
	Error string
	Code  int
	Name  string
}

type Uploader 

type Uploader struct {
	// contains filtered or unexported fields
}

Uploader 上传器

func NewUploader 

func NewUploader(c *Config) *Uploader

NewUploader 根据配置创建上传器

func NewUploaderV2 

func NewUploaderV2() *Uploader

NewUploaderV2 根据环境变量创建上传器

func (*Uploader) Upload 

func (p *Uploader) Upload(file string, key string) (err error)

Upload 上传指定文件到指定对象中

func (*Uploader) UploadData 

func (p *Uploader) UploadData(data []byte, key string) (err error)

UploadData 上传内存数据到指定对象中

func (*Uploader) UploadDataReader 

func (p *Uploader) UploadDataReader(data io.ReaderAt, size int, key string) (err error)

UploadDataReader 从 Reader 中阅读指定大小的数据并上传到指定对象中

func (*Uploader) UploadReader 

func (p *Uploader) UploadReader(reader io.Reader, key string) (err error)

UploadReader 从 Reader 中阅读全部数据并上传到指定对象中

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.