httpfs

Documentation

Overview

Package httpfs provides a (userspace) file system API over http. httpfs is used by mumax3-server to proved file system access to the compute nodes.

The API is similar to go's os package, but both local file names and URLs may be passed. When the file "name" starts with "http://", it is treated as a remote file, otherwise it is local. Hence, the same API is used for local and remote file access.

Index

• Constants
• Variables
• func Append(URL string, p []byte) error
• func AppendSize(URL string, p []byte, size int64) error
• func Log(msg ...interface{})
• func Mkdir(URL string) error
• func MustOpen(URL string) io.ReadCloser
• func Open(URL string) (io.ReadCloser, error)
• func Put(URL string, p []byte) error
• func Read(URL string) ([]byte, error)
• func ReadDir(URL string) ([]string, error)
• func RegisterHandlers()
• func Remove(URL string) error
• func SetWD(dir string)
• func Touch(URL string) error
• type WriteCloseFlusher
  • func Create(URL string) (WriteCloseFlusher, error)
  • func MustCreate(URL string) WriteCloseFlusher

Constants

const (
	DirPerm  = 0777 // permissions for new directory
	FilePerm = 0666 // permissions for new files
)

const (
	APPEND action = "append"
	LS     action = "ls"
	MKDIR  action = "mkdir"
	PUT    action = "put"
	READ   action = "read"
	RM     action = "rm"
	TOUCH  action = "touch"
)

httpfs actions, handled at /actionName/ (e.g. /ls/, /mkdir/, ...)

const BUFSIZE = 16 * 1024 * 1024 // bufio buffer size

Variables

var Logging = false // enables logging

Functions

func Append

func Append(URL string, p []byte) error

Append p to the file given by URL.

func AppendSize

func AppendSize(URL string, p []byte, size int64) error

Append p to the file given by URL, but first assure that the file had the expected size. Used to avoid accidental concurrent writes by two processes to the same file. Size < 0 disables size check.

func Log

func Log(msg ...interface{})

func Mkdir

func Mkdir(URL string) error

Mkdir creates a directory at specified URL.

func MustOpen

func MustOpen(URL string) io.ReadCloser

func Open

func Open(URL string) (io.ReadCloser, error)

open a file for reading

func Put

func Put(URL string, p []byte) error

Create file given by URL and put data from p there.

func Read

func Read(URL string) ([]byte, error)

Read the entire file and return its contents.

func ReadDir

func ReadDir(URL string) ([]string, error)

ReadDir reads and returns all file names in the directory at URL.

func RegisterHandlers

func RegisterHandlers()

RegisterHandlers sets up the http handlers needed for the httpfs protocol (calling go's http.Handle). After RegisterHandlers, http.ListenAndServe may be called.

func Remove

func Remove(URL string) error

Remove removes the file or directory at URL, and all children it may contain. Similar to os.RemoveAll.

func SetWD

func SetWD(dir string)

SetWD sets a "working directory" for the client side, prefixed to all relative local paths passed to client functions (Mkdir, Touch, Remove, ...). dir may start with "http://", turning local relative client paths into remote paths. E.g.:

	http://path -> http://path
	path/file   -> wd/path/file
 /path/file  -> /path/file

func Touch

func Touch(URL string) error

Touch creates an empty file at the specified URL.

Types

type WriteCloseFlusher

type WriteCloseFlusher interface {
	io.WriteCloser
	Flush() error
}

func Create

func Create(URL string) (WriteCloseFlusher, error)

create a file for writing, clobbers previous content if any.

func MustCreate

func MustCreate(URL string) WriteCloseFlusher