Documentation ¶
Overview ¶
Package server configures and launches http/rpc server and storage engines specific to the type of DVID platform: local (e.g., running on MacBook Pro), clustered, or using cloud-based services like Google Cloud.
Datatypes can use any of the three tiers of storage (MetaData, Mutable, Immutable) that provide a layer of storage semantics (latency, mutability, etc) on top of underlying storage engines.
The DVID web client is also managed from this package. For a DVID web console, see the repo:
https://github.com/janelia-flyem/dvid-console
The goal of a DVID web console is to provide a GUI for monitoring and performing a subset of operations in a nicely formatted view.
DVID command line interaction occurs via the rpc interface to a running server. Please see the main DVID documentation:
Index ¶
- Constants
- Variables
- func About() string
- func AboutJSON() (jsonStr string, err error)
- func BadAPIRequest(w http.ResponseWriter, r *http.Request, d dvid.Data)
- func BadRequest(w http.ResponseWriter, r *http.Request, format interface{}, ...)
- func BlockOnInteractiveRequests(caller ...string)
- func CacheSize(id string) int
- func CloseTest()
- func CreateTestInstance(t *testing.T, uuid dvid.UUID, typename, name string, config dvid.Config)
- func CreateTestReplaceSync(t *testing.T, uuid dvid.UUID, name string, syncs ...string)
- func CreateTestSync(t *testing.T, uuid dvid.UUID, name string, syncs ...string)
- func DecodeJSON(r *http.Request) (dvid.Config, error)
- func GitVersion() string
- func GotInteractiveRequest()
- func LoadConfig(filename string) (*tomlConfig, *storage.Backend, error)
- func OpenTest(configs ...TestConfig) error
- func SendRPC(addr string, req datastore.Request) error
- func Serve()
- func ServeSingleHTTP(w http.ResponseWriter, r *http.Request)
- func SetFullWrite(on bool)
- func SetReadOnly(on bool)
- func Shutdown()
- func TestBadHTTP(t *testing.T, method, urlStr string, payload io.Reader)
- func TestHTTP(t *testing.T, method, urlStr string, payload io.Reader) []byte
- func TestHTTPError(t *testing.T, method, urlStr string, payload io.Reader) ([]byte, error)
- func TestHTTPResponse(t *testing.T, method, urlStr string, payload io.Reader) *httptest.ResponseRecorder
- func ThrottledHTTP(w http.ResponseWriter) bool
- func ThrottledOpDone()
- type Config
- type ServerConfig
- type TestConfig
Constants ¶
const ( // DefaultWebAddress is the default URL of the DVID web server DefaultWebAddress = "localhost:8000" // DefaultRPCAddress is the default RPC address for command-line use of a remote DVID server DefaultRPCAddress = "localhost:8001" // ErrorLogFilename is the name of the server error log, stored in the datastore directory. ErrorLogFilename = "dvid-errors.log" )
const ( // WebAPIVersion is the string version of the API. Once DVID is somewhat stable, // this will be "v1/", "v2/", etc. WebAPIVersion = "" // WebAPIPath is the relative URL path to our Level 2 REST API WebAPIPath = "/api/" + WebAPIVersion // WriteTimeout is the maximum time in seconds DVID will wait to write data down HTTP connection. WriteTimeout = 600 * time.Second // ReadTimeout is the maximum time in seconds DVID will wait to read data from HTTP connection. ReadTimeout = 600 * time.Second )
Variables ¶
var ( // MaxDataRequest sets the limit on the amount of data that could be returned for a request MaxDataRequest = int64(3) * dvid.Giga // InteractiveOpsPer2Min gives the number of interactive-level requests // received over the last 2 minutes. This is useful for throttling "batch" // operations on a single DVID server. Note that this metric is an lower // bound on the number of interactive requests over the last minute since // we do non-blocking reports. InteractiveOpsPer2Min int // MaxInteractiveOpsBeforeBlock specifies the number of interactive requests // per minute that are allowed before batch-like computation (e.g., loading // of voxel volumes) is blocked. MaxInteractiveOpsBeforeBlock = 3 // ActiveHandlers is maximum number of active handlers over last second. ActiveHandlers int // MaxChunkHandlers sets the maximum number of chunk handlers (goroutines) that // can be multiplexed onto available cores. (See -numcpu setting in dvid.go) MaxChunkHandlers = runtime.NumCPU() // HandlerToken is buffered channel to limit spawning of goroutines. // See ProcessChunk() in datatype/imageblk for example. HandlerToken = make(chan int, MaxChunkHandlers) // LargeMutationMutex is a global lock for compute-intense processes that want to // spawn goroutines that consume handler tokens. This lets processes capture most // if not all available handler tokens in a FIFO basis rather than have multiple // concurrent requests launch a few goroutines each. LargeMutationMutex sync.Mutex // TimeoutSecs specifies the seconds waiting to open a datastore for exclusive access. TimeoutSecs int )
var (
// DefaultHost is the default most understandable alias for this server.
DefaultHost = "localhost"
)
Functions ¶
func About ¶
func About() string
About returns a chart of version identifiers for the DVID source code, DVID datastore, and all component data types for this executable.
func BadAPIRequest ¶ added in v0.8.3
BadAPIRequest writes a standard error message to http.ResponseWriter for a badly formatted API call.
func BadRequest ¶
func BadRequest(w http.ResponseWriter, r *http.Request, format interface{}, args ...interface{})
BadRequest writes an error message out to the http.ResponseWriter using format similar to fmt.Printf.
func BlockOnInteractiveRequests ¶
func BlockOnInteractiveRequests(caller ...string)
BlockOnInteractiveRequests will block this goroutine until the number of interactive requests dips below MaxInteractiveOpsBeforeBlock.
func CacheSize ¶ added in v0.8.17
CacheSize returns the number oF bytes reserved for the given identifier. If unset, will return 0.
func CreateTestInstance ¶
CreateTestInstance posts a new data instance for testing.
func CreateTestReplaceSync ¶ added in v0.8.5
CreateTestReplaceSync replaces syncs for a given data instance / uuid.
func CreateTestSync ¶ added in v0.8.0
CreateTestSync posts new syncs for a given data instance / uuid.
func DecodeJSON ¶
DecodeJSON decodes JSON passed in a request into a dvid.Config.
func GitVersion ¶
func GitVersion() string
GitVersion returns a git-derived string that allows recovery of the exact source code used for this DVID server.
func GotInteractiveRequest ¶
func GotInteractiveRequest()
GotInteractiveRequest can be called to track the # of interactive requests that require some amount of computation. Don't use this to track simple polling APIs. This routine will not block.
func LoadConfig ¶
LoadConfig loads DVID server configuration from a TOML file.
func OpenTest ¶ added in v0.8.17
func OpenTest(configs ...TestConfig) error
OpenTest initializes the server for testing, setting up caching, datastore, etc. Later configurations will override earlier ones.
func ServeSingleHTTP ¶
func ServeSingleHTTP(w http.ResponseWriter, r *http.Request)
ServeSingleHTTP fulfills one request using the default web Mux.
func SetFullWrite ¶ added in v0.8.7
func SetFullWrite(on bool)
SetFullWrite allows mutations on any version.
func Shutdown ¶
func Shutdown()
Shutdown handles graceful cleanup of server functions before exiting DVID. This may not be so graceful if the chunk handler uses cgo since the interrupt may be caught during cgo execution.
func TestBadHTTP ¶
TestBadHTTP expects a HTTP response with an error status code.
func TestHTTP ¶
TestHTTP returns the response body bytes for a test request, making sure any response has status OK.
func TestHTTPError ¶ added in v0.8.17
TestHTTPError returns the response body bytes for a test request, making sure any response has status OK.
func TestHTTPResponse ¶
func TestHTTPResponse(t *testing.T, method, urlStr string, payload io.Reader) *httptest.ResponseRecorder
TestHTTPResponse returns a response from a test run of the DVID server. Use TestHTTP if you just want the response body bytes.
func ThrottledHTTP ¶ added in v0.8.0
func ThrottledHTTP(w http.ResponseWriter) bool
ThrottledHTTP checks if a request can continue under throttling. If so, it returns false. If it cannot (throttled state), it sends a http.StatusServiceUnavailable and returns true. Throttling is controlled by
func ThrottledOpDone ¶ added in v0.8.0
func ThrottledOpDone()
ThrottledOpDone marks the end of a throttled operation, allowing another op blocked by ThrottledHTTP() to succeed.
Types ¶
type Config ¶
type Config interface { Host() string // Abbreviated host identifier + port. Note() string HTTPAddress() string RPCAddress() string // Path to web client files WebClient() string // Set timing in HTTP header AllowTiming() bool // Kafka servers, nil if not configured KafkaServers() []string }
Config provides server configuration parameters. Should be set by platform-specific implementations.
type ServerConfig ¶ added in v0.8.18
type ServerConfig struct { Host string HTTPAddress string RPCAddress string WebClient string Note string AllowTiming bool // If true, returns * for Timing-Allow-Origin in response headers. StartWebhook string // http address that should be called when server is started up. IIDGen string `toml:"instance_id_gen"` IIDStart uint32 `toml:"instance_id_start"` }
ServerConfig holds ports, host name, and other properties of this dvid server.
func (ServerConfig) DatastoreInstanceConfig ¶ added in v0.8.18
func (sc ServerConfig) DatastoreInstanceConfig() datastore.InstanceConfig
DatastoreInstanceConfig returns data instance configuration necessary to handle id generation.
func (ServerConfig) Initialize ¶ added in v0.8.18
func (sc ServerConfig) Initialize() error
Initialize POSTs data to any set webhook indicating the server configuration.