Documentation
¶
Overview ¶
Package request provides functionality to create an manage individual HTTP requests. Requests are serialized using Flatbuffers (See request.fbs for schema)
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ReadFull ¶
ReadFull is similar in functionality and uses stdlib's io.ReadFull (in that in tries to fill the buffer) but when an EOF is encountered it can optionally sleep & wait for data (if `waitForData` is true)
Think of the behavior akin to unix `tail -f`. This behavior only applies when `waitForData=true` waitForData is to allow reading from a file that has not yet been flushed completely from another process or thread.
`waitForData` is to be used in special circumstances when archive file is concurrently being filled from another process. WARNING: If the underlying the io.Reader is decompressor, it may misbehave in this kind of usage. Care must be taken to use this option only with plain text archive files.
Types ¶
type MarshalledRequest ¶
type MarshalledRequest struct {
// contains filtered or unexported fields
}
func CreateRequest ¶
func CreateRequest( id, method, uri, headers, body []byte) (mr *MarshalledRequest)
CreateRequest returns *MarshalledRequest ready to be saved A *MarshalledRequest contains pointers from a buffer pool. You must call `.Release()` on it as soon as you are done with it.
func CreateRequestFromFastHTTPCtx ¶
func CreateRequestFromFastHTTPCtx(ctx *fasthttp.RequestCtx) (mr *MarshalledRequest)
CreateRequestFromFastHTTPCtx returns *MarshalledRequest ready to be saved created from a *fasthttp.RequestCtx . This coupling with fasthttp format is to avoid unnecessary copying to other generic formats. A *MarshalledRequest contains pointers from a buffer pool. You must call `.Release()` on it as soon as you are done with it.
func (*MarshalledRequest) Bytes ¶
func (mr *MarshalledRequest) Bytes() []byte
Bytes returns underlying buffer. This is exposed *only* to be passed to an io.Writer TODO: Find a better way to encapsulate this
func (*MarshalledRequest) Release ¶
func (mr *MarshalledRequest) Release()
Release releases the object back to the pool
func (*MarshalledRequest) SaveRequest ¶
func (req *MarshalledRequest) SaveRequest(rf archive.Archive, flushNow bool) (err error)
SaveRequest saves the data held by MarshalledRequest to the archive file. MarshalledRequest typically holds a flatbuffer builder that is already
type UnmarshalledRequest ¶
type UnmarshalledRequest struct {
// contains filtered or unexported fields
}
UnmarshalledRequest holds an fbr.Request that has just been read from storage/archive. For flatbuffers, there isn't really an Unmarshal step. However I am using the terminology to indicate an object that has just been read from storage/archive. It needs a buffer that holds the actual DS. Compare this to MarshalledRequest where we need a flatbuffers builder instead MarshalledRequest is for writing, UnmarshalledRequest is for reading
func CreateUMRequest ¶
func CreateUMRequest() (umr *UnmarshalledRequest)
CreateUMRequest creates an UnmarshalledRequest object (No data in it yet) Used by GetNextRequest.
func GetNextRequest ¶
func GetNextRequest(rf archive.Archive, waitForData bool) (umr *UnmarshalledRequest, err error)
GetNextRequest reads an archived request from a stream (io.Reader), allocates a buffer from a Pool and returns a lease to that buffer. Caller must do the following
- Call umr, err := GetNextRequest() for the next item.
- Handle io.EOF properly as not an error, but end of processing.
- req := umr.Request() <- Use the flatbuff generated types and code
- Call umr.Release() with the original slice to free
Why do we return a `*UnmarshalledRequest` instead of a `fbr.Request` object?
Buffer pool needs to be managed at the level of `UnmarshalledRequest` to save allocations If `fbr.Request` is returned instead of the wrapper object, it may not work. Caller can't use a pointer to `fbr.Request` and return the original buffer to the pool. More research is needed to identity if the intermediate object `*UnmarshalledRequest` can be removed. Anyways this intermediate object does not cause inefficiency, except for the wierdness in the API
func (*UnmarshalledRequest) Bytes ¶
func (umr *UnmarshalledRequest) Bytes() []byte
Bytes returns underlying buffer. This is exposed *only* to be passed to readFull / io.ReadFull TODO: Find a better way to encapsulate this
func (*UnmarshalledRequest) Grow ¶
func (umr *UnmarshalledRequest) Grow(size int)
Grow can be used to grow the underlying buffer. This is required to for later use with readFull / io.ReadFull
func (*UnmarshalledRequest) Release ¶
func (umr *UnmarshalledRequest) Release()
Release releases the object back to the pool
func (*UnmarshalledRequest) Request ¶
func (umr *UnmarshalledRequest) Request() *fbr.Request
Request returns a pointer to the fbr.Request represented by the UnmarshalledRequest
Source Files