requests

package module
v0.0.0-...-3f59def Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 24, 2014 License: Unlicense Imports: 14 Imported by: 0

README

Build Status Coverage Status

requests

A request unmarshaller for web servers using the Go language.

Goals

  1. Use Content-Type to load request bodies.
  2. Provide methods for attempting to make all data loaded from a body function roughly the same (e.g. if only one value was provided for an input, store just the value instead of []string{value}, to make application/x-www-urlencoded and application/json work a little more similarly).
  3. Keep track of all input errors and return them in an easy to parse way, for providing details to the user about what went wrong.
  4. Provide functionality similar to json's Unmarshal process for generic web requests, regardless of Content-Type.

State

The actual logic within this repository has been in production use with a few projects for a while, now. However, this project is a massive refactor of that logic, and I haven't got any test coverage for it yet. You may want to play around with it, but I would suggest waiting until there is at least 80% test coverage before using it actively in production, yourself.

Summary

You will likely want to read the package documentation to get the most out of this package. However, the absolute basics are:

import "github.com/go-requests/requests"

type User struct {
	Name string `request:"username,required"`
    Pass password `request:"password,required"`
    Blurb string `request:"about_me"`
}

func HandleRequest(httpRequest *http.Request) error {
	target := new(User)
	return requests.New(httpRequest).Unmarshal(target)
}

The request body will be loaded into a map of parameters and then values from the request will be applied to fields on the target user. If either username or password are missing from the request, an error will be returned.

Installing

For projects with solid unit testing, or projects intending to follow continuous integration, I recommend getting directly from github:

go get github.com/nelsam/requests

However, for projects that need more assurances that nothing will ever change, this project does support versioning through gopkg.in (using a fork at github.com/go-requests/requests):

go get gopkg.in/requests.v0

Check the project tags for version updates. v1 will be released when test coverage exceeds 80%.

Documentation

Overview

The requests package contains logic for loading and unmarshalling data contained within web requests. The most common uses for this library are as follows: 

params, err := requests.New(request).Params()

err := requests.New(request).Unmarshal(structPtr)

Parameters will be loaded from the request body based on the request's Content-Type header. Some attempts are made to unify data structure, to make it easier to treat all requests the same (regardless of Content-Type).

For the Unmarshal process, the requests package uses a combination of reflection (to check field tags) and interfaces to figure out which values (from the above params) should be applied to which fields in the target struct. Unmarshalling to non-struct types is not supported.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AddCodec 

func AddCodec(codec codecs.Codec)

AddCodec adds a "github.com/stretchr/codecs".Codec to the "github.com/stretchr/codecs/services".CodecService currently in use by this library.

func AddFallbackTag 

func AddFallbackTag(newTag string)

AddFallbackTag adds a tag name to the list of fallback tag names to use when locating a field name. For example, if you use the "db" tag and find yourself duplicating the name in both the "db" and "request" tags as follows: 

type Example struct {
    User `db:"user_id" request:"user_id"`
}

Then you could update your code to the following: 

func init() {
    AddFallbackTag("db")
}

type Example struct {
    User `db:"user_id"`
}

Fallback tags will only be added once - duplicates will be ignored. Fallbacks will be used in the order that they are added, so if you add "db" before "response" (as an example), the "db" tag will be preferred over the "response" tag as a fallback.

In all cases, the "request" tag will be preferred over anything else. However, an empty name (but non-empty options) will cause the fallbacks to be used, as in the following example: 

func init() {
    AddFallbackTag("db")
}

type Example struct {
    // Use the name from the DB, but add the 'required' option
    // for requests.
    User `db:"user_id" request:",required"`
}

func ApplyOptions 

func ApplyOptions(field reflect.StructField, orig, input interface{}) (value interface{}, optionErr error)

ApplyOptions will attempt to apply any OptionFunc values registered with RegisterOption to a struct field. If any of the OptionFuncs return an error, the process will immediately return a nil value and the returned error.

func Codecs 

func Codecs() services.CodecService

Codecs returns the "github.com/stretchr/codecs/services".CodecService currently in use by this library.

func FallbackTags 

func FallbackTags() []string

FallbackTags returns a list of tag names that are used as fallbacks to locate field names. See AddFallbackTag for more details.

func MultipartMem 

func MultipartMem() int64

MultipartMem returns the current memory limit for multipart form data.

func OptionDefaults 

func OptionDefaults() map[string]string

OptionDefaults returns a list of default values for options. See SetOptionDefault for more details.

func ParseBody 

func ParseBody(request *http.Request) (interface{}, error)

ParseBody locates a codec matching the request's Content-Type header, then Unmarshals the request's body to an interface{} type. The resulting type is unpredictable and will be heavily based on the actual data in the request.

There are two exceptions to the above, where no codec lookup is used:

* application/x-www-form-urlencoded (or an empty Content-Type)

** The return value for this type will be the same as "net/http".Request.PostForm after calling ParseForm.

* multipart/form-data

** The return value for this type will be the same as "net/http".Request.MultipartForm after calling ParseMultipartForm.

func ParseParams 

func ParseParams(request *http.Request) (map[string]interface{}, error)

ParseParams returns a map[string]interface{} of values found in a request body. In most cases, this is the equivalent of ParseBody(request).(map[string]interface{}). However, there are two exceptions:

* application/x-www-form-urlencoded (or an empty Content-Type)

** Each value in request.PostForm that has a len() of 1 will be stored instead as the zeroeth index of the value.

* multipart/form-data

** In addition to the above, files will be stored at the same level as values. Each value in the resulting map could contain both string and *"mime/multipart".FileHeader values.

The resulting code to parse a form may look like the following: 

params, err := ParseParams(request)
// handle err
for key, value := range params {
    switch v := value.(type) {
    case string:
        // Do stuff with single string value
    case *multipart.FileHeader:
        // Do stuff with single file
    case []interface{}:
        // There were multiple string and/or file values at
        // this key, so deal with that.
    }
}

func RegisterOption 

func RegisterOption(name string, optionFunc OptionFunc) error

RegisterOption can be used to register functions that should be called for struct fields with matching option strings. For example: 

RegisterOption("allow-empty", func(value interface{}, optionValue string) (interface{}, error) {
    if optionValue == "false" {
        if value == nil || value.(string) == "" {
            return nil, errors.New("Cannot be empty")
        }
    }
    return value, nil
})

type Example struct {
    Name `request:",allow-empty=false"`
}

Any options without a value (e.g. request:",non-empty") will have their value set to "true".

An error will be returned if an OptionFunc is already registered for the provided name.

func SetCodecs 

func SetCodecs(newService services.CodecService)

SetCodecs can be used to change the "github.com/stretchr/codecs/services".CodecService used by this library.

func SetMultipartMem 

func SetMultipartMem(mem int64)

SetMultipartMem sets the memory limit for multipart form data.

func SetOptionDefault 

func SetOptionDefault(option, value string)

SetOptionDefault sets an option to default to a value. By default, OptionFuncs that are excluded from a field's options are never called, but if you set a default for that option, it will always be called with the default value if the option is not provided on the struct field. An example: 

func init() {
    SetOptionDefault("required", "true")
}

type Example struct {
    // user_id will be required in the request
    User `db:user_id`

    // description will be optional in the request
    Description `request:",required=false"`
}

Types

type ChangeReceiver 

type ChangeReceiver interface {
	// Receive takes a value and performs the same logic as
	// Receiver.Receive, but returns whether or not the new value is
	// different from the old value, as well as any errors encountered.
	Receive(interface{}) (valueChanged bool, err error)
}

A ChangeReceiver is a receiver that, in addition to performing its own logic for receiving input values, also returns whether or not the passed in value was different from the existing value.

This is used primarily for immutable option checking. Struct fields of type Receiver cannot support the "immutable" option, so types which are used in struct fields that need the "immutable" option should implement ChangeReceiver, instead.

Note that this will *not* be used if the current value of the field is equal to the empty value of the field - only if the field is set to a non-empty value will the immutable option care about the ChangeReceiver interface.

type Defaulter 

type Defaulter interface {
	// DefaultValue should return the default value of this type.
	DefaultValue() interface{}
}

Defaulter is a type that returns its own default value, for when it is not included in the request. This can be used as an alternative to the "default=something" tag option.

type InputErrors 

type InputErrors map[string]error

InputErrors is an error type that maps input names to errors encountered while parsing their value. A nil error will be stored for any input names that were parsed successfully.

func (InputErrors) Error 

func (errs InputErrors) Error() string

Error returns the InputError's full error string.

func (InputErrors) Errors 

func (errs InputErrors) Errors() InputErrors

Errors returns a clone of errs with all nil error indexes removed. If there are no non-nil errors, this method will return nil.

func (InputErrors) HasErrors 

func (errs InputErrors) HasErrors() bool

HasErrors returns whether or not any of the errors in errs are non-nil.

func (InputErrors) Merge 

func (errs InputErrors) Merge(newErrs InputErrors) InputErrors

Merge merges errs and newErrs, returning the resulting map. If errs is nil, nothing will be done and newErrs will be returned. If errs is non-nil, all keys and values in newErrs will be added to errs, and you can safely ignore the return value.

func (InputErrors) Set 

func (errs InputErrors) Set(input string, err error) bool

Set executes errs[input] = err. Returns true if err is non-nil, false otherwise.

type OptionFunc 

type OptionFunc func(originalValue, requestValue interface{}, optionValue string) (convertedValue interface{}, err error)

An OptionFunc is a function which takes a field's original value, new value (from a request), and a string of option values (anything to the right of the = sign in a field's tag option), and returns the final new value (parsed from the request value) and any errors encountered.

type PostReceiver 

type PostReceiver interface {
	// PostReceive performs final tasks subsequent to receiving a
	// value from input.
	PostReceive() error
}

A PostReceiver has an action to perform subsequent to receiving data from a user request.

type PostUnmarshaller 

type PostUnmarshaller interface {
	PostUnmarshal() error
}

A PostUnmarshaller is a type that performs certain actions subsequent to having data unmarshalled to it.

type PreReceiver 

type PreReceiver interface {
	// PreReceive performs initial tasks prior to receiving a value
	// from input.
	PreReceive() error
}

A PreReceiver has an action to perform prior to receiving data from a user request.

type PreUnmarshaller 

type PreUnmarshaller interface {
	PreUnmarshal() error
}

A PreUnmarshaller is a type that performs certain actions prior to having data unmarshalled to it.

type ReceiveTyper 

type ReceiveTyper interface {
	Receiver

	// ReceiveType should return a value (preferably empty) of the same
	// type as the ReceiveTyper's Receive method expects.  Any value
	// in a request destined to be an argument for the ReceiveTyper's
	// Receive method will first be converted to the same type as the
	// value returned by ReceiveType.
	ReceiveType() interface{}
}

A ReceiveTyper has methods for returning the type that its Receive method expects. It *must* implement Receiver as well, otherwise its ReceiveType method will be useless. If it does, then request data destined for the ReceiveTyper will be converted (if possible) to the same type as the return value of its ReceiveType method.

type Receiver 

type Receiver interface {
	// Receive takes a value and attempts to read it in to the
	// underlying type.  It should return an error if the passed in
	// value cannot be parsed to the underlying type.
	Receive(interface{}) error
}

A Receiver is a type that receives a value from a request and performs its own logic to apply the input value to itself.

Example: 

type Password string

func (pass *Password) Receive(rawPassword interface{}) error {
    *pass = hash(rawPassword.(string))
}

type Request 

type Request struct {
	// contains filtered or unexported fields
}

A Request is a type that stores data about an HTTP request and contains methods for reading that request's body.

func New 

func New(request *http.Request) *Request

New creates a new *Request based on the request parameter.

func (*Request) Body 

func (request *Request) Body() (interface{}, error)

Body returns the result of ParseBody for this request. ParseBody will only be called the first time Body is called; subsequent calls will return the same value as the first call.

func (*Request) Params 

func (request *Request) Params() (map[string]interface{}, error)

Params returns the result of ParseParams for this request. ParseParams will only be called the first time Params is called; subsequent calls will return the same value as the first call.

func (*Request) QueryParams 

func (request *Request) QueryParams() map[string]interface{}

QueryParams performs the same conversion for query parameters in the request's URL as Params does for body parameters (i.e. converting []string entries with a len() of 1 to string values). The intent is to allow you to treat parameters in GET requests the same way you treat parameters in POST or PATCH requests.

func (*Request) Unmarshal 

func (request *Request) Unmarshal(target interface{}) error

Unmarshal unmarshals a request to a struct, using field tags to locate corresponding values in the request and check/parse them before assigning them to struct fields. It acts similar to json's Unmarshal when used on a struct, but works with any codec registered with AddCodec().

Field tags are used as follows:

* All field tags are considered to be of the format name,option1,option2,...

* Options will *only* be parsed from the "request" tag.

* By default, name will only be checked in the "request" tag, but you can add fallback tag names using AddFallbackTag.

* If no non-empty name is found using field tags, the lowercase field name will be used instead.

* Once a name is found, if the name is "-", then the field will be treated as if it does not exist.

For an explanation on how options work, see the documentation for RegisterOption. For a list of tag options built in to this library, see the options package in this package.

Fields which have no data in the request will be left as their current value. They will still be passed through the option parser for the purposes of options like "required".

Fields which implement Receiver will have their Receive method called using the value from the request after calling all OptionFuncs matching the field's tag options.

An error will be returned if the target type is not a pointer to a struct, or if the target implements PreUnmarshaller, Unmarshaller, or PostUnmarshaller and the corresponding methods fail. An UnusedFields error will be returned if fields in the request had no corresponding fields on the target struct.

Panics from an Unmarshaller's Unmarshal method will be recovered and returned as error types. The error will be the same error as returned by request.Params() if the error returned from request.Params() is of type *services.ContentTypeNotSupportedError, or a generic error with the recover output otherwise.

Any errors encountered while attempting to apply input values to the target's fields will be stored in an error of type InputErrors. At the end of the Unmarshal process, the InputErrors error will be returned if any errors were encountered.

A simple example: 

type Example struct {
    Foo string `request:",required"`
    Bar string `response:"baz"`
    Baz string `response:"-"`
    Bacon string `response:"-" request:"bacon,required"`
}

func CreateExample(request *http.Request) (*Example, error) {
    target := new(Example)
    if err := requests.New(request).Unmarshal(target); err != nil {
        if inputErrs, ok := err.(InputErrors); ok {
            // inputErrs is a map of input names to error
            // messages, so send them to a function to turn
            // them into a proper user-friendly error message.
            return nil, userErrors(inputErrs)
        }
        return nil, err
    }
    return target, nil
}

func (*Request) UnmarshalReplace 

func (request *Request) UnmarshalReplace(target interface{}) error

UnmarshalReplace performs the same process as Unmarshal, except that values not found in the request will be updated to their zero value. For example, if foo.Bar == "baz" and foo.Bar has no corresponding data in a request, Unmarshal would leave it as "baz", but UnmarshalReplace will update it to "".

Exceptions are made for unexported fields and fields which are found to have a name of "-". Those are left alone.

type Unmarshaller 

type Unmarshaller interface {
	Unmarshal(body interface{}) error
}

An Unmarshaller is a type that is capable of unmarshalling data, itself, rather than relying on generic behavior. The calling function will make its best attempt at sending a map[string]interface{} to Unmarshal as the body parameter, but there are situations where it won't be.

Primarily, the type will depend on the request's Content-Type header. Any parsable Content-Type will be parsed, to the best of this library's ability, into a map[string]interface{} as you would expect to see from Request.Params(). However, unrecognized Content-Type headers will cause the raw request.Body to be passed along instead.

Most of the time, you should be able to assume that body is of type map[string]interface{} - the only time that body will be the raw request.Body *should* be when someone is performing a file upload using the file's raw bytes as the body of the request, such as the following example: https://developer.mozilla.org/en-US/docs/Using_files_from_web_applications#Example.3A_Uploading_a_user-selected_file

Except on resource endpoints where you want to support that, you can safely run params := body.(map[string]interface{}) - panics from Unmarshal will be caught and handled as errors.

type UnusedFields 

type UnusedFields struct {
	// contains filtered or unexported fields
}

UnusedFields is an error type for input values that were not used in a request.

func (*UnusedFields) Error 

func (err *UnusedFields) Error() string

Error returns an error message listing which fields could not be found in the target struct.

func (*UnusedFields) Fields 

func (err *UnusedFields) Fields() []string

Fields returns the request names of the fields that had no corresponding struct fields in a request.

func (*UnusedFields) HasMissing 

func (err *UnusedFields) HasMissing() bool

HasMissing returns whether or not this error knows about any input values that were not used in a request.

func (*UnusedFields) NumMissing 

func (err *UnusedFields) NumMissing() int

NumMissing returns the number of input values that were not used in a request that this error knows about.

Source Files

View all Source files

Directories

Path Synopsis
The options package includes some default option parsers for the requests package.
 The options package includes some default option parsers for the requests package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.