gorequest

package module
v0.2.20 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 10, 2018 License: MIT Imports: 22 Imported by: 0

README

GoRequest

GoRequest -- Simplified HTTP client ( inspired by famous SuperAgent lib in Node.js )

GopherGoRequest

"Shooting Requests like a Machine Gun" - Gopher

Sending request would never been fun and easier than this. It comes with lots of feature:

  • Get/Post/Put/Head/Delete/Patch/Options
  • Set - simple header setting
  • JSON - made it simple with JSON string as a parameter
  • Multipart-Support - send data and files as multipart request
  • Proxy - sending request via proxy
  • Timeout - setting timeout for a request
  • TLSClientConfig - taking control over tls where at least you can disable security check for https
  • RedirectPolicy
  • Cookie - setting cookies for your request
  • CookieJar - automatic in-memory cookiejar
  • BasicAuth - setting basic authentication header
  • more to come..

Installation

$ go get github.com/parnurzeal/gorequest

Documentation

See Go Doc or Go Walker for usage and details.

Status

Drone Build Status Travis Build Status

Why should you use GoRequest?

GoRequest makes thing much more simple for you, making http client more awesome and fun like SuperAgent + golang style usage.

This is what you normally do for a simple GET without GoRequest:

resp, err := http.Get("http://example.com/")

With GoRequest:

request := gorequest.New()
resp, body, errs := request.Get("http://example.com/").End()

Or below if you don't want to reuse it for other requests.

resp, body, errs := gorequest.New().Get("http://example.com/").End()

How about getting control over HTTP client headers, redirect policy, and etc. Things is getting more complicated in golang. You need to create a Client, setting header in different command, ... to do just only one GET

client := &http.Client{
  CheckRedirect: redirectPolicyFunc,
}

req, err := http.NewRequest("GET", "http://example.com", nil)

req.Header.Add("If-None-Match", `W/"wyzzy"`)
resp, err := client.Do(req)

Why making things ugly while you can just do as follows:

request := gorequest.New()
resp, body, errs := request.Get("http://example.com").
  RedirectPolicy(redirectPolicyFunc).
  Set("If-None-Match", `W/"wyzzy"`).
  End()

DELETE, HEAD, POST, PUT, PATCH are now supported and can be used the same way as GET:

request := gorequest.New()
resp, body, errs := request.Post("http://example.com").End()
// PUT -> request.Put("http://example.com").End()
// DELETE -> request.Delete("http://example.com").End()
// HEAD -> request.Head("http://example.com").End()
// ANYTHING -> request.CustomMethod("TRACE", "http://example.com").End()
JSON

For a JSON POST with standard libraries, you might need to marshal map data structure to json format, setting header to 'application/json' (and other headers if you need to) and declare http.Client. So, you code become longer and hard to maintain:

m := map[string]interface{}{
  "name": "backy",
  "species": "dog",
}
mJson, _ := json.Marshal(m)
contentReader := bytes.NewReader(mJson)
req, _ := http.NewRequest("POST", "http://example.com", contentReader)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Notes","GoRequest is coming!")
client := &http.Client{}
resp, _ := client.Do(req)

Compared to our GoRequest version, JSON is for sure a default. So, it turns out to be just one simple line!:

request := gorequest.New()
resp, body, errs := request.Post("http://example.com").
  Set("Notes","gorequst is coming!").
  Send(`{"name":"backy", "species":"dog"}`).
  End()

Moreover, it also supports struct type. So, you can have a fun Mix & Match sending the different data types for your request:

type BrowserVersionSupport struct {
  Chrome string
  Firefox string
}
ver := BrowserVersionSupport{ Chrome: "37.0.2041.6", Firefox: "30.0" }
request := gorequest.New()
resp, body, errs := request.Post("http://version.com/update").
  Send(ver).
  Send(`{"Safari":"5.1.10"}`).
  End()

Not only for Send() but Query() is also supported. Just give it a try! :)

Callback

Moreover, GoRequest also supports callback function. This gives you much more flexibility on using it. You can use it any way to match your own style! Let's see a bit of callback example:

func printStatus(resp gorequest.Response, body string, errs []error){
  fmt.Println(resp.Status)
}
gorequest.New().Get("http://example.com").End(printStatus)

Multipart/Form-Data

You can specify the content-type of the request to type multipart to send all data as multipart/form-data. This feature also allows you to send (multiple) files! Check the examples below!

gorequest.New().Post("http://example.com/").
  Type("multipart").
  Send(`{"query1":"test"}`).
  End()

The SendFile function accepts strings as path to a file, []byte slice or even a os.File! You can also combine them to send multiple files with either custom name and/or custom fieldname:

          f, _ := filepath.Abs("./file2.txt")
bytesOfFile, _ := ioutil.ReadFile(f)

gorequest.New().Post("http://example.com/").
  Type("multipart").
  SendFile("./file1.txt").
  SendFile(bytesOfFile, "file2.txt", "my_file_fieldname").
  End()

Check the docs for SendFile to get more information about the types of arguments.

Proxy

In the case when you are behind proxy, GoRequest can handle it easily with Proxy func:

request := gorequest.New().Proxy("http://proxy:999")
resp, body, errs := request.Get("http://example-proxy.com").End()
// To reuse same client with no_proxy, use empty string:
resp, body, errs = request.Proxy("").Get("http://example-no-proxy.com").End()

Basic Authentication

To add a basic authentication header:

request := gorequest.New().SetBasicAuth("username", "password")
resp, body, errs := request.Get("http://example-proxy.com").End()

Timeout

Timeout can be set in any time duration using time package:

request := gorequest.New().Timeout(2*time.Millisecond)
resp, body, errs:= request.Get("http://example.com").End()

Timeout func defines both dial + read/write timeout to the specified time parameter.

EndBytes

Thanks to @jaytaylor, we now have EndBytes to use when you want the body as bytes.

The callbacks work the same way as with End, except that a byte array is used instead of a string.

resp, bodyBytes, errs := gorequest.New().Get("http://example.com/").EndBytes()

EndStruct

We now have EndStruct to use when you want the body as struct.

The callbacks work the same way as with End, except that a struct is used instead of a string.

Supposing the URL http://example.com/ returns the body {"hey":"you"}

heyYou struct {
  Hey string `json:"hey"`
}

var heyYou heyYou

resp, _, errs := gorequest.New().Get("http://example.com/").EndStruct(&heyYou)

Retry

Supposing you need retry 3 times, with 5 seconds between each attempt when gets a BadRequest or a InternalServerError

request := gorequest.New()
resp, body, errs := request.Get("http://example.com/").
                    Retry(3, 5 * time.seconds, http.StatusBadRequest, http.StatusInternalServerError).
                    End()

Handling Redirects

Redirects can be handled with RedirectPolicy which behaves similarly to net/http Client's CheckRedirect function. Simply specify a function which takes the Request about to be made and a slice of previous Requests in order of oldest first. When this function returns an error, the Request is not made.

For example to redirect only to https endpoints:

request := gorequest.New()
resp, body, errs := request.Get("http://example.com/").
                    RedirectPolicy(func(req Request, via []*Request) error {
                      if req.URL.Scheme != "https" {
                        return http.ErrUseLastResponse
                      }
                    }).
                    End()

Debug

For debugging, GoRequest leverages httputil to dump details of every request/response. (Thanks to @dafang)

You can just use SetDebug or environment variable GOREQUEST_DEBUG=0|1 to enable/disable debug mode and SetLogger to set your own choice of logger.

Thanks to @QuentinPerez, we can see even how gorequest is compared to CURL by using SetCurlCommand.

Noted

As the underlying gorequest is based on http.Client in most use cases, gorequest.New() should be called once and reuse gorequest as much as possible.

Contributing to GoRequest:

If you find any improvement or issue you want to fix, feel free to send me a pull request with testing.

Thanks to all contributors thus far:

Contributors
https://github.com/alaingilbert
https://github.com/austinov
https://github.com/coderhaoxin
https://github.com/codegoalie
https://github.com/dafang
https://github.com/davyzhang
https://github.com/dickeyxxx
https://github.com/figlief
https://github.com/fraenky8
https://github.com/franciscocpg
https://github.com/heytitle
https://github.com/hownowstephen
https://github.com/kemadz
https://github.com/killix
https://github.com/jaytaylor
https://github.com/na-ga
https://github.com/piotrmiskiewicz
https://github.com/pencil001
https://github.com/pkopac
https://github.com/quangbuule
https://github.com/QuentinPerez
https://github.com/smallnest
https://github.com/WaveCutz
https://github.com/xild
https://github.com/yangmls
https://github.com/6david9

Also, co-maintainer is needed here. If anyone is interested, please email me (parnurzeal at gmail.com)

Credits

  • Renee French - the creator of Gopher mascot
  • Wisi Mongkhonsrisawat for providing an awesome GoRequest's Gopher image :)

License

GoRequest is MIT License.

Documentation

Overview

Package gorequest inspired by Nodejs SuperAgent provides easy-way to write http client

Index

Constants

View Source 
const (
	POST    = "POST"
	GET     = "GET"
	HEAD    = "HEAD"
	PUT     = "PUT"
	DELETE  = "DELETE"
	PATCH   = "PATCH"
	OPTIONS = "OPTIONS"
)

HTTP methods we support

Variables

View Source 
var DisableTransportSwap = false
View Source 
var Types = map[string]string{
	"html":       "text/html",
	"json":       "application/json",
	"xml":        "application/xml",
	"text":       "text/plain",
	"urlencoded": "application/x-www-form-urlencoded",
	"form":       "application/x-www-form-urlencoded",
	"form-data":  "application/x-www-form-urlencoded",
	"multipart":  "multipart/form-data",
}

Functions

func IsTimeout added in v0.2.16 

func IsTimeout(errs []error) bool

IsTimeout checks is it a net timeout error. Applicable only for go version >1.6

Types

type File added in v0.2.14 

type File struct {
	Filename  string
	Fieldname string
	Data      []byte
}

type Request 

type Request *http.Request

type Response 

type Response *http.Response

type SuperAgent 

type SuperAgent struct {
	Url               string
	Method            string
	Header            map[string]string
	TargetType        string
	ForceType         string
	Data              map[string]interface{}
	SliceData         []interface{}
	FormData          url.Values
	QueryData         url.Values
	FileData          []File
	BounceToRawString bool
	RawString         string
	Client            *http.Client
	Transport         *http.Transport
	Cookies           []*http.Cookie
	Errors            []error
	BasicAuth         struct{ Username, Password string }
	Debug             bool
	CurlCommand       bool

	Retryable struct {
		RetryableStatus []int
		RetryerTime     time.Duration
		RetryerCount    int
		Attempt         int
		Enable          bool
	}
	// contains filtered or unexported fields
}

A SuperAgent is a object storing all request data for client.

func New 

func New() *SuperAgent

Used to create a new SuperAgent object.

func (*SuperAgent) AddCookie added in v0.2.2 

func (s *SuperAgent) AddCookie(c *http.Cookie) *SuperAgent

AddCookie adds a cookie to the request. The behavior is the same as AddCookie on Request from net/http

func (*SuperAgent) AddCookies added in v0.2.9 

func (s *SuperAgent) AddCookies(cookies []*http.Cookie) *SuperAgent

AddCookies is a convenient method to add multiple cookies

func (*SuperAgent) AsCurlCommand added in v0.2.14 

func (s *SuperAgent) AsCurlCommand() (string, error)

AsCurlCommand returns a string representing the runnable `curl' command version of the request.

func (*SuperAgent) ClearSuperAgent 

func (s *SuperAgent) ClearSuperAgent()

Clear SuperAgent data for another new request.

func (*SuperAgent) CustomMethod added in v0.2.14 

func (s *SuperAgent) CustomMethod(method, targetUrl string) *SuperAgent

Just a wrapper to initialize SuperAgent instance by method string

func (*SuperAgent) Delete added in v0.2.0 

func (s *SuperAgent) Delete(targetUrl string) *SuperAgent

func (*SuperAgent) End 

func (s *SuperAgent) End(callback ...func(response Response, body string, errs []error)) (Response, string, []error)

End is the most important function that you need to call when ending the chain. The request won't proceed without calling it. End function returns Response which matchs the structure of Response type in Golang's http package (but without Body data). The body data itself returns as a string in a 2nd return value. Lastly but worth noticing, error array (NOTE: not just single error value) is returned as a 3rd value and nil otherwise.

For example: 

resp, body, errs := gorequest.New().Get("http://www.google.com").End()
if (errs != nil) {
  fmt.Println(errs)
}
fmt.Println(resp, body)

Moreover, End function also supports callback which you can put as a parameter. This extends the flexibility and makes GoRequest fun and clean! You can use GoRequest in whatever style you love!

For example: 

func printBody(resp gorequest.Response, body string, errs []error){
  fmt.Println(resp.Status)
}
gorequest.New().Get("http://www..google.com").End(printBody)

func (*SuperAgent) EndBytes added in v0.2.6 

func (s *SuperAgent) EndBytes(callback ...func(response Response, body []byte, errs []error)) (Response, []byte, []error)

EndBytes should be used when you want the body as bytes. The callbacks work the same way as with `End`, except that a byte array is used instead of a string.

func (*SuperAgent) EndStruct added in v0.2.14 

func (s *SuperAgent) EndStruct(v interface{}, callback ...func(response Response, v interface{}, body []byte, errs []error)) (Response, []byte, []error)

EndStruct should be used when you want the body as a struct. The callbacks work the same way as with `End`, except that a struct is used instead of a string.

func (*SuperAgent) Get 

func (s *SuperAgent) Get(targetUrl string) *SuperAgent

func (*SuperAgent) Head added in v0.2.0 

func (s *SuperAgent) Head(targetUrl string) *SuperAgent

func (*SuperAgent) MakeRequest added in v0.2.14 

func (s *SuperAgent) MakeRequest() (*http.Request, error)

func (*SuperAgent) Options added in v0.2.14 

func (s *SuperAgent) Options(targetUrl string) *SuperAgent

func (*SuperAgent) Param added in v0.2.9 

func (s *SuperAgent) Param(key string, value string) *SuperAgent

As Go conventions accepts ; as a synonym for &. (https://github.com/golang/go/issues/2210) Thus, Query won't accept ; in a querystring if we provide something like fields=f1;f2;f3 This Param is then created as an alternative method to solve this.

func (*SuperAgent) Patch added in v0.2.3 

func (s *SuperAgent) Patch(targetUrl string) *SuperAgent

func (*SuperAgent) Post 

func (s *SuperAgent) Post(targetUrl string) *SuperAgent

func (*SuperAgent) Proxy added in v0.2.0 

func (s *SuperAgent) Proxy(proxyUrl string) *SuperAgent

Proxy function accepts a proxy url string to setup proxy url for any request. It provides a convenience way to setup proxy which have advantages over usual old ways. One example is you might try to set `http_proxy` environment. This means you are setting proxy up for all the requests. You will not be able to send different request with different proxy unless you change your `http_proxy` environment again. Another example is using Golang proxy setting. This is normal prefer way to do but too verbase compared to GoRequest's Proxy: 

gorequest.New().Proxy("http://myproxy:9999").
  Post("http://www.google.com").
  End()

To set no_proxy, just put empty string to Proxy func: 

gorequest.New().Proxy("").
  Post("http://www.google.com").
  End()

func (*SuperAgent) Put added in v0.2.0 

func (s *SuperAgent) Put(targetUrl string) *SuperAgent

func (*SuperAgent) Query 

func (s *SuperAgent) Query(content interface{}) *SuperAgent

Query function accepts either json string or strings which will form a query-string in url of GET method or body of POST method. For example, making "/search?query=bicycle&size=50x50&weight=20kg" using GET method: 

gorequest.New().
  Get("/search").
  Query(`{ query: 'bicycle' }`).
  Query(`{ size: '50x50' }`).
  Query(`{ weight: '20kg' }`).
  End()

Or you can put multiple json values: 

gorequest.New().
  Get("/search").
  Query(`{ query: 'bicycle', size: '50x50', weight: '20kg' }`).
  End()

Strings are also acceptable: 

gorequest.New().
  Get("/search").
  Query("query=bicycle&size=50x50").
  Query("weight=20kg").
  End()

Or even Mixed! :) 

gorequest.New().
  Get("/search").
  Query("query=bicycle").
  Query(`{ size: '50x50', weight:'20kg' }`).
  End()

func (*SuperAgent) RedirectPolicy 

func (s *SuperAgent) RedirectPolicy(policy func(req Request, via []Request) error) *SuperAgent

RedirectPolicy accepts a function to define how to handle redirects. If the policy function returns an error, the next Request is not made and the previous request is returned.

The policy function's arguments are the Request about to be made and the past requests in order of oldest first.

func (*SuperAgent) Retry added in v0.2.15 

func (s *SuperAgent) Retry(retryerCount int, retryerTime time.Duration, statusCode ...int) *SuperAgent

gorequest.New(). 

Post("/gamelist").
Retry(3, 5 * time.seconds, http.StatusBadRequest, http.StatusInternalServerError).
End()

func (*SuperAgent) Send 

func (s *SuperAgent) Send(content interface{}) *SuperAgent

Send function accepts either json string or query strings which is usually used to assign data to POST or PUT method. Without specifying any type, if you give Send with json data, you are doing requesting in json format: 

gorequest.New().
  Post("/search").
  Send(`{ query: 'sushi' }`).
  End()

While if you use at least one of querystring, GoRequest understands and automatically set the Content-Type to `application/x-www-form-urlencoded` 

gorequest.New().
  Post("/search").
  Send("query=tonkatsu").
  End()

So, if you want to strictly send json format, you need to use Type func to set it as `json` (Please see more details in Type function). You can also do multiple chain of Send: 

gorequest.New().
  Post("/search").
  Send("query=bicycle&size=50x50").
  Send(`{ wheel: '4'}`).
  End()

From v0.2.0, Send function provide another convenience way to work with Struct type. You can mix and match it with json and query string: 

type BrowserVersionSupport struct {
  Chrome string
  Firefox string
}
ver := BrowserVersionSupport{ Chrome: "37.0.2041.6", Firefox: "30.0" }
gorequest.New().
  Post("/update_version").
  Send(ver).
  Send(`{"Safari":"5.1.10"}`).
  End()

If you have set Type to text or Content-Type to text/plain, content will be sent as raw string in body instead of form 

gorequest.New().
  Post("/greet").
  Type("text").
  Send("hello world").
  End()

func (*SuperAgent) SendFile added in v0.2.14 

func (s *SuperAgent) SendFile(file interface{}, args ...string) *SuperAgent

SendFile function works only with type "multipart". The function accepts one mandatory and up to two optional arguments. The mandatory (first) argument is the file. The function accepts a path to a file as string: 

gorequest.New().
  Post("http://example.com").
  Type("multipart").
  SendFile("./example_file.ext").
  End()

File can also be a []byte slice of a already file read by eg. ioutil.ReadFile: 

b, _ := ioutil.ReadFile("./example_file.ext")
gorequest.New().
  Post("http://example.com").
  Type("multipart").
  SendFile(b).
  End()

Furthermore file can also be a os.File: 

f, _ := os.Open("./example_file.ext")
gorequest.New().
  Post("http://example.com").
  Type("multipart").
  SendFile(f).
  End()

The first optional argument (second argument overall) is the filename, which will be automatically determined when file is a string (path) or a os.File. When file is a []byte slice, filename defaults to "filename". In all cases the automatically determined filename can be overwritten: 

b, _ := ioutil.ReadFile("./example_file.ext")
gorequest.New().
  Post("http://example.com").
  Type("multipart").
  SendFile(b, "my_custom_filename").
  End()

The second optional argument (third argument overall) is the fieldname in the multipart/form-data request. It defaults to fileNUMBER (eg. file1), where number is ascending and starts counting at 1. So if you send multiple files, the fieldnames will be file1, file2, ... unless it is overwritten. If fieldname is set to "file" it will be automatically set to fileNUMBER, where number is the greatest exsiting number+1. 

b, _ := ioutil.ReadFile("./example_file.ext")
gorequest.New().
  Post("http://example.com").
  Type("multipart").
  SendFile(b, "", "my_custom_fieldname"). // filename left blank, will become "example_file.ext"
  End()

func (*SuperAgent) SendMap added in v0.2.15 

func (s *SuperAgent) SendMap(content interface{}) *SuperAgent

func (*SuperAgent) SendSlice added in v0.2.11 

func (s *SuperAgent) SendSlice(content []interface{}) *SuperAgent

SendSlice (similar to SendString) returns SuperAgent's itself for any next chain and takes content []interface{} as a parameter. Its duty is to append slice of interface{} into s.SliceData ([]interface{}) which later changes into json array in the End() func.

func (*SuperAgent) SendString added in v0.2.0 

func (s *SuperAgent) SendString(content string) *SuperAgent

SendString returns SuperAgent's itself for any next chain and takes content string as a parameter. Its duty is to transform String into s.Data (map[string]interface{}) which later changes into appropriate format such as json, form, text, etc. in the End func. Send implicitly uses SendString and you should use Send instead of this.

func (*SuperAgent) SendStruct added in v0.2.11 

func (s *SuperAgent) SendStruct(content interface{}) *SuperAgent

SendStruct (similar to SendString) returns SuperAgent's itself for any next chain and takes content interface{} as a parameter. Its duty is to transfrom interface{} (implicitly always a struct) into s.Data (map[string]interface{}) which later changes into appropriate format such as json, form, text, etc. in the End() func.

func (*SuperAgent) Set 

func (s *SuperAgent) Set(param string, value string) *SuperAgent

Set is used for setting header fields. Example. To set `Accept` as `application/json` 

gorequest.New().
  Post("/gamelist").
  Set("Accept", "application/json").
  End()

func (*SuperAgent) SetBasicAuth added in v0.2.5 

func (s *SuperAgent) SetBasicAuth(username string, password string) *SuperAgent

SetBasicAuth sets the basic authentication header Example. To set the header for username "myuser" and password "mypass" 

gorequest.New()
  Post("/gamelist").
  SetBasicAuth("myuser", "mypass").
  End()

func (*SuperAgent) SetCurlCommand added in v0.2.12 

func (s *SuperAgent) SetCurlCommand(enable bool) *SuperAgent

Enable the curlcommand mode which display a CURL command line

func (*SuperAgent) SetDebug added in v0.2.8 

func (s *SuperAgent) SetDebug(enable bool) *SuperAgent

Enable the debug mode which logs request/response detail

func (*SuperAgent) SetLogger added in v0.2.8 

func (s *SuperAgent) SetLogger(logger *log.Logger) *SuperAgent

func (*SuperAgent) TLSClientConfig added in v0.2.2 

func (s *SuperAgent) TLSClientConfig(config *tls.Config) *SuperAgent

Set TLSClientConfig for underling Transport. One example is you can use it to disable security check (https): 

gorequest.New().TLSClientConfig(&tls.Config{ InsecureSkipVerify: true}).
  Get("https://disable-security-check.com").
  End()

func (*SuperAgent) Timeout added in v0.2.0 

func (s *SuperAgent) Timeout(timeout time.Duration) *SuperAgent

func (*SuperAgent) Type 

func (s *SuperAgent) Type(typeStr string) *SuperAgent

Type is a convenience function to specify the data type to send. For example, to send data as `application/x-www-form-urlencoded` : 

gorequest.New().
  Post("/recipe").
  Type("form").
  Send(`{ "name": "egg benedict", "category": "brunch" }`).
  End()

This will POST the body "name=egg benedict&category=brunch" to url /recipe

GoRequest supports 

"text/html" uses "html"
"application/json" uses "json"
"application/xml" uses "xml"
"text/plain" uses "text"
"application/x-www-form-urlencoded" uses "urlencoded", "form" or "form-data"

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.