httplib

README

httplib

httplib is an libs help you to curl remote url.

How to use?

GET

you can use Get to crawl data.

import "httplib"

str, err := httplib.Get("http://beego.me/").String()
if err != nil {
	t.Fatal(err)
}
fmt.Println(str)

POST

POST data to remote url

b:=httplib.Post("http://beego.me/")
b.Param("username","astaxie")
b.Param("password","123456")
str, err := b.String()
if err != nil {
	t.Fatal(err)
}
fmt.Println(str)

set timeout

you can set timeout in request.default is 60 seconds.

set Get timeout:

httplib.Get("http://beego.me/").SetTimeout(100 * time.Second, 30 * time.Second)

set post timeout:

httplib.Post("http://beego.me/").SetTimeout(100 * time.Second, 30 * time.Second)

• first param is connectTimeout.
• second param is readWriteTimeout

debug

if you want to debug the request info, set the debug on

httplib.Get("http://beego.me/").Debug(true)

support HTTPS client

if request url is https. You can set the client support TSL:

httplib.SetTLSClientConfig(&tls.Config{InsecureSkipVerify: true})

more info about the tls.Config please visit http://golang.org/pkg/crypto/tls/#Config

set cookie

some http request need setcookie. So set it like this:

cookie := &http.Cookie{}
cookie.Name = "username"
cookie.Value  = "astaxie"
httplib.Get("http://beego.me/").SetCookie(cookie)

upload file

httplib support mutil file upload, use b.PostFile()

b:=httplib.Post("http://beego.me/")
b.Param("username","astaxie")
b.Param("password","123456")
b.PostFile("uploadfile1", "httplib.pdf")
b.PostFile("uploadfile2", "httplib.txt")
str, err := b.String()
if err != nil {
	t.Fatal(err)
}
fmt.Println(str)

set HTTP version

some servers need to specify the protocol version of HTTP

httplib.Get("http://beego.me/").SetProtocolVersion("HTTP/1.1")

Documentation

Index

• func SetDefaultSetting(setting BeegoHttpSettings)
• func TimeoutDialer(cTimeout time.Duration, rwTimeout time.Duration) func(net, addr string) (c net.Conn, err error)
• type BeegoHttpRequest
  • func Delete(url string) *BeegoHttpRequest
  • func Get(url string) *BeegoHttpRequest
  • func Head(url string) *BeegoHttpRequest
  • func Post(url string) *BeegoHttpRequest
  • func Put(url string) *BeegoHttpRequest
  • func (b *BeegoHttpRequest) Body(data interface{}) *BeegoHttpRequest
  • func (b *BeegoHttpRequest) Bytes() ([]byte, error)
  • func (b *BeegoHttpRequest) Debug(isdebug bool) *BeegoHttpRequest
  • func (b *BeegoHttpRequest) Header(key, value string) *BeegoHttpRequest
  • func (b *BeegoHttpRequest) Param(key, value string) *BeegoHttpRequest
  • func (b *BeegoHttpRequest) PostFile(formname, filename string) *BeegoHttpRequest
  • func (b *BeegoHttpRequest) Response() (*http.Response, error)
  • func (b *BeegoHttpRequest) SetAgent(useragent string) *BeegoHttpRequest
  • func (b *BeegoHttpRequest) SetCookie(cookie *http.Cookie) *BeegoHttpRequest
  • func (b *BeegoHttpRequest) SetEnableCookie(enable bool) *BeegoHttpRequest
  • func (b *BeegoHttpRequest) SetProtocolVersion(vers string) *BeegoHttpRequest
  • func (b *BeegoHttpRequest) SetProxy(proxy func(*http.Request) (*url.URL, error)) *BeegoHttpRequest
  • func (b *BeegoHttpRequest) SetTLSClientConfig(config *tls.Config) *BeegoHttpRequest
  • func (b *BeegoHttpRequest) SetTimeout(connectTimeout, readWriteTimeout time.Duration) *BeegoHttpRequest
  • func (b *BeegoHttpRequest) SetTransport(transport http.RoundTripper) *BeegoHttpRequest
  • func (b *BeegoHttpRequest) Setting(setting BeegoHttpSettings) *BeegoHttpRequest
  • func (b *BeegoHttpRequest) String() (string, error)
  • func (b *BeegoHttpRequest) ToFile(filename string) error
  • func (b *BeegoHttpRequest) ToJson(v interface{}) error
  • func (b *BeegoHttpRequest) ToXML(v interface{}) error
• type BeegoHttpSettings

Functions

func SetDefaultSetting

func SetDefaultSetting(setting BeegoHttpSettings)

Overwrite default settings

func TimeoutDialer

func TimeoutDialer(cTimeout time.Duration, rwTimeout time.Duration) func(net, addr string) (c net.Conn, err error)

TimeoutDialer returns functions of connection dialer with timeout settings for http.Transport Dial field.

Types

type BeegoHttpRequest

type BeegoHttpRequest struct {
	// contains filtered or unexported fields
}

BeegoHttpRequest provides more useful methods for requesting one url than http.Request.

func Delete

func Delete(url string) *BeegoHttpRequest

Delete returns *BeegoHttpRequest DELETE GET method.

func Get

func Get(url string) *BeegoHttpRequest

Get returns *BeegoHttpRequest with GET method.

func Head

func Head(url string) *BeegoHttpRequest

Head returns *BeegoHttpRequest with HEAD method.

func Post

func Post(url string) *BeegoHttpRequest

Post returns *BeegoHttpRequest with POST method.

func Put

func Put(url string) *BeegoHttpRequest

Put returns *BeegoHttpRequest with PUT method.

func (*BeegoHttpRequest) Body

func (b *BeegoHttpRequest) Body(data interface{}) *BeegoHttpRequest

Body adds request raw body. it supports string and []byte.

func (*BeegoHttpRequest) Bytes

func (b *BeegoHttpRequest) Bytes() ([]byte, error)

Bytes returns the body []byte in response. it calls Response inner.

func (*BeegoHttpRequest) Debug

func (b *BeegoHttpRequest) Debug(isdebug bool) *BeegoHttpRequest

Debug sets show debug or not when executing request.

func (*BeegoHttpRequest) Header

func (b *BeegoHttpRequest) Header(key, value string) *BeegoHttpRequest

Header add header item string in request.

func (*BeegoHttpRequest) Param

func (b *BeegoHttpRequest) Param(key, value string) *BeegoHttpRequest

Param adds query param in to request. params build query string as ?key1=value1&key2=value2...

func (*BeegoHttpRequest) PostFile

func (b *BeegoHttpRequest) PostFile(formname, filename string) *BeegoHttpRequest

func (*BeegoHttpRequest) Response

func (b *BeegoHttpRequest) Response() (*http.Response, error)

Response executes request client gets response mannually.

func (*BeegoHttpRequest) SetAgent

func (b *BeegoHttpRequest) SetAgent(useragent string) *BeegoHttpRequest

SetUserAgent sets User-Agent header field

func (*BeegoHttpRequest) SetCookie

func (b *BeegoHttpRequest) SetCookie(cookie *http.Cookie) *BeegoHttpRequest

SetCookie add cookie into request.

func (*BeegoHttpRequest) SetEnableCookie

func (b *BeegoHttpRequest) SetEnableCookie(enable bool) *BeegoHttpRequest

SetEnableCookie sets enable/disable cookiejar

func (*BeegoHttpRequest) SetProtocolVersion

func (b *BeegoHttpRequest) SetProtocolVersion(vers string) *BeegoHttpRequest

Set the protocol version for incoming requests. Client requests always use HTTP/1.1.

func (*BeegoHttpRequest) SetProxy

func (b *BeegoHttpRequest) SetProxy(proxy func(*http.Request) (*url.URL, error)) *BeegoHttpRequest

Set http proxy example:

func(req *http.Request) (*url.URL, error) {
	u, _ := url.ParseRequestURI("http://127.0.0.1:8118")
	return u, nil
}

func (*BeegoHttpRequest) SetTLSClientConfig

func (b *BeegoHttpRequest) SetTLSClientConfig(config *tls.Config) *BeegoHttpRequest

SetTLSClientConfig sets tls connection configurations if visiting https url.

func (*BeegoHttpRequest) SetTimeout

func (b *BeegoHttpRequest) SetTimeout(connectTimeout, readWriteTimeout time.Duration) *BeegoHttpRequest

SetTimeout sets connect time out and read-write time out for BeegoRequest.

func (*BeegoHttpRequest) SetTransport

func (b *BeegoHttpRequest) SetTransport(transport http.RoundTripper) *BeegoHttpRequest

Set transport to

func (*BeegoHttpRequest) Setting

func (b *BeegoHttpRequest) Setting(setting BeegoHttpSettings) *BeegoHttpRequest

Change request settings

func (*BeegoHttpRequest) String

func (b *BeegoHttpRequest) String() (string, error)

String returns the body string in response. it calls Response inner.

func (*BeegoHttpRequest) ToFile

func (b *BeegoHttpRequest) ToFile(filename string) error

ToFile saves the body data in response to one file. it calls Response inner.

func (*BeegoHttpRequest) ToJson

func (b *BeegoHttpRequest) ToJson(v interface{}) error

ToJson returns the map that marshals from the body bytes as json in response . it calls Response inner.

func (*BeegoHttpRequest) ToXML

func (b *BeegoHttpRequest) ToXML(v interface{}) error

ToXml returns the map that marshals from the body bytes as xml in response . it calls Response inner.

type BeegoHttpSettings

type BeegoHttpSettings struct {
	ShowDebug        bool
	UserAgent        string
	ConnectTimeout   time.Duration
	ReadWriteTimeout time.Duration
	TlsClientConfig  *tls.Config
	Proxy            func(*http.Request) (*url.URL, error)
	Transport        http.RoundTripper
	EnableCookie     bool
}

BeegoHttpSettings