httpclient

README

go-httpclient

Advanced HTTP client for golang.

Features

• Simple API
• Direct file upload
• Timeout
• Proxy(HTTP, SOCKS5, SOCKS4, SOCKS4A)
• Cookie
• Redirect Policy

Why

There is already a built in http module for golang, and it's possible to achieve any tasks with it. But I found it painful to do some simple tasks. For example, set timeout, set proxy, upload files etc.

I used to be using CURL, and have built a PHP library on top of it. It turned out solving my problems. So I decided to built one for golang.

Installation

go get github.com/ddliu/go-httpclient

Usage

Initialize

Use NewHttpClient to create a client.

package main

import (
    "github.com/ddliu/go-httpclient"
)

var c := httpclient.NewHttpClient(map[int]interface{} {
    httpclient.OPT_COOKIEJAR: true,
    httpclient.OPT_USERAGENT: "my awsome httpclient",
})

The httpclient.OPT_XXX options define basic behaviours of this client, they are shared between different HTTP requests.

GET/POST

In most cases you just need the Get and Post method after initializing:

func (this *HttpClient) Get(url string, params map[string]string) (*httpclient.Response, error)

func (this *HttpClient) Post(url string, params map[string]string) (*httpclient.Response, error)

    res, err := c.Get("http://google.com/search", map[string]string{
        "q": "news",
    })

    fmt.Println(res.StatusCode, err)

    // post file
    res, err := c.Post("http://dropbox.com/upload", map[string]string {
        "@file": "/tmp/hello.pdf",
    })

    fmt.Println(res, err)

Customize Request

Before you start a new HTTP request with Get or Post method, you can specify options, headers or cookies.

c.
    WithHeader("User-Agent", "Super Robot").
    WithHeader("custom-header", "value").
    WithHeaders(map[string]string {
        "another-header": "another-value",
        "and-another-header": "another-value",
    }).
    WithOption(httpclent.OPT_TIMEOUT, 60).
    WithCookie(&http.Cookie{
        Name: "uid",
        Value: "123",
    }).
    Get("http://github.com", nil)

Response

The httpclient.Response is a thin wrap of http.Response.

// traditional
res, err := c.Get("http://google.com", nil)
bodyBytes, err := ioutil.ReadAll(res.Body)
res.Body.Close()

// ToString
res, err = c.Get("http://google.com", nil)
bodyString := res.ToString()

// ReadAll
res, err = c.Get("http://google.com", nil)
bodyBytes := res.ReadAll()

Concurrent Safe

If you've created one client and want to start many requests concurrently, remember to call the Begin method when you begin:

c := httpclient.NewHttpClient(nil)

go func() {
    c.
        Begin().
        WithHeader("Req-A", "a").
        Get("http://google.com")
}()
go func() {
    c.
        Begin().
        WithHeader("Req-B", "b").
        Get("http://google.com")
}()

All Together

package main

import (
    "github.com/ddliu/go-httpclient"
    "fmt"
)

const (
    USERAGENT = "my awsome httpclient"
    TIMEOUT = 30
    AVATAR = "data/avatar.jpg"
    SERVER = "http://example.com"
)

func main() {
    c := httpclient.New(map[int]interface{} {
        httpclient.OPT_USERAGENT: USERAGENT,
        httpclient.OPT_COOKIEJAR: true,
        httpclient.OPT_TIMEOUT: TIMEOUT,
    })

    res, _ := c.
        WithHeader("Accept-Language", "en-us").
        WithOption(httpclient.OPT_TIMEOUT, 10).
        Get(SERVER)

    fmt.Println(res.ToString())

    res, _ := c.Post(SERVER + "/login", map[string]string {
        "username": "dong",
        "password": "123456",
    })

    res, _ := c.Post(SERVER + "/edit", map[string]string {
        "email": "a@example.com",
        "@avatar": AVATAR,
    })
}

Options

Available options as below:

• OPT_FOLLOWLOCATION: TRUE to follow any "Location: " header that the server sends as part of the HTTP header
• OPT_CONNECTTIMEOUT: The number of seconds to wait while trying to connect. Use 0 to wait indefinitely.
• OPT_CONNECTTIMEOUT_MS: The number of milliseconds to wait while trying to connect. Use 0 to wait indefinitely.
• OPT_MAXREDIRS: The maximum amount of HTTP redirections to follow. Use this option alongside OPT_FOLLOWLOCATION.
• OPT_PROXYTYPE: Specify the proxy type. Valid options are PROXY_HTTP, PROXY_SOCKS4, PROXY_SOCKS5, PROXY_SOCKS4A. Only PROXY_HTTP is supported currently.
• OPT_TIMEOUT: The maximum number of seconds to allow httpclient functions to execute.
• OPT_TIMEOUT_MS: The maximum number of milliseconds to allow httpclient functions to execute.
• OPT_COOKIEJAR: Set to true to enable the default cookiejar, or you can set to a http.CookieJar instance to use a customized jar.
• OPT_INTERFACE: TODO
• OPT_PROXY: Proxy host and port(127.0.0.1:1080).
• OPT_REFERER: The Referer header of the request.
• OPT_USERAGENT: The User-Agent header of the request.
• OPT_REDIRECT_POLICY: Function to check redirect.
• OPT_PROXY_FUNC: Function to specify proxy.

TODO

• Socks proxy support

Changelog

v0.1.0 (2014-02-14)

Initial release

v0.2.0 (2014-02-17)

Rewrite API, make it simple

v0.2.1 (2014-05-18)

Make http.Client reusable

v0.3.0 (2014-05-20)

API improvements

Cookie support

Concurrent safe

v0.3.1 (2014-05-20)

Add shortcut for response

Documentation

Index

• Constants
• Variables
• func Option(o map[string]interface{}) map[int]interface{}
• type HttpClient
  • func NewHttpClient(options map[int]interface{}) *HttpClient
  • func (this *HttpClient) Begin() *HttpClient
  • func (this *HttpClient) Do(method string, url string, headers map[string]string, body io.Reader) (*Response, error)
  • func (this *HttpClient) Get(url string, params map[string]string) (*Response, error)
  • func (this *HttpClient) Post(url string, params map[string]string) (*Response, error)
  • func (this *HttpClient) PostMultipart(url string, params map[string]string) (*Response, error)
  • func (this *HttpClient) WithCookie(cookie *http.Cookie) *HttpClient
  • func (this *HttpClient) WithHeader(k string, v string) *HttpClient
  • func (this *HttpClient) WithHeaders(m map[string]string) *HttpClient
  • func (this *HttpClient) WithOption(k int, v interface{}) *HttpClient
  • func (this *HttpClient) WithOptions(m map[int]interface{}) *HttpClient
• type Response
  • func (this *Response) ReadAll() ([]byte, error)
  • func (this *Response) ToString() (string, error)

Constants

const (
	VERSION   = "0.3.1"
	USERAGENT = "go-httpclient v" + VERSION

	PROXY_HTTP    = 0
	PROXY_SOCKS4  = 4
	PROXY_SOCKS5  = 5
	PROXY_SOCKS4A = 6

	// CURL like OPT
	OPT_AUTOREFERER       = 58
	OPT_FOLLOWLOCATION    = 52
	OPT_CONNECTTIMEOUT    = 78
	OPT_CONNECTTIMEOUT_MS = 156
	OPT_MAXREDIRS         = 68
	OPT_PROXYTYPE         = 101
	OPT_TIMEOUT           = 13
	OPT_TIMEOUT_MS        = 155
	OPT_COOKIEJAR         = 10082
	OPT_INTERFACE         = 10062
	OPT_PROXY             = 10004
	OPT_REFERER           = 10016
	OPT_USERAGENT         = 10018

	// Other OPT
	OPT_REDIRECT_POLICY = 100000
	OPT_PROXY_FUNC      = 100001
)

https://github.com/bagder/curl/blob/169fedbdce93ecf14befb6e0e1ce6a2d480252a3/packages/OS400/curl.inc.in

Variables

var CONST = map[string]int{
	"OPT_AUTOREFERER":       58,
	"OPT_FOLLOWLOCATION":    52,
	"OPT_CONNECTTIMEOUT":    78,
	"OPT_CONNECTTIMEOUT_MS": 156,
	"OPT_MAXREDIRS":         68,
	"OPT_PROXYTYPE":         101,
	"OPT_TIMEOUT":           13,
	"OPT_TIMEOUT_MS":        155,
	"OPT_COOKIEJAR":         10082,
	"OPT_INTERFACE":         10062,
	"OPT_PROXY":             10004,
	"OPT_REFERER":           10016,
	"OPT_USERAGENT":         10018,

	"OPT_REDIRECT_POLICY": 100000,
	"OPT_PROXY_FUNC":      100001,
}

Functions

func Option

func Option(o map[string]interface{}) map[int]interface{}

Convert options with strin keys to desired format

Types

type HttpClient

type HttpClient struct {
	Options map[int]interface{}
	Headers map[string]string
	// contains filtered or unexported fields
}

func NewHttpClient

func NewHttpClient(options map[int]interface{}) *HttpClient

func (*HttpClient) Begin

func (this *HttpClient) Begin() *HttpClient

func (*HttpClient) Do

func (this *HttpClient) Do(method string, url string, headers map[string]string, body io.Reader) (*Response, error)

func (*HttpClient) Get

func (this *HttpClient) Get(url string, params map[string]string) (*Response, error)

The GET request

func (*HttpClient) Post

func (this *HttpClient) Post(url string, params map[string]string) (*Response, error)

The POST request

With multipart set to true, the request will be encoded as "multipart/form-data". If any of the params key starts with "@", it is considered as a form file (similar to CURL but different).

func (*HttpClient) PostMultipart

func (this *HttpClient) PostMultipart(url string, params map[string]string) (*Response, error)

Post with the request encoded as "multipart/form-data".

func (*HttpClient) WithCookie

func (this *HttpClient) WithCookie(cookie *http.Cookie) *HttpClient

func (*HttpClient) WithHeader

func (this *HttpClient) WithHeader(k string, v string) *HttpClient

func (*HttpClient) WithHeaders

func (this *HttpClient) WithHeaders(m map[string]string) *HttpClient

func (*HttpClient) WithOption

func (this *HttpClient) WithOption(k int, v interface{}) *HttpClient

change the oneTimeOptions

func (*HttpClient) WithOptions

func (this *HttpClient) WithOptions(m map[int]interface{}) *HttpClient

type Response

type Response struct {
	*http.Response
}

thin wrapper of http.Response

func (*Response) ReadAll

func (this *Response) ReadAll() ([]byte, error)

func (*Response) ToString

func (this *Response) ToString() (string, error)