httplib

package
v0.0.8 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 9, 2022 License: MIT Imports: 21 Imported by: 0

README

Bhojpur Web - Client HTTPlib

The HTTPlib is a library that helps you to curl remote url.

How to use?

GET

You can use GET to crawl the data.

import "github.com/bhojpur/web/pkg/client/httplib"

str, err := httplib.Get("http://bhojpur.net/").String()
if err != nil {
    	// error
}
fmt.Println(str)
POST

The POST data to remote url

req := httplib.Post("http://bhojpur.net/")
req.Param("username","bhojpur")
req.Param("password","123456")
str, err := req.String()
if err != nil {
    	// error
}
fmt.Println(str)
Set timeout

The default timeout is 60 seconds, function prototype:

SetTimeout(connectTimeout, readWriteTimeout time.Duration)

Example:

// GET
httplib.Get("http://bhojpur.net/").SetTimeout(100 * time.Second, 30 * time.Second)

// POST
httplib.Post("http://bhojpur.net/").SetTimeout(100 * time.Second, 30 * time.Second)
Debug

If you want to debug the request info, set the debug on

httplib.Get("http://bhojpur.net/").Debug(true)
Set HTTP Basic Auth
str, err := Get("http://bhojpur.net/").SetBasicAuth("user", "passwd").String()
if err != nil {
    	// error
}
fmt.Println(str)
Set HTTPS

If requested url is using https, then you can set the client support TLS:

httplib.SetTLSClientConfig(&tls.Config{InsecureSkipVerify: true})

More info about the tls.Config please visit http://golang.org/pkg/crypto/tls/#Config

Set HTTP Version

Some servers need to specify the protocol version of HTTP

httplib.Get("http://bhojpur.net/").SetProtocolVersion("HTTP/1.1")

Some HTTP requests need setcookie. So, set it like this:

cookie := &http.Cookie{}
cookie.Name = "username"
cookie.Value  = "bhojpur"
httplib.Get("http://bhojpur.net/").SetCookie(cookie)
Upload file

The HTTPlib supports mutil-file upload, use req.PostFile()

req := httplib.Post("http://bhojpur.net/")
req.Param("username","bhojpur")
req.PostFile("uploadfile1", "httplib.pdf")
str, err := req.String()
if err != nil {
    	// error
}
fmt.Println(str)

Documentation

Index

Constants

This section is empty.

Variables

View Source
var CloseFileFailed = berror.DefineCode(5001004, moduleName, "CloseFileFailed", `
After handling files, Bhojpur try to close file but failed. Usually it was caused by bad file descriptor.
`)
View Source
var CopyFileFailed = berror.DefineCode(5001003, moduleName, "CopyFileFailed", `
When we try to read file content and then copy it to another writer, and failed.
1. Unexpected EOF;
2. Bad file descriptor;
3. Write conflict;

Please check your file content, and confirm that file is not processed by other process (or by user manually).
`)
View Source
var CreateFileIfNotExistFailed = berror.DefineCode(5001007, moduleName, "CreateFileIfNotExist", `
Bhojpur want to create file if not exist and failed. 
In most cases, it means that Bhojpur doesn't have the privilege to create this file.
Please change file mode to ensure that Bhojpur is able to create files on specific directory.
Or you can run Bhojpur with higher authority.
In some cases, you pass invalid filename. Make sure that the file name is valid on your system.
`)
View Source
var CreateFormFileFailed = berror.DefineCode(5001001, moduleName, "CreateFormFileFailed", `
In normal case than handling files with BhojpurRequest, you should not see this error code.
Unexpected EOF, invalid characters, bad file descriptor may cause this error.
`)
View Source
var InvalidJSONBody = berror.DefineCode(4001006, moduleName, "InvalidJSONBody", `
You pass invalid data which could not be converted to JSON documents. In general, if you pass structure, it works well.
Sometimes you got JSON document and you want to make it as request body. So you call JSONBody.
If you do this, you got this code. Instead, you should call Header to set Content-type and call Body to set body data.
`)
View Source
var InvalidUrl = berror.DefineCode(4001001, moduleName, "InvalidUrl", `
You pass a invalid url to httplib module. Please check your url, be careful about special character. 
`)
View Source
var InvalidUrlProtocolVersion = berror.DefineCode(4001002, moduleName, "InvalidUrlProtocolVersion", `
You pass a invalid protocol version. In practice, we use HTTP/1.0, HTTP/1.1, HTTP/1.2
But something like HTTP/3.2 is valid for client, and the major version is 3, minor version is 2.
but you must confirm that server support those abnormal protocol version.
`)
View Source
var InvalidXMLBody = berror.DefineCode(4001004, moduleName, "InvalidXMLBody", `
You pass invalid data which could not be converted to XML documents. In general, if you pass structure, it works well.
Sometimes you got XML document and you want to make it as request body. So you call XMLBody.
If you do this, you got this code. Instead, you should call Header to set Content-type and call Body to set body data.
`)
View Source
var InvalidYAMLBody = berror.DefineCode(4001005, moduleName, "InvalidYAMLBody", `
You pass invalid data which could not be converted to YAML documents. In general, if you pass structure, it works well.
Sometimes you got YAML document and you want to make it as request body. So you call YAMLBody.
If you do this, you got this code. Instead, you should call Header to set Content-type and call Body to set body data.
`)
View Source
var ReadFileFailed = berror.DefineCode(5001002, moduleName, "ReadFileFailed", `
There are several cases that cause this error:
1. file not found. Please check the file name;
2. file not found, but file name is correct. If you use relative file path, it's very possible for you to see this code.
make sure that this file is in correct directory which Bhojpur looks for;
3. Bhojpur don't have the privilege to read this file, please change file mode; 
`)
View Source
var ReadGzipBodyFailed = berror.DefineCode(5001006, moduleName, "BuildGzipReaderFailed", `
Bhojpur parse gzip-encode body failed. Usually Bhojpur got invalid response.
Please confirm that server returns gzip data.
`)
View Source
var SendRequestFailed = berror.DefineCode(5001005, moduleName, "SendRequestRetryExhausted", `
Bhojpur send HTTP request, but it failed.
If you config retry times, it means that Bhojpur had retried and failed.
When you got this error, there are vary kind of reason:
1. Network unstable and timeout. In this case, sometimes server has received the request.
2. Server error. Make sure that server works well.
3. The request is invalid, which means that you pass some invalid parameter.
`)
View Source
var UnmarshalJSONResponseToObjectFailed = berror.DefineCode(5001008, moduleName,
	"UnmarshalResponseToObjectFailed", `
Bhojpur trying to unmarshal response's body to structure but failed.
Make sure that:
1. You pass valid structure pointer to the function;
2. The body is valid json document
`)
View Source
var UnmarshalResponseToObjectFailed = berror.DefineCode(5001011, moduleName,
	"UnmarshalResponseToObjectFailed", `
Bhojpur trying to unmarshal response's body to structure but failed.
There are several cases that cause this error:
1. You pass valid structure pointer to the function;
2. The body is valid json, Yaml or XML document
`)
View Source
var UnmarshalXMLResponseToObjectFailed = berror.DefineCode(5001009, moduleName,
	"UnmarshalResponseToObjectFailed", `
Bhojpur trying to unmarshal response's body to structure but failed.
Make sure that:
1. You pass valid structure pointer to the function;
2. The body is valid XML document
`)
View Source
var UnmarshalYAMLResponseToObjectFailed = berror.DefineCode(5001010, moduleName,
	"UnmarshalResponseToObjectFailed", `
Bhojpur trying to unmarshal response's body to structure but failed.
Make sure that:
1. You pass valid structure pointer to the function;
2. The body is valid YAML document
`)
View Source
var UnsupportedBodyType = berror.DefineCode(4001003, moduleName, "UnsupportedBodyType", `
You use a invalid data as request body.
For now, we only support type string and byte[].
`)

Functions

func AddDefaultFilter added in v0.0.4

func AddDefaultFilter(fc FilterChain)

AddDefaultFilter add a new filter into defaultSetting Be careful about using this method if you invoke SetDefaultSetting somewhere

func NewHttpResponseWithJsonBody added in v0.0.4

func NewHttpResponseWithJsonBody(data interface{}) *http.Response

NewHttpResponseWithJsonBody will try to convert the data to json format usually you only use this when you want to mock http Response

func SetDefaultSetting

func SetDefaultSetting(setting BhojpurHTTPSettings)

SetDefaultSetting overwrites default settings Keep in mind that when you invoke the SetDefaultSetting some methods invoked before SetDefaultSetting

func TimeoutDialer

func TimeoutDialer(cTimeout time.Duration, rwTimeout time.Duration) func(net, addr string) (c net.Conn, err error)

TimeoutDialer returns functions of connection dialer with timeout settings for http.Transport Dial field. Deprecated we will move this at the end of 2021 please use TimeoutDialerCtx

func TimeoutDialerCtx added in v0.0.4

func TimeoutDialerCtx(cTimeout time.Duration,
	rwTimeout time.Duration) func(ctx context.Context, net, addr string) (c net.Conn, err error)

Types

type BhojpurHTTPRequest

type BhojpurHTTPRequest struct {
	// contains filtered or unexported fields
}

BhojpurHTTPRequest provides more useful methods than http.Request for requesting a url.

func Delete

func Delete(url string) *BhojpurHTTPRequest

Delete returns *BhojpurHttpRequest DELETE method.

func Get

func Get(url string) *BhojpurHTTPRequest

Get returns *BhojpurHttpRequest with GET method.

func Head(url string) *BhojpurHTTPRequest

Head returns *BhojpurHttpRequest with HEAD method.

func NewBhojpurRequest

func NewBhojpurRequest(rawurl, method string) *BhojpurHTTPRequest

NewBhojpurRequest returns *BhojpurHttpRequest with specific method TODO add error as return value I think if we don't return error users are hard to check whether we create Bhojpur Web request successfully

func Post

func Post(url string) *BhojpurHTTPRequest

Post returns *BhojpurHttpRequest with POST method.

func Put

func Put(url string) *BhojpurHTTPRequest

Put returns *BhojpurHttpRequest with PUT method.

func (*BhojpurHTTPRequest) AddFilters

func (b *BhojpurHTTPRequest) AddFilters(fcs ...FilterChain) *BhojpurHTTPRequest

AddFilters adds filter

func (*BhojpurHTTPRequest) Body

func (b *BhojpurHTTPRequest) Body(data interface{}) *BhojpurHTTPRequest

Body adds request raw body. Supports string and []byte. TODO return error if data is invalid

func (*BhojpurHTTPRequest) Bytes

func (b *BhojpurHTTPRequest) Bytes() ([]byte, error)

Bytes returns the body []byte in response. Calls Response inner.

func (*BhojpurHTTPRequest) DoRequest

func (b *BhojpurHTTPRequest) DoRequest() (resp *http.Response, err error)

DoRequest executes client.Do

func (*BhojpurHTTPRequest) DoRequestWithCtx

func (b *BhojpurHTTPRequest) DoRequestWithCtx(ctx context.Context) (resp *http.Response, err error)

func (*BhojpurHTTPRequest) GetRequest

func (b *BhojpurHTTPRequest) GetRequest() *http.Request

GetRequest returns the request object

func (*BhojpurHTTPRequest) Header

func (b *BhojpurHTTPRequest) Header(key, value string) *BhojpurHTTPRequest

Header adds header item string in request.

func (*BhojpurHTTPRequest) JSONBody

func (b *BhojpurHTTPRequest) JSONBody(obj interface{}) (*BhojpurHTTPRequest, error)

JSONBody adds the request raw body encoded in JSON.

func (*BhojpurHTTPRequest) JSONMarshal added in v0.0.4

func (b *BhojpurHTTPRequest) JSONMarshal(obj interface{}) ([]byte, error)

func (*BhojpurHTTPRequest) Param

func (b *BhojpurHTTPRequest) Param(key, value string) *BhojpurHTTPRequest

Param adds query param in to request. params build query string as ?key1=value1&key2=value2...

func (*BhojpurHTTPRequest) PostFile

func (b *BhojpurHTTPRequest) PostFile(formname, filename string) *BhojpurHTTPRequest

PostFile adds a post file to the request

func (*BhojpurHTTPRequest) Response

func (b *BhojpurHTTPRequest) Response() (*http.Response, error)

Response executes request client gets response manually.

func (*BhojpurHTTPRequest) Retries

func (b *BhojpurHTTPRequest) Retries(times int) *BhojpurHTTPRequest

Retries sets Retries times. default is 0 (never retry) -1 retry indefinitely (forever) Other numbers specify the exact retry amount

func (*BhojpurHTTPRequest) RetryDelay

func (b *BhojpurHTTPRequest) RetryDelay(delay time.Duration) *BhojpurHTTPRequest

RetryDelay sets the time to sleep between reconnection attempts

func (*BhojpurHTTPRequest) SetBasicAuth

func (b *BhojpurHTTPRequest) SetBasicAuth(username, password string) *BhojpurHTTPRequest

SetBasicAuth sets the request's Authorization header to use HTTP Basic Authentication with the provided username and password.

func (*BhojpurHTTPRequest) SetCheckRedirect

func (b *BhojpurHTTPRequest) SetCheckRedirect(redirect func(req *http.Request, via []*http.Request) error) *BhojpurHTTPRequest

SetCheckRedirect specifies the policy for handling redirects.

If CheckRedirect is nil, the Client uses its default policy, which is to stop after 10 consecutive requests.

func (*BhojpurHTTPRequest) SetCookie

func (b *BhojpurHTTPRequest) SetCookie(cookie *http.Cookie) *BhojpurHTTPRequest

SetCookie adds a cookie to the request.

func (*BhojpurHTTPRequest) SetEnableCookie

func (b *BhojpurHTTPRequest) SetEnableCookie(enable bool) *BhojpurHTTPRequest

SetEnableCookie sets enable/disable cookiejar

func (*BhojpurHTTPRequest) SetEscapeHTML added in v0.0.4

func (b *BhojpurHTTPRequest) SetEscapeHTML(isEscape bool) *BhojpurHTTPRequest

SetEscapeHTML is used to set the flag whether escape HTML special characters during processing

func (*BhojpurHTTPRequest) SetFilters

func (b *BhojpurHTTPRequest) SetFilters(fcs ...FilterChain) *BhojpurHTTPRequest

SetFilters will use the filter as the invocation filters

func (*BhojpurHTTPRequest) SetHost

func (b *BhojpurHTTPRequest) SetHost(host string) *BhojpurHTTPRequest

SetHost set the request host

func (*BhojpurHTTPRequest) SetProtocolVersion

func (b *BhojpurHTTPRequest) SetProtocolVersion(vers string) *BhojpurHTTPRequest

SetProtocolVersion sets the protocol version for incoming requests. Client requests always use HTTP/1.1

func (*BhojpurHTTPRequest) SetProxy

func (b *BhojpurHTTPRequest) SetProxy(proxy func(*http.Request) (*url.URL, error)) *BhojpurHTTPRequest

SetProxy sets the HTTP proxy example:

func(req *http.Request) (*url.URL, error) {
	u, _ := url.ParseRequestURI("http://127.0.0.1:8118")
	return u, nil
}

func (*BhojpurHTTPRequest) SetTLSClientConfig

func (b *BhojpurHTTPRequest) SetTLSClientConfig(config *tls.Config) *BhojpurHTTPRequest

SetTLSClientConfig sets TLS connection configuration if visiting HTTPS url.

func (*BhojpurHTTPRequest) SetTimeout

func (b *BhojpurHTTPRequest) SetTimeout(connectTimeout, readWriteTimeout time.Duration) *BhojpurHTTPRequest

SetTimeout sets connect time out and read-write time out for BhojpurRequest.

func (*BhojpurHTTPRequest) SetTransport

func (b *BhojpurHTTPRequest) SetTransport(transport http.RoundTripper) *BhojpurHTTPRequest

SetTransport sets the transport field

func (*BhojpurHTTPRequest) SetUserAgent

func (b *BhojpurHTTPRequest) SetUserAgent(useragent string) *BhojpurHTTPRequest

SetUserAgent sets User-Agent header field

func (*BhojpurHTTPRequest) Setting

Setting changes request settings

func (*BhojpurHTTPRequest) String

func (b *BhojpurHTTPRequest) String() (string, error)

String returns the body string in response. Calls Response inner.

func (*BhojpurHTTPRequest) ToFile

func (b *BhojpurHTTPRequest) ToFile(filename string) error

ToFile saves the body data in response to one file. Calls Response inner.

func (*BhojpurHTTPRequest) ToJSON

func (b *BhojpurHTTPRequest) ToJSON(v interface{}) error

ToJSON returns the map that marshals from the body bytes as json in response. Calls Response inner.

func (*BhojpurHTTPRequest) ToValue added in v0.0.4

func (b *BhojpurHTTPRequest) ToValue(value interface{}) error

ToValue attempts to resolve the response body to value using an existing method. Calls Response inner. If response header contain Content-Type, func will call ToJSON\ToXML\ToYAML. Else it will try to parse body as json\yaml\xml, If all attempts fail, an error will be returned

func (*BhojpurHTTPRequest) ToXML

func (b *BhojpurHTTPRequest) ToXML(v interface{}) error

ToXML returns the map that marshals from the body bytes as xml in response . Calls Response inner.

func (*BhojpurHTTPRequest) ToYAML

func (b *BhojpurHTTPRequest) ToYAML(v interface{}) error

ToYAML returns the map that marshals from the body bytes as yaml in response . Calls Response inner.

func (*BhojpurHTTPRequest) XMLBody

func (b *BhojpurHTTPRequest) XMLBody(obj interface{}) (*BhojpurHTTPRequest, error)

XMLBody adds the request raw body encoded in XML.

func (*BhojpurHTTPRequest) YAMLBody

func (b *BhojpurHTTPRequest) YAMLBody(obj interface{}) (*BhojpurHTTPRequest, error)

YAMLBody adds the request raw body encoded in YAML.

type BhojpurHTTPRequestOption added in v0.0.4

type BhojpurHTTPRequestOption func(request *BhojpurHTTPRequest)

func WithBasicAuth added in v0.0.4

func WithBasicAuth(basicAuth func() (string, string)) BhojpurHTTPRequestOption

WithBasicAuth adds a custom function to set basic auth

func WithContentType added in v0.0.4

func WithContentType(contentType string) BhojpurHTTPRequestOption

WithContentType adds ContentType in header

func WithCookie added in v0.0.4

func WithCookie(cookie *http.Cookie) BhojpurHTTPRequestOption

WithCookie adds a cookie to the request.

func WithFilters added in v0.0.4

func WithFilters(fcs ...FilterChain) BhojpurHTTPRequestOption

WithFilters will use the filter as the invocation filters

func WithHeader added in v0.0.4

func WithHeader(key, value string) BhojpurHTTPRequestOption

WithHeader adds header item string in request.

func WithParam added in v0.0.4

func WithParam(key, value string) BhojpurHTTPRequestOption

WithParam adds query param in to request.

func WithRetry added in v0.0.4

func WithRetry(times int, delay time.Duration) BhojpurHTTPRequestOption

WithRetry set retry times and delay for the request default is 0 (never retry) -1 retry indefinitely (forever) Other numbers specify the exact retry amount

func WithTimeout added in v0.0.4

func WithTimeout(connectTimeout, readWriteTimeout time.Duration) BhojpurHTTPRequestOption

WithTimeout sets connect time out and read-write time out for BhojpurRequest.

func WithTokenFactory added in v0.0.4

func WithTokenFactory(tokenFactory func() string) BhojpurHTTPRequestOption

Withtokenfactory adds a custom function to set Authorization

type BhojpurHTTPSettings

type BhojpurHTTPSettings struct {
	UserAgent        string
	ConnectTimeout   time.Duration
	ReadWriteTimeout time.Duration
	TLSClientConfig  *tls.Config
	Proxy            func(*http.Request) (*url.URL, error)
	Transport        http.RoundTripper
	CheckRedirect    func(req *http.Request, via []*http.Request) error
	EnableCookie     bool
	Gzip             bool
	Retries          int // if set to -1 means will retry forever
	RetryDelay       time.Duration
	FilterChains     []FilterChain
	EscapeHTML       bool // if set to false means will not escape escape HTML special characters during processing, default true
}

BhojpurHTTPSettings is the http.Client setting

func GetDefaultSetting added in v0.0.4

func GetDefaultSetting() BhojpurHTTPSettings

GetDefaultSetting return current default setting

type Client added in v0.0.4

type Client struct {
	Name       string
	Endpoint   string
	CommonOpts []BhojpurHTTPRequestOption

	Setting BhojpurHTTPSettings
}

Client provides an HTTP client supporting chain call

func NewClient added in v0.0.4

func NewClient(name string, endpoint string, opts ...ClientOption) (*Client, error)

NewClient return a new http client

func (*Client) Delete added in v0.0.4

func (c *Client) Delete(value interface{}, path string, opts ...BhojpurHTTPRequestOption) error

Delete Send a Delete request and try to give its result value

func (*Client) Get added in v0.0.4

func (c *Client) Get(value interface{}, path string, opts ...BhojpurHTTPRequestOption) error

Get Send a GET request and try to give its result value

func (*Client) Head added in v0.0.4

func (c *Client) Head(value interface{}, path string, opts ...BhojpurHTTPRequestOption) error

Head Send a Head request and try to give its result value

func (*Client) Post added in v0.0.4

func (c *Client) Post(value interface{}, path string, body interface{}, opts ...BhojpurHTTPRequestOption) error

Post Send a POST request and try to give its result value

func (*Client) Put added in v0.0.4

func (c *Client) Put(value interface{}, path string, body interface{}, opts ...BhojpurHTTPRequestOption) error

Put Send a Put request and try to give its result value

type ClientOption added in v0.0.4

type ClientOption func(client *Client)

func WithCheckRedirect added in v0.0.4

func WithCheckRedirect(redirect func(req *http.Request, via []*http.Request) error) ClientOption

WithCheckRedirect will specifies the policy for handling redirects in all subsequent request

func WithEnableCookie added in v0.0.4

func WithEnableCookie(enable bool) ClientOption

WithEnableCookie will enable cookie in all subsequent request

func WithEnableGzip added in v0.0.4

func WithEnableGzip(enable bool) ClientOption

WithEnableGzip will enable gzip in all subsequent request

func WithHTTPSetting added in v0.0.4

func WithHTTPSetting(setting BhojpurHTTPSettings) ClientOption

WithHTTPSetting can replace BhojpurHTTPSeting

func WithProxy added in v0.0.4

func WithProxy(proxy func(*http.Request) (*url.URL, error)) ClientOption

WithProxy will set http proxy field in all subsequent request

func WithTLSClientConfig added in v0.0.4

func WithTLSClientConfig(config *tls.Config) ClientOption

WithTLSClientConfig will adds tls config in all subsequent request

func WithTransport added in v0.0.4

func WithTransport(transport http.RoundTripper) ClientOption

WithTransport will set transport field in all subsequent request

func WithUserAgent added in v0.0.4

func WithUserAgent(userAgent string) ClientOption

WithEnableCookie will adds UA in all subsequent request

type Filter

type Filter func(ctx context.Context, req *BhojpurHTTPRequest) (*http.Response, error)

type FilterChain

type FilterChain func(next Filter) Filter

type HTTPBodyCarrier added in v0.0.4

type HTTPBodyCarrier interface {
	SetReader(r io.ReadCloser)
}

HTTPBodyCarrier If value implement HTTPBodyCarrier. http.Response.Body will pass to SetReader

type HTTPBytesCarrier added in v0.0.4

type HTTPBytesCarrier interface {
	SetBytes(bytes []byte)
}

HTTPBytesCarrier If value implement HTTPBytesCarrier. All the byte in http.Response.Body will pass to SetBytes

type HTTPHeadersCarrier added in v0.0.4

type HTTPHeadersCarrier interface {
	SetHeader(header map[string][]string)
}

HttpHeaderCarrier If value implement HttpHeaderCarrier. http.Response.Header will pass to SetHeader

type HTTPResponseCarrier added in v0.0.4

type HTTPResponseCarrier interface {
	SetHTTPResponse(resp *http.Response)
}

HTTPResponseCarrier If value implement HTTPResponseCarrier. http.Response will pass to SetHTTPResponse

type HTTPStatusCarrier added in v0.0.4

type HTTPStatusCarrier interface {
	SetStatusCode(status int)
}

HTTPStatusCarrier If value implement HTTPStatusCarrier. http.Response.StatusCode will pass to SetStatusCode

Directories

Path Synopsis
filter
log

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL