httplib

package
v1.0.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 4, 2024 License: Apache-2.0 Imports: 20 Imported by: 0

README

httplib

httplib is an libs help you to curl remote url.

How to use?

GET

you can use Get to crawl data.

import "github.com/rachelos/rungo/client/httplib"

str, err := httplib.Get("http://rungo.vip/").String()
if err != nil {
    	// error
}
fmt.Println(str)

POST

POST data to remote url

req := httplib.Post("http://rungo.vip/")
req.Param("username","astaxie")
req.Param("password","123456")
str, err := req.String()
if err != nil {
    	// error
}
fmt.Println(str)

Set timeout

The default timeout is 60 seconds, function prototype:

SetTimeout(connectTimeout, readWriteTimeout time.Duration)

Example:

// GET
httplib.Get("http://rungo.vip/").SetTimeout(100 * time.Second, 30 * time.Second)

// POST
httplib.Post("http://rungo.vip/").SetTimeout(100 * time.Second, 30 * time.Second)

Debug

If you want to debug the request info, set the debug on

httplib.Get("http://rungo.vip/").Debug(true)

Set HTTP Basic Auth

str, err := Get("http://rungo.vip/").SetBasicAuth("user", "passwd").String()
if err != nil {
    	// error
}
fmt.Println(str)

Set HTTPS

If request url is https, You can set the client support TSL:

httplib.SetTLSClientConfig(&tls.Config{InsecureSkipVerify: true})

More info about the tls.Config please visit http://golang.org/pkg/crypto/tls/#Config

Set HTTP Version

some servers need to specify the protocol version of HTTP

httplib.Get("http://rungo.vip/").SetProtocolVersion("HTTP/1.1")

some http request need setcookie. So set it like this:

cookie := &http.Cookie{}
cookie.Name = "username"
cookie.Value  = "astaxie"
httplib.Get("http://rungo.vip/").SetCookie(cookie)

Upload file

httplib support mutil file upload, use req.PostFile()

req := httplib.Post("http://rungo.vip/")
req.Param("username","astaxie")
req.PostFile("uploadfile1", "httplib.pdf")
str, err := req.String()
if err != nil {
    	// error
}
fmt.Println(str)

See godoc for further documentation and examples.

Documentation

Overview

Package httplib is used as http.Client Usage:

import "github.com/rachelos/rungo/client/httplib" 

b := httplib.Post("http://rungo.vip/")
b.Param("username","astaxie")
b.Param("password","123456")
b.PostFile("uploadfile1", "httplib.pdf")
b.PostFile("uploadfile2", "httplib.txt")
str, err := b.String()
if err != nil {
	t.Fatal(err)
}
fmt.Println(str)

Index

Constants

This section is empty.

Variables

View Source 
var CloseFileFailed = berror.DefineCode(5001004, moduleName, "CloseFileFailed", `
After handling files, rungo try to close file but failed. Usually it was caused by bad file descriptor.
`)
View Source 
var CopyFileFailed = berror.DefineCode(5001003, moduleName, "CopyFileFailed", `
When we try to read file content and then copy it to another writer, and failed.
1. Unexpected EOF;
2. Bad file descriptor;
3. Write conflict;

Please check your file content, and confirm that file is not processed by other process (or by user manually).
`)
View Source 
var CreateFileIfNotExistFailed = berror.DefineCode(5001007, moduleName, "CreateFileIfNotExist", `
rungo want to create file if not exist and failed. 
In most cases, it means that rungo doesn't have the privilege to create this file.
Please change file mode to ensure that rungo is able to create files on specific directory.
Or you can run rungo with higher authority.
In some cases, you pass invalid filename. Make sure that the file name is valid on your system.
`)
View Source 
var CreateFormFileFailed = berror.DefineCode(5001001, moduleName, "CreateFormFileFailed", `
In normal case than handling files with rungoRequest, you should not see this error code.
Unexpected EOF, invalid characters, bad file descriptor may cause this error.
`)
View Source 
var InvalidJSONBody = berror.DefineCode(4001006, moduleName, "InvalidJSONBody", `
You pass invalid data which could not be converted to JSON documents. In general, if you pass structure, it works well.
Sometimes you got JSON document and you want to make it as request body. So you call JSONBody.
If you do this, you got this code. Instead, you should call Header to set Content-type and call Body to set body data.
`)
View Source 
var InvalidURLOrMethod = berror.DefineCode(4001007, moduleName, "InvalidURLOrMethod", `
You pass invalid url or method to httplib module. Please check the url and method, be careful about special characters.
`)
View Source 
var InvalidUrl = berror.DefineCode(4001001, moduleName, "InvalidUrl", `
You pass an invalid url to httplib module. Please check your url, be careful about special character. 
`)
View Source 
var InvalidUrlProtocolVersion = berror.DefineCode(4001002, moduleName, "InvalidUrlProtocolVersion", `
You pass an invalid protocol version. In practice, we use HTTP/1.0, HTTP/1.1, HTTP/1.2
But something like HTTP/3.2 is valid for client, and the major version is 3, minor version is 2.
but you must confirm that server support those abnormal protocol version.
`)
View Source 
var InvalidXMLBody = berror.DefineCode(4001004, moduleName, "InvalidXMLBody", `
You pass invalid data which could not be converted to XML documents. In general, if you pass structure, it works well.
Sometimes you got XML document and you want to make it as request body. So you call XMLBody.
If you do this, you got this code. Instead, you should call Header to set Content-type and call Body to set body data.
`)
View Source 
var InvalidYAMLBody = berror.DefineCode(4001005, moduleName, "InvalidYAMLBody", `
You pass invalid data which could not be converted to YAML documents. In general, if you pass structure, it works well.
Sometimes you got YAML document and you want to make it as request body. So you call YAMLBody.
If you do this, you got this code. Instead, you should call Header to set Content-type and call Body to set body data.
`)
View Source 
var ReadFileFailed = berror.DefineCode(5001002, moduleName, "ReadFileFailed", `
There are several cases that cause this error:
1. file not found. Please check the file name;
2. file not found, but file name is correct. If you use relative file path, it's very possible for you to see this code.
make sure that this file is in correct directory which rungo looks for;
3. rungo don't have the privilege to read this file, please change file mode; 
`)
View Source 
var ReadGzipBodyFailed = berror.DefineCode(5001006, moduleName, "BuildGzipReaderFailed", `
rungo parse gzip-encode body failed. Usually rungo got invalid response.
Please confirm that server returns gzip data.
`)
View Source 
var SendRequestFailed = berror.DefineCode(5001005, moduleName, "SendRequestRetryExhausted", `
rungo send HTTP request, but it failed.
If you config retry times, it means that rungo had retried and failed.
When you got this error, there are vary kind of reason:
1. Network unstable and timeout. In this case, sometimes server has received the request.
2. Server error. Make sure that server works well.
3. The request is invalid, which means that you pass some invalid parameter.
`)
View Source 
var UnmarshalJSONResponseToObjectFailed = berror.DefineCode(5001008, moduleName,
	"UnmarshalResponseToObjectFailed", `
rungo trying to unmarshal response's body to structure but failed.
Make sure that:
1. You pass valid structure pointer to the function;
2. The body is valid json document
`)
View Source 
var UnmarshalResponseToObjectFailed = berror.DefineCode(5001011, moduleName,
	"UnmarshalResponseToObjectFailed", `
rungo trying to unmarshal response's body to structure but failed.
There are several cases that cause this error:
1. You pass valid structure pointer to the function;
2. The body is valid json, Yaml or XML document
`)
View Source 
var UnmarshalXMLResponseToObjectFailed = berror.DefineCode(5001009, moduleName,
	"UnmarshalResponseToObjectFailed", `
rungo trying to unmarshal response's body to structure but failed.
Make sure that:
1. You pass valid structure pointer to the function;
2. The body is valid XML document
`)
View Source 
var UnmarshalYAMLResponseToObjectFailed = berror.DefineCode(5001010, moduleName,
	"UnmarshalResponseToObjectFailed", `
rungo trying to unmarshal response's body to structure but failed.
Make sure that:
1. You pass valid structure pointer to the function;
2. The body is valid YAML document
`)
View Source 
var UnsupportedBodyType = berror.DefineCode(4001003, moduleName, "UnsupportedBodyType", `
You use an invalid data as request body.
For now, we only support type string and byte[].
`)

Functions

func AddDefaultFilter 

func AddDefaultFilter(fc FilterChain)

AddDefaultFilter add a new filter into defaultSetting Be careful about using this method if you invoke SetDefaultSetting somewhere

func GetDefaultSetting 

func GetDefaultSetting() rungoHTTPSettings

GetDefaultSetting return current default setting

func NewHttpResponseWithJsonBody 

func NewHttpResponseWithJsonBody(data interface{}) *http.Response

NewHttpResponseWithJsonBody will try to convert the data to json format usually you only use this when you want to mock http Response

func SetDefaultSetting 

func SetDefaultSetting(setting rungoHTTPSettings)

SetDefaultSetting overwrites default settings Keep in mind that when you invoke the SetDefaultSetting some methods invoked before SetDefaultSetting

func TimeoutDialer 

func TimeoutDialer(cTimeout time.Duration, rwTimeout time.Duration) func(net, addr string) (c net.Conn, err error)

TimeoutDialer returns functions of connection dialer with timeout settings for http.Transport Dial field. Deprecated we will move this at the end of 2021 please use TimeoutDialerCtx

func TimeoutDialerCtx 

func TimeoutDialerCtx(cTimeout time.Duration,
	rwTimeout time.Duration) func(ctx context.Context, net, addr string) (c net.Conn, err error)

Types

type Client 

type Client struct {
	Name       string
	Endpoint   string
	CommonOpts []RungoHTTPRequestOption

	Setting rungoHTTPSettings
}

Client provides an HTTP client supporting chain call

func NewClient 

func NewClient(name string, endpoint string, opts ...ClientOption) (*Client, error)

NewClient return a new http client

func (*Client) Delete 

func (c *Client) Delete(value interface{}, path string, opts ...RungoHTTPRequestOption) error

Delete Send a Delete request and try to give its result value

func (*Client) Get 

func (c *Client) Get(value interface{}, path string, opts ...RungoHTTPRequestOption) error

Get Send a GET request and try to give its result value

func (*Client) Head 

func (c *Client) Head(value interface{}, path string, opts ...RungoHTTPRequestOption) error

Head Send a Head request and try to give its result value

func (*Client) Post 

func (c *Client) Post(value interface{}, path string, body interface{}, opts ...RungoHTTPRequestOption) error

Post Send a POST request and try to give its result value

func (*Client) Put 

func (c *Client) Put(value interface{}, path string, body interface{}, opts ...RungoHTTPRequestOption) error

Put Send a Put request and try to give its result value

type ClientOption 

type ClientOption func(client *Client)

func WithCheckRedirect 

func WithCheckRedirect(redirect func(req *http.Request, via []*http.Request) error) ClientOption

WithCheckRedirect will specifies the policy for handling redirects in all subsequent request

func WithEnableCookie 

func WithEnableCookie(enable bool) ClientOption

WithEnableCookie will enable cookie in all subsequent request

func WithEnableGzip 

func WithEnableGzip(enable bool) ClientOption

WithEnableGzip will enable gzip in all subsequent request

func WithHTTPSetting 

func WithHTTPSetting(setting rungoHTTPSettings) ClientOption

WithHTTPSetting can replace rungoHTTPSeting

func WithProxy 

func WithProxy(proxy func(*http.Request) (*url.URL, error)) ClientOption

WithProxy will set http proxy field in all subsequent request

func WithTLSClientConfig 

func WithTLSClientConfig(config *tls.Config) ClientOption

WithTLSClientConfig will adds tls config in all subsequent request

func WithTransport 

func WithTransport(transport http.RoundTripper) ClientOption

WithTransport will set transport field in all subsequent request

func WithUserAgent 

func WithUserAgent(userAgent string) ClientOption

WithUserAgent will adds UA in all subsequent request

type Filter 

type Filter func(ctx context.Context, req *RungoHTTPRequest) (*http.Response, error)

type FilterChain 

type FilterChain func(next Filter) Filter

type HTTPBodyCarrier 

type HTTPBodyCarrier interface {
	SetReader(r io.ReadCloser)
}

HTTPBodyCarrier If value implement HTTPBodyCarrier. http.Response.Body will pass to SetReader

type HTTPBytesCarrier 

type HTTPBytesCarrier interface {
	SetBytes(bytes []byte)
}

HTTPBytesCarrier If value implement HTTPBytesCarrier. All the byte in http.Response.Body will pass to SetBytes

type HTTPHeadersCarrier 

type HTTPHeadersCarrier interface {
	SetHeader(header map[string][]string)
}

HTTPHeadersCarrier If value implement HttpHeaderCarrier. http.Response.Header will pass to SetHeader

type HTTPResponseCarrier 

type HTTPResponseCarrier interface {
	SetHTTPResponse(resp *http.Response)
}

HTTPResponseCarrier If value implement HTTPResponseCarrier. http.Response will pass to SetHTTPResponse

type HTTPStatusCarrier 

type HTTPStatusCarrier interface {
	SetStatusCode(status int)
}

HTTPStatusCarrier If value implement HTTPStatusCarrier. http.Response.StatusCode will pass to SetStatusCode

type RungoHTTPRequest added in v1.0.2 

type RungoHTTPRequest struct {
	// contains filtered or unexported fields
}

RungoHTTPRequest provides more useful methods than http.Request for requesting an url.

func Delete 

func Delete(url string) *RungoHTTPRequest

Delete returns *RungoHTTPRequest DELETE method.

func Get 

func Get(url string) *RungoHTTPRequest

Get returns *RungoHTTPRequest with GET method. 

func Head(url string) *RungoHTTPRequest

Head returns *RungoHTTPRequest with HEAD method.

func NewrungoRequest added in v1.0.2 

func NewrungoRequest(rawurl, method string) *RungoHTTPRequest

NewrungoRequest returns *RungoHTTPRequest with specific method TODO add error as return value I think if we don't return error users are hard to check whether we create rungo request successfully

func NewrungoRequestWithCtx added in v1.0.2 

func NewrungoRequestWithCtx(ctx context.Context, rawurl, method string) *RungoHTTPRequest

NewrungoRequestWithCtx returns a new RungoHTTPRequest given a method, URL

func Post 

func Post(url string) *RungoHTTPRequest

Post returns *RungoHTTPRequest with POST method.

func Put 

func Put(url string) *RungoHTTPRequest

Put returns *RungoHTTPRequest with PUT method.

func (*RungoHTTPRequest) AddFilters added in v1.0.2 

func (b *RungoHTTPRequest) AddFilters(fcs ...FilterChain) *RungoHTTPRequest

AddFilters adds filter

func (*RungoHTTPRequest) Body added in v1.0.2 

func (b *RungoHTTPRequest) Body(data interface{}) *RungoHTTPRequest

Body adds request raw body. Supports string and []byte. TODO return error if data is invalid

func (*RungoHTTPRequest) Bytes added in v1.0.2 

func (b *RungoHTTPRequest) Bytes() ([]byte, error)

Bytes returns the body []byte in response. Calls Response inner.

func (*RungoHTTPRequest) DoRequest added in v1.0.2 

func (b *RungoHTTPRequest) DoRequest() (resp *http.Response, err error)

DoRequest executes client.Do

func (*RungoHTTPRequest) DoRequestWithCtx deprecated added in v1.0.2 

func (b *RungoHTTPRequest) DoRequestWithCtx(ctx context.Context) (resp *http.Response, err error)

Deprecated: please use NewrungoRequestWithContext

func (*RungoHTTPRequest) GetRequest added in v1.0.2 

func (b *RungoHTTPRequest) GetRequest() *http.Request

GetRequest returns the request object

func (*RungoHTTPRequest) Header added in v1.0.2 

func (b *RungoHTTPRequest) Header(key, value string) *RungoHTTPRequest

Header adds header item string in request.

func (*RungoHTTPRequest) JSONBody added in v1.0.2 

func (b *RungoHTTPRequest) JSONBody(obj interface{}) (*RungoHTTPRequest, error)

JSONBody adds the request raw body encoded in JSON.

func (*RungoHTTPRequest) JSONMarshal added in v1.0.2 

func (b *RungoHTTPRequest) JSONMarshal(obj interface{}) ([]byte, error)

func (*RungoHTTPRequest) Param added in v1.0.2 

func (b *RungoHTTPRequest) Param(key, value string) *RungoHTTPRequest

Param adds query param in to request. params build query string as ?key1=value1&key2=value2...

func (*RungoHTTPRequest) PostFile added in v1.0.2 

func (b *RungoHTTPRequest) PostFile(formname, filename string) *RungoHTTPRequest

PostFile adds a post file to the request

func (*RungoHTTPRequest) Response added in v1.0.2 

func (b *RungoHTTPRequest) Response() (*http.Response, error)

Response executes request client gets response manually.

func (*RungoHTTPRequest) Retries added in v1.0.2 

func (b *RungoHTTPRequest) Retries(times int) *RungoHTTPRequest

Retries sets Retries times. default is 0 (never retry) -1 retry indefinitely (forever) Other numbers specify the exact retry amount

func (*RungoHTTPRequest) RetryDelay added in v1.0.2 

func (b *RungoHTTPRequest) RetryDelay(delay time.Duration) *RungoHTTPRequest

RetryDelay sets the time to sleep between reconnection attempts

func (*RungoHTTPRequest) SetBasicAuth added in v1.0.2 

func (b *RungoHTTPRequest) SetBasicAuth(username, password string) *RungoHTTPRequest

SetBasicAuth sets the request's Authorization header to use HTTP Basic Authentication with the provided username and password.

func (*RungoHTTPRequest) SetCheckRedirect added in v1.0.2 

func (b *RungoHTTPRequest) SetCheckRedirect(redirect func(req *http.Request, via []*http.Request) error) *RungoHTTPRequest

SetCheckRedirect specifies the policy for handling redirects.

If CheckRedirect is nil, the Client uses its default policy, which is to stop after 10 consecutive requests.

func (*RungoHTTPRequest) SetCookie added in v1.0.2 

func (b *RungoHTTPRequest) SetCookie(cookie *http.Cookie) *RungoHTTPRequest

SetCookie adds a cookie to the request.

func (*RungoHTTPRequest) SetEnableCookie added in v1.0.2 

func (b *RungoHTTPRequest) SetEnableCookie(enable bool) *RungoHTTPRequest

SetEnableCookie sets enable/disable cookiejar

func (*RungoHTTPRequest) SetEscapeHTML added in v1.0.2 

func (b *RungoHTTPRequest) SetEscapeHTML(isEscape bool) *RungoHTTPRequest

SetEscapeHTML is used to set the flag whether escape HTML special characters during processing

func (*RungoHTTPRequest) SetFilters added in v1.0.2 

func (b *RungoHTTPRequest) SetFilters(fcs ...FilterChain) *RungoHTTPRequest

SetFilters will use the filter as the invocation filters

func (*RungoHTTPRequest) SetHost added in v1.0.2 

func (b *RungoHTTPRequest) SetHost(host string) *RungoHTTPRequest

SetHost set the request host

func (*RungoHTTPRequest) SetProtocolVersion added in v1.0.2 

func (b *RungoHTTPRequest) SetProtocolVersion(vers string) *RungoHTTPRequest

SetProtocolVersion sets the protocol version for incoming requests. Client requests always use HTTP/1.1

func (*RungoHTTPRequest) SetProxy added in v1.0.2 

func (b *RungoHTTPRequest) SetProxy(proxy func(*http.Request) (*url.URL, error)) *RungoHTTPRequest

SetProxy sets the HTTP proxy example: 

func(req *http.Request) (*url.URL, error) {
	u, _ := url.ParseRequestURI("http://127.0.0.1:8118")
	return u, nil
}

func (*RungoHTTPRequest) SetTLSClientConfig added in v1.0.2 

func (b *RungoHTTPRequest) SetTLSClientConfig(config *tls.Config) *RungoHTTPRequest

SetTLSClientConfig sets TLS connection configuration if visiting HTTPS url.

func (*RungoHTTPRequest) SetTimeout added in v1.0.2 

func (b *RungoHTTPRequest) SetTimeout(connectTimeout, readWriteTimeout time.Duration) *RungoHTTPRequest

SetTimeout sets connect time out and read-write time out for rungoRequest.

func (*RungoHTTPRequest) SetTransport added in v1.0.2 

func (b *RungoHTTPRequest) SetTransport(transport http.RoundTripper) *RungoHTTPRequest

SetTransport sets the transport field

func (*RungoHTTPRequest) SetUserAgent added in v1.0.2 

func (b *RungoHTTPRequest) SetUserAgent(useragent string) *RungoHTTPRequest

SetUserAgent sets User-Agent header field

func (*RungoHTTPRequest) Setting added in v1.0.2 

func (b *RungoHTTPRequest) Setting(setting rungoHTTPSettings) *RungoHTTPRequest

Setting changes request settings

func (*RungoHTTPRequest) String added in v1.0.2 

func (b *RungoHTTPRequest) String() (string, error)

String returns the body string in response. Calls Response inner.

func (*RungoHTTPRequest) ToFile added in v1.0.2 

func (b *RungoHTTPRequest) ToFile(filename string) error

ToFile saves the body data in response to one file. Calls Response inner.

func (*RungoHTTPRequest) ToJSON added in v1.0.2 

func (b *RungoHTTPRequest) ToJSON(v interface{}) error

ToJSON returns the map that marshals from the body bytes as json in response. Calls Response inner.

func (*RungoHTTPRequest) ToValue added in v1.0.2 

func (b *RungoHTTPRequest) ToValue(value interface{}) error

ToValue attempts to resolve the response body to value using an existing method. Calls Response inner. If response header contain Content-Type, func will call ToJSON\ToXML\ToYAML. Else it will try to parse body as json\yaml\xml, If all attempts fail, an error will be returned

func (*RungoHTTPRequest) ToXML added in v1.0.2 

func (b *RungoHTTPRequest) ToXML(v interface{}) error

ToXML returns the map that marshals from the body bytes as xml in response . Calls Response inner.

func (*RungoHTTPRequest) ToYAML added in v1.0.2 

func (b *RungoHTTPRequest) ToYAML(v interface{}) error

ToYAML returns the map that marshals from the body bytes as yaml in response . Calls Response inner.

func (*RungoHTTPRequest) XMLBody added in v1.0.2 

func (b *RungoHTTPRequest) XMLBody(obj interface{}) (*RungoHTTPRequest, error)

XMLBody adds the request raw body encoded in XML.

func (*RungoHTTPRequest) YAMLBody added in v1.0.2 

func (b *RungoHTTPRequest) YAMLBody(obj interface{}) (*RungoHTTPRequest, error)

YAMLBody adds the request raw body encoded in YAML.

type RungoHTTPRequestOption added in v1.0.2 

type RungoHTTPRequestOption func(request *RungoHTTPRequest)

func WithBasicAuth 

func WithBasicAuth(basicAuth func() (string, string)) RungoHTTPRequestOption

WithBasicAuth adds a custom function to set basic auth

func WithContentType 

func WithContentType(contentType string) RungoHTTPRequestOption

WithContentType adds ContentType in header

func WithCookie 

func WithCookie(cookie *http.Cookie) RungoHTTPRequestOption

WithCookie adds a cookie to the request.

func WithFilters 

func WithFilters(fcs ...FilterChain) RungoHTTPRequestOption

WithFilters will use the filter as the invocation filters

func WithHeader 

func WithHeader(key, value string) RungoHTTPRequestOption

WithHeader adds header item string in request.

func WithParam 

func WithParam(key, value string) RungoHTTPRequestOption

WithParam adds query param in to request.

func WithRetry 

func WithRetry(times int, delay time.Duration) RungoHTTPRequestOption

WithRetry set retry times and delay for the request default is 0 (never retry) -1 retry indefinitely (forever) Other numbers specify the exact retry amount

func WithTimeout 

func WithTimeout(connectTimeout, readWriteTimeout time.Duration) RungoHTTPRequestOption

WithTimeout sets connect time out and read-write time out for rungoRequest.

func WithTokenFactory 

func WithTokenFactory(tokenFactory func() string) RungoHTTPRequestOption

WithTokenFactory adds a custom function to set Authorization

Source Files

View all Source files

Directories

Path Synopsis
filter
log
opentelemetry
opentracing
prometheus

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.