httplib

package module
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 24, 2021 License: MIT Imports: 21 Imported by: 0

README

httplib

httplib is an libs help you to curl remote url.

How to use?

GET

you can use Get to crawl data.

import "github.com/Pow-Duck/httplib"

str, err := httplib.Get("http://beego.me/").String()
if err != nil {
    	// error
}
fmt.Println(str)

POST

POST data to remote url

req := httplib.Post("http://beego.me/")
req.Param("username","astaxie")
req.Param("password","123456")
str, err := req.String()
if err != nil {
    	// error
}
fmt.Println(str)

Set timeout

The default timeout is 60 seconds, function prototype:

SetTimeout(connectTimeout, readWriteTimeout time.Duration)

Example:

// GET
httplib.Get("http://beego.me/").SetTimeout(100 * time.Second, 30 * time.Second)

// POST
httplib.Post("http://beego.me/").SetTimeout(100 * time.Second, 30 * time.Second)

Debug

If you want to debug the request info, set the debug on

httplib.Get("http://beego.me/").Debug(true)

Set HTTP Basic Auth

str, err := Get("http://beego.me/").SetBasicAuth("user", "passwd").String()
if err != nil {
    	// error
}
fmt.Println(str)

Set HTTPS

If request url is https, You can set the client support TSL:

httplib.SetTLSClientConfig(&tls.Config{InsecureSkipVerify: true})

More info about the tls.Config please visit http://golang.org/pkg/crypto/tls/#Config

Set HTTP Version

some servers need to specify the protocol version of HTTP

httplib.Get("http://beego.me/").SetProtocolVersion("HTTP/1.1")

some http request need setcookie. So set it like this:

cookie := &http.Cookie{}
cookie.Name = "username"
cookie.Value  = "astaxie"
httplib.Get("http://beego.me/").SetCookie(cookie)

Upload file

httplib support mutil file upload, use req.PostFile()

req := httplib.Post("http://beego.me/")
req.Param("username","astaxie")
req.PostFile("uploadfile1", "httplib.pdf")
str, err := req.String()
if err != nil {
    	// error
}
fmt.Println(str)

See godoc for further documentation and examples.

Documentation

Overview

Package httplib is used as http.Client Usage:

import "github.com/Pow-Duck/httplib" 

	b := httplib.Post("http://beego.me/")
	b.Param("username","astaxie")
	b.Param("password","123456")
	b.PostFile("uploadfile1", "httplib.pdf")
	b.PostFile("uploadfile2", "httplib.txt")
	str, err := b.String()
	if err != nil {
		t.Fatal(err)
	}
	fmt.Println(str)

 more docs http://beego.me/docs/module/httplib.md

Index

Constants

This section is empty.

Variables

View Source 
var CloseFileFailed = berror.DefineCode(5001004, moduleName, "CloseFileFailed", `
After handling files, Beego try to close file but failed. Usually it was caused by bad file descriptor.
`)
View Source 
var CopyFileFailed = berror.DefineCode(5001003, moduleName, "CopyFileFailed", `
When we try to read file content and then copy it to another writer, and failed.
1. Unexpected EOF;
2. Bad file descriptor;
3. Write conflict;

Please check your file content, and confirm that file is not processed by other process (or by user manually).
`)
View Source 
var CreateFileIfNotExistFailed = berror.DefineCode(5001007, moduleName, "CreateFileIfNotExist", `
Beego want to create file if not exist and failed. 
In most cases, it means that Beego doesn't have the privilege to create this file.
Please change file mode to ensure that Beego is able to create files on specific directory.
Or you can run Beego with higher authority.
In some cases, you pass invalid filename. Make sure that the file name is valid on your system.
`)
View Source 
var CreateFormFileFailed = berror.DefineCode(5001001, moduleName, "CreateFormFileFailed", `
In normal case than handling files with BeegoRequest, you should not see this error code.
Unexpected EOF, invalid characters, bad file descriptor may cause this error.
`)
View Source 
var InvalidJSONBody = berror.DefineCode(4001006, moduleName, "InvalidJSONBody", `
You pass invalid data which could not be converted to JSON documents. In general, if you pass structure, it works well.
Sometimes you got JSON document and you want to make it as request body. So you call JSONBody.
If you do this, you got this code. Instead, you should call Header to set Content-type and call Body to set body data.
`)
View Source 
var InvalidUrl = berror.DefineCode(4001001, moduleName, "InvalidUrl", `
You pass a invalid url to httplib module. Please check your url, be careful about special character. 
`)
View Source 
var InvalidUrlProtocolVersion = berror.DefineCode(4001002, moduleName, "InvalidUrlProtocolVersion", `
You pass a invalid protocol version. In practice, we use HTTP/1.0, HTTP/1.1, HTTP/1.2
But something like HTTP/3.2 is valid for client, and the major version is 3, minor version is 2.
but you must confirm that server support those abnormal protocol version.
`)
View Source 
var InvalidXMLBody = berror.DefineCode(4001004, moduleName, "InvalidXMLBody", `
You pass invalid data which could not be converted to XML documents. In general, if you pass structure, it works well.
Sometimes you got XML document and you want to make it as request body. So you call XMLBody.
If you do this, you got this code. Instead, you should call Header to set Content-type and call Body to set body data.
`)
View Source 
var InvalidYAMLBody = berror.DefineCode(4001005, moduleName, "InvalidYAMLBody", `
You pass invalid data which could not be converted to YAML documents. In general, if you pass structure, it works well.
Sometimes you got YAML document and you want to make it as request body. So you call YAMLBody.
If you do this, you got this code. Instead, you should call Header to set Content-type and call Body to set body data.
`)
View Source 
var ReadFileFailed = berror.DefineCode(5001002, moduleName, "ReadFileFailed", `
There are several cases that cause this error:
1. file not found. Please check the file name;
2. file not found, but file name is correct. If you use relative file path, it's very possible for you to see this code.
make sure that this file is in correct directory which Beego looks for;
3. Beego don't have the privilege to read this file, please change file mode; 
`)
View Source 
var ReadGzipBodyFailed = berror.DefineCode(5001006, moduleName, "BuildGzipReaderFailed", `
Beego parse gzip-encode body failed. Usually Beego got invalid response.
Please confirm that server returns gzip data.
`)
View Source 
var SendRequestFailed = berror.DefineCode(5001005, moduleName, "SendRequestRetryExhausted", `
Beego send HTTP request, but it failed.
If you config retry times, it means that Beego had retried and failed.
When you got this error, there are vary kind of reason:
1. Network unstable and timeout. In this case, sometimes server has received the request.
2. Server error. Make sure that server works well.
3. The request is invalid, which means that you pass some invalid parameter.
`)
View Source 
var UnmarshalJSONResponseToObjectFailed = berror.DefineCode(5001008, moduleName,
	"UnmarshalResponseToObjectFailed", `
Beego trying to unmarshal response's body to structure but failed.
Make sure that:
1. You pass valid structure pointer to the function;
2. The body is valid json document
`)
View Source 
var UnmarshalXMLResponseToObjectFailed = berror.DefineCode(5001009, moduleName,
	"UnmarshalResponseToObjectFailed", `
Beego trying to unmarshal response's body to structure but failed.
Make sure that:
1. You pass valid structure pointer to the function;
2. The body is valid XML document
`)
View Source 
var UnmarshalYAMLResponseToObjectFailed = berror.DefineCode(5001010, moduleName,
	"UnmarshalResponseToObjectFailed", `
Beego trying to unmarshal response's body to structure but failed.
Make sure that:
1. You pass valid structure pointer to the function;
2. The body is valid YAML document
`)
View Source 
var UnsupportedBodyType = berror.DefineCode(4001003, moduleName, "UnsupportedBodyType", `
You use a invalid data as request body.
For now, we only support type string and byte[].
`)

Functions

func AddDefaultFilter 

func AddDefaultFilter(fc FilterChain)

AddDefaultFilter add a new filter into defaultSetting Be careful about using this method if you invoke SetDefaultSetting somewhere

func NewHttpResponseWithJsonBody 

func NewHttpResponseWithJsonBody(data interface{}) *http.Response

NewHttpResponseWithJsonBody will try to convert the data to json format usually you only use this when you want to mock http Response

func SetDefaultSetting 

func SetDefaultSetting(setting BeegoHTTPSettings)

SetDefaultSetting overwrites default settings Keep in mind that when you invoke the SetDefaultSetting some methods invoked before SetDefaultSetting

func TimeoutDialer 

func TimeoutDialer(cTimeout time.Duration, rwTimeout time.Duration) func(net, addr string) (c net.Conn, err error)

TimeoutDialer returns functions of connection dialer with timeout settings for http.Transport Dial field. Deprecated we will move this at the end of 2021 please use TimeoutDialerCtx

func TimeoutDialerCtx 

func TimeoutDialerCtx(cTimeout time.Duration,
	rwTimeout time.Duration) func(ctx context.Context, net, addr string) (c net.Conn, err error)

Types

type BeegoHTTPRequest 

type BeegoHTTPRequest struct {
	// contains filtered or unexported fields
}

BeegoHTTPRequest provides more useful methods than http.Request for requesting a url.

func Delete 

func Delete(url string) *BeegoHTTPRequest

Delete returns *BeegoHttpRequest DELETE method.

func Get 

func Get(url string) *BeegoHTTPRequest

Get returns *BeegoHttpRequest with GET method. 

func Head(url string) *BeegoHTTPRequest

Head returns *BeegoHttpRequest with HEAD method.

func NewBeegoRequest 

func NewBeegoRequest(rawurl, method string) *BeegoHTTPRequest

NewBeegoRequest returns *BeegoHttpRequest with specific method TODO add error as return value I think if we don't return error users are hard to check whether we create Beego request successfully

func Post 

func Post(url string) *BeegoHTTPRequest

Post returns *BeegoHttpRequest with POST method.

func Put 

func Put(url string) *BeegoHTTPRequest

Put returns *BeegoHttpRequest with PUT method.

func (*BeegoHTTPRequest) AddFilters 

func (b *BeegoHTTPRequest) AddFilters(fcs ...FilterChain) *BeegoHTTPRequest

AddFilters adds filter

func (*BeegoHTTPRequest) Body 

func (b *BeegoHTTPRequest) Body(data interface{}) *BeegoHTTPRequest

Body adds request raw body. Supports string and []byte. TODO return error if data is invalid

func (*BeegoHTTPRequest) Bytes 

func (b *BeegoHTTPRequest) Bytes() ([]byte, error)

Bytes returns the body []byte in response. Calls Response inner.

func (*BeegoHTTPRequest) DoRequest 

func (b *BeegoHTTPRequest) DoRequest() (resp *http.Response, err error)

DoRequest executes client.Do

func (*BeegoHTTPRequest) DoRequestWithCtx 

func (b *BeegoHTTPRequest) DoRequestWithCtx(ctx context.Context) (resp *http.Response, err error)

func (*BeegoHTTPRequest) GetRequest 

func (b *BeegoHTTPRequest) GetRequest() *http.Request

GetRequest returns the request object

func (*BeegoHTTPRequest) Header 

func (b *BeegoHTTPRequest) Header(key, value string) *BeegoHTTPRequest

Header adds header item string in request.

func (*BeegoHTTPRequest) JSONBody 

func (b *BeegoHTTPRequest) JSONBody(obj interface{}) (*BeegoHTTPRequest, error)

JSONBody adds the request raw body encoded in JSON.

func (*BeegoHTTPRequest) Param 

func (b *BeegoHTTPRequest) Param(key, value string) *BeegoHTTPRequest

Param adds query param in to request. params build query string as ?key1=value1&key2=value2...

func (*BeegoHTTPRequest) PostFile 

func (b *BeegoHTTPRequest) PostFile(formname, filename string) *BeegoHTTPRequest

PostFile adds a post file to the request

func (*BeegoHTTPRequest) Response 

func (b *BeegoHTTPRequest) Response() (*http.Response, error)

Response executes request client gets response manually.

func (*BeegoHTTPRequest) Retries 

func (b *BeegoHTTPRequest) Retries(times int) *BeegoHTTPRequest

Retries sets Retries times. default is 0 (never retry) -1 retry indefinitely (forever) Other numbers specify the exact retry amount

func (*BeegoHTTPRequest) RetryDelay 

func (b *BeegoHTTPRequest) RetryDelay(delay time.Duration) *BeegoHTTPRequest

RetryDelay sets the time to sleep between reconnection attempts

func (*BeegoHTTPRequest) SetBasicAuth 

func (b *BeegoHTTPRequest) SetBasicAuth(username, password string) *BeegoHTTPRequest

SetBasicAuth sets the request's Authorization header to use HTTP Basic Authentication with the provided username and password.

func (*BeegoHTTPRequest) SetCheckRedirect 

func (b *BeegoHTTPRequest) SetCheckRedirect(redirect func(req *http.Request, via []*http.Request) error) *BeegoHTTPRequest

SetCheckRedirect specifies the policy for handling redirects.

If CheckRedirect is nil, the Client uses its default policy, which is to stop after 10 consecutive requests.

func (*BeegoHTTPRequest) SetCookie 

func (b *BeegoHTTPRequest) SetCookie(cookie *http.Cookie) *BeegoHTTPRequest

SetCookie adds a cookie to the request.

func (*BeegoHTTPRequest) SetEnableCookie 

func (b *BeegoHTTPRequest) SetEnableCookie(enable bool) *BeegoHTTPRequest

SetEnableCookie sets enable/disable cookiejar

func (*BeegoHTTPRequest) SetFilters 

func (b *BeegoHTTPRequest) SetFilters(fcs ...FilterChain) *BeegoHTTPRequest

SetFilters will use the filter as the invocation filters

func (*BeegoHTTPRequest) SetHost 

func (b *BeegoHTTPRequest) SetHost(host string) *BeegoHTTPRequest

SetHost set the request host

func (*BeegoHTTPRequest) SetProtocolVersion 

func (b *BeegoHTTPRequest) SetProtocolVersion(vers string) *BeegoHTTPRequest

SetProtocolVersion sets the protocol version for incoming requests. Client requests always use HTTP/1.1

func (*BeegoHTTPRequest) SetProxy 

func (b *BeegoHTTPRequest) SetProxy(proxy func(*http.Request) (*url.URL, error)) *BeegoHTTPRequest

SetProxy sets the HTTP proxy example: 

func(req *http.Request) (*url.URL, error) {
	u, _ := url.ParseRequestURI("http://127.0.0.1:8118")
	return u, nil
}

func (*BeegoHTTPRequest) SetTLSClientConfig 

func (b *BeegoHTTPRequest) SetTLSClientConfig(config *tls.Config) *BeegoHTTPRequest

SetTLSClientConfig sets TLS connection configuration if visiting HTTPS url.

func (*BeegoHTTPRequest) SetTimeout 

func (b *BeegoHTTPRequest) SetTimeout(connectTimeout, readWriteTimeout time.Duration) *BeegoHTTPRequest

SetTimeout sets connect time out and read-write time out for BeegoRequest.

func (*BeegoHTTPRequest) SetTransport 

func (b *BeegoHTTPRequest) SetTransport(transport http.RoundTripper) *BeegoHTTPRequest

SetTransport sets the transport field

func (*BeegoHTTPRequest) SetUserAgent 

func (b *BeegoHTTPRequest) SetUserAgent(useragent string) *BeegoHTTPRequest

SetUserAgent sets User-Agent header field

func (*BeegoHTTPRequest) Setting 

func (b *BeegoHTTPRequest) Setting(setting BeegoHTTPSettings) *BeegoHTTPRequest

Setting changes request settings

func (*BeegoHTTPRequest) String 

func (b *BeegoHTTPRequest) String() (string, error)

String returns the body string in response. Calls Response inner.

func (*BeegoHTTPRequest) ToFile 

func (b *BeegoHTTPRequest) ToFile(filename string) error

ToFile saves the body data in response to one file. Calls Response inner.

func (*BeegoHTTPRequest) ToJSON 

func (b *BeegoHTTPRequest) ToJSON(v interface{}) error

ToJSON returns the map that marshals from the body bytes as json in response. Calls Response inner.

func (*BeegoHTTPRequest) ToXML 

func (b *BeegoHTTPRequest) ToXML(v interface{}) error

ToXML returns the map that marshals from the body bytes as xml in response . Calls Response inner.

func (*BeegoHTTPRequest) ToYAML 

func (b *BeegoHTTPRequest) ToYAML(v interface{}) error

ToYAML returns the map that marshals from the body bytes as yaml in response . Calls Response inner.

func (*BeegoHTTPRequest) XMLBody 

func (b *BeegoHTTPRequest) XMLBody(obj interface{}) (*BeegoHTTPRequest, error)

XMLBody adds the request raw body encoded in XML.

func (*BeegoHTTPRequest) YAMLBody 

func (b *BeegoHTTPRequest) YAMLBody(obj interface{}) (*BeegoHTTPRequest, error)

YAMLBody adds the request raw body encoded in YAML.

type BeegoHTTPSettings 

type BeegoHTTPSettings struct {
	UserAgent        string
	ConnectTimeout   time.Duration
	ReadWriteTimeout time.Duration
	TLSClientConfig  *tls.Config
	Proxy            func(*http.Request) (*url.URL, error)
	Transport        http.RoundTripper
	CheckRedirect    func(req *http.Request, via []*http.Request) error
	EnableCookie     bool
	Gzip             bool
	Retries          int // if set to -1 means will retry forever
	RetryDelay       time.Duration
	FilterChains     []FilterChain
}

BeegoHTTPSettings is the http.Client setting

type Filter 

type Filter func(ctx context.Context, req *BeegoHTTPRequest) (*http.Response, error)

type FilterChain 

type FilterChain func(next Filter) Filter

Source Files

View all Source files

Directories

Path Synopsis
filter
log
Package logs provide a general log interface Usage: import "github.com/Pow-Duck/httplib/logs" log := NewLogger(10000) log.SetLogger("console", "") > the first params stand for how many channel Use it like this: log.Trace("trace") log.Info("info") log.Warn("warning") log.Debug("debug") log.Critical("critical") more docs http://beego.me/docs/module/logs.md
 Package logs provide a general log interface Usage: import "github.com/Pow-Duck/httplib/logs" log := NewLogger(10000) log.SetLogger("console", "") > the first params stand for how many channel Use it like this: log.Trace("trace") log.Info("info") log.Warn("warning") log.Debug("debug") log.Critical("critical") more docs http://beego.me/docs/module/logs.md
alils
Package alils implements the SDK(v0.5.0) of Simple Log Service(abbr.
 Package alils implements the SDK(v0.5.0) of Simple Log Service(abbr.
es

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.