requests

package module
v0.0.0-...-f099ba3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 24, 2024 License: Apache-2.0 Imports: 20 Imported by: 2

README

requests

Requests is a simple, yet elegant, Go HTTP client and server library for Humans™ ✨🍰✨
Go Reference Apache V2 License Build status go report requests on Sourcegraph
API Reference and User Guide available on Read the Docs
Supported Features & Best–Practices
  • Safe Close resp.Body
  • Only depends on standard library
  • Streaming Downloads
  • Chunked HTTP Requests
  • Keep-Alive & Connection Pooling
  • Sessions with Cookie Persistence
  • Basic & Digest Authentication
  • Implement http.RoundTripper fully compatible net/http
  • Offer File System to upload or download files easily
Quick Start
Get Started
cat github.com/golang-io/examples/example_1/main.go

package main

import (
	"context"
	"github.com/golang-io/requests"
)

func main() {
	sess := requests.New(requests.URL("https://httpbin.org/uuid"), requests.TraceLv(4))
	_, _ = sess.DoRequest(context.TODO())
}


$ go run github.com/golang-io/examples/example_1/main.go
* Connect: httpbin.org:80
* Got Conn: <nil> -> <nil>
* Connect: httpbin.org:443
* Resolved Host: httpbin.org
* Resolved DNS: [50.16.63.240 107.21.176.221 3.220.97.10 18.208.241.22], Coalesced: false, err=<nil>
* Trying ConnectStart tcp 50.16.63.240:443...
* Completed connection: tcp 50.16.63.240:443, err=<nil>
* SSL HandshakeComplete: true
* Got Conn: 192.168.255.10:64170 -> 50.16.63.240:443
> GET /uuid HTTP/1.1
> Host: httpbin.org
> User-Agent: Go-http-client/1.1
> Accept-Encoding: gzip
> 
> 

< HTTP/1.1 200 OK
< Content-Length: 53
< Access-Control-Allow-Credentials: true
< Access-Control-Allow-Origin: *
< Connection: keep-alive
< Content-Type: application/json
< Date: Fri, 22 Mar 2024 12:16:04 GMT
< Server: gunicorn/19.9.0
< 
< {
<   "uuid": "ba0a69b3-25d0-415e-b998-030120052f4d"
< }
< 

*
  • use requests.New() method to create a global session for http client.
  • use requests.URL() method to define a sever address to request.
  • use requests.TraceLv() method to open http trace mode.
  • use DoRequest() method to send a request from local to remote server.

Also, you can using simple method like requests.Get(), requests.Post() etc. to send a request, and return *http.Response, error. This is fully compatible with net/http method.

Simple Get
package main

import (
	"bytes"
	"github.com/golang-io/requests"
	"log"
)

func main() {
	resp, err := requests.Get("https://httpbin.org/get")
	if err != nil {
		log.Fatalln(err)
	}
	defer resp.Body.Close()
	var buf bytes.Buffer
	if _, err := buf.ReadFrom(resp.Body); err != nil {
		log.Fatalln(err)
	}
	log.Println(buf.String())
}

% go run github.com/golang-io/examples/example_2/main.go
2024/03/22 20:31:12 {
  "args": {}, 
  "headers": {
    "Host": "httpbin.org", 
    "User-Agent": "Go-http-client/1.1", 
    "X-Amzn-Trace-Id": "Root=1-65fd7a10-781981cc111ac4662510a87e"
  }, 
  "origin": "43.132.141.21", 
  "url": "https://httpbin.org/get"
}
Auto handle response.Body

There are many negative cases, network connections not released or memory cannot be released, because the response.Body is not closed correctly. To solve this problem, requests offers type *requests.Response. The response body sample read from response.Content. There is no need to declare a bunch of variables and duplicate code just for reading the response.Body. Additionally, the body will be safely closed, regardless of whether you need to read it or not.

For example:

package main

import (
	"context"
	"github.com/golang-io/requests"
	"log"
)

func main() {
	sess := requests.New(requests.URL("http://httpbin.org/post"))
	resp, err := sess.DoRequest(context.TODO(), requests.MethodPost, requests.Body("Hello world"))
	log.Printf("resp=%s, err=%v", resp.Content, err)
}


% go run github.com/golang-io/examples/example_3/main.go
2024/03/22 20:43:25 resp={
  "args": {}, 
  "data": "Hello world", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Content-Length": "11", 
    "Host": "httpbin.org", 
    "User-Agent": "Go-http-client/1.1", 
    "X-Amzn-Trace-Id": "Root=1-65fd7ced-718974b7528527911b977e1e"
  }, 
  "json": null, 
  "origin": "127.0.0.1", 
  "url": "http://httpbin.org/post"
}
, err=<nil>
Usage
Common Rules

All params can set by two level, session and request. session params is persistence, which is set by all request from this session. Request params is provisional, which is set by one request from this session.

Such as session params: ``

Such as request params: ``

Debugging - Log/Trace
Set Body
Set Header
Authentication
Gzip compress
Request and Response Middleware
Client and Transport Middleware
Proxy
Retry
Transport and RoundTripper
Example

Server

package main

import (
	"context"
	"fmt"
	"github.com/go-chi/chi/v5/middleware"
	"github.com/golang-io/requests"
	"github.com/gorilla/websocket"
	"io"
	"log"
	"net/http"
)

var upgrader = websocket.Upgrader{
	ReadBufferSize:  1024,
	WriteBufferSize: 1024,
	CheckOrigin: func(r *http.Request) bool {
		return true
	},
}

func ws(w http.ResponseWriter, r *http.Request) {
	conn, err := upgrader.Upgrade(w, r, nil)
	if err != nil {
		http.Error(w, err.Error(), http.StatusInternalServerError)
		return
	}
	defer conn.Close()
	for {
		messageType, message, err := conn.ReadMessage()
		if err != nil {
			log.Printf("Failed to read message: %v", err)
			break
		}

		log.Printf("Received message: %s", message)

		err = conn.WriteMessage(messageType, message)
		if err != nil {
			log.Printf("Failed to write message: %v", err)
			break
		}
	}
}

func main() {
	r := requests.NewServeMux(
		requests.URL("0.0.0.0:1234"),
		requests.Use(middleware.Recoverer, middleware.Logger),
		requests.Use(func(next http.Handler) http.Handler {
			return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
				next.ServeHTTP(w, r)
			})
		}),
	)
	r.Route("/panic", func(w http.ResponseWriter, r *http.Request) {
		panic("panic test")
	})
	r.Route("/echo", func(w http.ResponseWriter, r *http.Request) {
		_, _ = io.Copy(w, r.Body)
	})
	r.Route("/ping", func(w http.ResponseWriter, r *http.Request) {
		_, _ = fmt.Fprintf(w, "pong\n")
	}, requests.Use(func(next http.Handler) http.Handler {
		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			next.ServeHTTP(w, r)
		})
	}))
	r.Route("/ws", ws)
	err := requests.ListenAndServe(context.Background(), r)
	fmt.Println(err)
}

Then, do some requests...

% curl http://127.0.0.1:1234/echo
% curl http://127.0.0.1:1234/ping
pong

And, there are some logs from server.

% go run github.com/golang-io/examples/server/example_1/main.go
2024-03-27 02:47:47 http serve 0.0.0.0:1234
2024/03/27 02:47:59 "GET http://127.0.0.1:1234/echo HTTP/1.1" from 127.0.0.1:60922 - 000 0B in 9.208µs
path use {}
2024/03/27 02:48:06 "GET http://127.0.0.1:1234/ping HTTP/1.1" from 127.0.0.1:60927 - 200 5B in 5.125µs

Documentation

Index

Constants

View Source 
const RequestId = "Request-Id"

Variables

View Source 
var (
	MethodGet  = Method("GET")
	MethodPost = Method("POST")
)

Method http method

View Source 
var ErrHandler = func(err string, code int) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		http.Error(w, err, code)
	})
}

ErrHandler handler err

Functions

func CopyBody 

func CopyBody(b io.ReadCloser) (*bytes.Buffer, io.ReadCloser, error)

CopyBody reads all of b to memory and then returns two equivalent ReadClosers yielding the same bytes.

It returns an error if the initial slurp of all bytes fails. It does not attempt to make the returned ReadClosers have identical error-matching behavior.

func Delete 

func Delete(url, contentType string, body io.Reader) (*http.Response, error)

Delete send delete request

func GenId 

func GenId(id ...string) string

GenId gen random id.

func Get 

func Get(url string) (*http.Response, error)

Get send get request 

func Head(url string) (resp *http.Response, err error)

Head send post request

func ListenAndServe 

func ListenAndServe(ctx context.Context, h http.Handler, opts ...Option) error

ListenAndServe listens on the TCP network address addr and then calls [Serve] with handler to handle requests on incoming connections. Accepted connections are configured to enable TCP keep-alives.

The handler is typically nil, in which case [DefaultServeMux] is used.

ListenAndServe always returns a non-nil error.

func Log 

func Log(format string, v ...any)

Log print

func NewRequestWithContext 

func NewRequestWithContext(ctx context.Context, options Options) (*http.Request, error)

NewRequestWithContext request

func PUT 

func PUT(url, contentType string, body io.Reader) (*http.Response, error)

PUT send put request

func ParseBody 

func ParseBody(r io.ReadCloser) (*bytes.Buffer, error)

ParseBody parse body from `Request.Body`.

func Post 

func Post(url string, contentType string, body io.Reader) (*http.Response, error)

Post send post request

func PostForm 

func PostForm(url string, data url.Values) (*http.Response, error)

PostForm send post request, content-type = application/x-www-form-urlencoded

func WarpHandler 

func WarpHandler(next http.Handler) func(http.Handler) http.Handler

WarpHandler warp `http.Handler`.

func WarpRoundTripper 

func WarpRoundTripper(next http.RoundTripper) func(http.RoundTripper) http.RoundTripper

WarpRoundTripper warp `http.RoundTripper`.

Types

type Node 

type Node struct {
	// contains filtered or unexported fields
}

Node trie node

func NewNode 

func NewNode(path string, h http.Handler, opts ...Option) *Node

NewNode new

func (*Node) Add 

func (node *Node) Add(path string, h http.HandlerFunc, opts ...Option)

Add node

func (*Node) Find 

func (node *Node) Find(path string) *Node

Find 按照最长的匹配原则，/a/b/c/会优先返回/a/b/c/,其次返回/a/b/c，再返回/a/b，再返回/a，再返回/

func (*Node) Print 

func (node *Node) Print()

Print print trie tree struct

type Option 

type Option func(*Options)

Option func

func BasicAuth 

func BasicAuth(username, password string) Option

BasicAuth base auth

func Body 

func Body(body any) Option

Body request body

func CertKey 

func CertKey(cert, key string) Option

CertKey is cert and key file. 

func Cookie(cookie http.Cookie) Option

Cookie cookie

func Cookies 

func Cookies(cookies ...http.Cookie) Option

Cookies cookies

func Form 

func Form(form url.Values) Option

Form set form, content-type is

func Gzip 

func Gzip(body any) Option

Gzip request gzip compressed 

func Header(k, v string) Option

Header header

func Headers 

func Headers(kv map[string]string) Option

Headers headers

func Host 

func Host(host string) Option

Host set net/http.Request.Host. 在客户端，请求的Host字段（可选地）用来重写请求的Host头。 如过该字段为""，Request.Write方法会使用URL字段的Host。

func LocalAddr 

func LocalAddr(addr net.Addr) Option

LocalAddr local ip for tcp: &net.TCPAddr{IP: ip} for unix: &net.UnixAddr{Net: "unix", Name: "xxx")}

func Logf 

func Logf(f func(ctx context.Context, stat *Stat)) Option

Logf print log

func MaxConns 

func MaxConns(conn int) Option

MaxConns set max connections

func Method 

func Method(method string) Option

Method set method

func Param 

func Param(k string, v ...string) Option

Param params

func Params 

func Params(query map[string]string) Option

Params add query args

func Path 

func Path(path string) Option

Path set path

func Proxy 

func Proxy(addr string) Option

Proxy set proxy addr os.Setenv("HTTP_PROXY", "http://127.0.0.1:9743") os.Setenv("HTTPS_PROXY", "https://127.0.0.1:9743") https://stackoverflow.com/questions/14661511/setting-up-proxy-for-http-client

func RoundTripper 

func RoundTripper(tr http.RoundTripper) Option

RoundTripper set default `*http.Transport` by customer define.

func Setup 

func Setup(fn ...func(tripper http.RoundTripper) http.RoundTripper) Option

Setup is used for client middleware

func Stream 

func Stream(stream func(int64, []byte) error) Option

Stream handle func

func Timeout 

func Timeout(timeout time.Duration) Option

Timeout client timeout duration

func Trace 

func Trace(mLimit ...int) Option

Trace trace print

func URL 

func URL(url string) Option

URL set client to dial connection use http transport or unix socket. IF using socket connection. you must set unix in session, and set http in request. For example, sess := requests.New(requests.URL("unix:///tmp/requests.sock")) sess.DoRequest(context.Background(), requests.URL("http://path?k=v"), requests.Body("12345")) https://old.lubui.com/2021/07/26/golang-socket-file/

func Use 

func Use(fn ...func(http.Handler) http.Handler) Option

Use is used for server middleware

func Verify 

func Verify(verify bool) Option

Verify verify

type Options 

type Options struct {
	Method   string
	URL      string
	Path     []string
	RawQuery url.Values

	Header   http.Header
	Cookies  []http.Cookie
	Timeout  time.Duration
	MaxConns int
	Verify   bool
	Stream   func(int64, []byte) error

	Transport        http.RoundTripper
	HttpRoundTripper []func(http.RoundTripper) http.RoundTripper

	Handler     http.Handler
	HttpHandler []func(http.Handler) http.Handler

	// client session used
	LocalAddr net.Addr
	Proxy     func(*http.Request) (*url.URL, error)
	// contains filtered or unexported fields
}

Options request

type Response 

type Response struct {
	*http.Request
	*http.Response
	StartAt time.Time
	Cost    time.Duration
	Content *bytes.Buffer
	Err     error
}

Response wrap `http.Response` struct.

func (*Response) Error 

func (resp *Response) Error() string

Error implement error interface.

func (*Response) Stat 

func (resp *Response) Stat() *Stat

Stat stat

func (*Response) String 

func (resp *Response) String() string

String implement fmt.Stringer interface.

type ResponseWriter 

type ResponseWriter struct {
	http.ResponseWriter

	StatusCode int
	Content    []byte
	// contains filtered or unexported fields
}

ResponseWriter wrap `http.ResponseWriter` interface.

func (*ResponseWriter) Flush 

func (w *ResponseWriter) Flush()

func (*ResponseWriter) Hijack 

func (w *ResponseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error)

func (*ResponseWriter) Push 

func (w *ResponseWriter) Push(target string, opts *http.PushOptions) error

func (*ResponseWriter) Write 

func (w *ResponseWriter) Write(buf []byte) (int, error)

func (*ResponseWriter) WriteHeader 

func (w *ResponseWriter) WriteHeader(statusCode int)

type RoundTripperFunc 

type RoundTripperFunc func(*http.Request) (*http.Response, error)

RoundTripperFunc is a http.RoundTripper implementation, which is a simple function.

func (RoundTripperFunc) RoundTrip 

func (fn RoundTripperFunc) RoundTrip(r *http.Request) (*http.Response, error)

RoundTrip implements http.RoundTripper.

type ServeMux 

type ServeMux struct {
	// contains filtered or unexported fields
}

ServeMux implement ServeHTTP interface.

func NewServeMux 

func NewServeMux(opts ...Option) *ServeMux

NewServeMux new router.

func (*ServeMux) Handle 

func (mux *ServeMux) Handle(path string, h http.Handler, opts ...Option)

Handle set handler pattern path to handler

func (*ServeMux) HandleFunc 

func (mux *ServeMux) HandleFunc(path string, f func(http.ResponseWriter, *http.Request), opts ...Option)

HandleFunc set func pattern path to handle path cannot override, so if your path not work, maybe it is already exists!

func (*ServeMux) Pprof 

func (mux *ServeMux) Pprof()

Pprof debug, 必须使用这个路径访问：/debug/pprof/

func (*ServeMux) Print 

func (mux *ServeMux) Print()

Print print trie tree struct.

func (*ServeMux) Redirect 

func (mux *ServeMux) Redirect(source, target string)

Redirect set redirect path to handle

func (*ServeMux) Route 

func (mux *ServeMux) Route(path string, v any, opts ...Option)

Route set any pattern path to handle

func (*ServeMux) ServeHTTP 

func (mux *ServeMux) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP implement http.Handler interface 首先对路由进行校验,不满足的话直接404 其次执行RequestEach对`http.Request`进行处理,如果处理失败的话，直接返回400 最后处理中间件`func(next http.Handler) http.Handler`

func (*ServeMux) Use 

func (mux *ServeMux) Use(fn ...func(http.Handler) http.Handler)

Use can set middleware which compatible with net/http.ServeMux.

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server server

func NewServer 

func NewServer(ctx context.Context, h http.Handler, opts ...Option) *Server

NewServer new server, opts is not add to ServeMux

func (*Server) ListenAndServe 

func (s *Server) ListenAndServe() (err error)

ListenAndServe listens on the TCP network address srv.Addr and then calls [Serve] or [ServeTLS] to handle requests on incoming (TLS) connections. Accepted connections are configured to enable TCP keep-alives.

If srv.Addr is blank, ":http" is used.

Filenames containing a certificate and matching private key for the server must be provided if neither the Server's TLSConfig.Certificates nor TLSConfig.GetCertificate are populated. If the certificate is signed by a certificate authority, the certFile should be the concatenation of the server's certificate, any intermediates, and the CA's certificate.

ListenAndServe(TLS) always returns a non-nil error. After [Server.Shutdown] or [Server.Close], the returned error is [ErrServerClosed].

func (*Server) OnShutdown 

func (s *Server) OnShutdown(f func(s *http.Server))

OnShutdown do something after serve shutdown

func (*Server) OnStartup 

func (s *Server) OnStartup(f func(s *http.Server))

OnStartup do something before serve startup

type Session 

type Session struct {
	// contains filtered or unexported fields
}

Session httpclient session Clients and Transports are safe for concurrent use by multiple goroutines for efficiency should only be created once and re-used. so, session is also safe for concurrent use by multiple goroutines.

func New 

func New(opts ...Option) *Session

New session

func (*Session) Do 

func (s *Session) Do(ctx context.Context, opts ...Option) (*http.Response, error)

Do send a request and return `http.Response`. DO NOT forget close `resp.Body`.

func (*Session) DoRequest 

func (s *Session) DoRequest(ctx context.Context, opts ...Option) (*Response, error)

DoRequest send a request and return a response, and is safely close `resp.Body`.

func (*Session) HTTPClient 

func (s *Session) HTTPClient() *http.Client

HTTPClient returns the http.Client that is configured to be used for HTTP requests.

func (*Session) RoundTrip 

func (s *Session) RoundTrip(r *http.Request) (*http.Response, error)

RoundTrip implements the RoundTripper interface. Like the `http.RoundTripper` interface, the error types returned by RoundTrip are unspecified.

func (*Session) RoundTripper 

func (s *Session) RoundTripper(opts ...Option) http.RoundTripper

RoundTripper return http.RoundTripper. Setup: session.Setup -> request.Setup

type Stat 

type Stat struct {
	RequestId string `json:"RequestId"`
	StartAt   string `json:"StartAt"`
	Cost      int64  `json:"Cost"`

	Request struct {
		// RemoteAddr is remote addr in server side,
		// For client requests, it is unused.
		RemoteAddr string `json:"RemoteAddr"`

		// URL is Request.URL
		// For client requests, is request addr. contains schema://ip:port/path/xx
		// For server requests, is only path. eg: /api/v1/xxx
		URL    string            `json:"URL"`
		Method string            `json:"Method"`
		Header map[string]string `json:"Header"`
		Body   any               `json:"Body"`
	} `json:"Request"`
	Response struct {

		// URL is server addr(http://127.0.0.1:8080).
		// For client requests, it is unused.
		URL           string            `json:"URL"`
		Header        map[string]string `json:"Header"`
		Body          any               `json:"Body"`
		StatusCode    int               `json:"StatusCode"`
		ContentLength int64             `json:"ContentLength"`
	} `json:"Response"`
	Err string `json:"Err"`
}

Stat stats

func (*Stat) Print 

func (stat *Stat) Print() string

Print is used for server side

func (*Stat) String 

func (stat *Stat) String() string

String implement fmt.Stringer interface.

type Transport 

type Transport struct {
	*http.Transport
	// contains filtered or unexported fields
}

func (*Transport) RoundTrip 

func (t *Transport) RoundTrip(r *http.Request) (*http.Response, error)

RoundTrip implements the RoundTripper interface. Like the `http.RoundTripper` interface, the error types returned by RoundTrip are unspecified.

func (*Transport) RoundTripper 

func (t *Transport) RoundTripper(opts ...Option) http.RoundTripper

RoundTripper return http.RoundTripper. Setup: session.Setup -> request.Setup

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.