httpbulb

README

httpbulb

HTTP Request & Response Testing Tool & Service.

An implementation of httpbin for Go.

This is a http mux handler based on go-chi.

It is useful for testing purposes. Actually, it was created to work with httptest package, because httptest package allows you to run http server for your tests without touching the external server.

[!NOTE] This handler follows the original httpbin API but in some cases it is more strict.

Requirements

Go 1.18

Installation

go get -u github.com/niklak/httpbulb

The main differences between httpbin and httpbulb

• args, form, files and headers fields are represented by map[string][]string.
• /status/{code} endpoint does not handle status codes lesser than 200 or greater than 599.
• /cookies-list -- a new endpoint that returns a cookie list ([]http.Cookie) in the same order as it was received and parsed on the go http server.
• /images, /encoding/utf8, /html, /json, /xml endpoints support Range requests.
• /delete, /get, /patch, /post, /put endpoints also return field proto which can help to detect HTTP protocol version in the client-server connection.

Examples

The main approach is to use httpbulb with httptest.Server.

Testing client's http2 support

package h2

import (
	"crypto/tls"
	"encoding/json"
	"io"
	"net/http"
	"net/http/httptest"
	"net/url"
	"strings"
	"testing"

	"github.com/niklak/httpbulb"
	"github.com/stretchr/testify/require"
)

func Test_Http2Client(t *testing.T) {
	// This test checks http client configuration to send HTTP/2.0 requests to the server with tls
	type testArgs struct {
		name          string
		client        *http.Client
		wantProto     string
		wantClientErr bool
	}

	// a representation of a response from the testing server,
	// for the test i require only an `url` and a `proto` fields, so i skip the rest.
	type serverResponse struct {
		URL   string `json:"url"`
		Proto string `json:"proto"`
	}

	// starting a tls test server with HTTP/2.0 support
	handleFunc := httpbulb.NewRouter()
	testServer := httptest.NewUnstartedServer(handleFunc)
	testServer.EnableHTTP2 = true
	testServer.StartTLS()
	defer testServer.Close()

	// prepare the url
	u, err := url.Parse(testServer.URL)
	require.NoError(t, err)
	u.Path = "/get"
	testURL := u.String()

	// this client is forced to use HTTP/2.0
	h2Client := testServer.Client()

	// this client is expected to fail, because i didn't install CA certificate for this server
	h1Client := &http.Client{}

	// a client that skips TLS
	h1ClientInsecure := &http.Client{
		Transport: &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: true}},
	}

	tests := []testArgs{
		{name: "HTTP/2.0", client: h2Client, wantProto: "HTTP/2.0"},
		{name: "HTTP/1.1 - Insecure", client: h1ClientInsecure, wantProto: "HTTP/1.1"},
		{name: "HTTP/1.1 - Without CA Certs", client: h1Client, wantClientErr: true},
	}

	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {

			req, err := http.NewRequest("GET", testURL, nil)
			require.NoError(t, err)
			resp, err := tt.client.Do(req)
			if tt.wantClientErr {
				// we expect an error, exit the test
				require.Error(t, err)
				return
			}
			require.NoError(t, err)

			// in this test i always expect 200
			require.Equal(t, http.StatusOK, resp.StatusCode)

			// reading the body, decode the body into result

			body, err := io.ReadAll(resp.Body)
			require.NoError(t, err)
			resp.Body.Close()

			result := new(serverResponse)

			err = json.Unmarshal(body, result)
			require.NoError(t, err)

			// ensure that we got a secure url
			require.True(t, strings.HasPrefix(result.URL, "https://"))
			// ensure that client's proto is what we expect
			require.Equal(t, tt.wantProto, result.Proto)
		})
	}
}

It is also possible to use httpbulb as a web-server.

The binary can be built with from github.com/niklak/httpbulb/cmd/bulb.

Also you can simply use docker to run a web-server:

docker run -p 8080:8080 ghcr.io/niklak/httpbulb:latest.

Running a web-server using docker-compose

name: httpbulb

services:
  web:
    image: ghcr.io/niklak/httpbulb:latest
    restart: unless-stopped
    volumes:
    # If you require an HTTPS server, you should link the directories with tls certificates.
      - "./data/certs:/certs"
    ports:
      # map the bulb port to your external port
      - :4443:8080
    environment:
      # you can set server address as HOST:PORT or just :PORT
      - SERVER_ADDR=:8080
      # If you require an HTTPS server, you should set SERVER_CERT_PATH and SERVER_KEY_PATH.
      # If you work with self-signed certificates, you need to ensure that `root CA` is installed on the requesting machine.
      # Or you need to mark your requests as insecure. (InsecureSkipVerify: true for Go, or --insecure flag for curl)
      # If server unable to load certificates, it will produce a warning, but start serving an HTTP server. 
      - SERVER_CERT_PATH=/certs/server-host.pem
      - SERVER_KEY_PATH=/certs/server-host-key.pem
      - SERVER_READ_TIMEOUT=120s
      - SERVER_WRITE_TIMEOUT=120s

After starting the server with docker compose its ready to accept requests.

# with no certificates
curl -v http://localhost:4443/get

# with self-signed certificates and installed root CA.
curl -v https://localhost:4443/get

# with self-signed certificates but without installed root CA on requesting machine.
curl -v --insecure https://localhost:4443/get

# with real certificates
curl -v https://example.com:4443/get

Live example

Live example is available at codesandbox:

Endpoints

Route	Methods	Description
/delete /get /patch /post /put	DELETE GET PATCH POST PUT	These are basic endpoints. They return a response with request's common information. Unlike the original httpbin implementation, this handler doesn't read the request body for DELETE and GET requests. args, files, form, and headers are always represented by a map of string lists (slices).
/basic-auth	GET	Prompts the user for authorization using HTTP Basic Auth. Returns 401 if authorization is failed.
/hidden-basic-auth	GET	Prompts the user for authorization using HTTP Basic Auth. Returns 404 if authorization is failed.
/digest-auth/{qop}/{user}/{passwd} /digest-auth/{qop}/{user}/{passwd}/{algorithm} /digest-auth/{qop}/{user}/{passwd}/{algorithm}/{stale_after}	GET	Prompts the user for authorization using HTTP Digest Auth. Returns 401 or 403 if authorization is failed.
/bearer	GET	Prompts the user for authorization using bearer authentication. Returns 401 if authorization is failed.
/status/{codes}	DELETE GET PATCH POST PUT	Returns status code or random status code if more than one are given. This handler does not handle status codes lesser than 200 or greater than 599.
/headers	GET	Return the incoming request's HTTP headers.
/ip	GET	Returns the requester's IP Address.
/user-agent	GET	Return the incoming requests's User-Agent header.
/cache	GET	Returns a 304 if an If-Modified-Since header or If-None-Match is present. Returns the same as a /get otherwise.
/cache/{value}	GET	Sets a Cache-Control header for n seconds.
/etag/{etag}	GET	Assumes the resource has the given etag and responds to If-None-Match and If-Match headers appropriately.
/response-headers	GET POST	Returns a set of response headers from the query string.
/brotli	GET	Returns Brotli-encoded data.
/deflate	GET	Returns Deflate-encoded data.
/deny	GET	Returns page denied by robots.txt rules.
/encoding/utf8	GET	Returns a UTF-8 encoded body.
/gzip	GET	Returns Gzip-encoded data.
/html	GET	Returns a simple HTML document.
/json	GET	Returns a simple JSON document.
/robots.txt	GET	Returns some robots.txt rules.
/xml	GET	Returns a simple XML document.
/base64/{value}	GET	Decodes base64url-encoded string.
/bytes/{n}	GET	Returns n random bytes generated with given seed.
/delay/{delay}	DELETE GET PATCH POST PUT	Returns a delayed response (max of 10 seconds).
/drip	GET	Drips data over a duration after an optional initial delay.
/links/{n}/{offset}	GET	Generates a page containing n links to other pages which do the same.
/range/{numbytes}	GET	Streams n random bytes generated with given seed, at given chunk size per packet. Supports Accept-Ranges and Content-Range headers.
/stream-bytes/{n}	GET	Streams n random bytes generated with given seed, at given chunk size per packet.
/stream/{n}	GET	Streams n json messages.
/uuid	GET	Returns a UUID4.
/cookies	GET	Returns cookie data.
/cookies-list	GET	Returns a cookie list ([]http.Cookie) in the same order as it was received and parsed on the server.
/cookies/delete	GET	Deletes cookie(s) as provided by the query string and redirects to cookie list.
/cookies/set	GET	Sets cookie(s) as provided by the query string and redirects to cookie list.
/cookies/set/{name}/{value}	GET	Sets a cookie and redirects to cookie list.
/image	GET	Returns a simple image of the type suggest by the Accept header. Also supports a Range requests
/image/{format:svg|png|jpeg|webp|avif}	GET	Returns an image with the given format. If the format is not matched it returns 404
/absolute-redirect/{n}	GET	Absolutely 302 Redirects n times. Location header will be an absolute URL.
/redirect-to	DELETE GET PATCH POST PUT	302/3XX Redirects to the given URL. url parameter is required and status parameter is optional.
/redirect/{n}	GET	302 Redirects n times. Location header will be an absolute if absolute=true was sent as a query parameter.
/relative-redirect/{n}	GET	Relatively 302 Redirects n times. Location header will be a relative URL.
/anything /anything/{anything}	DELETE GET PATCH POST PUT	Returns anything passed in request data.

Documentation

Index

• Variables
• func AbsoluteRedirectHandle(w http.ResponseWriter, r *http.Request)
• func Base64DecodeHandle(w http.ResponseWriter, r *http.Request)
• func BasicAuthHandle(w http.ResponseWriter, r *http.Request)
• func BearerAuthHandle(w http.ResponseWriter, r *http.Request)
• func BrotliHandle(w http.ResponseWriter, r *http.Request)
• func CacheControlHandle(w http.ResponseWriter, r *http.Request)
• func CacheHandle(w http.ResponseWriter, r *http.Request)
• func CookiesHandle(w http.ResponseWriter, r *http.Request)
• func CookiesListHandle(w http.ResponseWriter, r *http.Request)
• func Cors(next http.Handler) http.Handler
• func DeflateHandle(w http.ResponseWriter, r *http.Request)
• func DelayHandle(w http.ResponseWriter, r *http.Request)
• func DeleteCookiesHandle(w http.ResponseWriter, r *http.Request)
• func DenyHandle(w http.ResponseWriter, r *http.Request)
• func DigestAuthHandle(w http.ResponseWriter, r *http.Request)
• func DripHandle(w http.ResponseWriter, r *http.Request)
• func EtagHandle(w http.ResponseWriter, r *http.Request)
• func GzipHandle(w http.ResponseWriter, r *http.Request)
• func HeadersHandle(w http.ResponseWriter, r *http.Request)
• func HiddenBasicAuthHandle(w http.ResponseWriter, r *http.Request)
• func HtmlSampleHandle(w http.ResponseWriter, r *http.Request)
• func ImageAcceptHandle(w http.ResponseWriter, r *http.Request)
• func ImageHandle(w http.ResponseWriter, r *http.Request)
• func IndexHandle(w http.ResponseWriter, r *http.Request)
• func IpHandle(w http.ResponseWriter, r *http.Request)
• func JSONSampleHandle(w http.ResponseWriter, r *http.Request)
• func JsonError(w http.ResponseWriter, err string, code int)
• func LinkPageHandle(w http.ResponseWriter, r *http.Request)
• func LinksHandle(w http.ResponseWriter, r *http.Request)
• func MethodsHandle(w http.ResponseWriter, r *http.Request)
• func NewRouter(middlewares ...func(http.Handler) http.Handler) *chi.Mux
• func RandomBytesHandle(w http.ResponseWriter, r *http.Request)
• func RangeHandle(w http.ResponseWriter, r *http.Request)
• func RedirectHandle(w http.ResponseWriter, r *http.Request)
• func RedirectToHandle(w http.ResponseWriter, r *http.Request)
• func RelativeRedirectHandle(w http.ResponseWriter, r *http.Request)
• func ResponseHeadersHandle(w http.ResponseWriter, r *http.Request)
• func RobotsHandle(w http.ResponseWriter, r *http.Request)
• func SetCookieHandle(w http.ResponseWriter, r *http.Request)
• func SetCookiesHandle(w http.ResponseWriter, r *http.Request)
• func StatusCodeHandle(w http.ResponseWriter, r *http.Request)
• func StreamNMessagesHandle(w http.ResponseWriter, r *http.Request)
• func StreamRandomBytesHandle(w http.ResponseWriter, r *http.Request)
• func TextError(w http.ResponseWriter, err string, code int)
• func UUIDHandle(w http.ResponseWriter, r *http.Request)
• func UserAgentHandle(w http.ResponseWriter, r *http.Request)
• func Utf8SampleHandle(w http.ResponseWriter, r *http.Request)
• func XMLSampleHandle(w http.ResponseWriter, r *http.Request)
• type AuthResponse
• type CookiesListResponse
• type CookiesResponse
• type ErrorResponse
• type HeadersResponse
• type IpResponse
• type MethodsResponse
• type StatusResponse
• type StreamResponse
• type UUIDResponse
• type UserAgentResponse

Variables

var RenderError func(http.ResponseWriter, string, int) = JsonError

RenderError renders the error message. It renders JSON by default and it can be overridden on program's `init` function.

var RenderResponse func(http.ResponseWriter, int, interface{}) = renderJson

RenderResponse is the default renderer function used by httpbulb. It renders JSON by default and it can be overridden on program's `init` function.

Functions

func AbsoluteRedirectHandle

func AbsoluteRedirectHandle(w http.ResponseWriter, r *http.Request)

AbsoluteRedirectHandle is a http handler that makes 302/3XX redirects `n` times. `Location` header is an absolute URL.

func Base64DecodeHandle

func Base64DecodeHandle(w http.ResponseWriter, r *http.Request)

Base64DecodeHandle decodes a base64 encoded string and returns the decoded value

func BasicAuthHandle

func BasicAuthHandle(w http.ResponseWriter, r *http.Request)

BasicAuthHandle prompts the user for authorization using HTTP Basic Auth. It returns 401 if not authorized.

func BearerAuthHandle

func BearerAuthHandle(w http.ResponseWriter, r *http.Request)

BearerAuthHandle prompts the user for authorization using bearer authentication

func BrotliHandle

func BrotliHandle(w http.ResponseWriter, r *http.Request)

BrotliHandle is the handler that returns a response compressed with brotli

func CacheControlHandle

func CacheControlHandle(w http.ResponseWriter, r *http.Request)

CacheControlHandle sets a Cache-Control header for n seconds

func CacheHandle

func CacheHandle(w http.ResponseWriter, r *http.Request)

CacheHandle returns a 304 if an If-Modified-Since header or If-None-Match is present. Returns the same as a `MethodsHandle` (/get) otherwise.

func CookiesHandle

func CookiesHandle(w http.ResponseWriter, r *http.Request)

CookiesHandle returns the cookies list, sent by the client

func CookiesListHandle

func CookiesListHandle(w http.ResponseWriter, r *http.Request)

CookiesListResponse returns a **list** with request cookies

func Cors

func Cors(next http.Handler) http.Handler

Cors middleware to handle Cross Origin Resource Sharing (CORS).

func DeflateHandle

func DeflateHandle(w http.ResponseWriter, r *http.Request)

DeflateHandle is the handler that returns a response compressed with zlib

func DelayHandle

func DelayHandle(w http.ResponseWriter, r *http.Request)

DelayHandle returns the same response as the MethodsHandle, but with a delay

func DeleteCookiesHandle

func DeleteCookiesHandle(w http.ResponseWriter, r *http.Request)

DeleteCookiesHandle deletes the cookies passed in the query parameters, then redirects to /cookies

func DenyHandle

func DenyHandle(w http.ResponseWriter, r *http.Request)

DenyHandle returns a page denied by robots.txt rules.

func DigestAuthHandle

func DigestAuthHandle(w http.ResponseWriter, r *http.Request)

DigestAuthHandle prompts the user for authorization using HTTP Digest Auth.

func DripHandle

func DripHandle(w http.ResponseWriter, r *http.Request)

DripHandle drips data over a duration after an optional initial delay

func EtagHandle

func EtagHandle(w http.ResponseWriter, r *http.Request)

EtagHandle assumes the resource has the given etag and responds to If-None-Match and If-Match headers appropriately.

func GzipHandle

func GzipHandle(w http.ResponseWriter, r *http.Request)

GzipHandle is the handler that returns a response compressed with gzip

func HeadersHandle

func HeadersHandle(w http.ResponseWriter, r *http.Request)

HeadersHandle returns only the request headers. Check `HeadersResponse`.

func HiddenBasicAuthHandle

func HiddenBasicAuthHandle(w http.ResponseWriter, r *http.Request)

HiddenBasicAuthHandle prompts the user for authorization using HTTP Basic Auth. It returns 404 if not authorized.

func HtmlSampleHandle

func HtmlSampleHandle(w http.ResponseWriter, r *http.Request)

HtmlSampleHandle serves the html file

func ImageAcceptHandle

func ImageAcceptHandle(w http.ResponseWriter, r *http.Request)

ImageAcceptHandle returns an image based on the client's Accept header.

func ImageHandle

func ImageHandle(w http.ResponseWriter, r *http.Request)

ImageHandle returns a simple image of the type specified in the URL path. The supported image types are svg, jpeg, png, and webp.

func IndexHandle

func IndexHandle(w http.ResponseWriter, r *http.Request)

IndexHandle serves the index.html file

func IpHandle

func IpHandle(w http.ResponseWriter, r *http.Request)

IpHandle returns only the IP address. Check `IpResponse`.

func JSONSampleHandle

func JSONSampleHandle(w http.ResponseWriter, r *http.Request)

JSONSampleHandle serves the json file

func JsonError

func JsonError(w http.ResponseWriter, err string, code int)

JsonError is a shortcut func for writing an error in `application/json`

func LinkPageHandle

func LinkPageHandle(w http.ResponseWriter, r *http.Request)

LinkPageHandle generates a page containing n links to other pages which do the same.

func LinksHandle

func LinksHandle(w http.ResponseWriter, r *http.Request)

LinksHandle redirects to first links page.

func MethodsHandle

func MethodsHandle(w http.ResponseWriter, r *http.Request)

MethodsHandle is the basic handler for the methods endpoint (GET, POST, PUT, PATCH, DELETE)

func NewRouter

func NewRouter(middlewares ...func(http.Handler) http.Handler) *chi.Mux

NewRouter returns a new chi.Mux with predefined http handlers. It also accepts chi middlewares.

func RandomBytesHandle

func RandomBytesHandle(w http.ResponseWriter, r *http.Request)

RandomBytesHandle returns `n` random bytes generated with given `seed`.

func RangeHandle

func RangeHandle(w http.ResponseWriter, r *http.Request)

RangeHandle streams n random bytes generated with given seed, at given chunk size per packet

func RedirectHandle

func RedirectHandle(w http.ResponseWriter, r *http.Request)

RedirectHandle is a http handler that makes 302/3XX redirects `n` times. `n` is a number in the URL path, if `n` is 1, it will redirect to `/get`. if `absolute` query param is true, `Location` header will be an absolute URL.

func RedirectToHandle

func RedirectToHandle(w http.ResponseWriter, r *http.Request)

RedirectToHandle is a http handler that makes 302/3XX redirects to the given URL

func RelativeRedirectHandle

func RelativeRedirectHandle(w http.ResponseWriter, r *http.Request)

RelativeRedirectHandle is a http handler that makes 302/3XX redirects `n` times. `Location` header is a relative URL.

func ResponseHeadersHandle

func ResponseHeadersHandle(w http.ResponseWriter, r *http.Request)

ResponseHeadersHandle returns the response headers as JSON response

func RobotsHandle

func RobotsHandle(w http.ResponseWriter, r *http.Request)

RobotsHandle returns the `text/plain` content for the robots.txt

func SetCookieHandle

func SetCookieHandle(w http.ResponseWriter, r *http.Request)

SetCookieHandle sets a cookie with the name and value passed in the URL path, then redirects to /cookies

func SetCookiesHandle

func SetCookiesHandle(w http.ResponseWriter, r *http.Request)

SetCookiesHandle sets the cookies passed from the query parameters, then redirects to /cookies

func StatusCodeHandle

func StatusCodeHandle(w http.ResponseWriter, r *http.Request)

StatusCodesHandle returns status code or random status code if more than one are given. This handler does not handle status codes lesser than 200 or greater than 599.

func StreamNMessagesHandle

func StreamNMessagesHandle(w http.ResponseWriter, r *http.Request)

StreamNMessagesHandle streams N json messages

func StreamRandomBytesHandle

func StreamRandomBytesHandle(w http.ResponseWriter, r *http.Request)

StreamRandomBytesHandle streams `n` random bytes generated with given `seed`, at given `chunk_size` per packet.

func TextError

func TextError(w http.ResponseWriter, err string, code int)

TextError is a shortcut func for writing an error in `text/plain`

func UUIDHandle

func UUIDHandle(w http.ResponseWriter, r *http.Request)

UUIDHandle returns a new UUID version 4

func UserAgentHandle

func UserAgentHandle(w http.ResponseWriter, r *http.Request)

UserAgentHandle returns only the user agent. Check `UserAgentResponse`.

func Utf8SampleHandle

func Utf8SampleHandle(w http.ResponseWriter, r *http.Request)

Utf8SampleHandle serves the utf8-encoded file

func XMLSampleHandle

func XMLSampleHandle(w http.ResponseWriter, r *http.Request)

XMLSampleHandle serves the xml file

Types

type AuthResponse

type AuthResponse struct {
	Authenticated bool   `json:"authenticated"`
	User          string `json:"user,omitempty"`
	Token         string `json:"token,omitempty"`
}

AuthResponse is the response for the basic-auth endpoint

type CookiesListResponse

type CookiesListResponse struct {
	Cookies []*http.Cookie `json:"cookies"`
}

CookiesListResponse represents a response for the cookies-list endpoint. In this case, cookies are represented as a list (slice) of `http.Cookie`.

type CookiesResponse

type CookiesResponse struct {
	Cookies map[string][]string `json:"cookies"`
}

CookiesResponse represents a response for the cookies endpoint It contains a map of cookies, which is cookie name and a list of its values.

type ErrorResponse

type ErrorResponse struct {
	Error string `json:"error"`
}

ErrorResponse represents an error response from the server.

It is used to return errors in JSON format. It contains a single field, "error", which holds the error message.

type HeadersResponse

type HeadersResponse struct {
	Headers http.Header `json:"headers"`
}

HeadersResponse is the response for the headers endpoint

type IpResponse

type IpResponse struct {
	Origin string `json:"origin"`
}

IpResponse is the response for the ip endpoint

type MethodsResponse

type MethodsResponse struct {
	// Args is a map of query parameters
	Args map[string][]string `json:"args"`
	// Data is the raw body of the request
	Data string `json:"data"`
	// Files is a map of files sent in the request
	Files map[string][]string `json:"files"`
	// Form is a map of form values sent in the request
	Form map[string][]string `json:"form"`
	// Headers is a map of headers sent in the request
	Headers map[string][]string `json:"headers"`
	// JSON is the parsed JSON body of the request
	JSON interface{} `json:"json"`
	// Origin is the IP address of the requester
	Origin string `json:"origin"`
	// URL is the full URL of the request
	URL string `json:"url"`
	// Gzipped is true if the request was compressed with gzip
	Gzipped bool `json:"gzipped,omitempty"`
	// Brotli is true if the request was compressed with brotli
	Brotli bool `json:"brotli,omitempty"`
	// Deflated is true if the request was compressed with zlib
	Deflated bool `json:"deflated,omitempty"`
	// Proto is the protocol of the request
	Proto string `json:"proto"`
}

MethodsResponse is the response for the methods endpoint

type StatusResponse

type StatusResponse struct {
	StatusText string `json:"status_text"`
}

StatusResponse is the response for the status endpoint

type StreamResponse

type StreamResponse struct {
	// ID is the ID of the message
	ID int `json:"id"`
	// Args is a map of query parameters
	Args map[string][]string `json:"args"`
	// Data is the raw body of the request
	Headers map[string][]string `json:"headers"`
	// Origin is the IP address of the requester
	Origin string `json:"origin"`
	// URL is the full URL of the request
	URL string `json:"url"`
}

StreamResponse represents a response for the stream endpoint

type UUIDResponse

type UUIDResponse struct {
	UUID string `json:"uuid"`
}

UUIDResponse represents a response with a UUID string.

type UserAgentResponse

type UserAgentResponse struct {
	UserAgent string `json:"user-agent"`
}

UserAgentResponse is the response for the user-agent endpoint