client

README

go-client

This repository contains a generic HTTP client which can be adapted to provide:

• General HTTP methods for GET and POST of data
• Ability to send and receive JSON, plaintext and XML data
• Ability to send files and data of type multipart/form-data
• Ability to send data of type application/x-www-form-urlencoded
• Debugging capabilities to see the request and response data
• Streaming text events

API Documentation: https://pkg.go.dev/github.com/mutablelogic/go-client

There are also some example clients which use this library:

• Anthropic API Client for Claude LLM
• Bitwarden API Client
• Elevenlabs API Client for Text-to-Speech
• Home Assistant API Client
• IPify Client
• Mistral API Client for Mistral LLM
• NewsAPI client
• Ollama API client for locally-hosted LLM's
• OpenAI API client for OpenAI LLM's
• WeatherAPI client

Aiming to have compatibility with go version 1.21 and above.

Basic Usage

The following example shows how to decode a response from a GET request to a JSON endpoint:

package main

import (
    client "github.com/mutablelogic/go-client"
)

func main() {
    // Create a new client
    c := client.New(client.OptEndpoint("https://api.example.com/api/v1"))

    // Send a GET request, populating a struct with the response
    var response struct {
        Message string `json:"message"`
    }
    if err := c.Do(nil, &response, OptPath("test")); err != nil {
        // Handle error
    }

    // Print the response
    fmt.Println(response.Message)
}

Various options can be passed to the client New method to control its behaviour:

• OptEndpoint(value string) sets the endpoint for all requests
• OptTimeout(value time.Duration) sets the timeout on any request, which defaults to 30 seconds. Timeouts can be ignored on a request-by-request basis using the OptNoTimeout option (see below).
• OptUserAgent(value string) sets the user agent string on each API request.
• OptTrace(w io.Writer, verbose bool) allows you to debug the request and response data. When verbose is set to true, it also displays the payloads.
• OptStrict() turns on strict content type checking on anything returned from the API.
• OptRateLimit(value float32) sets the limit on number of requests per second and the API will sleep to regulate the rate limit when exceeded.
• OptReqToken(value Token) sets a request token for all client requests. This can be overridden by the client for individual requests using OptToken (see below).
• OptSkipVerify() skips TLS certificate domain verification.
• OptHeader(key, value string) appends a custom header to each request.

Usage with a payload

The first argument to the Do method is the payload to send to the server, when set. You can create a payload using the following methods:

• client.NewRequest() returns a new empty payload which defaults to GET.
• client.NewJSONRequest(payload any, accept string) returns a new request with a JSON payload which defaults to POST.
• client.NewMultipartRequest(payload any, accept string) returns a new request with a Multipart Form data payload which defaults to POST.
• client.NewFormRequest(payload any, accept string) returns a new request with a Form data payload which defaults to POST.

For example,

package main

import (
    client "github.com/mutablelogic/go-client"
)

func main() {
    // Create a new client
    c := client.New(client.OptEndpoint("https://api.example.com/api/v1"))

    // Send a GET request, populating a struct with the response
    var request struct {
        Prompt string `json:"prompt"`
    }
    var response struct {
        Reply string `json:"reply"`
    }
    request.Prompt = "Hello, world!"
    payload := client.NewJSONRequest(request)
    if err := c.Do(payload, &response, OptPath("test")); err != nil {
        // Handle error
    }

    // Print the response
    fmt.Println(response.Reply)
}

You can also implement your own payload by implementing the Payload interface:

type Payload interface {
  io.Reader

  // The method to use to send the payload
  Method() string

  // The content type of the payload
  Type() string

  // The content type which is accepted as a response, or empty string if any
  Accept() string
}

Request options

The signature of the Do method is as follows:

type Client interface {
    // Perform request and wait for response
    Do(in Payload, out any, opts ...RequestOpt) error

    // Perform request and wait for response, with context for cancellation
    DoWithContext(ctx context.Context, in Payload, out any, opts ...RequestOpt) error
}

If you pass a context to the DoWithContext method, then the request can be cancelled using the context in addition to the timeout. Various options can be passed to modify each individual request when using the Do method:

• OptReqEndpoint(value string) sets the endpoint for the request
• OptPath(value ...string) appends path elements onto a request endpoint
• OptToken(value Token) adds an authorization header (overrides the client OptReqToken option)
• OptQuery(value url.Values) sets the query parameters to a request
• OptHeader(key, value string) sets a custom header to the request
• OptNoTimeout() disables the timeout on the request, which is useful for long running requests
• OptTextStreamCallback(func(TextStreamCallback) error) allows you to set a callback function to process a streaming text response of type text/event-stream. See below for more details.

Authentication

The authentication token can be set as follows:

package main

import (
    client "github.com/mutablelogic/go-client"
)

func main() {
    // Create a new client
    c := client.New(
        client.OptEndpoint("https://api.example.com/api/v1"),
        client.OptReqToken(client.Token{
            Scheme: "Bearer",
            Value: os.GetEnv("API_TOKEN"),
        }),
    )

    // ...
}

You can also set the token on a per-request basis using the OptToken option in call to the Do method.

Form submission

You can create a payload with form data:

• client.NewFormRequest(payload any, accept string) returns a new request with a Form data payload which defaults to POST.
• client.NewMultipartRequest(payload any, accept string) returns a new request with a Multipart Form data payload which defaults to POST. This is useful for file uploads.

The payload should be a struct where the fields are converted to form tuples. File uploads require a field of type multipart.File. For example,

package main

import (
    client "github.com/mutablelogic/go-client"
    multipart "github.com/mutablelogic/go-client/pkg/multipart"
)

type FileUpload struct {
    File multipart.File `json:"file"`
}

func main() {
    // Create a new client
    c := client.New(client.OptEndpoint("https://api.example.com/api/v1"))

    // Create a file upload request
    request := FileUpload{
        File: multipart.File{
            Path: "helloworld.txt",
            Body: strings.NewReader("Hello, world!"),
        }
    }

    // Upload a file
    if payload, err := client.NewMultipartRequest(request, "*/*"); err != nil {
        // Handle error
    } else if err := c.Do(payload, &response, OptPath("upload")); err != nil {
        // Handle error
    }
}

Unmarshalling responses

You can implement your own unmarshalling of responses by implementing the Unmarshaler interface:

type Unmarshaler interface {
  Unmarshal(mimetype string, r io.Reader) error
}

Streaming Responses

The client implements a streaming text event callback which can be used to process a stream of text events, as per the Mozilla specification.

In order to process streamed events, pass the OptTextStreamCallback() option to the request with a callback function, which should have the following signature:

func Callback(event client.TextStreamEvent) error {
    // Finish processing successfully
    if event.Event == "close" {
        return io.EOF
    }

    // Decode the data into a JSON object
    var data map[string]any
    if err := event.Json(data); err != nil {
        return err
    }

    // Return success - continue streaming
    return nil
}

The TextStreamEvent object has the following

If you return an error of type io.EOF from the callback, then the stream will be closed. Similarly, if you return any other error the stream will be closed and the error returned.

Usually, you would pair this option with OptNoTimeout to prevent the request from timing out.

Documentation

Overview

Implements a generic REST API client which can be used for creating gateway-specific clients. Basic usage:

package main

import (
	client "github.com/mutablelogic/go-client"
)

func main() {
	// Create a new client
	c := client.New(client.OptEndpoint("https://api.example.com/api/v1"))

	// Send a GET request, populating a struct with the response
	var response struct {
		Message string `json:"message"`
	}
	if err := c.Do(nil, &response, OptPath("test")); err != nil {
		// Handle error
	}

	// Print the response
	fmt.Println(response.Message)
}

Index

• Constants
• Variables
• type Client
  • func New(opts ...ClientOpt) (*Client, error)
  • func (client *Client) Debugf(f string, args ...any)
  • func (client *Client) Do(in Payload, out any, opts ...RequestOpt) error
  • func (client *Client) DoWithContext(ctx context.Context, in Payload, out any, opts ...RequestOpt) error
  • func (client *Client) Request(req *http.Request, out any, opts ...RequestOpt) error
  • func (client *Client) String() string
• type ClientOpt
  • func OptEndpoint(value string) ClientOpt
  • func OptHeader(key, value string) ClientOpt
  • func OptParent(v any) ClientOpt
  • func OptRateLimit(value float32) ClientOpt
  • func OptReqToken(value Token) ClientOpt
  • func OptSkipVerify() ClientOpt
  • func OptStrict() ClientOpt
  • func OptTimeout(value time.Duration) ClientOpt
  • func OptTrace(w io.Writer, verbose bool) ClientOpt
  • func OptUserAgent(value string) ClientOpt
• type Payload
  • func NewFormRequest(payload any, accept string) (Payload, error)
  • func NewJSONRequest(payload any) (Payload, error)
  • func NewJSONRequestEx(method string, payload any, accept string) (Payload, error)
  • func NewMultipartRequest(payload any, accept string) (Payload, error)
  • func NewRequest() Payload
  • func NewRequestEx(method, accept string) Payload
• type RequestOpt
  • func OptNoTimeout() RequestOpt
  • func OptPath(value ...string) RequestOpt
  • func OptQuery(value url.Values) RequestOpt
  • func OptReqEndpoint(value string) RequestOpt
  • func OptReqHeader(name, value string) RequestOpt
  • func OptTextStreamCallback(fn TextStreamCallback) RequestOpt
  • func OptToken(value Token) RequestOpt
• type TextStream
  • func NewTextStream() *TextStream
  • func (t *TextStream) Decode(r io.Reader, callback TextStreamCallback) error
• type TextStreamCallback
• type TextStreamEvent
  • func (t *TextStreamEvent) IsZero() bool
  • func (t *TextStreamEvent) Json(v any) error
  • func (t TextStreamEvent) String() string
• type Token
  • func (token Token) String() string
• type Unmarshaler

Constants

const (
	DefaultTimeout            = time.Second * 30
	DefaultUserAgent          = "github.com/mutablelogic/go-client"
	PathSeparator             = string(os.PathSeparator)
	ContentTypeAny            = "*/*"
	ContentTypeJson           = "application/json"
	ContentTypeTextXml        = "text/xml"
	ContentTypeApplicationXml = "application/xml"
	ContentTypeTextPlain      = "text/plain"
	ContentTypeTextHTML       = "text/html"
	ContentTypeBinary         = "application/octet-stream"
)

const (
	Bearer = "Bearer"
)

const (
	// Mime type for text stream
	ContentTypeTextStream = "text/event-stream"
)

Variables

var (
	MethodGet    = NewRequestEx(http.MethodGet, ContentTypeAny)
	MethodDelete = NewRequestEx(http.MethodDelete, ContentTypeAny)
	MethodPut    = NewRequestEx(http.MethodPut, ContentTypeAny)
)

Types

type Client

type Client struct {
	sync.Mutex
	*http.Client

	// Parent object for client options
	Parent any
	// contains filtered or unexported fields
}

func New

func New(opts ...ClientOpt) (*Client, error)

New creates a new client with options. OptEndpoint is required as an option to set the endpoint for all requests.

func (*Client) Debugf

func (client *Client) Debugf(f string, args ...any)

Debugf outputs debug information

func (*Client) Do

func (client *Client) Do(in Payload, out any, opts ...RequestOpt) error

Do a JSON request with a payload, populate an object with the response and return any errors

func (*Client) DoWithContext

func (client *Client) DoWithContext(ctx context.Context, in Payload, out any, opts ...RequestOpt) error

Do a JSON request with a payload, populate an object with the response and return any errors. The context can be used to cancel the request

func (*Client) Request

func (client *Client) Request(req *http.Request, out any, opts ...RequestOpt) error

Do a HTTP request and decode it into an object

func (*Client) String

func (client *Client) String() string

type ClientOpt

type ClientOpt func(*Client) error

func OptEndpoint

func OptEndpoint(value string) ClientOpt

OptEndpoint sets the endpoint for all requests.

func OptHeader

func OptHeader(key, value string) ClientOpt

OptHeader appends a custom header to each request

func OptParent

func OptParent(v any) ClientOpt

OptParent sets the parent client for this client, which is used for setting additional client options in the parent

func OptRateLimit

func OptRateLimit(value float32) ClientOpt

OptRateLimit sets the limit on number of requests per second and the API will sleep when exceeded. For account tokens this is 1 per second

func OptReqToken

func OptReqToken(value Token) ClientOpt

OptReqToken sets a request token for all client requests. This can be overridden by the client for individual requests using OptToken.

func OptSkipVerify

func OptSkipVerify() ClientOpt

OptSkipVerify skips TLS certificate domain verification

func OptStrict

func OptStrict() ClientOpt

OptStrict turns on strict content type checking on anything returned from the API

func OptTimeout

func OptTimeout(value time.Duration) ClientOpt

OptTimeout sets the timeout on any request. By default, a timeout of 10 seconds is used if OptTimeout is not set

func OptTrace

func OptTrace(w io.Writer, verbose bool) ClientOpt

OptTrace allows you to be the "man in the middle" on any requests so you can see traffic move back and forth. Setting verbose to true also displays the JSON response

func OptUserAgent

func OptUserAgent(value string) ClientOpt

OptUserAgent sets the user agent string on each API request It is set to the default if empty string is passed

type Payload

type Payload interface {
	io.Reader

	Method() string
	Accept() string
	Type() string
}

func NewFormRequest

func NewFormRequest(payload any, accept string) (Payload, error)

Return a new request with a Form data payload which defaults to POST. The accept parameter is the accepted mime-type of the response.

func NewJSONRequest

func NewJSONRequest(payload any) (Payload, error)

Return a new request with a JSON payload which defaults to POST.

func NewJSONRequestEx

func NewJSONRequestEx(method string, payload any, accept string) (Payload, error)

Return a new request with a JSON payload with method. The accept parameter is the accepted mime-type of the response.

func NewMultipartRequest

func NewMultipartRequest(payload any, accept string) (Payload, error)

Return a new request with a Multipart Form data payload which defaults to POST. The accept parameter is the accepted mime-type of the response.

func NewRequest

func NewRequest() Payload

Return a new empty request which defaults to GET

func NewRequestEx

func NewRequestEx(method, accept string) Payload

Return a new empty request. The accept parameter is the accepted mime-type of the response.

type RequestOpt

type RequestOpt func(*requestOpts) error

func OptNoTimeout

func OptNoTimeout() RequestOpt

OptNoTimeout disables the timeout for this request, useful for long-running requests. The context can be used instead for cancelling requests

func OptPath

func OptPath(value ...string) RequestOpt

OptPath appends path elements onto a request

func OptQuery

func OptQuery(value url.Values) RequestOpt

OptQuery adds query parameters to a request

func OptReqEndpoint

func OptReqEndpoint(value string) RequestOpt

OptReqEndpoint modifies the request endpoint for this request only

func OptReqHeader

func OptReqHeader(name, value string) RequestOpt

OptReqHeader sets a header value to the request

func OptTextStreamCallback

func OptTextStreamCallback(fn TextStreamCallback) RequestOpt

OptTextStreamCallback is called for each event in a text stream

func OptToken

func OptToken(value Token) RequestOpt

OptToken adds an authorization header. The header format is "Authorization: Bearer <token>"

type TextStream

type TextStream struct {
	// contains filtered or unexported fields
}

Implementation of a text stream, as per https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events

func NewTextStream

func NewTextStream() *TextStream

Create a new text stream decoder

func (*TextStream) Decode

func (t *TextStream) Decode(r io.Reader, callback TextStreamCallback) error

Decode a text stream. The reader should be a stream of text/event-stream data and the method will return when all the data has been scanned, or the callback returns an error

type TextStreamCallback

type TextStreamCallback func(TextStreamEvent) error

Callback for text stream events, return an error if you want to return from the Decode method

type TextStreamEvent

type TextStreamEvent struct {
	// The event ID to set the EventSource object's last event ID value.
	Id string `json:"id,omitempty"`

	// A string identifying the type of event described
	Event string `json:"event,omitempty"`

	// The data field for the message
	Data string `json:"data"`

	// The reconnection time. If the connection to the server is lost,
	// the client should wait for the specified time before attempting to reconnect.
	Retry time.Duration `json:"retry,omitempty"`
}

Implementation of a text stream, as per https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#event_stream_format

func (*TextStreamEvent) IsZero

func (t *TextStreamEvent) IsZero() bool

Return true if the event contains no content

func (*TextStreamEvent) Json

func (t *TextStreamEvent) Json(v any) error

Decode the text stream event data as JSON

func (TextStreamEvent) String

func (t TextStreamEvent) String() string

Return the text stream event as a string

type Token

type Token struct {
	Scheme string
	Value  string
}

func (Token) String

func (token Token) String() string

Stringify the token value

type Unmarshaler

type Unmarshaler interface {
	Unmarshal(mimetype string, r io.Reader) error
}

Unmarshaler is an interface which can be implemented by a type to unmarshal a response body