gateway

package
v0.10.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 17, 2023 License: Apache-2.0, MIT Imports: 75 Imported by: 5

README

IPFS Gateway

A reference implementation of HTTP Gateway Specifications.

Documentation

Example

This example shows how you can start your own gateway, assuming you have an IPFSBackend implementation.

// Initialize your headers and apply the default headers.
headers := map[string][]string{}
gateway.AddAccessControlHeaders(headers)

conf := gateway.Config{
  Headers:  headers,
}

// Initialize an IPFSBackend interface for both an online and offline versions.
// The offline version should not make any network request for missing content.
ipfsBackend := ...

// Create http mux and setup path gateway handler.
mux := http.NewServeMux()
handler := gateway.NewHandler(conf, ipfsBackend)
mux.Handle("/ipfs/", handler)
mux.Handle("/ipns/", handler)

// Start the server on :8080 and voilá! You have a basic IPFS gateway running
// in http://localhost:8080.
_ = http.ListenAndServe(":8080", mux)

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrInternalServerError = NewErrorStatusCodeFromStatus(http.StatusInternalServerError)
	ErrGatewayTimeout      = NewErrorStatusCodeFromStatus(http.StatusGatewayTimeout)
	ErrBadGateway          = NewErrorStatusCodeFromStatus(http.StatusBadGateway)
	ErrServiceUnavailable  = NewErrorStatusCodeFromStatus(http.StatusServiceUnavailable)
	ErrTooManyRequests     = NewErrorStatusCodeFromStatus(http.StatusTooManyRequests)
)

Functions

func AddAccessControlHeaders 

func AddAccessControlHeaders(headers map[string][]string)

AddAccessControlHeaders ensures safe default HTTP headers are used for controlling cross-origin requests. This function adds several values to the Access-Control-Allow-Headers and Access-Control-Expose-Headers entries to be exposed on GET and OPTIONS responses, including CORS Preflight.

If the Access-Control-Allow-Origin entry is missing, a default value of '*' is added, indicating that browsers should allow requesting code from any origin to access the resource.

If the Access-Control-Allow-Methods entry is missing a value, 'GET, HEAD, OPTIONS' is added, indicating that browsers may use them when issuing cross origin requests.

func NewDNSResolver 

func NewDNSResolver(resolvers map[string]string, dohOpts ...doh.Option) (*madns.Resolver, error)

NewDNSResolver creates a new DNS resolver based on the default resolvers and the provided resolvers.

The argument 'resolvers' is a map of FQDNs to URLs for custom DNS resolution. URLs starting with "https://" indicate DoH endpoints. Support for other resolver types may be added in the future.

Example:

func NewHandler 

func NewHandler(c Config, backend IPFSBackend) http.Handler

NewHandler returns an http.Handler that provides the functionality of an IPFS HTTP Gateway based on a Config and IPFSBackend.

func NewHostnameHandler 

func NewHostnameHandler(c Config, backend IPFSBackend, next http.Handler) http.HandlerFunc

NewHostnameHandler is a middleware that wraps an http.Handler in order to parse the Host header and translate it into the content path. This is useful for creating Subdomain Gateways or DNSLink Gateways.

Types

type BlocksBackend 

type BlocksBackend struct {
	// contains filtered or unexported fields
}

BlocksBackend is an IPFSBackend implementation based on a blockservice.BlockService.

func NewBlocksBackend 

func NewBlocksBackend(blockService blockservice.BlockService, opts ...BlocksBackendOption) (*BlocksBackend, error)

func (*BlocksBackend) Get 

func (bb *BlocksBackend) Get(ctx context.Context, path ImmutablePath, ranges ...ByteRange) (ContentPathMetadata, *GetResponse, error)

func (*BlocksBackend) GetAll 

func (bb *BlocksBackend) GetAll(ctx context.Context, path ImmutablePath) (ContentPathMetadata, files.Node, error)

func (*BlocksBackend) GetBlock 

func (bb *BlocksBackend) GetBlock(ctx context.Context, path ImmutablePath) (ContentPathMetadata, files.File, error)

func (*BlocksBackend) GetCAR 

func (bb *BlocksBackend) GetCAR(ctx context.Context, p ImmutablePath, params CarParams) (ContentPathMetadata, io.ReadCloser, error)

func (*BlocksBackend) GetDNSLinkRecord 

func (bb *BlocksBackend) GetDNSLinkRecord(ctx context.Context, hostname string) (ifacepath.Path, error)

func (*BlocksBackend) GetIPNSRecord 

func (bb *BlocksBackend) GetIPNSRecord(ctx context.Context, c cid.Cid) ([]byte, error)

func (*BlocksBackend) Head 

func (bb *BlocksBackend) Head(ctx context.Context, path ImmutablePath) (ContentPathMetadata, files.Node, error)

func (*BlocksBackend) IsCached 

func (bb *BlocksBackend) IsCached(ctx context.Context, p ifacepath.Path) bool

func (*BlocksBackend) ResolveMutable 

func (bb *BlocksBackend) ResolveMutable(ctx context.Context, p ifacepath.Path) (ImmutablePath, error)

func (*BlocksBackend) ResolvePath 

func (bb *BlocksBackend) ResolvePath(ctx context.Context, path ImmutablePath) (ContentPathMetadata, error)

type BlocksBackendOption 

type BlocksBackendOption func(options *blocksBackendOptions) error

func WithNameSystem 

func WithNameSystem(ns namesys.NameSystem) BlocksBackendOption

WithNameSystem sets the name system to use with the BlocksBackend. If not set it will use the default DNSLink resolver generated by NewDNSResolver along with any configured routing.ValueStore.

func WithValueStore 

func WithValueStore(vs routing.ValueStore) BlocksBackendOption

WithValueStore sets the routing.ValueStore to use with the BlocksBackend.

type ByteRange 

type ByteRange struct {
	From uint64
	To   *int64
}

ByteRange describes a range request within a UnixFS file. "From" and "To" mostly follow HTTP Byte Range Request semantics:

  • From >= 0 and To = nil: Get the file (From, Length)
  • From >= 0 and To >= 0: Get the range (From, To)
  • From >= 0 and To <0: Get the range (From, Length - To)

type CarParams 

type CarParams struct {
	Range *DagByteRange
	Scope DagScope
}

type Config 

type Config struct {
	// Headers is a map containing all the headers that should be sent by default
	// in all requests. You can define custom headers, as well as add the recommended
	// headers via AddAccessControlHeaders.
	Headers map[string][]string

	// DeserializedResponses configures this gateway to support returning data
	// in deserialized format. By default, the gateway will only support
	// trustless, verifiable [application/vnd.ipld.raw] and
	// [application/vnd.ipld.car] responses, operating as a [Trustless Gateway].
	//
	// This global flag can be overridden per FQDN in PublicGateways map.
	//
	// [application/vnd.ipld.raw]: https://www.iana.org/assignments/media-types/application/vnd.ipld.raw
	// [application/vnd.ipld.car]: https://www.iana.org/assignments/media-types/application/vnd.ipld.car
	// [Trustless Gateway]: https://specs.ipfs.tech/http-gateways/trustless-gateway/
	DeserializedResponses bool

	// NoDNSLink configures the gateway to _not_ perform DNS TXT record lookups in
	// response to requests with values in `Host` HTTP header. This flag can be
	// overridden per FQDN in PublicGateways. To be used with WithHostname.
	NoDNSLink bool

	// PublicGateways configures the behavior of known public gateways. Each key is
	// a fully qualified domain name (FQDN). To be used with WithHostname.
	PublicGateways map[string]*PublicGateway

	// Menu adds items to the gateway menu that are shown in pages, such as
	// directory listings, DAG previews and errors. These will be displayed to the
	// right of "About IPFS" and "Install IPFS".
	Menu []assets.MenuItem
}

Config is the configuration used when creating a new gateway handler.

type ContentPathMetadata 

type ContentPathMetadata struct {
	PathSegmentRoots []cid.Cid
	LastSegment      path.Resolved
	ContentType      string // Only used for UnixFS requests
}

type DagByteRange 

type DagByteRange struct {
	From int64
	To   *int64
}

DagByteRange describes a range request within a UnixFS file. "From" and "To" mostly follow the HTTP Byte Range Request semantics:

  • From >= 0 and To = nil: Get the file (From, Length)
  • From >= 0 and To >= 0: Get the range (From, To)
  • From >= 0 and To <0: Get the range (From, Length - To)
  • From < 0 and To = nil: Get the file (Length - From, Length)
  • From < 0 and To >= 0: Get the range (Length - From, To)
  • From < 0 and To <0: Get the range (Length - From, Length - To)

func NewDagByteRange 

func NewDagByteRange(rangeStr string) (DagByteRange, error)

type DagScope 

type DagScope string

DagScope describes the scope of the requested DAG, as per the Trustless Gateway specification. 

const (
	DagScopeAll    DagScope = "all"
	DagScopeEntity DagScope = "entity"
	DagScopeBlock  DagScope = "block"
)

type ErrorRetryAfter 

type ErrorRetryAfter struct {
	Err        error
	RetryAfter time.Duration
}

ErrorRetryAfter wraps any error with "retry after" hint. When an error of this type returned to the gateway handler by an IPFSBackend, the retry after value will be passed to the HTTP client in a Retry-After HTTP header.

func NewErrorRetryAfter 

func NewErrorRetryAfter(err error, retryAfter time.Duration) *ErrorRetryAfter

func (*ErrorRetryAfter) Error 

func (e *ErrorRetryAfter) Error() string

func (*ErrorRetryAfter) Is 

func (e *ErrorRetryAfter) Is(err error) bool

func (*ErrorRetryAfter) RetryAfterHeader 

func (e *ErrorRetryAfter) RetryAfterHeader() string

RetryAfterHeader returns the Retry-After header value as a string, representing the number of seconds to wait before making a new request, rounded to the nearest second. This function follows the Retry-After header definition as specified in RFC 9110.

func (*ErrorRetryAfter) Unwrap 

func (e *ErrorRetryAfter) Unwrap() error

type ErrorStatusCode 

type ErrorStatusCode struct {
	StatusCode int
	Err        error
}

ErrorStatusCode wraps any error with a specific HTTP status code. When an error of this type is returned to the gateway handler by an IPFSBackend, the status code will be used for the response status.

func NewErrorStatusCode 

func NewErrorStatusCode(err error, statusCode int) *ErrorStatusCode

func NewErrorStatusCodeFromStatus 

func NewErrorStatusCodeFromStatus(statusCode int) *ErrorStatusCode

func (*ErrorStatusCode) Error 

func (e *ErrorStatusCode) Error() string

func (*ErrorStatusCode) Is 

func (e *ErrorStatusCode) Is(err error) bool

func (*ErrorStatusCode) Unwrap 

func (e *ErrorStatusCode) Unwrap() error

type GetResponse 

type GetResponse struct {
	// contains filtered or unexported fields
}

func NewGetResponseFromDirectoryListing 

func NewGetResponseFromDirectoryListing(dagSize uint64, entries <-chan unixfs.LinkResult) *GetResponse

func NewGetResponseFromFile 

func NewGetResponseFromFile(file files.File) *GetResponse

type IPFSBackend 

type IPFSBackend interface {
	// Get returns a [GetResponse] with UnixFS file, directory or a block in IPLD
	// format, e.g. (DAG-)CBOR/JSON.
	//
	// Returned Directories are preferably a minimum info required for enumeration: Name, Size, and Cid.
	//
	// Optional ranges follow [HTTP Byte Ranges] notation and can be used for
	// pre-fetching specific sections of a file or a block.
	//
	// Range notes:
	//   - Generating response to a range request may require additional data
	//     beyond the passed ranges (e.g. a single byte range from the middle of a
	//     file will still need magic bytes from the very beginning for content
	//     type sniffing).
	//   - A range request for a directory currently holds no semantic meaning.
	//
	// [HTTP Byte Ranges]: https://httpwg.org/specs/rfc9110.html#rfc.section.14.1.2
	Get(context.Context, ImmutablePath, ...ByteRange) (ContentPathMetadata, *GetResponse, error)

	// GetAll returns a UnixFS file or directory depending on what the path is that has been requested. Directories should
	// include all content recursively.
	GetAll(context.Context, ImmutablePath) (ContentPathMetadata, files.Node, error)

	// GetBlock returns a single block of data
	GetBlock(context.Context, ImmutablePath) (ContentPathMetadata, files.File, error)

	// Head returns a file or directory depending on what the path is that has been requested.
	// For UnixFS files should return a file which has the correct file size and either returns the ContentType in ContentPathMetadata or
	// enough data (e.g. 3kiB) such that the content type can be determined by sniffing.
	// For all other data types returning just size information is sufficient
	// TODO: give function more explicit return types
	Head(context.Context, ImmutablePath) (ContentPathMetadata, files.Node, error)

	// ResolvePath resolves the path using UnixFS resolver. If the path does not
	// exist due to a missing link, it should return an error of type:
	// NewErrorResponse(fmt.Errorf("no link named %q under %s", name, cid), http.StatusNotFound)
	ResolvePath(context.Context, ImmutablePath) (ContentPathMetadata, error)

	// GetCAR returns a CAR file for the given immutable path. It returns an error
	// if there was an issue before the CAR streaming begins.
	GetCAR(context.Context, ImmutablePath, CarParams) (ContentPathMetadata, io.ReadCloser, error)

	// IsCached returns whether or not the path exists locally.
	IsCached(context.Context, path.Path) bool

	// GetIPNSRecord retrieves the best IPNS record for a given CID (libp2p-key)
	// from the routing system.
	GetIPNSRecord(context.Context, cid.Cid) ([]byte, error)

	// ResolveMutable takes a mutable path and resolves it into an immutable one. This means recursively resolving any
	// DNSLink or IPNS records.
	//
	// For example, given a mapping from `/ipns/dnslink.tld -> /ipns/ipns-id/mydirectory` and `/ipns/ipns-id` to
	// `/ipfs/some-cid`, the result of passing `/ipns/dnslink.tld/myfile` would be `/ipfs/some-cid/mydirectory/myfile`.
	ResolveMutable(context.Context, path.Path) (ImmutablePath, error)

	// GetDNSLinkRecord returns the DNSLink TXT record for the provided FQDN.
	// Unlike ResolvePath, it does not perform recursive resolution. It only
	// checks for the existence of a DNSLink TXT record with path starting with
	// /ipfs/ or /ipns/ and returns the path as-is.
	GetDNSLinkRecord(context.Context, string) (path.Path, error)
}

IPFSBackend is the required set of functionality used to implement the IPFS HTTP Gateway specification.

The error returned by the implementer influences the status code that the user receives. By default, the gateway is able to handle a few error types, such as context.DeadlineExceeded, cid.ErrInvalidCid and various IPLD-pathing related errors.

To signal custom error types to the gateway, such that not everything is an 500 Internal Server Error, implementers can return errors wrapped in either ErrorRetryAfter or ErrorStatusCode.

type ImmutablePath 

type ImmutablePath struct {
	// contains filtered or unexported fields
}

ImmutablePath represents a path.Path that is not mutable.

TODO: Is this what we want for ImmutablePath?

func NewImmutablePath 

func NewImmutablePath(p path.Path) (ImmutablePath, error)

func (ImmutablePath) IsValid 

func (i ImmutablePath) IsValid() error

func (ImmutablePath) Mutable 

func (i ImmutablePath) Mutable() bool

func (ImmutablePath) Namespace 

func (i ImmutablePath) Namespace() string

func (ImmutablePath) String 

func (i ImmutablePath) String() string

type PublicGateway 

type PublicGateway struct {
	// Paths is explicit list of path prefixes that should be handled by
	// this gateway. Example: `["/ipfs", "/ipns"]`
	// Useful if you only want to support immutable `/ipfs`.
	Paths []string

	// UseSubdomains indicates whether or not this is a [Subdomain Gateway].
	//
	// If this flag is set, any `/ipns/$id` and/or `/ipfs/$id` paths in Paths
	// will be permanently redirected to `http(s)://$id.[ipns|ipfs].$gateway/`.
	//
	// We do not support using both paths and subdomains for a single domain
	// for security reasons ([Origin isolation]).
	//
	// [Subdomain Gateway]: https://specs.ipfs.tech/http-gateways/subdomain-gateway/
	// [Origin isolation]: https://en.wikipedia.org/wiki/Same-origin_policy
	UseSubdomains bool

	// NoDNSLink configures this gateway to _not_ resolve DNSLink for the
	// specific FQDN provided in `Host` HTTP header. Useful when you want to
	// explicitly allow or refuse hosting a single hostname. To refuse all
	// DNSLinks in `Host` processing, set NoDNSLink in Config instead. This setting
	// overrides the global setting.
	NoDNSLink bool

	// InlineDNSLink configures this gateway to always inline DNSLink names
	// (FQDN) into a single DNS label in order to interop with wildcard TLS certs
	// and Origin per CID isolation provided by rules like https://publicsuffix.org
	//
	// This should be set to true if you use HTTPS.
	InlineDNSLink bool

	// DeserializedResponses configures this gateway to support returning data
	// in deserialized format. This setting overrides the global setting.
	DeserializedResponses bool
}

PublicGateway is the specification of an IPFS Public Gateway.

type RequestContextKey 

type RequestContextKey string

RequestContextKey is a type representing a context.Context value key. 

const (
	// GatewayHostnameKey is the key for the hostname at which the gateway is
	// operating. It may be a DNSLink, Subdomain or Regular gateway.
	GatewayHostnameKey RequestContextKey = "gw-hostname"

	// DNSLinkHostnameKey is the key for the hostname of a [DNSLink Gateway].
	//
	// [DNSLink Gateway]: https://specs.ipfs.tech/http-gateways/dnslink-gateway/
	DNSLinkHostnameKey RequestContextKey = "dnslink-hostname"

	// SubdomainHostnameKey is the key for the hostname of a [Subdomain Gateway].
	//
	// [Subdomain Gateway]: https://specs.ipfs.tech/http-gateways/subdomain-gateway/
	SubdomainHostnameKey RequestContextKey = "subdomain-hostname"

	// ContentPathKey is the key for the original [http.Request] URL Path, as an [ipath.Path].
	ContentPathKey RequestContextKey = "content-path"
)

Source Files

View all Source files

Directories

Path Synopsis
test

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.