web3protocol

package module
v0.2.13 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 13, 2024 License: MIT Imports: 31 Imported by: 1

README

web3protocol-go

Parse and execute ERC-6860 / ERC-4804 web3:// protocol URLs.

Usage

import "github.com/web3-protocol/web3protocol-go"

config := web3protocol.Config{ ... } // Fill with the example in the configuration section
client := NewClient(&config)

fetchedWeb3Url, err := client.FetchUrl("web3://terraformnavigator.eth/view/1234")

fmt.Println("HTTP return code:", fetchedWeb3Url.HttpCode)
fmt.Printf("HTTP return headers: %+v\n", fetchedWeb3Url.HttpHeaders)

output, err := ioutil.ReadAll(fetchedWeb3Url.Output)
fmt.Println("Output bytes", output)

On top of the result, fetchedWeb3Url contains a lot more details on the processing of the call.

Configuration

Right now, by default, this does not come with chains and domain name definitions; they will need to be provided. Here is an example for a basic Ethereum mainnet + ENS config, using the publicnode.com RPCs :

config := Config {
    Chains: map[int]ChainConfig{
        1: ChainConfig{
            ChainId: 1,
            ShortName: "eth",
            RPC: "https://ethereum.publicnode.com/",
            DomainNameServices: map[DomainNameService]DomainNameServiceChainConfig{
                DomainNameServiceENS: DomainNameServiceChainConfig{
                    Id: DomainNameServiceENS,
                    ResolverAddress: common.HexToAddress("0x00000000000C2E074eC69A0dFb2997BA6C7d2e1e"),
                },
            },
        },
    },
    DomainNameServices: map[DomainNameService]DomainNameServiceConfig{
        DomainNameServiceENS: DomainNameServiceConfig{
            Id: DomainNameServiceENS,
            Suffix: "eth",
            DefaultChainId: 1,
        },
    },
}

Supported standards

Implemented features
  • ERC-6860 (draft) : the base web3:// protocol with auto and manual mode, basic ENS support. This updates ERC-4804 (final) with clarifications, small fixes and changes.
  • ERC-6821 (draft) : ENS resolution : support for the contentcontract TXT field to point to a contract in another chain
  • ERC-6944 (draft) / ERC-5219 (final) : New mode offloading some parsing processing on the browser side
  • ERC-7617 (draft): Add chunk support in ERC-6944 resource request mode
  • ERC-7618 (draft): Add Content-encoding handling in ERC-6944 resource request mode
Partially implemented features
  • ERC-7087 (draft) : Auto mode : Add MIME type support (dataURLs not supported yet)
  • ERC-7774 (pending): Cache invalidation in ERC-5219 mode Web3 URL (only ETag supported, wildcards not yet supported)

Testing

Web3:// test files are located in their own git repository and are imported as a git submodule. After cloning, please run git submodule init and then git submodule update.

Testing is then launched with go test

Documentation

Index

Constants

View Source 
const (
	DomainNameServiceENS  = "ens"
	DomainNameServiceW3NS = "w3ns"
)
View Source 
const (
	ResolveModeAuto             = "auto"
	ResolveModeManual           = "manual"
	ResolveModeResourceRequests = "resourceRequest"
)
View Source 
const (
	ContractCallModeCalldata = "calldata"
	ContractCallModeMethod   = "method"
)
View Source 
const (
	// Expect the whole returned data to be ABI-encoded bytes. Decode.
	ContractReturnProcessingDecodeABIEncodedBytes = "decodeABIEncodedBytes"
	// JSON-encode the raw bytes of the returned data
	ContractReturnProcessingRawBytesJsonEncoded = "jsonEncodeRawBytes"
	// JSON-encode the different return values
	ContractReturnProcessingJsonEncodeValues = "jsonEncodeValues"
	// Expect a string as first return value, parse it as a dataUrl
	// ContractReturnProcessingDataUrl = "dataUrl" // To implement
	// Expect a return following the erc5219 spec, will decode it using this spec
	ContractReturnProcessingDecodeErc5219Request = "decodeErc5219Request"
)

Variables

View Source 
var (
	EmptyString  = strings.Repeat("0", 62) + "20" + strings.Repeat("0", 64)
	EmptyAddress = strings.Repeat("0", 64)
)

Functions

func GetCacheControlHeaderDirectives added in v0.2.5 

func GetCacheControlHeaderDirectives(headerValue string) (directives map[string]string)

Parse the Cache-Control header value into a map of directives

func JsonEncodeAbiTypeValue 

func JsonEncodeAbiTypeValue(arg abi.Type, value interface{}) (result interface{}, err error)

Used for auto mode returning JSON : For a given ABI type and a value, convert it to a string, with the data formatted according to the spec

func LabelHash 

func LabelHash(label string) (hash [32]byte, err error)

LabelHash generates a simple hash for a piece of a name.

func NameHash 

func NameHash(name string) (hash [32]byte, err error)

NameHash generates a hash from a name that can be used to look up the name in ENS

func Normalize 

func Normalize(input string) (output string, err error)

Normalize normalizes a name according to the ENS rules

func SerializeResourceRequestMethodArgValues added in v0.2.5 

func SerializeResourceRequestMethodArgValues(argValues []interface{}) (pathQuery string)

Serialize the arguments of a resource request method, for the purpose of making a cache key

Types

type AddressPathQuery added in v0.2.7 

type AddressPathQuery struct {
	ContractAddress common.Address
	PathQuery       string
}

type ArgInfo 

type ArgInfo struct {
	// contains filtered or unexported fields
}

type ChainConfig 

type ChainConfig struct {
	ChainId int

	// A mapping of chain "short name" (from https://github.com/ethereum-lists/chains) to their chain id
	// Used by ERC-6821 which relies on ERC-3770 addresses
	ShortName string

	// The RPC URL to use to call the chain
	RPC ChainRPCConfig

	// System RPC : RPC used by system workers (right now, ERC-7774 and its event caching checks)
	// It will not be used for user requests, and has to be different from the main RPCs
	// Aim : If the main RPCs are down, the system workers can still work
	// If empty, the default RPC is used
	SystemRPC string

	// A chain-specific config per domain name service. Key is their short name.
	DomainNameServices map[DomainNameService]DomainNameServiceChainConfig
}

type ChainRPCConfig added in v0.2.12 

type ChainRPCConfig struct {
	// The RPC URL to use to call the chain
	Url string
	// The maximum number of parralel requests to the RPC
	MaxConcurrentRequests int
}

type Client 

type Client struct {
	Config *Config
	Logger *logrus.Logger

	// The list of RPCs per chain. They are filled with the content of the config,
	// and contains their state (available, too many requests, unauthorized)
	Rpcs      map[int][]*Rpc
	RpcsMutex sync.RWMutex

	// The request queue, and their channel to notify the caller with the response
	// Used to aggregate requests to the same URL
	RequestQueue      map[RequestQueueKey][]chan *RequestQueueResponse
	RequestQueueMutex sync.Mutex

	// Cache for domain name resolution
	DomainNameResolutionCache *localCache

	// Cache for resolve mode determination
	ResolveModeCache *golanglru2.LRU[ResolveModeCacheKey, ResolveMode]

	// ERC-7774: Resource request mode : Cache invalidation tracking
	ResourceRequestCachingTracker ResourceRequestCachingTracker
}

func NewClient 

func NewClient(config *Config) (client *Client)

*

  • You'll need to instantiate a client to make calls.

func (*Client) AttemptEarlyResourceRequestModeResponse added in v0.2.5 

func (client *Client) AttemptEarlyResourceRequestModeResponse(web3Url *Web3URL) (fetchedWeb3Url FetchedWeb3URL, success bool, err error)

Step 2 : Attempt early response

func (*Client) AttemptEarlyResponse added in v0.2.5 

func (client *Client) AttemptEarlyResponse(web3Url *Web3URL) (fetchedWeb3Url FetchedWeb3URL, success bool, err error)

*

  • Step 2: Attempt an early response which bypass a contract call.

func (*Client) CheckTooManyRequestsStateWorker added in v0.2.12 

func (client *Client) CheckTooManyRequestsStateWorker(rpc *Rpc)

When a RPC is returning 429, we start a worker to check if it is available again

func (*Client) FetchContractReturn 

func (client *Client) FetchContractReturn(web3Url *Web3URL) (contractReturn []byte, err error)

*

  • Step 3: Make the call to the main contract.

func (*Client) FetchUrl 

func (client *Client) FetchUrl(url string, httpHeaders map[string]string) (fetchedUrl *FetchedWeb3URL, err error)

*

  • The main function of the package.
  • For a given full web3:// url ("web3://xxxx"), returns a structure containing
  • the bytes output and the HTTP code and headers, as well as plenty of informations on
  • how the processing was done.

func (*Client) FetchUrlDirect added in v0.2.12 

func (client *Client) FetchUrlDirect(url string, httpHeaders map[string]string) (fetchedUrl FetchedWeb3URL, err error)

*

  • The main function executing a web3:// URL call.
  • For a given full web3:// url ("web3://xxxx"), returns a structure containing
  • the bytes output and the HTTP code and headers, as well as plenty of informations on
  • how the processing was done. *
  • Unlike FetchUrl(), this function does not use the built-in worker system which
  • aggregates identical requests and limits the number of parallel requests.
  • On the other hand, it does have the per-RPC limit of parralel requests.

func (*Client) GetSystemRpcUrl added in v0.2.12 

func (client *Client) GetSystemRpcUrl(chain int) (rpcUrl string, err error)

Get the RPC to be used by system workers (e.g. ERC-7774), by chain

func (*Client) ParseUrl 

func (client *Client) ParseUrl(url string, httpHeaders map[string]string) (web3Url Web3URL, err error)

*

  • Step 1 : Parse the URL and determine how we are going to call the main contract.

func (*Client) ProcessContractReturn 

func (client *Client) ProcessContractReturn(web3Url *Web3URL, contractReturn []byte) (fetchedWeb3Url FetchedWeb3URL, err error)

*

  • Step 4 : Process the data returned by the main contract.

func (*Client) ProcessResourceRequestContractReturn 

func (client *Client) ProcessResourceRequestContractReturn(fetchedWeb3Url *FetchedWeb3URL, web3Url *Web3URL, contractReturn []byte) (err error)

Step 4 : We have the contract return, process it

type Config 

type Config struct {
	// A config per chain. Key is the chain id
	Chains map[int]ChainConfig

	// A config per domain name service. Key is their short name
	DomainNameServices map[DomainNameService]DomainNameServiceConfig

	// There is an internal domain name resolution cache
	NameAddrCacheDurationInMinutes int
}

The config used by the client to make the web3:// calls

func (*Config) GetChainIdByShortName 

func (config *Config) GetChainIdByShortName(shortName string) (result int)

func (*Config) GetDomainNameServiceBySuffix 

func (config *Config) GetDomainNameServiceBySuffix(suffix string) (result DomainNameService)

type ContractCallMode 

type ContractCallMode string

type ContractReturnProcessing 

type ContractReturnProcessing string

type DomainNameService 

type DomainNameService string

type DomainNameServiceChainConfig 

type DomainNameServiceChainConfig struct {
	Id DomainNameService

	// The URL to the contract of the resolver
	ResolverAddress common.Address
}

Attributes of a domain name service specific to a chain

type DomainNameServiceConfig 

type DomainNameServiceConfig struct {
	Id DomainNameService

	// "eth", ...
	Suffix string

	// The default home chain of a domain name service; e.g. 1 for ENS, 333 for W3NS
	DefaultChainId int
}

Attributes of a domain name service

type FetchedWeb3URL 

type FetchedWeb3URL struct {
	// The web3 URL, parsed
	ParsedUrl *Web3URL

	// The raw data returned by the contract
	ContractReturn []byte

	// The processed output, to be returned by the browser
	Output io.Reader
	// The HTTP code to be returned by the browser
	HttpCode int
	// The HTTP headers to be returned by the browser
	HttpHeaders map[string]string
}

This contains the result of a web3:// URL call : the parsed URL, the raw contract return, and the bytes output, HTTP code and headers for the browser.

type JsonError added in v0.2.12 

type JsonError interface {
	Error() string
	ErrorCode() int
	ErrorData() interface{}
}

Go-Ethereum use an internal jsonError type; to be able to read it we redeclare it here

type ParsedWeb3Url added in v0.2.5 

type ParsedWeb3Url struct {
	Protocol string
	Hostname string
	ChainId  string

	// The PathQuery is the full path, including the Pathname and Query
	PathQuery string
	Path      string
	Query     string

	Fragment string
}

A raw splitting of the web3 URL parts

type PrefixDecompressionErrorReader added in v0.2.3 

type PrefixDecompressionErrorReader struct {
	Reader io.Reader
}

func (*PrefixDecompressionErrorReader) Read added in v0.2.3 

func (r *PrefixDecompressionErrorReader) Read(p []byte) (readBytes int, err error)

type QueryParameter 

type QueryParameter struct {
	Name  string
	Value string
}

URL.parseQuery does not preserve the order of query attributes This is a version which keep order

type QueryParameters 

type QueryParameters []QueryParameter

func ParseQuery 

func ParseQuery(query string) (params QueryParameters, err error)

type RequestQueueKey added in v0.2.12 

type RequestQueueKey struct {
	// The URL of the request
	URL string
	// Some specific request headers, which may alter the response
	HttpHeaderIfNoneMatch     string
	HttpHeaderIfModifiedSince string
	HttpHeaderAcceptEncoding  string
}

Requests are grouped by URL and some specific request headers

type RequestQueueResponse added in v0.2.12 

type RequestQueueResponse struct {
	// contains filtered or unexported fields
}

The response to a request queue entry

type ResolveMode 

type ResolveMode string

type ResolveModeCacheKey added in v0.2.6 

type ResolveModeCacheKey struct {
	ChainId         int
	ContractAddress common.Address
}

type ResourceCachingInfos added in v0.2.5 

type ResourceCachingInfos struct {
	// The last modified date of the resource
	LastModified time.Time

	// The ETag of the resource
	ETag string

	// Others contract can proxy paths of this contract, and they can indicate with
	// '"Cache-Control: evem-events="<contractAddress><pathQuery>"'
	// to listen for cache clearing events for this contract
	// So here we store the list of external contracts paths that proxy this resource
	CacheClearEventListeners []AddressPathQuery
}

Caching infos for a specific resource

type ResourceRequestCachingTracker added in v0.2.5 

type ResourceRequestCachingTracker struct {
	// Per-chain caching trackers
	ChainCachingTrackers map[int]*ResourceRequestChainCachingTracker

	// Pointer to the Client, to access the config and make requests
	Client *Client

	// Mutex to protect the struct
	Mutex sync.RWMutex
}

Contains the caching infos of all the resources requested by the client that implements ERC-7774 (includes "evm-events" in the Cache-Control header)

func NewResourceRequestCachingTracker added in v0.2.5 

func NewResourceRequestCachingTracker(client *Client) ResourceRequestCachingTracker

func (*ResourceRequestCachingTracker) GetChainCachingTracker added in v0.2.5 

func (c *ResourceRequestCachingTracker) GetChainCachingTracker(chainId int) (chainCachingTracker *ResourceRequestChainCachingTracker, ok bool)

func (*ResourceRequestCachingTracker) GetOrCreateChainCachingTracker added in v0.2.5 

func (c *ResourceRequestCachingTracker) GetOrCreateChainCachingTracker(chainId int) (chainCachingTracker *ResourceRequestChainCachingTracker)

type ResourceRequestChainCachingTracker added in v0.2.5 

type ResourceRequestChainCachingTracker struct {
	// The chain ID of this tracker
	ChainId int
	// The last block number we have processed
	LastBlockNumber uint64
	// The time of the last successfull event check
	LastEventsCheck time.Time
	// Is it active? It is not active if the event check worker has trouble and is
	// not able to track events
	IsActive bool

	// The cache of indifidual resources requested by the client
	// map[contractAddress][pathQuery]
	ResourcesCachingInfos map[common.Address]map[string]*ResourceCachingInfos
	// Last read date of the cache
	LastRead time.Time

	// There is a goroutine worker that checks for events to be processed
	// This is the channel to stop it
	EventsCheckWorkerStopChan chan bool

	// Mutex to protect access to the struct
	Mutex sync.RWMutex

	// Pointer to the global caching tracker
	GlobalCachingTracker *ResourceRequestCachingTracker
}

Like ResourceRequestCachingTracker, but for a specific chain

func (*ResourceRequestChainCachingTracker) Activate added in v0.2.5 

func (c *ResourceRequestChainCachingTracker) Activate()

The worker that checks for events to be processed is now able to track events

func (*ResourceRequestChainCachingTracker) ActivateUnsafe added in v0.2.8 

func (c *ResourceRequestChainCachingTracker) ActivateUnsafe()

Activate() without the mutex

func (*ResourceRequestChainCachingTracker) DeleteResourceCachingInfos added in v0.2.5 

func (c *ResourceRequestChainCachingTracker) DeleteResourceCachingInfos(contractAddress common.Address, pathQuery string)

Delete the caching infos by pathQuery. Support wildcard pathQuery.

func (*ResourceRequestChainCachingTracker) Desactivate added in v0.2.5 

func (c *ResourceRequestChainCachingTracker) Desactivate()

The worker that checks for events to be processed had issues and is not able to track events, so we mark the tracker as inactive, and clear cache

func (*ResourceRequestChainCachingTracker) GetResourceCachingInfos added in v0.2.5 

func (c *ResourceRequestChainCachingTracker) GetResourceCachingInfos(contractAddress common.Address, pathQuery string) (resourceCachingInfos *ResourceCachingInfos, ok bool)

func (*ResourceRequestChainCachingTracker) GetResourceCachingInfosByPattern added in v0.2.9 

func (c *ResourceRequestChainCachingTracker) GetResourceCachingInfosByPattern(contractAddress common.Address, pathQueryPattern string) (resourceCachingInfos map[string]*ResourceCachingInfos, err error)

func (*ResourceRequestChainCachingTracker) SetResourceCachingInfos added in v0.2.5 

func (c *ResourceRequestChainCachingTracker) SetResourceCachingInfos(contractAddress common.Address, pathQuery string, resourceCachingInfos *ResourceCachingInfos)

type ResourceRequestReader added in v0.2.0 

type ResourceRequestReader struct {
	Client         *Client
	FetchedWeb3URL *FetchedWeb3URL
	// Content of the last chunk call
	Chunk        []byte
	Cursor       int
	NextChunkUrl string
}

func (*ResourceRequestReader) Read added in v0.2.0 

func (rrr *ResourceRequestReader) Read(p []byte) (readBytes int, err error)

Return the result of the method call Implements ERC-7617: Support for chunking

type Rpc added in v0.2.12 

type Rpc struct {
	// The RPC config
	Config ChainRPCConfig

	// The state of the RPC
	State RpcState

	// We authorize X parralel requests to the RPC
	RequestSemaphone chan struct{}
}

A RPC, containing its URL and state

type RpcState added in v0.2.12 

type RpcState string
const (
	RpcStateAvailable       RpcState = "available"
	RpcStateTooManyRequests RpcState = "tooManyRequests"
	RpcStateUnauthorized    RpcState = "unauthorized"
)

type SharedOutput added in v0.2.13 

type SharedOutput struct {
	// The original output
	OriginalOutput io.Reader
	// The fetched byte
	FetchedBytes []byte
	// Is ended?
	IsEnded bool

	Mutex sync.Mutex
}

When a request is shared between multiple receivers, we need to duplicate the response This contains the original output, and ongoing fetched bytes

type SharedOutputReader added in v0.2.13 

type SharedOutputReader struct {
	// The shared output
	SharedOutput *SharedOutput
	// The current position in the output
	Position int
}

This is a io.Reader that reads from a SharedOutput

func (*SharedOutputReader) Read added in v0.2.13 

func (r *SharedOutputReader) Read(p []byte) (n int, err error)

type Web3ProtocolError added in v0.2.12 

type Web3ProtocolError struct {
	Type Web3ProtocolErrorType

	// The HTTP code to return to the client
	HttpCode int

	// If the type is RPC error, this is the HTTP code and message from the RPC
	RpcHttpCode int

	// If the type is RPC JSON error, this is the JSON error from the RPC
	JsonErrorCode int
	JsonErrorData interface{}

	// The original error, if any
	Err error
}

Web3 protocol error

func (*Web3ProtocolError) Error added in v0.2.12 

func (e *Web3ProtocolError) Error() string

type Web3ProtocolErrorType added in v0.2.12 

type Web3ProtocolErrorType string // The type of the error

const (
	// The RPC call itself failed (bad HTTP code)
	Web3ProtocolErrorTypeRPCError Web3ProtocolErrorType = "rpcError"
	// The RPC call succeeded, but the JSON returned by the RPC is an error
	Web3ProtocolErrorTypeRPCJsonError Web3ProtocolErrorType = "rpcJsonError"
	// Other
	Web3ProtocolErrorTypeOther Web3ProtocolErrorType = ""
)

type Web3URL 

type Web3URL struct {
	// The actual url string "web3://...."
	Url string
	// The request HTTP headers
	HttpHeaders map[string]string

	// A raw splitting of the web3 URL parts, to be used by the processing
	// You should not use this directly outside of this package
	UrlParts ParsedWeb3Url

	// If the host was a domain name, what domain name service was used?
	HostDomainNameResolver DomainNameService
	// Chain of the name resolution service
	HostDomainNameResolverChainId int

	// The contract address (after optional domain name resolution) that is going to be called,
	// and its chain location
	ContractAddress common.Address // actual address
	ChainId         int

	// The ERC-4804 resolve mode
	ResolveMode ResolveMode

	// How do we call the smartcontract
	// 'calldata' : We use a raw calldata
	// 'method': We use the specified method and method parameters
	ContractCallMode ContractCallMode
	// Attributes for ContractCallModeCalldata
	Calldata []byte
	// Attributes for ContractCallModeMethod
	MethodName      string
	MethodArgs      []abi.Type
	MethodArgValues []interface{}

	// How to process the return of the contract. See enum for doc
	ContractReturnProcessing ContractReturnProcessing
	// In case of contractReturnProcessing being decodeABIEncodedBytes,
	// this will set the mime type to return
	DecodedABIEncodedBytesMimeType string
	// In case of ContractReturnProcessing being jsonEncodeValues,
	// this will tell us how to ABI-decode the returned data
	JsonEncodedValueTypes []abi.Type
}

This contains a web3:// URL parsed and ready to call the main smartcontract

func (*Web3URL) ComputeCalldata 

func (web3Url *Web3URL) ComputeCalldata() (calldata []byte, err error)

If ContractCallMode is calldata, returned the stored calldata If ContractCallMode is method, compute and return it

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.