Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type OptionFunc ¶
type OptionFunc func(*Options)
OptionFunc is a function that applies an option to the Uploader.
func WithCallbackURL ¶
func WithCallbackURL(url string) OptionFunc
WithCallbackURL sets the callback URL.
func WithHTTPTimeout ¶
func WithHTTPTimeout(timeout time.Duration) OptionFunc
WithHTTPTimeout sets the HTTP timeout.
func WithLRUCacheSize ¶
func WithLRUCacheSize(size int) OptionFunc
WithLRUCacheSize sets the LRU cache size.
type Options ¶
type Options struct {
// contains filtered or unexported fields
}
Options holds optional parameters for the Uploader.
func DefaultOptions ¶
DefaultOptions returns the default options.
type Request ¶
type Request struct {
// Vertex is the name of the vertex
Vertex string `json:"vertex"`
// Pipeline is the name of the pipeline
Pipeline string `json:"pipeline"`
// UUID is the unique identifier of the message
ID string `json:"id"`
// CbTime is the time when the callback was made
CbTime int64 `json:"cb_time"`
// Tags is the list of tags associated with the message
Tags []string `json:"tags,omitempty"` // Add 'omitempty' here
// FromVertex is the name of the vertex from which the message was sent
FromVertex string `json:"from_vertex"`
}
Request is the struct that holds the data to be sent in the POST request
type Uploader ¶
type Uploader struct {
// contains filtered or unexported fields
}
Uploader uploads the callback messages to the callback endpoint.
func NewUploader ¶
func NewUploader(ctx context.Context, vertexName string, pipelineName string, opts ...OptionFunc) *Uploader
NewUploader creates a new callback Uploader.
func (*Uploader) GetClient ¶
GetClient returns a client for the given URL from the cache If the client is not in the cache, a new one is created.
func (*Uploader) NonSinkVertexCallback ¶
func (u *Uploader) NonSinkVertexCallback(ctx context.Context, messagePairs []isb.ReadWriteMessagePair) error
NonSinkVertexCallback groups callbacks based on their callback URL and makes a POST request for each callback group for non-sink vertices. If the callbackHeaderKey is set, it writes all the callback requests to the callbackURL. In case of failure while writing the url from the headers, it writes all the callback requests to the callbackURL. The callback URL could change because of the affinity towards colocating the callbacks to the same endpoint server that initiated the request (each request could be from a different server in the pool).
func (*Uploader) SinkVertexCallback ¶
SinkVertexCallback groups callbacks based on their callback URL present in the headers and sends a POST request for each callback group for sink vertices. If the callback header is not set, it writes all the callback requests to the callbackURL. In case of failure while writing the url from the headers, it writes all the callback requests to the callbackURL.
Source Files