eth

Documentation

Index

• func Read(scope beam.Scope, wsURL string, filter filter.Filter) beam.PCollection
• type EthClient
  • func Connect(wsURL string) *EthClient
• type SubscriptionRTracker
  • func NewSubscriptionRTracker(entry string) *SubscriptionRTracker
  • func (s *SubscriptionRTracker) GetError() error
  • func (s *SubscriptionRTracker) GetProgress() (done float64, remaining float64)
  • func (s *SubscriptionRTracker) GetRestriction() any
  • func (s *SubscriptionRTracker) IsBounded() bool
  • func (s *SubscriptionRTracker) IsDone() bool
  • func (s *SubscriptionRTracker) TryClaim(pos any) bool
  • func (s *SubscriptionRTracker) TrySplit(frac float64) (primary, residual any, err error)

Functions

func Read

func Read(scope beam.Scope, wsURL string, filter filter.Filter) beam.PCollection

Types

type EthClient

type EthClient struct {
	// contains filtered or unexported fields
}

func Connect

func Connect(wsURL string) *EthClient

type SubscriptionRTracker

type SubscriptionRTracker struct {
	Subscription string
	Done         bool
}

The SubscriptionRTracker maintains a single entry string to keep up with the PubSub subscription being used in the NativeRead SDF.

func NewSubscriptionRTracker

func NewSubscriptionRTracker(entry string) *SubscriptionRTracker

NewSubscriptionRTracker returns an RTracker wrapping the provided subscription and a "Done" boolean.

func (*SubscriptionRTracker) GetError

func (s *SubscriptionRTracker) GetError() error

GetError is a no-op.

func (*SubscriptionRTracker) GetProgress

func (s *SubscriptionRTracker) GetProgress() (done float64, remaining float64)

GetProgress returns complete just so the runner doesn't try to do much in the way of splitting.

func (*SubscriptionRTracker) GetRestriction

func (s *SubscriptionRTracker) GetRestriction() any

GetRestriction returns the name of the subscription.

func (*SubscriptionRTracker) IsBounded

func (s *SubscriptionRTracker) IsBounded() bool

IsBounded always returns false, as the StaticRTracker represents an unbounded number of reads from PubSub.

func (*SubscriptionRTracker) IsDone

func (s *SubscriptionRTracker) IsDone() bool

IsDone returns whether or not the StaticRTracker is complete (e.g. has stopped processing.)

func (*SubscriptionRTracker) TryClaim

func (s *SubscriptionRTracker) TryClaim(pos any) bool

TryClaim returns true iff the given position is a string and matches the underlying subscription ID.

func (*SubscriptionRTracker) TrySplit

func (s *SubscriptionRTracker) TrySplit(frac float64) (primary, residual any, err error)

TrySplit is a no-op for the StaticRTracker in the normal case and moves the subscription to the residual in the checkpointing case, marking itself as done to keep the logical checks around SDF data loss happy.