readmatcher

Documentation

Index

• func New(offsetReader func(int64) ioctx.ReadCloser, opts ...Opt) interface{ ... }
• type Opt
  • func SoftMaxReaders(n int) Opt

Functions

func New

func New(offsetReader func(int64) ioctx.ReadCloser, opts ...Opt) interface {
	ioctx.ReaderAt
	ioctx.Closer
}

New returns a ReaderAt that "multiplexes" incoming reads onto one of a collection of "backend" readers. It matches read to backend based on last read position; a reader is selected if its last request ended near where the new read starts.

It is intended for use with biofs+S3. S3 readers have high initialization costs vs. subsequently reading bytes, because that is S3's performance characteristic. ReaderAt maps incoming reads to a backend S3 reader that may be able to efficiently serve it. Otherwise, it opens a new reader. Our intention is that this will adapt to non-S3-aware clients' read patterns (small reads). S3-aware clients can always choose to read big chunks to avoid performance worst-cases. But, the Linux kernel limits FUSE read requests to 128 KiB, and we can't feasibly change that, so we adapt.

To performantly handle Linux kernel readahead requests, the matching algorithm allows out-of-order positions within a small window (see trailingbuf).

offsetReader opens a reader into the underlying file.

Types

type Opt

type Opt func(*readerAt)

func SoftMaxReaders

func SoftMaxReaders(n int) Opt