limitio

README

LimitIO

io.Reader and io.Writer with limit.

go get github.com/nanmu42/limitio

Rationale and Usage

There are times when a limited reader or writer comes in handy.

1. wrap upstream so that reading is metered and limited:

// request is an incoming http.Request
request.Body = limitio.NewReadCloser(c.Request.Body, maxRequestBodySize, false)

// deal with the body now with easy mind. It's maximum size is assured.

Yes, io.LimitReader works the same way, but throws EOF on exceeding limit, which is confusing.

LimitIO provides error that can be identified.

decoder := json.NewDecoder(request.Body)
err := decoder.Decode(&myStruct)
if err != nil {
    if errors.Is(err, limitio.ErrThresholdExceeded) {
        // oops, we reached the limit
    }

    err = fmt.Errorf("other error happened: %w", err)
    return
}

1. wrap downstream so that writing is metered and limited(or instead, just pretending writing):

// request is an incoming http.Request.
// Say, we want to record its body somewhere in the middleware,
// but feeling uneasy since its body might be HUGE, which may
// result in OOM and a successful DDOS...

var reqBuf bytes.buffer

// a limited writer comes to rescue!
// `true` means after reaching `RequestBodyMaxLength`,
// `limitedReqBuf` will start pretending writing so that
// io.TeeReader continues working while reqBuf stays unmodified.
limitedReqBuf := limitio.NewWriter(&reqBuf, RequestBodyMaxLength, true)

request.Body = &readCloser{
    Reader: io.TeeReader(request.Body, limitedReqBuf), 
    Closer: c.Request.Body,
}

LimitIO provides Reader, Writer and their Closer versions, for details, see docs.

Status: Stable

LimitIO is well battle tested under production environment.

APIs are subjected to change in backward compatible way during 1.x releases.

License

MIT License

Copyright (c) 2021 LI Zhennan

Documentation

Overview

Package limitio brings `io.Reader` and `io.Writer` with limit.

Index

• Variables
• func AtMostFirstNBytes(s []byte, n int) []byte
• type ReadCloser
  • func NewReadCloser(r io.ReadCloser, limit int, regardOverSizeEOF bool) *ReadCloser
• type Reader
  • func NewReader(r io.Reader, limit int, regardOverSizeEOF bool) *Reader
  • func (lr *Reader) Read(p []byte) (n int, err error)
• type WriteCloser
  • func NewWriteCloser(w io.WriteCloser, n int, silentWhenOverSize bool) *WriteCloser
• type Writer
  • func NewWriter(w io.Writer, n int, regardOverSizeNormal bool) *Writer
  • func (lw *Writer) Write(p []byte) (n int, err error)
  • func (lw *Writer) Written() int

Variables

var ErrThresholdExceeded = errors.New("stream size exceeds threshold")

ErrThresholdExceeded indicates stream size exceeds threshold

Functions

func AtMostFirstNBytes

func AtMostFirstNBytes(s []byte, n int) []byte

AtMostFirstNBytes takes at more n bytes from s

Types

type ReadCloser

type ReadCloser struct {
	*Reader
	io.Closer
}

ReadCloser works like io.LimitedReader( but with a Close() method) but may be tuned to report oversize read as a distinguishable error other than io.EOF.

User NewReadCloser() to create ReadCloser.

func NewReadCloser

func NewReadCloser(r io.ReadCloser, limit int, regardOverSizeEOF bool) *ReadCloser

NewReadCloser creates a ReadCloser that works like io.LimitedReader(but with a Close() method) and it may be tuned to report oversize read as a distinguishable error other than io.EOF.

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

Reader works like io.LimitedReader but may be tuned to report oversize read as a distinguishable error other than io.EOF.

Use NewReader() to create Reader.

func NewReader

func NewReader(r io.Reader, limit int, regardOverSizeEOF bool) *Reader

NewReader creates a Reader works like io.LimitedReader but may be tuned to report oversize read as a distinguishable error other than io.EOF.

func (*Reader) Read

func (lr *Reader) Read(p []byte) (n int, err error)

Read implements io.Reader

type WriteCloser

type WriteCloser struct {
	*Writer
	io.Closer
}

WriteCloser wraps w with writing length limit.

To create WriteCloser, use NewWriteCloser().

func NewWriteCloser

func NewWriteCloser(w io.WriteCloser, n int, silentWhenOverSize bool) *WriteCloser

NewWriteCloser create a WriteCloser that writes at most n bytes.

regardOverSizeNormal controls whether Writer.Write() returns error when writing totally more bytes than n, or do no-op to inner w, pretending writing is processed normally.

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

Writer wraps w with writing length limit.

To create Writer, use NewWriter().

func NewWriter

func NewWriter(w io.Writer, n int, regardOverSizeNormal bool) *Writer

NewWriter create a writer that writes at most n bytes.

regardOverSizeNormal controls whether Writer.Write() returns error when writing totally more bytes than n, or do no-op to inner w, pretending writing is processed normally.

func (*Writer) Write

func (lw *Writer) Write(p []byte) (n int, err error)

Writer implements io.Writer

func (*Writer) Written

func (lw *Writer) Written() int

Written returns number of bytes written