mem

package module
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 15, 2022 License: MIT Imports: 5 Imported by: 6

README

Go Reference Github CI

The mem package provides types and functions for measuring and displaying memory throughput and capacity.

Example

Walk the directory tree of the current directory and list all (non-hidden) files and the corresponding file sizes.

package main

import (
    "fmt"
    "io/fs"
    "path/filepath"
    "strings"

    "aead.dev/mem"
)

func main() {
    const Directory = "."
    filepath.WalkDir(Directory, func(path string, entry fs.DirEntry, err error) error {
        if entry.IsDir() {
            if entry.Name() != Directory && strings.HasPrefix(entry.Name(), ".") { // Skip hidden sub directories
                return filepath.SkipDir
            }
            fmt.Println(entry.Name())
            return nil
        }

        stat, err := entry.Info()
        if err == nil {
            var size mem.Size = mem.Size(stat.Size())
            fmt.Printf("  %-20s %10s\n", stat.Name(), mem.FormatSize(size, 'D', 2))
        }
        return nil
    })
}
Example Ouput 
.
  .gitignore              269.00B
  .golangci.yml           420.00B
  LICENSE                  1.08KB
  benchmark_test.go        1.47KB
  bitsize.go               2.79KB
  bitsize_test.go          3.38KB
cmd
example
  main.go                 605.00B
  doc.go                   2.91KB
  examples_test.go         1.76KB
  format.go               10.15KB
  format_test.go           4.24KB
  go.mod                   29.00B
  math.go                 831.00B
  math_test.go             2.28KB
  progress.go             842.00B
  size.go                  3.00KB
  size_test.go             3.73KB

Install

Add aead.dev/mem as dependency to your go.mod file:

go mod download aead.dev/mem@latest

Documentation

Overview

Package mem provides functionality for measuring and displaying memory throughput and capacity.

Units

Package mem defines two different types that represent quantities of data - Size and BitSize. The former represents an amount of data as bytes while the later represents an amount of data as bits.

Not each BitSize can be represented as Size and vice versa. For example 65 bit is neither 8 bytes (64 bit) nor 9 bytes (72 bit). However, Size and BitSize provide APIs for converting one to the other.

Some computer metrics, like networking speed, are usually measured in bits while others, like storage capacity, are measured in bytes. Large quantities of bits / bytes are commonly displayed using the decimal prefixes for powers of 10. However, some systems use binary prefixes (2^10 = 1024). 

BitSize (decimal)     Size (decimal)        Size (binary)
┌──────┬───────────┐  ┌──────┬───────────┐  ┌──────┬───────────┐
│  Bit │    1      │  │ Byte │    1      │  │ Byte │    1      │
│ Kbit │ 1000  Bit │  │ KB   │ 1000 Byte │  │ KiB  │ 1024 Byte │
│ Mbit │ 1000 Kbit │  │ MB   │ 1000 KB   │  │ MiB  │ 1024 KiB  │
│ Gbit │ 1000 Mbit │  │ GB   │ 1000 MB   │  │ GiB  │ 1024 MiB  │
│ Tbit │ 1000 Gbit │  │ TB   │ 1000 GB   │  │ TiB  │ 1024 GiB  │
│      │           │  │ PB   │ 1000 TB   │  │ PiB  │ 1024 TiB  │
└──────┴───────────┘  └──────┴───────────┘  └──────┴───────────┘

Formatting

Sizes can be formatted and displayed in various units and with various precisions. For example: 

a := mem.FormatSize(1*mem.MB, 'd', -1)      // "1mb"
b := mem.FormatSize(1*mem.MB, 'b', -1)      // "976.5625kib"
c := mem.FormatBitSize(1*mem.MBit, 'd', -1) // "1mbit"

Parsing

String representation of sizes can be parsed using the corresponding Parse functions. For example: 

a, err := mem.ParseSize("1mb")
b, err := mem.ParseSize("976.5625KiB")
c, err := mem.ParseBitSize("1mbit")

Buffers and Limits

Often, code has to pre-allocate a buffer of a certain size or limit the amount of data from an io.Reader. With the mem package this can be done in a human readable way. For example: 

buffer := make([]byte, 1 * mem.MB)        // Allocate a 1MB buffer
reader := io.LimitReader(r, 512 * mem.MB) // Limit the reader to 512MB

// Limit an HTTP request body to 5 MB.
req.Body = http.MaxBytesReader(w, req.Body, 5 * mem.MB)

Index

Examples

Constants

View Source 
const (
	Bit  BitSize = 1
	KBit         = 1000 * Bit
	MBit         = 1000 * KBit
	GBit         = 1000 * MBit
	TBit         = 1000 * GBit
)

Common sizes when measuring amounts of data in bits.

To count the number of units in a BitSize, divide: 

mbits := mem.MBit
fmt.Print(int64(mbits / mem.KBit)) // prints 1000

To convert an integer of units to a BitSize, multiply: 

mbits := 10
fmt.Print(mem.BitSize(mbits)*mem.MBit) // prints 10MBit
View Source 
const (
	Byte Size = 1

	KB Size = 1000 * Byte
	MB      = 1000 * KB
	GB      = 1000 * MB
	TB      = 1000 * GB
	PB      = 1000 * TB

	KiB Size = 1024 * Byte
	MiB      = 1024 * KiB
	GiB      = 1024 * MiB
	TiB      = 1024 * GiB
	PiB      = 1024 * TiB
)

Common sizes for measuring memory and disk capacity.

To count the number of units in a Size, divide: 

megabyte := mem.MB
fmt.Print(int64(megabyte / mem.KB)) // prints 1000

To convert an integer of units to a Size, multiply: 

megabytes := 10
fmt.Print(mem.Size(megabytes)*mem.MB) // prints 10MB

Variables

This section is empty.

Functions

func FormatBitSize 

func FormatBitSize(s BitSize, fmt byte, prec int) string

FormatBitSize converts the bit size s to a string, according to the format fmt and precision prec.

The format fmt specifies how to format the size s. Valid values are:

  • 'd' formats s as "-ddd.dddddmbit" using the decimal byte units.
  • 'D' formats s as "-ddd.dddddMbit" using the decimal byte units.

The precision prec controls the number of digits after the decimal point printed by the 'd' and 'D' formats. The special precision -1 uses the smallest number of digits necessary such that ParseBitSize will return s exactly.

Example 
package main

import (
	"fmt"

	"aead.dev/mem"
)

func main() {
	fmt.Println(mem.FormatBitSize(1*mem.MBit, 'D', -1))
	fmt.Println(mem.FormatBitSize(1*mem.MBit+111*mem.KBit, 'd', 2))
	fmt.Println(mem.FormatBitSize(2*mem.TBit+512*mem.MBit, 'D', 4))
}

Output:

1Mbit
1.11mbit
2.0005Tbit

func FormatSize 

func FormatSize(s Size, fmt byte, prec int) string

FormatSize converts the size s to a string, according to the format fmt and precision prec.

The format fmt specifies how to format the size s. Valid values are:

  • 'd' formats s as "-ddd.dddddmb" using the decimal byte units.
  • 'b' formats s as "-ddd.dddddmib" using the binary byte units.

In addition, 'D' and 'B' format s similar to 'd' and 'b' but with partially uppercase unit strings. In particular:

  • 'D' formats s as "-ddd.dddddMB" using the decimal byte units.
  • 'B' formats s as "-ddd.dddddMiB" using the binary byte units.

The precision prec controls the number of digits after the decimal point printed by the 'd' and 'b' formats. The special precision -1 uses the smallest number of digits necessary such that ParseSize will return s exactly.

Example 
package main

import (
	"fmt"

	"aead.dev/mem"
)

func main() {
	fmt.Println(mem.FormatSize(1*mem.MB, 'D', -1))
	fmt.Println(mem.FormatSize(1*mem.MB+111*mem.KB, 'd', 2))
	fmt.Println(mem.FormatSize(2*mem.TiB+512*mem.MiB, 'B', 4))
}

Output:

1MB
1.11mb
2.0005TiB

func LimitReader added in v0.2.0 

func LimitReader(r io.Reader, n Size) *io.LimitedReader

LimitReader returns a io.LimitedReader that reads from r but stops with io.EOF after n bytes.

Types

type BitSize 

type BitSize int64

BitSize represents an amount of data as int64 number of bits. The largest representable size is approximately 9223372 Tbit.

func ParseBitSize 

func ParseBitSize(s string) (BitSize, error)

ParseBitSize parses a bit size string. A bit size string is a possibly signed decimal number with an optional fraction and a unit suffix, such as "64Kbit" or "1mbit".

A string may be a decimal size representation. Valid units are "bit", "kbit", "mbit", "gbit" and "tbit".

Example 
package main

import (
	"fmt"
	"log"

	"aead.dev/mem"
)

func main() {
	a, err := mem.ParseBitSize("1.123Mbit")
	if err != nil {
		log.Fatalln(err)
	}
	b, err := mem.ParseBitSize("3.877Mbit")
	if err != nil {
		log.Fatalln(err)
	}
	fmt.Println(a + b)
}

Output:

5Mbit

func (BitSize) Abs 

func (b BitSize) Abs() BitSize

Abs returns the absolute value of b. As a special case, math.MinInt64 is converted to math.MaxInt64.

func (BitSize) Bytes 

func (b BitSize) Bytes() (Size, BitSize)

Bytes returns b as number of bytes and any remaining bits, if any. It guarantees that: 

bytes, bits := b.Bytes()
fmt.Print(b == bytes.Bits() + bits) // true
fmt.Print(-7 <= bits && bits <= 7) // true
Example 
package main

import (
	"fmt"

	"aead.dev/mem"
)

func main() {
	b := mem.MBit
	fmt.Println(b.Bytes())

	b = 1*mem.MBit + 4*mem.Bit
	bytes, bits := b.Bytes()

	fmt.Println(b == bytes.Bits()+bits && -7 <= bits && bits <= 7)
}

Output:

125KB 0Bit
true

func (BitSize) Gigabits 

func (b BitSize) Gigabits() float64

Gigabits returns the size as floating point number of gigabits (Gbit).

func (BitSize) Kilobits 

func (b BitSize) Kilobits() float64

Kilobits returns the size as floating point number of kilobits (Kbit).

func (BitSize) Megabits 

func (b BitSize) Megabits() float64

Megabits returns the size as floating point number of megabits (Mbit).

func (BitSize) Round 

func (b BitSize) Round(m BitSize) BitSize

Round returns the result of rounding b to the nearest multiple of m. The rounding behavior for halfway values is to round away from zero. If the result exceeds the maximum (or minimum) value that can be stored in a Size, Round returns the maximum (or minimum) size. If m <= 0, Round returns b unchanged.

func (BitSize) String 

func (b BitSize) String() string

String returns a string representing the bit size in the form "1.25Mbit". The zero size formats as 0Bit.

Example 
package main

import (
	"fmt"

	"aead.dev/mem"
)

func main() {
	fmt.Println(1 * mem.MBit)
	fmt.Println(1*mem.GBit + 500*mem.MBit)
	fmt.Println(5*mem.KBit + 880*mem.Bit)
}

Output:

1Mbit
1.5Gbit
5.88Kbit

func (BitSize) Terabits 

func (b BitSize) Terabits() float64

Terabits returns the size as floating point number of terabits (Tbit).

func (BitSize) Truncate 

func (b BitSize) Truncate(m BitSize) BitSize

Truncate returns the result of rounding b towards zero to a multiple of m. If m <= 0, Truncate returns b unchanged.

type Progress added in v0.2.0 

type Progress struct {
	// N is the number of bytes since the last progress
	// update.
	N Size

	// Total is the number of bytes since the start
	// of the operation.
	Total Size

	// Err is any error that occurred during the operation.
	// Once the operation completes, Err is io.EOF.
	Err error
}

Progress represents the progress of an I/O operation, like reading data from a file or network connection.

func (*Progress) Done added in v0.2.0 

func (p *Progress) Done() bool

Done reports whether the operation has been completed.

type ProgressReader added in v0.2.0 

type ProgressReader struct {
	R io.Reader // The underlying io.Reader

	// Update, if non-nil, is called with the current
	// progress whenever a read from R completes and
	// either the UpdateEvery period has ellapsed
	// since the last update or UpdateAfter bytes have
	// been read.
	//
	// As a special case, Update is called after every
	// read from R if UpdateEvery and UpdateAfter
	// are both <= 0.
	//
	// The passed progress contains the number of bytes
	// read since the last Update call, the number of
	// bytes read in total so far and any error that
	// has occurred while reading from R.
	// Once reading from R returns an non-nil error,
	// including io.EOF, Update is called immediately
	// one more time and then never again.
	//
	// Update is called by the goroutine reading from
	// R. A long-running or blocking Update function
	// defers or blocks reads, and therefore, impacts
	// read performance.
	// In such cases, sending the progress to another
	// concurrently running goroutine via a channel
	// may be viable solution.
	Update func(Progress)

	// UpdateEvery is the duration that has to ellapse
	// between two Update calls.
	//
	// If UpdateEvery <= 0, Update may be called after
	// every read.
	UpdateEvery time.Duration

	// UpdateAfter is the number of bytes that have to
	// be read from R before Update is called again.
	//
	// If UpdateAfter <= 0, Update may be called after
	// every read.
	UpdateAfter Size
	// contains filtered or unexported fields
}

ProgressReader wraps an io.Reader and calls Update with the current status when reading makes progress.

Example 
package main

import (
	"bytes"
	"fmt"
	"io"
	"log"
	"time"

	"aead.dev/mem"
)

func main() {
	r := bytes.NewReader(make([]byte, 1*mem.MB))
	p := mem.NewProgressReader(r, 500*time.Millisecond, func(p mem.Progress) {
		fmt.Printf("Copied %s/%s\n", p.Total, mem.Size(r.Size()))
		if p.Done() {
			fmt.Println("Done")
		}
	})
	if _, err := io.Copy(io.Discard, p); err != nil {
		log.Fatal(err)
	}
}

Output:

Copied 8.192KB/1MB
Copied 1MB/1MB
Done
Example (Concurrent) 
package main

import (
	"bytes"
	"fmt"
	"io"
	"log"
	"sync"
	"time"

	"aead.dev/mem"
)

func main() {
	r := bytes.NewReader(make([]byte, 1*mem.MB))
	progress := make(chan mem.Progress, 1)

	var wg sync.WaitGroup
	wg.Add(1)
	go func() {
		defer wg.Done()

		for p := range progress {
			fmt.Printf("Copied %s/%s\n", p.Total, mem.Size(r.Size()))
			if p.Done() {
				fmt.Println("Done")
				break
			}
		}
	}()

	p := mem.NewProgressReader(r, 500*time.Millisecond, func(p mem.Progress) {
		// Sending the progress to a channel blocks reads if the
		// channel is full. A select with a default cause reads
		// won't block but progress updates will get dropped when
		// the channel is full.
		progress <- p
	})
	if _, err := io.Copy(io.Discard, p); err != nil {
		log.Fatal(err)
	}

	close(progress)
	wg.Wait() // Wait until all progress updates got printed
}

Output:

Copied 8.192KB/1MB
Copied 1MB/1MB
Done

func NewProgressReader added in v0.2.0 

func NewProgressReader(r io.Reader, d time.Duration, update func(Progress)) *ProgressReader

NewProgressReader returns a new ProgressReader that wraps r and calls update periodically with the current progress while reading.

func (*ProgressReader) Progress added in v0.2.0 

func (r *ProgressReader) Progress() Progress

Progress returns the current progress.

It contains the number of bytes read since the last invocation of Update by Read, the total number of bytes read so far and any error that has occurred while reading from R.

func (*ProgressReader) Read added in v0.2.0 

func (r *ProgressReader) Read(p []byte) (int, error)

type Size 

type Size int64

Size represents an amount of data as int64 number of bytes. The largest representable size is approximately 8192 PiB.

func ParseSize 

func ParseSize(s string) (Size, error)

ParseSize parses a size string. A size string is a possibly signed decimal number with an optional fraction and a unit suffix, such as "64KB" or "1MiB".

A string may be a decimal or binary size representation. Valid units are:

  • decimal: "b", "kb", "mb", "gb", "tb", "pb"
  • binary: "b", "kib", "mib", "gib", "tib", "pib"
Example 
package main

import (
	"fmt"
	"log"

	"aead.dev/mem"
)

func main() {
	a, err := mem.ParseSize("1.123MB")
	if err != nil {
		log.Fatalln(err)
	}
	b, err := mem.ParseSize("3.877MB")
	if err != nil {
		log.Fatalln(err)
	}
	fmt.Println(a + b)
}

Output:

5MB

func (Size) Abs 

func (s Size) Abs() Size

Abs returns the absolute value of s. As a special case, math.MinInt64 is converted to math.MaxInt64.

func (Size) Bits 

func (s Size) Bits() BitSize

Bits returns s as number of bits. As special cases, if s would be greater resp. smaller than the max. resp. min. representable BitSize it returns math.MaxInt64 resp. math.MinInt64.

func (Size) Gibibytes 

func (s Size) Gibibytes() float64

Gibibytes returns the size as floating point number of gibibytes (GiB).

func (Size) Gigabytes 

func (s Size) Gigabytes() float64

Gigabytes returns the size as floating point number of gigabytes (GB).

func (Size) Kibibytes 

func (s Size) Kibibytes() float64

Kibibytes returns the size as floating point number of kibibytes (KiB).

func (Size) Kilobytes 

func (s Size) Kilobytes() float64

Kilobytes returns the size as floating point number of kilobytes (KB).

func (Size) Mebibytes 

func (s Size) Mebibytes() float64

Mebibytes returns the size as floating point number of mebibytes (MiB).

func (Size) Megabytes 

func (s Size) Megabytes() float64

Megabytes returns the size as floating point number of megabytes (MB).

func (Size) Pebibytes 

func (s Size) Pebibytes() float64

Pebibytes returns the size as floating point number of pebibytes (PiB).

func (Size) Petabytes 

func (s Size) Petabytes() float64

Petabytes returns the size as floating point number of petabytes (PB).

func (Size) Round 

func (s Size) Round(m Size) Size

Round returns the result of rounding s to the nearest multiple of m. The rounding behavior for halfway values is to round away from zero. If the result exceeds the maximum (or minimum) value that can be stored in a Size, Round returns the maximum (or minimum) size. If m <= 0, Round returns s unchanged.

func (Size) String 

func (s Size) String() string

String returns a string representing the size in the form "1.25MB". The zero size formats as 0B.

Example 
package main

import (
	"fmt"

	"aead.dev/mem"
)

func main() {
	fmt.Println(1 * mem.MB)
	fmt.Println(1*mem.GB + 500*mem.MB)
	fmt.Println(5*mem.KiB + 880*mem.Byte)
}

Output:

1MB
1.5GB
6KB

func (Size) Tebibytes 

func (s Size) Tebibytes() float64

Tebibytes returns the size as floating point number of tebibytes (TiB).

func (Size) Terabytes 

func (s Size) Terabytes() float64

Terabytes returns the size as floating point number of terabytes (TB).

func (Size) Truncate 

func (s Size) Truncate(m Size) Size

Truncate returns the result of rounding s towards zero to a multiple of m. If m <= 0, Truncate returns s unchanged.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.