reedsolomon

package
v2.0.10 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 11, 2025 License: MIT, MIT Imports: 15 Imported by: 0

README

Reed-Solomon

Go Reference Go

Reed-Solomon Erasure Coding in Go, with speeds exceeding 1GB/s/cpu core implemented in pure Go.

This is a Go port of the JavaReedSolomon library released by Backblaze, with some additional optimizations.

For an introduction on erasure coding, see the post on the Backblaze blog.

For encoding high shard counts (>256) a Leopard implementation is used. For most platforms this performs close to the original Leopard implementation in terms of speed.

Package home: https://github.com/klauspost/reedsolomon

Godoc: https://pkg.go.dev/github.com/klauspost/reedsolomon

Installation

To get the package use the standard:

go get -u github.com/klauspost/reedsolomon

Using Go modules is recommended.

Changes

2024

  • Auto-generation of SVE and NEON routines for ARM based on AVX2 code. This results in a speedup of 2x for SVE (as measured using Graviton 3 on AWS) and a speedup of 1.5x as compared to the existing NEON-accelerated code.

2022

2021

  • Use GOAMD64=v4 to enable faster AVX2.
  • Add progressive shard encoding.
  • Wider AVX2 loops
  • Limit concurrency on AVX2, since we are likely memory bound.
  • Allow 0 parity shards.
  • Allow disabling inversion cache.
  • Faster AVX2 encoding.
See older changes

May 2020

  • ARM64 optimizations, up to 2.5x faster.
  • Added WithFastOneParityMatrix for faster operation with 1 parity shard.
  • Much better performance when using a limited number of goroutines.
  • AVX512 is now using multiple cores.
  • Stream processing overhaul, big speedups in most cases.
  • AVX512 optimizations

March 6, 2019

The pure Go implementation is about 30% faster. Minor tweaks to assembler implementations.

February 8, 2019

AVX512 accelerated version added for Intel Skylake CPUs. This can give up to a 4x speed improvement as compared to AVX2. See here for more details.

December 18, 2018

Assembly code for ppc64le has been contributed, this boosts performance by about 10x on this platform.

November 18, 2017

Added WithAutoGoroutines which will attempt to calculate the optimal number of goroutines to use based on your expected shard size and detected CPU.

October 1, 2017

  • Cauchy Matrix is now an option. Thanks to templexxx for the basis of this.

  • Default maximum number of goroutines has been increased for better multi-core scaling.

  • After several requests the Reconstruct and ReconstructData now slices of zero length but sufficient capacity to be used instead of allocating new memory.

August 26, 2017

  • The Encoder() now contains an Update function contributed by chenzhongtao.

  • Frank Wessels kindly contributed ARM 64 bit assembly, which gives a huge performance boost on this platform.

July 20, 2017

ReconstructData added to Encoder interface. This can cause compatibility issues if you implement your own Encoder. A simple workaround can be added:

func (e *YourEnc) ReconstructData(shards [][]byte) error {
	return ReconstructData(shards)
}

You can of course also do your own implementation. The StreamEncoder handles this without modifying the interface. This is a good lesson on why returning interfaces is not a good design.

Usage

This section assumes you know the basics of Reed-Solomon encoding. A good start is this Backblaze blog post.

This package performs the calculation of the parity sets. The usage is therefore relatively simple.

First of all, you need to choose your distribution of data and parity shards. A 'good' distribution is very subjective, and will depend a lot on your usage scenario.

To create an encoder with 10 data shards (where your data goes) and 3 parity shards (calculated):

    enc, err := reedsolomon.New(10, 3)

This encoder will work for all parity sets with this distribution of data and parity shards.

If you will primarily be using it with one shard size it is recommended to use WithAutoGoroutines(shardSize) as an additional parameter. This will attempt to calculate the optimal number of goroutines to use for the best speed. It is not required that all shards are this size.

Then you send and receive data that is a simple slice of byte slices; [][]byte. In the example above, the top slice must have a length of 13.

    data := make([][]byte, 13)

You should then fill the 10 first slices with equally sized data, and create parity shards that will be populated with parity data. In this case we create the data in memory, but you could for instance also use mmap to map files.

    // Create all shards, size them at 50000 each
    for i := range input {
      data[i] := make([]byte, 50000)
    }
    
    // The above allocations can also be done by the encoder:
    // data := enc.(reedsolomon.Extended).AllocAligned(50000)
    
    // Fill some data into the data shards
    for i, in := range data[:10] {
      for j:= range in {
         in[j] = byte((i+j)&0xff)
      }
    }

To populate the parity shards, you simply call Encode() with your data.

    err = enc.Encode(data)

The only cases where you should get an error is, if the data shards aren't of equal size. The last 3 shards now contain parity data. You can verify this by calling Verify():

    ok, err = enc.Verify(data)

The final (and important) part is to be able to reconstruct missing shards. For this to work, you need to know which parts of your data is missing. The encoder does not know which parts are invalid, so if data corruption is a likely scenario, you need to implement a hash check for each shard.

If a byte has changed in your set, and you don't know which it is, there is no way to reconstruct the data set.

To indicate missing data, you set the shard to nil before calling Reconstruct():

    // Delete two data shards
    data[3] = nil
    data[7] = nil
    
    // Reconstruct the missing shards
    err := enc.Reconstruct(data)

The missing data and parity shards will be recreated. If more than 3 shards are missing, the reconstruction will fail.

If you are only interested in the data shards (for reading purposes) you can call ReconstructData():

    // Delete two data shards
    data[3] = nil
    data[7] = nil
    
    // Reconstruct just the missing data shards
    err := enc.ReconstructData(data)

If you don't need all data shards you can use ReconstructSome():

    // Delete two data shards
    data[3] = nil
    data[7] = nil
    
    // Reconstruct just the shard 3
    err := enc.ReconstructSome(data, []bool{false, false, false, true, false, false, false, false})

So to sum up reconstruction:

  • The number of data/parity shards must match the numbers used for encoding.
  • The order of shards must be the same as used when encoding.
  • You may only supply data you know is valid.
  • Invalid shards should be set to nil.

For complete examples of an encoder and decoder see the examples folder.

Splitting/Joining Data

You might have a large slice of data. To help you split this, there are some helper functions that can split and join a single byte slice.

   bigfile, _ := ioutil.Readfile("myfile.data")
   
   // Split the file
   split, err := enc.Split(bigfile)

This will split the file into the number of data shards set when creating the encoder and create empty parity shards.

An important thing to note is that you have to keep track of the exact input size. If the size of the input isn't divisible by the number of data shards, extra zeros will be inserted in the last shard.

To join a data set, use the Join() function, which will join the shards and write it to the io.Writer you supply:

   // Join a data set and write it to io.Discard.
   err = enc.Join(io.Discard, data, len(bigfile))

Aligned Allocations

For AMD64 aligned inputs can make a big speed difference.

This is an example of the speed difference when inputs are unaligned/aligned:

BenchmarkEncode100x20x10000-32    	    7058	    172648 ns/op	6950.57 MB/s
BenchmarkEncode100x20x10000-32    	    8406	    137911 ns/op	8701.24 MB/s

This is mostly the case when dealing with odd-sized shards.

To facilitate this the package provides an AllocAligned(shards, each int) [][]byte. This will allocate a number of shards, each with the size each. Each shard will then be aligned to a 64 byte boundary.

Each encoder also has a AllocAligned(each int) [][]byte as an extended interface which will return the same, but with the shard count configured in the encoder.

It is not possible to re-aligned already allocated slices, for example when using Split. When it is not possible to write to aligned shards, you should not copy to them.

Progressive encoding

It is possible to encode individual shards using EncodeIdx:

	// EncodeIdx will add parity for a single data shard.
	// Parity shards should start out as 0. The caller must zero them.
	// Data shards must be delivered exactly once. There is no check for this.
	// The parity shards will always be updated and the data shards will remain the same.
	EncodeIdx(dataShard []byte, idx int, parity [][]byte) error

This allows progressively encoding the parity by sending individual data shards. There is no requirement on shards being delivered in order, but when sent in order it allows encoding shards one at the time, effectively allowing the operation to be streaming.

The result will be the same as encoding all shards at once. There is a minor speed penalty using this method, so send shards at once if they are available.

Example

func test() {
    // Create an encoder with 7 data and 3 parity slices.
    enc, _ := reedsolomon.New(7, 3)

    // This will be our output parity.
    parity := make([][]byte, 3)
    for i := range parity {
        parity[i] = make([]byte, 10000)
    }

    for i := 0; i < 7; i++ {
        // Send data shards one at the time.
        _ = enc.EncodeIdx(make([]byte, 10000), i, parity)
    }

    // parity now contains parity, as if all data was sent in one call.
}

Streaming/Merging

It might seem like a limitation that all data should be in memory, but an important property is that as long as the number of data/parity shards are the same, you can merge/split data sets, and they will remain valid as a separate set.

    // Split the data set of 50000 elements into two of 25000
    splitA := make([][]byte, 13)
    splitB := make([][]byte, 13)
    
    // Merge into a 100000 element set
    merged := make([][]byte, 13)
    
    for i := range data {
      splitA[i] = data[i][:25000]
      splitB[i] = data[i][25000:]
      
      // Concatenate it to itself
	  merged[i] = append(make([]byte, 0, len(data[i])*2), data[i]...)
	  merged[i] = append(merged[i], data[i]...)
    }
    
    // Each part should still verify as ok.
    ok, err := enc.Verify(splitA)
    if ok && err == nil {
        log.Println("splitA ok")
    }
    
    ok, err = enc.Verify(splitB)
    if ok && err == nil {
        log.Println("splitB ok")
    }
    
    ok, err = enc.Verify(merge)
    if ok && err == nil {
        log.Println("merge ok")
    }

This means that if you have a data set that may not fit into memory, you can split processing into smaller blocks. For the best throughput, don't use too small blocks.

This also means that you can divide big input up into smaller blocks, and do reconstruction on parts of your data. This doesn't give the same flexibility of a higher number of data shards, but it will be much more performant.

Streaming API

There has been added support for a streaming API, to help perform fully streaming operations, which enables you to do the same operations, but on streams. To use the stream API, use NewStream function to create the encoding/decoding interfaces.

You can use WithConcurrentStreams to ready an interface that reads/writes concurrently from the streams.

You can specify the size of each operation using WithStreamBlockSize. This will set the size of each read/write operation.

Input is delivered as []io.Reader, output as []io.Writer, and functionality corresponds to the in-memory API. Each stream must supply the same amount of data, similar to how each slice must be similar size with the in-memory API. If an error occurs in relation to a stream, a StreamReadError or StreamWriteError will help you determine which stream was the offender.

There is no buffering or timeouts/retry specified. If you want to add that, you need to add it to the Reader/Writer.

For complete examples of a streaming encoder and decoder see the examples folder.

GF16 (more than 256 shards) is not supported by the streaming interface.

Advanced Options

You can modify internal options which affects how jobs are split between and processed by goroutines.

To create options, use the WithXXX functions. You can supply options to New, NewStream. If no Options are supplied, default options are used.

Example of how to supply options:

    enc, err := reedsolomon.New(10, 3, WithMaxGoroutines(25))

Leopard Compatible GF16

When you encode more than 256 shards the library will switch to a Leopard-RS implementation.

This allows encoding up to 65536 shards (data+parity) with the following limitations, similar to leopard:

  • The original and recovery data must not exceed 65536 pieces.
  • The shard size must each be a multiple of 64 bytes.
  • Each buffer should have the same number of bytes.
  • Even the last shard must be rounded up to the block size.
Regular Leopard
Encode ✓ ✓
EncodeIdx ✓ -
Verify ✓ ✓
Reconstruct ✓ ✓
ReconstructData ✓ ✓
ReconstructSome ✓ ✓ (+)
Update ✓ -
Split ✓ ✓
Join ✓ ✓
  • (+) Same as calling ReconstructData.

The Split/Join functions will help to split an input to the proper sizes.

Speed can be expected to be O(N*log(N)), compared to the O(N*N). Reconstruction matrix calculation is more time-consuming, so be sure to include that as part of any benchmark you run.

For now SSSE3, AVX2 and AVX512 assembly are available on AMD64 platforms.

Leopard mode currently always runs as a single goroutine, since multiple goroutines doesn't provide any worthwhile speedup.

Leopard GF8

It is possible to replace the default reed-solomon encoder with a leopard compatible one. This will typically be faster when dealing with more than 20-30 shards. Note that the limitations listed above also applies to this mode. See table below for speed with different number of shards.

To enable Leopard GF8 mode use WithLeopardGF(true).

Benchmark Encoding and Reconstructing 1KB shards with variable number of shards. All implementation use inversion cache when available. Speed is total shard size for each operation. Data shard throughput is speed/2. AVX2 is used.

Encoder Shards Encode Recover All Recover One
Cauchy 4+4 23076.83 MB/s 5444.02 MB/s 10834.67 MB/s
Cauchy 8+8 15206.87 MB/s 4223.42 MB/s 16181.62 MB/s
Cauchy 16+16 7427.47 MB/s 3305.84 MB/s 22480.41 MB/s
Cauchy 32+32 3785.64 MB/s 2300.07 MB/s 26181.31 MB/s
Cauchy 64+64 1911.93 MB/s 1368.51 MB/s 27992.93 MB/s
Cauchy 128+128 963.83 MB/s 1327.56 MB/s 32866.86 MB/s
Leopard GF8 4+4 17061.28 MB/s 3099.06 MB/s 4096.78 MB/s
Leopard GF8 8+8 10546.67 MB/s 2925.92 MB/s 3964.00 MB/s
Leopard GF8 16+16 10961.37 MB/s 2328.40 MB/s 3110.22 MB/s
Leopard GF8 32+32 7111.47 MB/s 2374.61 MB/s 3220.75 MB/s
Leopard GF8 64+64 7468.57 MB/s 2055.41 MB/s 3061.81 MB/s
Leopard GF8 128+128 5479.99 MB/s 1953.21 MB/s 2815.15 MB/s
Leopard GF16 256+256 6158.66 MB/s 454.14 MB/s 506.70 MB/s
Leopard GF16 512+512 4418.58 MB/s 685.75 MB/s 801.63 MB/s
Leopard GF16 1024+1024 4778.05 MB/s 814.51 MB/s 1080.19 MB/s
Leopard GF16 2048+2048 3417.05 MB/s 911.64 MB/s 1179.48 MB/s
Leopard GF16 4096+4096 3209.41 MB/s 729.13 MB/s 1135.06 MB/s
Leopard GF16 8192+8192 2034.11 MB/s 604.52 MB/s 842.13 MB/s
Leopard GF16 16384+16384 1525.88 MB/s 486.74 MB/s 750.01 MB/s
Leopard GF16 32768+32768 1138.67 MB/s 482.81 MB/s 712.73 MB/s

"Traditional" encoding is faster until somewhere between 16 and 32 shards. Leopard provides fast encoding in all cases, but shows a significant overhead for reconstruction.

Calculating the reconstruction matrix takes a significant amount of computation. With bigger shards that will be smaller. Arguably, fewer shards typically also means bigger shards. Due to the high shard count caching reconstruction matrices generally isn't feasible for Leopard.

Performance

Performance depends mainly on the number of parity shards. In rough terms, doubling the number of parity shards will double the encoding time.

Here are the throughput numbers with some different selections of data and parity shards. For reference each shard is 1MB random data, and 16 CPU cores are used for encoding.

Data Parity Go MB/s SSSE3 MB/s AVX2 MB/s
5 2 20,772 66,355 108,755
8 8 6,815 38,338 70,516
10 4 9,245 48,237 93,875
50 20 2,063 12,130 22,828

The throughput numbers here is the size of the encoded data and parity shards.

If runtime.GOMAXPROCS() is set to a value higher than 1, the encoder will use multiple goroutines to perform the calculations in Verify, Encode and Reconstruct.

Benchmarking Reconstruct() followed by a Verify() (=all) versus just calling ReconstructData() (=data) gives the following result:

benchmark                            all MB/s     data MB/s    speedup
BenchmarkReconstruct10x2x10000-8     2011.67      10530.10     5.23x
BenchmarkReconstruct50x5x50000-8     4585.41      14301.60     3.12x
BenchmarkReconstruct10x2x1M-8        8081.15      28216.41     3.49x
BenchmarkReconstruct5x2x1M-8         5780.07      28015.37     4.85x
BenchmarkReconstruct10x4x1M-8        4352.56      14367.61     3.30x
BenchmarkReconstruct50x20x1M-8       1364.35      4189.79      3.07x
BenchmarkReconstruct10x4x16M-8       1484.35      5779.53      3.89x

The package will use GFNI instructions combined with AVX512 when these are available. This further improves speed by up to 3x over AVX2 code paths.

ARM64 NEON

By exploiting NEON instructions the performance for ARM has been accelerated. Below are the performance numbers for a single core on an EC2 m6g.16xlarge (Graviton2) instance (Amazon Linux 2):

BenchmarkGalois128K-64        119562     10028 ns/op        13070.78 MB/s
BenchmarkGalois1M-64           14380     83424 ns/op        12569.22 MB/s
BenchmarkGaloisXor128K-64      96508     12432 ns/op        10543.29 MB/s
BenchmarkGaloisXor1M-64        10000    100322 ns/op        10452.13 MB/s

Performance on ppc64le

The performance for ppc64le has been accelerated. This gives roughly a 10x performance improvement on this architecture as can be seen below:

benchmark                      old MB/s     new MB/s     speedup
BenchmarkGalois128K-160        948.87       8878.85      9.36x
BenchmarkGalois1M-160          968.85       9041.92      9.33x
BenchmarkGaloisXor128K-160     862.02       7905.00      9.17x
BenchmarkGaloisXor1M-160       784.60       6296.65      8.03x

None of section below is legal advice. Seek your own legal counsel. As stated by the LICENSE the authors will not be held reliable for any use of this library. Users are encouraged to independently verify they comply with all legal requirements.

As can be seen in recent news there has been lawsuits related to possible patents of aspects of erasure coding functionality.

As a possible mitigation it is possible to use the tag nopshufb when compiling any code which includes this package. This will remove all inclusion and use of PSHUFB and equivalent on other platforms.

This is done by adding -tags=nopshufb to go build and similar commands that produce binary output.

The removed code may not be infringing and even after -tags=nopshufb there may still be infringing code left.

License

This code, as the original JavaReedSolomon is published under an MIT license. See LICENSE file for more information.

Documentation

Overview

Package reedsolomon 提供Go语言的纠删码功能

使用方法和示例请参考 https://github.com/klauspost/reedsolomon

Package reedsolomon 实现了Reed-Solomon编码算法

Index

Examples

Constants

This section is empty.

Variables

View Source 
var ErrInvShardNum = errors.New("cannot create Encoder with less than one data shard or less than zero parity shards")

ErrInvShardNum 在以下情况下由 New() 函数返回: - 尝试创建数据分片数小于1的编码器 - 尝试创建奇偶校验分片数小于0的编码器 说明: - 数据分片数必须大于等于1,因为至少需要1个数据分片来存储原始数据 - 奇偶校验分片数必须大于等于0,表示可以不使用纠错功能

View Source 
var ErrInvalidInput = errors.New("invalid input")

ErrInvalidInput 当Update函数的输入参数无效时返回此错误

View Source 
var ErrInvalidShardSize = errors.New("分片大小无效")

ErrInvalidShardSize 在分片长度不满足要求时返回, 通常要求分片长度是N的倍数

View Source 
var ErrMaxShardNum = errors.New("cannot create Encoder with more than 256 data+parity shards")

ErrMaxShardNum 在以下情况下由 New() 函数返回: - 尝试创建数据分片数+奇偶校验分片数超过256的编码器 说明: - 由于使用GF(2^8)有限域,分片总数不能超过2^8=256 - 这是Reed-Solomon编码的数学特性决定的限制

View Source 
var ErrNotSupported = errors.New("operation not supported")

ErrNotSupported 在操作不被支持时返回 说明: - 当尝试执行编码器不支持的操作时返回此错误 - 例如在不支持的平台上使用特定的SIMD优化

View Source 
var ErrReconstructMismatch = errors.New("valid shards and fill shards are mutually exclusive")

ErrReconstructMismatch 当在同一个索引位置同时提供了"valid"和"fill"流时返回此错误 这种情况下无法判断是将该分片视为有效还是需要重建

View Source 
var ErrReconstructRequired = errors.New("需要重建,因为一个或多个必需的数据分片为空")

ErrReconstructRequired 在数据分片不完整需要重建时返回 当一个或多个必需的数据分片为空时,需要先进行重建才能成功合并分片

View Source 
var ErrShardNoData = errors.New("没有分片数据")

ErrShardNoData 在以下情况下返回: - 没有分片数据 - 所有分片的长度都为0

View Source 
var ErrShardSize = errors.New("分片大小不一致")

ErrShardSize 在分片长度不一致时返回

View Source 
var ErrShortData = errors.New("数据不足以填充请求的分片数量")

ErrShortData 当数据不足以填充所需分片数量时,Split()函数将返回此错误

View Source 
var ErrTooFewShards = errors.New("too few shards given")

ErrTooFewShards 在以下情况下返回: - 传入Encode/Verify/Reconstruct/Update的分片数量不足 - 在Reconstruct中,可用分片数量不足以重建丢失的数据

Functions

func AllocAligned 

func AllocAligned(shards, each int) [][]byte

AllocAligned allocates 'shards' slices, with 'each' bytes. Each slice will start on a 64 byte aligned boundary.

func StreamDecodeFile 

func StreamDecodeFile(originalFilePath string, dataShards, parityShards int) (string, error)

StreamDecodeFile 从分片文件中解码并恢复原始文件 参数:

  • originalFilePath: 原始文件路径
  • dataShards: 数据分片数量
  • parityShards: 校验分片数量

返回值:

  • string: 解码后的文件路径
  • error: 错误信息

func StreamEncodeWithFiles 

func StreamEncodeWithFiles(r io.Reader, dataShards, parityShards int, size int64) (dataFiles, parityFiles []*os.File, err error)

StreamEncodeWithFiles 对输入流进行Reed-Solomon编码，并将结果写入临时文件 参数:

  • r: 输入数据的读取器
  • dataShards: 数据分片的数量
  • parityShards: 奇偶校验分片的数量
  • size: 输入数据的总大小

返回值:

  • dataFiles: 数据分片的临时文件切片
  • parityFiles: 奇偶校验分片的临时文件切片
  • err: 编码过程中的错误，如果成功则为nil

Types

type Encoder 

type Encoder interface {
	// Encode 为一组数据分片生成奇偶校验
	// 参数:
	//   - shards: 包含数据分片和奇偶校验分片的切片数组
	// 返回值:
	//   - error: 编码过程中的错误信息
	// 说明:
	// - 分片数量必须与New()函数指定的数量匹配
	// - 每个分片都是字节数组,且大小必须相同
	// - 奇偶校验分片会被覆盖,数据分片保持不变
	// - 在编码过程中可以安全地读取数据分片
	Encode(shards [][]byte) error

	// EncodeIdx 为单个数据分片添加奇偶校验
	// 参数:
	//   - dataShard: 数据分片
	//   - idx: 分片索引
	//   - parity: 奇偶校验分片数组
	// 返回值:
	//   - error: 编码过程中的错误信息
	// 说明:
	// - 奇偶校验分片初始值应为0,调用者必须将其置零
	// - 数据分片必须只传递一次,不会对此进行检查
	// - 奇偶校验分片会被更新,数据分片保持不变
	EncodeIdx(dataShard []byte, idx int, parity [][]byte) error

	// Verify 验证奇偶校验分片是否包含正确的数据
	// 参数:
	//   - shards: 包含数据分片和奇偶校验分片的切片数组
	// 返回值:
	//   - bool: 验证是否通过
	//   - error: 验证过程中的错误信息
	// 说明:
	// - 数据格式与Encode相同
	// - 不会修改任何数据,可以在验证过程中安全读取
	Verify(shards [][]byte) (bool, error)

	// Reconstruct 尝试重建丢失的分片
	// 参数:
	//   - shards: 包含部分数据的分片数组
	// 返回值:
	//   - error: 重建过程中的错误信息
	// 说明:
	// - 数组长度必须等于分片总数
	// - 通过将分片设置为nil或零长度来表示丢失的分片
	// - 如果分片容量足够,将使用现有内存,否则分配新内存
	// - 如果可用分片太少,将返回ErrTooFewShards错误
	// - 重建后的分片集合是完整的,但未验证完整性
	Reconstruct(shards [][]byte) error

	// ReconstructData 仅重建丢失的数据分片
	// 参数:
	//   - shards: 包含部分数据的分片数组
	// 返回值:
	//   - error: 重建过程中的错误信息
	// 说明:
	// - 数组长度必须等于分片数
	// - 通过将分片设置为nil或零长度来表示丢失的分片
	// - 如果分片容量足够,将使用现有内存,否则分配新内存
	// - 如果可用分片太少,将返回ErrTooFewShards错误
	// - 由于重建后可能缺少奇偶校验分片,验证可能会失败
	ReconstructData(shards [][]byte) error

	// ReconstructSome 仅重建指定的分片
	// 参数:
	//   - shards: 包含部分数据的分片数组
	//   - required: 指示需要重建的分片的布尔数组
	// 返回值:
	//   - error: 重建过程中的错误信息
	// 说明:
	// - required数组长度必须等于分片总数或数据分片数
	// - 如果长度等于数据分片数,将忽略奇偶校验分片的重建
	// - shards数组长度必须等于分片总数
	ReconstructSome(shards [][]byte, required []bool) error

	// Update 用于更新部分数据分片并重新计算奇偶校验
	// 参数:
	//   - shards: 包含旧数据分片和旧奇偶校验分片的数组
	//   - newDatashards: 更改后的数据分片数组
	// 返回值:
	//   - error: 更新过程中的错误信息
	// 说明:
	// - 新的奇偶校验分片将存储在shards[DataShards:]中
	// - 当数据分片远多于奇偶校验分片且变更较少时,此方法比Encode更快
	Update(shards [][]byte, newDatashards [][]byte) error

	// Split 将数据切片分割成指定数量的分片
	// 参数:
	//   - data: 要分割的数据
	// 返回值:
	//   - [][]byte: 分割后的分片数组
	//   - error: 分割过程中的错误信息
	// 说明:
	// - 数据将被均匀分割
	// - 如果数据大小不能被分片数整除,最后一个分片将补零
	// - 如果提供的数据切片有额外容量,将用于分配奇偶校验分片
	Split(data []byte) ([][]byte, error)

	// Join 将分片合并并写入目标
	// 参数:
	//   - dst: 写入目标
	//   - shards: 分片数组
	//   - outSize: 期望的输出大小
	// 返回值:
	//   - error: 合并过程中的错误信息
	// 说明:
	// - 仅考虑数据分片
	// - 必须提供准确的输出大小
	// - 如果分片数量不足,将返回ErrTooFewShards错误
	// - 如果总数据大小小于outSize,将返回ErrShortData错误
	Join(dst io.Writer, shards [][]byte, outSize int) error
}

Encoder 是用于对数据进行Reed-Solomon奇偶校验编码的接口

Example

ExampleEncoder 演示了如何使用编码器的所有功能 注意：为了保持简洁，所有错误检查都已被移除 

package main

import (
	"fmt"
	"math/rand"

	"github.com/bpfs/defs/v2/reedsolomon"
)

// fillRandom 用随机数据填充字节切片
// 参数:
//   - p: 要填充的字节切片
func fillRandom(p []byte) {
	for i := 0; i < len(p); i += 7 {
		val := rand.Int63()
		for j := 0; i+j < len(p) && j < 7; j++ {
			p[i+j] = byte(val)
			val >>= 8
		}
	}
}

func main() {
	// 创建一些示例数据
	var data = make([]byte, 250000)
	fillRandom(data)

	// 创建一个具有17个数据分片和3个奇偶校验分片的编码器
	enc, _ := reedsolomon.New(17, 3)

	// 将数据分割成分片
	shards, _ := enc.Split(data)

	// 编码奇偶校验集
	_ = enc.Encode(shards)

	// 验证奇偶校验集
	ok, _ := enc.Verify(shards)
	if ok {
		fmt.Println("ok")
	}

	// 删除两个分片
	shards[10], shards[11] = nil, nil

	// 重建分片
	_ = enc.Reconstruct(shards)

	// 验证数据集
	ok, _ = enc.Verify(shards)
	if ok {
		fmt.Println("ok")
	}
}

Output:

ok
ok
Example (Slicing)

ExampleEncoder_slicing 演示了分片可以被任意切片和合并，并且仍然保持有效 

package main

import (
	"fmt"
	"math/rand"

	"github.com/bpfs/defs/v2/reedsolomon"
)

// fillRandom 用随机数据填充字节切片
// 参数:
//   - p: 要填充的字节切片
func fillRandom(p []byte) {
	for i := 0; i < len(p); i += 7 {
		val := rand.Int63()
		for j := 0; i+j < len(p) && j < 7; j++ {
			p[i+j] = byte(val)
			val >>= 8
		}
	}
}

func main() {
	// 创建一些示例数据
	var data = make([]byte, 250000)
	fillRandom(data)

	// 创建5个各包含50000个元素的数据分片
	enc, _ := reedsolomon.New(5, 3)
	shards, _ := enc.Split(data)
	err := enc.Encode(shards)
	if err != nil {
		panic(err)
	}

	// 检查是否验证通过
	ok, err := enc.Verify(shards)
	if ok && err == nil {
		fmt.Println("encode ok")
	}

	// 将50000个元素的数据集分割成两个25000个元素的集合
	splitA := make([][]byte, 8)
	splitB := make([][]byte, 8)

	// 合并成一个100000个元素的集合
	merged := make([][]byte, 8)

	// 分割/合并分片
	for i := range shards {
		splitA[i] = shards[i][:25000]
		splitB[i] = shards[i][25000:]

		// 将其与自身连接
		merged[i] = append(make([]byte, 0, len(shards[i])*2), shards[i]...)
		merged[i] = append(merged[i], shards[i]...)
	}

	// 每个部分应该仍然验证为ok
	ok, err = enc.Verify(shards)
	if ok && err == nil {
		fmt.Println("splitA ok")
	}

	ok, err = enc.Verify(splitB)
	if ok && err == nil {
		fmt.Println("splitB ok")
	}

	ok, err = enc.Verify(merged)
	if ok && err == nil {
		fmt.Println("merge ok")
	}
}

Output:

encode ok
splitA ok
splitB ok
merge ok
Example (Xor)

ExampleEncoder_xor 演示了分片可以进行异或操作并且仍然保持有效集合

每个分片中的第'n'个元素的异或值必须相同， 除非你与类似大小的编码分片集进行异或 

package main

import (
	"fmt"
	"math/rand"

	"github.com/bpfs/defs/v2/reedsolomon"
)

// fillRandom 用随机数据填充字节切片
// 参数:
//   - p: 要填充的字节切片
func fillRandom(p []byte) {
	for i := 0; i < len(p); i += 7 {
		val := rand.Int63()
		for j := 0; i+j < len(p) && j < 7; j++ {
			p[i+j] = byte(val)
			val >>= 8
		}
	}
}

func main() {
	// 创建一些示例数据
	var data = make([]byte, 25000)
	fillRandom(data)

	// 创建5个各包含5000个元素的数据分片
	enc, _ := reedsolomon.New(5, 3)
	shards, _ := enc.Split(data)
	err := enc.Encode(shards)
	if err != nil {
		panic(err)
	}

	// 检查是否验证通过
	ok, err := enc.Verify(shards)
	if !ok || err != nil {
		fmt.Println("falied initial verify", err)
	}

	// 创建一个异或后的集合
	xored := make([][]byte, 8)

	// 我们按索引进行异或，所以你可以看到异或可以改变，
	// 但是它应该在你的分片中垂直保持恒定
	for i := range shards {
		xored[i] = make([]byte, len(shards[i]))
		for j := range xored[i] {
			xored[i][j] = shards[i][j] ^ byte(j&0xff)
		}
	}

	// 每个部分应该仍然验证为ok
	ok, err = enc.Verify(xored)
	if ok && err == nil {
		fmt.Println("verified ok after xor")
	}
}

Output:

verified ok after xor

func New 

func New(dataShards, parityShards int, opts ...Option) (Encoder, error)

New 创建一个新的编码器并初始化 参数:

  • dataShards: 数据分片数量
  • parityShards: 校验分片数量
  • opts: 可选的配置选项

返回值:

  • Encoder: 创建的编码器实例
  • error: 创建过程中的错误

说明:

  • 总分片数量(数据分片+校验分片)最大为65536
  • 当总分片数大于256时有以下限制:
  • 分片大小必须是64的倍数
  • 不支持Join/Split/Update/EncodeIdx方法
  • 如果没有提供选项,将使用默认配置

type Extensions 

type Extensions interface {
	// ShardSizeMultiple 返回分片大小必须是其倍数的值
	ShardSizeMultiple() int

	// DataShards 返回数据分片的数量
	DataShards() int

	// ParityShards 返回奇偶校验分片的数量
	ParityShards() int

	// TotalShards 返回分片总数
	TotalShards() int

	// AllocAligned 分配TotalShards数量的对齐内存切片
	// 参数:
	//   - each: 每个分片的大小
	// 返回值:
	//   - [][]byte: 分配的内存切片数组
	AllocAligned(each int) [][]byte
}

Extensions 是一个可选接口 所有返回的实例都将支持此接口

type FileEncoder 

type FileEncoder interface {
	// EncodeFile 对输入文件数组进行编码,生成校验分片文件
	// 参数:
	//   - shards: 输入文件数组,包含数据分片和校验分片
	// 返回值:
	//   - error: 编码失败时返回相应错误,成功返回nil
	// 说明:
	//   - 分片数量必须与New()函数指定的数量匹配
	//   - 每个分片文件大小必须相同
	//   - 校验分片文件会被覆盖,数据分片文件保持不变
	EncodeFile(shards []*os.File) error

	// VerifyFile 验证文件分片数据的完整性
	// 参数:
	//   - shards: 文件分片数组,包含数据分片和校验分片
	// 返回值:
	//   - bool: 验证通过返回true,否则返回false
	//   - error: 验证过程中出现错误时返回相应错误,成功返回nil
	// 说明:
	//   - 分片数量必须与New()函数指定的数量匹配
	//   - 每个分片件大小必须相同
	//   - 不会修改何数据
	VerifyFile(shards []*os.File) (bool, error)

	// ReconstructFile 重建所有丢失的文件分片
	// 参数:
	//   - shards: 文件分片数组,包含数据分片和校验分片
	// 返回值:
	//   - error: 重建失败时返回相应错误,成功返回nil
	// 说明:
	//   - 分片数量必须等于总分片数
	//   - 通过将分片设置为nil表示丢失的分片
	//   - 如果可用分片太少,将返回ErrTooFewShards错误
	//   - 重建后的分片集合是完整的,但未验证完整性
	ReconstructFile(shards []*os.File) error

	// ReconstructDataFile 仅重建丢失的数据文件分片
	// 参数:
	//   - shards: 文件分片数组,包含数据分片和校验分片
	// 返回值:
	//   - error: 重建失败时返回相应错误,成功返回nil
	// 说明:
	//   - 只重建数据分片,不重建校验分片
	//   - 其他说明同ReconstructFile
	ReconstructDataFile(shards []*os.File) error

	// SplitFile 将输入文件分割成编码器指定数量的临时文件
	// 参数:
	//   - dataFile: 需要分割的输入文件
	// 返回值:
	//   - []*os.File: 分割后的临时文件数组
	//   - error: 分割失败时返回相应错误,成功返回nil
	// 说明:
	//   - 文件将被分割成大小相等的分片
	//   - 如果文件大小不能被分片数整除,最后一个分片将补零
	//   - 使用系统临时目录存储分片文件
	SplitFile(dataFile *os.File) ([]*os.File, error)

	// JoinFile 将文件分片合并到一个输出文件中
	// 参数:
	//   - dst: 输出文件
	//   - shards: 文件分片数组,包含数据分片和校验分片
	//   - outSize: 输出文件的大小
	// 返回值:
	//   - error: 合并失败时返回相应错误,成功返回nil
	// 说明:
	//   - 只考虑数据分片
	//   - 必须提供准确的输出大小
	//   - 如果分片数量不足,将返回ErrTooFewShards错误
	//   - 如果总数据大小小于outSize,将返回ErrShortData错误
	JoinFile(dst *os.File, shards []*os.File, outSize int) error
}

FileEncoder 提供基于文件的Reed-Solomon编解码功能 用于处理大文件的流式编码和解码

func NewFile 

func NewFile(dataShards, parityShards int, opts ...Option) (FileEncoder, error)

NewFile 创建一个新的文件编码器并初始化 参数:

  • dataShards: 数据分片数量
  • parityShards: 校验分片数量
  • opts: 可选的配置选项

返回值:

  • FileEncoder: 创建的文件编码器实例
  • error: 创建过程中的错误

说明:

  • 总分片数量(数据分片+校验分片)最大为65536
  • 当总分片数大于256时有以下限制:
  • 分片大小必须是64的倍数
  • 如果没有提供选项,将使用默认配置

type Option 

type Option func(*options)

Option allows to override processing parameters.

func WithAVX2 

func WithAVX2(enabled bool) Option

WithAVX2 allows to enable/disable AVX2 instructions. If not set, AVX will be turned on or off automatically based on CPU ID information. This will also disable AVX GFNI instructions.

func WithAVX512 

func WithAVX512(enabled bool) Option

WithAVX512 allows to enable/disable AVX512 (and GFNI) instructions.

func WithAVXGFNI 

func WithAVXGFNI(enabled bool) Option

WithAVXGFNI allows to enable/disable GFNI with AVX instructions. If not set, GFNI will be turned on or off automatically based on CPU ID information.

func WithAutoGoroutines 

func WithAutoGoroutines(shardSize int) Option

WithAutoGoroutines will adjust the number of goroutines for optimal speed with a specific shard size. Send in the shard size you expect to send. Other shard sizes will work, but may not run at the optimal speed. Overwrites WithMaxGoroutines. If shardSize <= 0, it is ignored.

func WithCauchyMatrix 

func WithCauchyMatrix() Option

WithCauchyMatrix will make the encoder build a Cauchy style matrix. The output of this is not compatible with the standard output. A Cauchy matrix is faster to generate. This does not affect data throughput, but will result in slightly faster start-up time.

func WithConcurrentStreamReads 

func WithConcurrentStreamReads(enabled bool) Option

WithConcurrentStreamReads will enable concurrent reads from the input streams. Default: Disabled, meaning only one stream will be read at the time. Ignored if not used on a stream input.

func WithConcurrentStreamWrites 

func WithConcurrentStreamWrites(enabled bool) Option

WithConcurrentStreamWrites will enable concurrent writes to the the output streams. Default: Disabled, meaning only one stream will be written at the time. Ignored if not used on a stream input.

func WithConcurrentStreams 

func WithConcurrentStreams(enabled bool) Option

WithConcurrentStreams will enable concurrent reads and writes on the streams. Default: Disabled, meaning only one stream will be read/written at the time. Ignored if not used on a stream input.

func WithCustomMatrix 

func WithCustomMatrix(customMatrix [][]byte) Option

WithCustomMatrix causes the encoder to use the manually specified matrix. customMatrix represents only the parity chunks. customMatrix must have at least ParityShards rows and DataShards columns. It can be used for interoperability with libraries which generate the matrix differently or to implement more complex coding schemes like LRC (locally reconstructible codes).

func WithFastOneParityMatrix 

func WithFastOneParityMatrix() Option

WithFastOneParityMatrix will switch the matrix to a simple xor if there is only one parity shard. The PAR1 matrix already has this property so it has little effect there.

func WithGFNI 

func WithGFNI(enabled bool) Option

WithGFNI allows to enable/disable AVX512+GFNI instructions. If not set, GFNI will be turned on or off automatically based on CPU ID information.

func WithInversionCache 

func WithInversionCache(enabled bool) Option

WithInversionCache allows to control the inversion cache. This will cache reconstruction matrices so they can be reused. Enabled by default, or <= 64 shards for Leopard encoding.

func WithJerasureMatrix 

func WithJerasureMatrix() Option

WithJerasureMatrix causes the encoder to build the Reed-Solomon-Vandermonde matrix in the same way as done by the Jerasure library. The first row and column of the coding matrix only contains 1's in this method so the first parity chunk is always equal to XOR of all data chunks.

func WithLeopardGF 

func WithLeopardGF(enabled bool) Option

WithLeopardGF will use leopard GF for encoding, even when there are fewer than 256 shards. This will likely improve reconstruction time for some setups. Note that Leopard places certain restrictions on use see other documentation.

func WithLeopardGF16 

func WithLeopardGF16(enabled bool) Option

WithLeopardGF16 will always use leopard GF16 for encoding, even when there is less than 256 shards. This will likely improve reconstruction time for some setups. This is not compatible with Leopard output for <= 256 shards. Note that Leopard places certain restrictions on use see other documentation.

func WithMaxGoroutines 

func WithMaxGoroutines(n int) Option

WithMaxGoroutines is the maximum number of goroutines number for encoding & decoding. Jobs will be split into this many parts, unless each goroutine would have to process less than minSplitSize bytes (set with WithMinSplitSize). For the best speed, keep this well above the GOMAXPROCS number for more fine grained scheduling. If n <= 0, it is ignored.

func WithMinSplitSize 

func WithMinSplitSize(n int) Option

WithMinSplitSize is the minimum encoding size in bytes per goroutine. By default this parameter is determined by CPU cache characteristics. See WithMaxGoroutines on how jobs are split. If n <= 0, it is ignored.

func WithPAR1Matrix 

func WithPAR1Matrix() Option

WithPAR1Matrix causes the encoder to build the matrix how PARv1 does. Note that the method they use is buggy, and may lead to cases where recovery is impossible, even if there are enough parity shards.

func WithSSE2 

func WithSSE2(enabled bool) Option

WithSSE2 allows to enable/disable SSE2 instructions. If not set, SSE2 will be turned on or off automatically based on CPU ID information.

func WithSSSE3 

func WithSSSE3(enabled bool) Option

WithSSSE3 allows to enable/disable SSSE3 instructions. If not set, SSSE3 will be turned on or off automatically based on CPU ID information.

func WithStreamBlockSize 

func WithStreamBlockSize(n int) Option

WithStreamBlockSize allows to set a custom block size per round of reads/writes. If not set, any shard size set with WithAutoGoroutines will be used. If WithAutoGoroutines is also unset, 4MB will be used. Ignored if not used on stream.

type StreamEncoder 

type StreamEncoder interface {
	// Encode 为一组数据分片编码奇偶校验分片。
	//
	// 参数:
	//   - data: 包含数据分片的读取器切片
	//   - parity: 用于写入奇偶校验分片的写入器切片
	//
	// 返回值:
	//   - error: 如果编码过程中出现错误则返回，否则返回nil
	//
	// 注意:
	//   - 分片数量必须与NewStream()中给定的数量匹配
	//   - 每个读取器必须提供相同数量的字节
	//   - 如果数据流返回错误,将返回StreamReadError类型的错误
	//   - 如果奇偶校验写入器返回错误,将返回StreamWriteError
	Encode(data []io.Reader, parity []io.Writer) error

	// Verify 验证奇偶校验分片是否包含正确的数据。
	//
	// 参数:
	//   - shards: 包含所有数据和奇偶校验分片的读取器切片
	//
	// 返回值:
	//   - bool: 如果奇偶校验正确则返回true，否则返回false
	//   - error: 如果验证过程中出现错误则返回，否则返回nil
	//
	// 注意:
	//   - 分片数量必须与NewStream()中给定的总数据+奇偶校验分片数量匹配
	//   - 每个读取器必须提供相同数量的字节
	//   - 如果分片流返回错误,将返回StreamReadError类型的错误
	Verify(shards []io.Reader) (bool, error)

	// Reconstruct 尝试重建丢失的分片。
	//
	// 参数:
	//   - valid: 有效分片的读取器切片，丢失的分片用nil表示
	//   - fill: 用于写入重建分片的写入器切片，不需要重建的分片用nil表示
	//
	// 返回值:
	//   - error: 如果重建过程中出现错误则返回，否则返回nil
	//
	// 注意:
	//   - 如果分片太少而无法重建丢失的分片,将返回ErrTooFewShards
	//   - 重建的分片集是完整的,但未验证完整性
	//   - 使用Verify函数检查数据集是否正常
	Reconstruct(valid []io.Reader, fill []io.Writer) error

	// Split 将输入流分割为给定编码器的分片数。
	//
	// 参数:
	//   - data: 输入数据流
	//   - dst: 用于写入分割后分片的写入器切片
	//   - size: 输入数据的总大小
	//
	// 返回值:
	//   - error: 如果分割过程中出现错误则返回，否则返回nil
	//
	// 注意:
	//   - 如果数据大小不能被分片数整除,最后一个分片将包含额外的零
	//   - 如果无法检索指定的字节数,将返回'ErrShortData'
	Split(data io.Reader, dst []io.Writer, size int64) (err error)

	// Join 将分片合并并将数据段写入dst。
	//
	// 参数:
	//   - dst: 用于写入合并后数据的写入器
	//   - shards: 包含所有分片的读取器切片
	//   - outSize: 期望的输出数据大小
	//
	// 返回值:
	//   - error: 如果合并过程中出现错误则返回，否则返回nil
	//
	// 注意:
	//   - 只考虑数据分片
	//   - 如果给定的分片太少,将返回ErrTooFewShards
	//   - 如果总数据大小小于outSize,将返回ErrShortData
	Join(dst io.Writer, shards []io.Reader, outSize int64) error
}

StreamEncoder 是一个接口,用于对数据进行Reed-Solomon奇偶校验编码。 它提供了完全流式的接口,并以最大4MB的块处理数据。

对于10MB及以下的小分片大小,建议使用内存接口, 因为流式接口有启动开销。

对于所有操作,读取器和写入器不应假定任何单个读/写的顺序/大小。

使用示例请参见examples文件夹中的"stream-encoder.go"和"streamdecoder.go"。

Example

ExampleStreamEncoder 展示了一个简单的流编码器，我们从包含每个分片的读取器的[]io.Reader中进行编码

输入和输出可以与文件、网络流或适合你需求的任何东西交换 

package main

import (
	"bytes"
	"fmt"
	"io"
	"io/ioutil"
	"log"
	"math/rand"

	"github.com/bpfs/defs/v2/reedsolomon"
)

// fillRandom 用随机数据填充字节切片
// 参数:
//   - p: 要填充的字节切片
func fillRandom(p []byte) {
	for i := 0; i < len(p); i += 7 {
		val := rand.Int63()
		for j := 0; i+j < len(p) && j < 7; j++ {
			p[i+j] = byte(val)
			val >>= 8
		}
	}
}

func main() {
	dataShards := 5
	parityShards := 2

	// 创建一个具有指定数据和奇偶校验分片数量的StreamEncoder
	rs, err := reedsolomon.NewStream(dataShards, parityShards)
	if err != nil {
		log.Fatal(err)
	}

	shardSize := 50000

	// 创建输入数据分片
	input := make([][]byte, dataShards)
	for s := range input {
		input[s] = make([]byte, shardSize)
		fillRandom(input[s])
	}

	// 将我们的缓冲区转换为io.Readers
	readers := make([]io.Reader, dataShards)
	for i := range readers {
		readers[i] = io.Reader(bytes.NewBuffer(input[i]))
	}

	// 创建我们的输出io.Writers
	out := make([]io.Writer, parityShards)
	for i := range out {
		out[i] = ioutil.Discard
	}

	// 从输入编码到输出
	err = rs.Encode(readers, out)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println("ok")
}

Output:

ok

func NewStream 

func NewStream(dataShards, parityShards int, o ...Option) (StreamEncoder, error)

NewStream 创建一个新的编码器并初始化它 参数:

  • dataShards: int, 数据分片的数量
  • parityShards: int, 奇偶校验分片的数量
  • o: ...Option, 可选的配置选项

返回值:

  • StreamEncoder: 创建的流编码器
  • error: 如果创建过程中出现错误则返回，否则为 nil

注意:

  • 数据分片的最大数量为 256
  • 可以重复使用此编码器

func NewStreamC 

func NewStreamC(dataShards, parityShards int, conReads, conWrites bool, o ...Option) (StreamEncoder, error)

NewStreamC 创建一个新的流编码器并初始化数据分片和校验分片数量

参数:

  • dataShards: 数据分片数量
  • parityShards: 校验分片数量
  • conReads: 是否启用并发读取
  • conWrites: 是否启用并发写入
  • o: ...Option, 可选的配置选项

返回值:

  • StreamEncoder: 创建的流编码器
  • error: 如果创建过程中出现错误则返回，否则为 nil

说明:

  • 此函数功能与 NewStream 相同，但允许启用并发读写

type StreamReadError 

type StreamReadError struct {
	Err    error // 具体的错误信息
	Stream int   // 发生错误的流序号
}

StreamReadError 表示在读取流时遇到的错误。 它可以帮助定位哪个读取器失败了。

字段:

  • Err: 具体的错误信息
  • Stream: 发生错误的流序号

func (StreamReadError) Error 

func (s StreamReadError) Error() string

Error 返回格式化的错误字符串。

返回值:

  • string: 包含流序号和错误信息的格式化字符串

func (StreamReadError) String 

func (s StreamReadError) String() string

String 返回错误的字符串表示。

返回值:

  • string: 与Error()方法返回相同的错误字符串

type StreamWriteError 

type StreamWriteError struct {
	Err    error // 具体的错误信息
	Stream int   // 发生错误的流序号
}

StreamWriteError 表示在写入流时遇到的错误。 它可以帮助定位哪个写入器失败了。

字段:

  • Err: 具体的错误信息
  • Stream: 发生错误的流序号

func (StreamWriteError) Error 

func (s StreamWriteError) Error() string

Error 返回格式化的错误字符串。

返回值:

  • string: 包含流序号和错误信息的格式化字符串

func (StreamWriteError) String 

func (s StreamWriteError) String() string

String 返回错误的字符串表示。

返回值:

  • string: 与Error()方法返回相同的错误字符串

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.