Documentation ¶
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ChunkSizePolicy ¶
type ChunkSizePolicy interface {
ChunkSize() int
}
ChunkSizePolicy is something that can tell how much data to fetch in single request for given S3 Object. With more advanced policies, Visit methods will be integrated.
type FixedChunkSizePolicy ¶
type FixedChunkSizePolicy struct {
Size int
}
FixedChunkSizePolicy always returns same chunk size.
func (FixedChunkSizePolicy) ChunkSize ¶
func (s FixedChunkSizePolicy) ChunkSize() int
type S3ReadSeeker ¶
type S3ReadSeeker struct {
// contains filtered or unexported fields
}
S3ReadSeeker is a reader of given S3 Object. It utilizes HTTP Byte Ranges to read chunks of data from S3 Object. It uses zero-memory copy from underlying HTTP Body response. It uses early HTTP Body termination, if seeks are beyond current HTTP Body. It uses adaptive policy for chunk size fetching. This is useful for iterating over very large S3 Objects.
Example ¶
package main import ( "io" "github.com/aws/aws-sdk-go/aws" "github.com/aws/aws-sdk-go/aws/credentials" "github.com/aws/aws-sdk-go/aws/session" "github.com/aws/aws-sdk-go/service/s3" awss3reader "github.com/nikolaydubina/aws-s3-reader" ) func main() { s3client := s3.New(session.Must(session.NewSession( aws.NewConfig(). WithRegion("ap-southeast-1"). WithCredentials(credentials.AnonymousCredentials), ))) r := awss3reader.NewS3ReadSeeker( s3client, "nikolaydubina-blog-public", "videos/2024-02-22.mov", awss3reader.FixedChunkSizePolicy{Size: 1 << 20 * 40}, ) defer r.Close() r.Seek(100, io.SeekCurrent) res, err := io.ReadAll(r) if err != nil || len(res) == 0 { panic(err) } }
Output:
func NewS3ReadSeeker ¶
func NewS3ReadSeeker( s3client *s3.S3, bucket string, key string, chunkSizePolicy ChunkSizePolicy, ) *S3ReadSeeker
func (*S3ReadSeeker) Close ¶
func (s *S3ReadSeeker) Close() error
Click to show internal directories.
Click to hide internal directories.