reisen

package module
v0.0.0-...-91d6583 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 15, 2024 License: MIT Imports: 5 Imported by: 1

README

Reisen GoDoc

A simple library to extract video and audio frames from media containers (based on libav, i.e. ffmpeg).

Dependencies

The library requires libav components to work, specifically version 6.x -- version 7 does not work!

  • libavformat
  • libavcodec
  • libavutil
  • libswresample
  • libswscale

For Arch-based Linux distributions:

sudo pacman -S ffmpeg

For Debian-based Linux distributions:

sudo add-apt-repository ppa:savoury1/ffmpeg4
sudo apt install libswscale-dev libavcodec-dev libavformat-dev libswresample-dev libavutil-dev

For macOS:

brew install ffmpeg@6
brew link ffmpeg@6  # key to actually being able to use the 6 version

For Windows see the detailed tutorial.

Installation

Just casually run this command:

go get github.com/cogentcore/reisen

Usage

Any media file is composed of streams containing media data, e.g. audio, video and subtitles. The whole presentation data of the file is divided into packets. Each packet belongs to one of the streams and represents a single frame of its data. The process of decoding implies reading packets and decoding them into either video frames or audio frames.

The library provides read video frames as RGBA pictures. The audio samples are provided as raw byte slices in the format of AV_SAMPLE_FMT_DBL (i.e. 8 bytes per sample for one channel, the data type is float64). The channel layout is stereo (2 channels). The byte order is little-endian. The detailed scheme of the audio samples sequence is given below.

Audio sample structure

You are welcome to look at the examples to understand how to work with the library. Also please take a look at the detailed tutorial.

Documentation

Index

Constants

View Source 
const (
	// StandardChannelCount is used for
	// audio conversion while decoding
	// audio frames.
	StandardChannelCount = 2
)
View Source 
const (
	// TimeBase is a global time base
	// used for describing media containers.
	TimeBase int = C.AV_TIME_BASE
)

Variables

This section is empty.

Functions

func NetworkDeinitialize 

func NetworkDeinitialize() error

func NetworkInitialize 

func NetworkInitialize() error

Types

type AudioFrame 

type AudioFrame struct {
	// contains filtered or unexported fields
}

AudioFrame is a data frame obtained from an audio stream.

func (*AudioFrame) Data 

func (frame *AudioFrame) Data() []byte

Data returns a raw slice of audio frame samples.

func (*AudioFrame) IndexCoded 

func (frame *AudioFrame) IndexCoded() int

IndexCoded returns the index of the frame in the bitstream order.

func (*AudioFrame) IndexDisplay 

func (frame *AudioFrame) IndexDisplay() int

IndexDisplay returns the index of the frame in the display order.

func (*AudioFrame) PresentationOffset 

func (frame *AudioFrame) PresentationOffset() (time.Duration, error)

PresentationOffset returns the duration offset since the start of the media at which the frame should be played.

type AudioStream 

type AudioStream struct {
	// contains filtered or unexported fields
}

AudioStream is a stream containing audio frames consisting of audio samples.

func (*AudioStream) ApplyFilter 

func (stream *AudioStream) ApplyFilter(args string) error

ApplyFilter applies a filter defined by the given string to the stream.

func (*AudioStream) BitRate 

func (stream *AudioStream) BitRate() int64

BitRate returns the bit rate of the stream (in bps).

func (*AudioStream) ChannelCount 

func (audio *AudioStream) ChannelCount() int

ChannelCount returns the number of channels (1 for mono, 2 for stereo, etc.).

func (*AudioStream) Close 

func (audio *AudioStream) Close() error

Close closes the audio stream and stops decoding audio frames.

func (*AudioStream) CodecLongName 

func (stream *AudioStream) CodecLongName() string

CodecName returns the long name of the codec that was used for encoding the stream.

func (*AudioStream) CodecName 

func (stream *AudioStream) CodecName() string

CodecName returns the name of the codec that was used for encoding the stream.

func (*AudioStream) Duration 

func (stream *AudioStream) Duration() (time.Duration, error)

Duration returns the duration of the stream.

func (*AudioStream) Filter 

func (stream *AudioStream) Filter() string

Filter returns the name and arguments of the filter currently applied to the stream or "" if no filter applied.

func (*AudioStream) FrameCount 

func (stream *AudioStream) FrameCount() int64

FrameCount returns the total number of frames in the stream.

func (*AudioStream) FrameRate 

func (stream *AudioStream) FrameRate() (int, int)

FrameRate returns the frame rate of the stream as a fraction with a numerator and a denominator.

func (*AudioStream) FrameSize 

func (audio *AudioStream) FrameSize() int

FrameSize returns the number of samples contained in one frame of the audio.

func (*AudioStream) Index 

func (stream *AudioStream) Index() int

Index returns the index of the stream.

func (*AudioStream) Open 

func (audio *AudioStream) Open() error

Open opens the audio stream to decode audio frames and samples from it.

func (*AudioStream) Opened 

func (stream *AudioStream) Opened() bool

Opened returns 'true' if the stream is opened for decoding, and 'false' otherwise.

func (*AudioStream) ReadAudioFrame 

func (audio *AudioStream) ReadAudioFrame() (*AudioFrame, bool, error)

ReadAudioFrame reads a new audio frame from the stream.

func (*AudioStream) ReadFrame 

func (audio *AudioStream) ReadFrame() (Frame, bool, error)

ReadFrame reads a new frame from the stream.

func (*AudioStream) RemoveFilter 

func (stream *AudioStream) RemoveFilter() error

RemoveFilter removes the currently applied filter from the stream and frees its memory.

func (*AudioStream) Rewind 

func (stream *AudioStream) Rewind(t time.Duration) error

Rewind rewinds the stream to the specified time position.

Can be used on all the types of streams. However, it's better to use it on the video stream of the media file if you don't want the streams of the playback to desynchronyze.

func (*AudioStream) SampleRate 

func (audio *AudioStream) SampleRate() int

SampleRate returns the sample rate of the audio stream.

func (*AudioStream) TimeBase 

func (stream *AudioStream) TimeBase() (int, int)

TimeBase the numerator and the denominator of the stream time base factor fraction.

All the duration values of the stream are multiplied by this factor to get duration in seconds.

func (*AudioStream) Type 

func (stream *AudioStream) Type() StreamType

Type returns the stream media data type.

type ErrorType 

type ErrorType int
const (
	// ErrorAgain is returned when
	// the decoder needs more data
	// to serve the frame.
	ErrorAgain ErrorType = -11
	// ErrorInvalidValue is returned
	// when the function call argument
	// is invalid.
	ErrorInvalidValue ErrorType = -22
	// ErrorEndOfFile is returned upon
	// reaching the end of the media file.
	ErrorEndOfFile ErrorType = -541478725
)

type Frame 

type Frame interface {
	Data() []byte
	PresentationOffset() (time.Duration, error)
}

Frame is an abstract data frame.

type InterpolationAlgorithm 

type InterpolationAlgorithm int

InterpolationAlgorithm is used when we scale a video frame in a different resolution. 

const (
	InterpolationFastBilinear    InterpolationAlgorithm = InterpolationAlgorithm(C.SWS_FAST_BILINEAR)
	InterpolationBilinear        InterpolationAlgorithm = InterpolationAlgorithm(C.SWS_BILINEAR)
	InterpolationBicubic         InterpolationAlgorithm = InterpolationAlgorithm(C.SWS_BICUBIC)
	InterpolationX               InterpolationAlgorithm = InterpolationAlgorithm(C.SWS_X)
	InterpolationPoint           InterpolationAlgorithm = InterpolationAlgorithm(C.SWS_POINT)
	InterpolationArea            InterpolationAlgorithm = InterpolationAlgorithm(C.SWS_AREA)
	InterpolationBicubicBilinear InterpolationAlgorithm = InterpolationAlgorithm(C.SWS_BICUBLIN)
	InterpolationGauss           InterpolationAlgorithm = InterpolationAlgorithm(C.SWS_GAUSS)
	InterpolationSinc            InterpolationAlgorithm = InterpolationAlgorithm(C.SWS_SINC)
	InterpolationLanczos         InterpolationAlgorithm = InterpolationAlgorithm(C.SWS_LANCZOS)
	InterpolationSpline          InterpolationAlgorithm = InterpolationAlgorithm(C.SWS_SPLINE)
)

func (InterpolationAlgorithm) String 

func (interpolationAlg InterpolationAlgorithm) String() string

String returns the name of the interpolation algorithm.

type Media 

type Media struct {
	// contains filtered or unexported fields
}

Media is a media file containing audio, video and other types of streams.

func NewMedia 

func NewMedia(filename string) (*Media, error)

NewMedia returns a new media container analyzer for the specified media file.

func (*Media) AudioStreams 

func (media *Media) AudioStreams() []*AudioStream

AudioStreams returns all the audio streams of the media file.

func (*Media) Close 

func (media *Media) Close()

Close closes the media container.

func (*Media) CloseDecode 

func (media *Media) CloseDecode() error

CloseDecode closes the media container for decoding.

func (*Media) Duration 

func (media *Media) Duration() (time.Duration, error)

Duration returns the overall duration of the media file.

func (*Media) FormatLongName 

func (media *Media) FormatLongName() string

FormatLongName returns the long name of the media container.

func (*Media) FormatMIMEType 

func (media *Media) FormatMIMEType() string

FormatMIMEType returns the MIME type name of the media container.

func (*Media) FormatName 

func (media *Media) FormatName() string

FormatName returns the name of the media format.

func (*Media) OpenDecode 

func (media *Media) OpenDecode() error

OpenDecode opens the media container for decoding.

CloseDecode() should be called afterwards.

func (*Media) ReadPacket 

func (media *Media) ReadPacket() (*Packet, bool, error)

ReadPacket reads the next packet from the media stream.

func (*Media) StreamCount 

func (media *Media) StreamCount() int

StreamCount returns the number of streams.

func (*Media) Streams 

func (media *Media) Streams() []Stream

Streams returns a slice of all the available media data streams.

func (*Media) VideoStreams 

func (media *Media) VideoStreams() []*VideoStream

VideoStreams returns all the video streams of the media file.

type Packet 

type Packet struct {
	// contains filtered or unexported fields
}

Packet is a piece of encoded data acquired from the media container.

It can be either a video frame or an audio frame.

func (*Packet) Data 

func (pkt *Packet) Data() []byte

Data returns the data encoded in the packet.

func (*Packet) Size 

func (pkt *Packet) Size() int

Returns the size of the packet data.

func (*Packet) StreamIndex 

func (pkt *Packet) StreamIndex() int

StreamIndex returns the index of the stream the packet belongs to.

func (*Packet) Type 

func (pkt *Packet) Type() StreamType

Type returns the type of the packet (video or audio).

type Stream 

type Stream interface {

	// Index returns the index
	// number of the stream.
	Index() int
	// Type returns the type
	// identifier of the stream.
	//
	// It's either video or audio.
	Type() StreamType
	// CodecName returns the
	// shortened name of the stream codec.
	CodecName() string
	// CodecLongName returns the
	// long name of the stream codec.
	CodecLongName() string
	// BitRate returns the stream
	// bitrate (in bps).
	BitRate() int64
	// Duration returns the time
	// duration of the stream
	Duration() (time.Duration, error)
	// TimeBase returns the numerator
	// and the denominator of the stream
	// time base fraction to convert
	// time duration in time base units
	// of the stream.
	TimeBase() (int, int)
	// FrameRate returns the approximate
	// frame rate (FPS) of the stream.
	FrameRate() (int, int)
	// FrameCount returns the total number
	// of frames in the stream.
	FrameCount() int64
	// Open opens the stream for decoding.
	Open() error
	// Rewind rewinds the whole media to the
	// specified time location based on the stream.
	Rewind(time.Duration) error
	// ApplyFilter applies a filter defined
	// by the given string to the stream.
	ApplyFilter(string) error
	// Filter returns the name and arguments
	// of the filter currently applied to the
	// stream or "" if no filter applied.
	Filter() string
	// RemoveFilter removes the currently applied
	// filter from the stream and frees its memory.
	RemoveFilter() error
	// ReadFrame decodes the next frame from the stream.
	ReadFrame() (Frame, bool, error)
	// Closes the stream for decoding.
	Close() error
	// contains filtered or unexported methods
}

Stream is an abstract media data stream.

type StreamType 

type StreamType int

StreamType is a type of a media stream. 

const (
	// StreamVideo denotes the stream keeping video frames.
	StreamVideo StreamType = C.AVMEDIA_TYPE_VIDEO
	// StreamAudio denotes the stream keeping audio frames.
	StreamAudio StreamType = C.AVMEDIA_TYPE_AUDIO
)

func (StreamType) String 

func (streamType StreamType) String() string

String returns the string representation of stream type identifier.

type UnknownStream 

type UnknownStream struct {
	// contains filtered or unexported fields
}

UnknownStream is a stream containing frames consisting of unknown data.

func (*UnknownStream) ApplyFilter 

func (stream *UnknownStream) ApplyFilter(args string) error

ApplyFilter applies a filter defined by the given string to the stream.

func (*UnknownStream) BitRate 

func (stream *UnknownStream) BitRate() int64

BitRate returns the bit rate of the stream (in bps).

func (*UnknownStream) Close 

func (unknown *UnknownStream) Close() error

Close is just a stub.

func (*UnknownStream) CodecLongName 

func (stream *UnknownStream) CodecLongName() string

CodecName returns the long name of the codec that was used for encoding the stream.

func (*UnknownStream) CodecName 

func (stream *UnknownStream) CodecName() string

CodecName returns the name of the codec that was used for encoding the stream.

func (*UnknownStream) Duration 

func (stream *UnknownStream) Duration() (time.Duration, error)

Duration returns the duration of the stream.

func (*UnknownStream) Filter 

func (stream *UnknownStream) Filter() string

Filter returns the name and arguments of the filter currently applied to the stream or "" if no filter applied.

func (*UnknownStream) FrameCount 

func (stream *UnknownStream) FrameCount() int64

FrameCount returns the total number of frames in the stream.

func (*UnknownStream) FrameRate 

func (stream *UnknownStream) FrameRate() (int, int)

FrameRate returns the frame rate of the stream as a fraction with a numerator and a denominator.

func (*UnknownStream) Index 

func (stream *UnknownStream) Index() int

Index returns the index of the stream.

func (*UnknownStream) Open 

func (unknown *UnknownStream) Open() error

Open is just a stub.

func (*UnknownStream) Opened 

func (stream *UnknownStream) Opened() bool

Opened returns 'true' if the stream is opened for decoding, and 'false' otherwise.

func (*UnknownStream) ReadFrame 

func (unknown *UnknownStream) ReadFrame() (Frame, bool, error)

ReadFrame is just a stub.

func (*UnknownStream) RemoveFilter 

func (stream *UnknownStream) RemoveFilter() error

RemoveFilter removes the currently applied filter from the stream and frees its memory.

func (*UnknownStream) Rewind 

func (stream *UnknownStream) Rewind(t time.Duration) error

Rewind rewinds the stream to the specified time position.

Can be used on all the types of streams. However, it's better to use it on the video stream of the media file if you don't want the streams of the playback to desynchronyze.

func (*UnknownStream) TimeBase 

func (stream *UnknownStream) TimeBase() (int, int)

TimeBase the numerator and the denominator of the stream time base factor fraction.

All the duration values of the stream are multiplied by this factor to get duration in seconds.

func (*UnknownStream) Type 

func (stream *UnknownStream) Type() StreamType

Type returns the stream media data type.

type VideoFrame 

type VideoFrame struct {
	// contains filtered or unexported fields
}

VideoFrame is a single frame of a video stream.

func (*VideoFrame) Data 

func (frame *VideoFrame) Data() []byte

Data returns a byte slice of RGBA pixels of the frame image.

func (*VideoFrame) Image 

func (frame *VideoFrame) Image() *image.RGBA

Image returns the RGBA image of the frame.

func (*VideoFrame) IndexCoded 

func (frame *VideoFrame) IndexCoded() int

IndexCoded returns the index of the frame in the bitstream order.

func (*VideoFrame) IndexDisplay 

func (frame *VideoFrame) IndexDisplay() int

IndexDisplay returns the index of the frame in the display order.

func (*VideoFrame) PresentationOffset 

func (frame *VideoFrame) PresentationOffset() (time.Duration, error)

PresentationOffset returns the duration offset since the start of the media at which the frame should be played.

type VideoStream 

type VideoStream struct {
	// contains filtered or unexported fields
}

VideoStream is a streaming holding video frames.

func (*VideoStream) ApplyFilter 

func (stream *VideoStream) ApplyFilter(args string) error

ApplyFilter applies a filter defined by the given string to the stream.

func (*VideoStream) AspectRatio 

func (video *VideoStream) AspectRatio() (int, int)

AspectRatio returns the fraction of the video stream frame aspect ratio (1/0 if unknown).

func (*VideoStream) BitRate 

func (stream *VideoStream) BitRate() int64

BitRate returns the bit rate of the stream (in bps).

func (*VideoStream) Close 

func (video *VideoStream) Close() error

Close closes the video stream for decoding.

func (*VideoStream) CodecLongName 

func (stream *VideoStream) CodecLongName() string

CodecName returns the long name of the codec that was used for encoding the stream.

func (*VideoStream) CodecName 

func (stream *VideoStream) CodecName() string

CodecName returns the name of the codec that was used for encoding the stream.

func (*VideoStream) Duration 

func (stream *VideoStream) Duration() (time.Duration, error)

Duration returns the duration of the stream.

func (*VideoStream) Filter 

func (stream *VideoStream) Filter() string

Filter returns the name and arguments of the filter currently applied to the stream or "" if no filter applied.

func (*VideoStream) FrameCount 

func (stream *VideoStream) FrameCount() int64

FrameCount returns the total number of frames in the stream.

func (*VideoStream) FrameRate 

func (stream *VideoStream) FrameRate() (int, int)

FrameRate returns the frame rate of the stream as a fraction with a numerator and a denominator.

func (*VideoStream) Height 

func (video *VideoStream) Height() int

Height returns the height of the video stream frame.

func (*VideoStream) Index 

func (stream *VideoStream) Index() int

Index returns the index of the stream.

func (*VideoStream) Open 

func (video *VideoStream) Open() error

OpenDecode opens the video stream for decoding with default parameters.

func (*VideoStream) OpenDecode 

func (video *VideoStream) OpenDecode(width, height int, alg InterpolationAlgorithm) error

OpenDecode opens the video stream for decoding with the specified parameters.

func (*VideoStream) Opened 

func (stream *VideoStream) Opened() bool

Opened returns 'true' if the stream is opened for decoding, and 'false' otherwise.

func (*VideoStream) ReadFrame 

func (video *VideoStream) ReadFrame() (Frame, bool, error)

ReadFrame reads the next frame from the stream.

func (*VideoStream) ReadVideoFrame 

func (video *VideoStream) ReadVideoFrame() (*VideoFrame, bool, error)

ReadVideoFrame reads the next video frame from the video stream.

func (*VideoStream) RemoveFilter 

func (stream *VideoStream) RemoveFilter() error

RemoveFilter removes the currently applied filter from the stream and frees its memory.

func (*VideoStream) Rewind 

func (stream *VideoStream) Rewind(t time.Duration) error

Rewind rewinds the stream to the specified time position.

Can be used on all the types of streams. However, it's better to use it on the video stream of the media file if you don't want the streams of the playback to desynchronyze.

func (*VideoStream) TimeBase 

func (stream *VideoStream) TimeBase() (int, int)

TimeBase the numerator and the denominator of the stream time base factor fraction.

All the duration values of the stream are multiplied by this factor to get duration in seconds.

func (*VideoStream) Type 

func (stream *VideoStream) Type() StreamType

Type returns the stream media data type.

func (*VideoStream) Width 

func (video *VideoStream) Width() int

Width returns the width of the video stream frame.

Source Files

View all Source files

Directories

Path Synopsis
examples
analyze

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.