reisen

package module
v0.0.0-...-91d6583 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 15, 2024 License: MIT Imports: 5 Imported by: 1

README

Reisen GoDoc

A simple library to extract video and audio frames from media containers (based on libav, i.e. ffmpeg).

Dependencies

The library requires libav components to work, specifically version 6.x -- version 7 does not work!

  • libavformat
  • libavcodec
  • libavutil
  • libswresample
  • libswscale

For Arch-based Linux distributions:

sudo pacman -S ffmpeg

For Debian-based Linux distributions:

sudo add-apt-repository ppa:savoury1/ffmpeg4
sudo apt install libswscale-dev libavcodec-dev libavformat-dev libswresample-dev libavutil-dev

For macOS:

brew install ffmpeg@6
brew link ffmpeg@6  # key to actually being able to use the 6 version

For Windows see the detailed tutorial.

Installation

Just casually run this command:

go get github.com/cogentcore/reisen

Usage

Any media file is composed of streams containing media data, e.g. audio, video and subtitles. The whole presentation data of the file is divided into packets. Each packet belongs to one of the streams and represents a single frame of its data. The process of decoding implies reading packets and decoding them into either video frames or audio frames.

The library provides read video frames as RGBA pictures. The audio samples are provided as raw byte slices in the format of AV_SAMPLE_FMT_DBL (i.e. 8 bytes per sample for one channel, the data type is float64). The channel layout is stereo (2 channels). The byte order is little-endian. The detailed scheme of the audio samples sequence is given below.

Audio sample structure

You are welcome to look at the examples to understand how to work with the library. Also please take a look at the detailed tutorial.

Documentation

Index

Constants

View Source
const (
	// StandardChannelCount is used for
	// audio conversion while decoding
	// audio frames.
	StandardChannelCount = 2
)
View Source
const (
	// TimeBase is a global time base
	// used for describing media containers.
	TimeBase int = C.AV_TIME_BASE
)

Variables

This section is empty.

Functions

func NetworkDeinitialize

func NetworkDeinitialize() error

func NetworkInitialize

func NetworkInitialize() error

Types

type AudioFrame

type AudioFrame struct {
	// contains filtered or unexported fields
}

AudioFrame is a data frame obtained from an audio stream.

func (*AudioFrame) Data

func (frame *AudioFrame) Data() []byte

Data returns a raw slice of audio frame samples.

func (*AudioFrame) IndexCoded

func (frame *AudioFrame) IndexCoded() int

IndexCoded returns the index of the frame in the bitstream order.

func (*AudioFrame) IndexDisplay

func (frame *AudioFrame) IndexDisplay() int

IndexDisplay returns the index of the frame in the display order.

func (*AudioFrame) PresentationOffset

func (frame *AudioFrame) PresentationOffset() (time.Duration, error)

PresentationOffset returns the duration offset since the start of the media at which the frame should be played.

type AudioStream

type AudioStream struct {
	// contains filtered or unexported fields
}

AudioStream is a stream containing audio frames consisting of audio samples.

func (*AudioStream) ApplyFilter

func (stream *AudioStream) ApplyFilter(args string) error

ApplyFilter applies a filter defined by the given string to the stream.

func (*AudioStream) BitRate

func (stream *AudioStream) BitRate() int64

BitRate returns the bit rate of the stream (in bps).

func (*AudioStream) ChannelCount

func (audio *AudioStream) ChannelCount() int

ChannelCount returns the number of channels (1 for mono, 2 for stereo, etc.).

func (*AudioStream) Close

func (audio *AudioStream) Close() error

Close closes the audio stream and stops decoding audio frames.

func (*AudioStream) CodecLongName

func (stream *AudioStream) CodecLongName() string

CodecName returns the long name of the codec that was used for encoding the stream.

func (*AudioStream) CodecName

func (stream *AudioStream) CodecName() string

CodecName returns the name of the codec that was used for encoding the stream.

func (*AudioStream) Duration

func (stream *AudioStream) Duration() (time.Duration, error)

Duration returns the duration of the stream.

func (*AudioStream) Filter

func (stream *AudioStream) Filter() string

Filter returns the name and arguments of the filter currently applied to the stream or "" if no filter applied.

func (*AudioStream) FrameCount

func (stream *AudioStream) FrameCount() int64

FrameCount returns the total number of frames in the stream.

func (*AudioStream) FrameRate

func (stream *AudioStream) FrameRate() (int, int)

FrameRate returns the frame rate of the stream as a fraction with a numerator and a denominator.

func (*AudioStream) FrameSize

func (audio *AudioStream) FrameSize() int

FrameSize returns the number of samples contained in one frame of the audio.

func (*AudioStream) Index

func (stream *AudioStream) Index() int

Index returns the index of the stream.

func (*AudioStream) Open

func (audio *AudioStream) Open() error

Open opens the audio stream to decode audio frames and samples from it.

func (*AudioStream) Opened

func (stream *AudioStream) Opened() bool

Opened returns 'true' if the stream is opened for decoding, and 'false' otherwise.

func (*AudioStream) ReadAudioFrame

func (audio *AudioStream) ReadAudioFrame() (*AudioFrame, bool, error)

ReadAudioFrame reads a new audio frame from the stream.

func (*AudioStream) ReadFrame

func (audio *AudioStream) ReadFrame() (Frame, bool, error)

ReadFrame reads a new frame from the stream.

func (*AudioStream) RemoveFilter

func (stream *AudioStream) RemoveFilter() error

RemoveFilter removes the currently applied filter from the stream and frees its memory.

func (*AudioStream) Rewind

func (stream *AudioStream) Rewind(t time.Duration) error

Rewind rewinds the stream to the specified time position.

Can be used on all the types of streams. However, it's better to use it on the video stream of the media file if you don't want the streams of the playback to desynchronyze.

func (*AudioStream) SampleRate

func (audio *AudioStream) SampleRate() int

SampleRate returns the sample rate of the audio stream.

func (*AudioStream) TimeBase

func (stream *AudioStream) TimeBase() (int, int)

TimeBase the numerator and the denominator of the stream time base factor fraction.

All the duration values of the stream are multiplied by this factor to get duration in seconds.

func (*AudioStream) Type

func (stream *AudioStream) Type() StreamType

Type returns the stream media data type.

type ErrorType

type ErrorType int
const (
	// ErrorAgain is returned when
	// the decoder needs more data
	// to serve the frame.
	ErrorAgain ErrorType = -11
	// ErrorInvalidValue is returned
	// when the function call argument
	// is invalid.
	ErrorInvalidValue ErrorType = -22
	// ErrorEndOfFile is returned upon
	// reaching the end of the media file.
	ErrorEndOfFile ErrorType = -541478725
)

type Frame

type Frame interface {
	Data() []byte
	PresentationOffset() (time.Duration, error)
}

Frame is an abstract data frame.

type InterpolationAlgorithm

type InterpolationAlgorithm int

InterpolationAlgorithm is used when we scale a video frame in a different resolution.

func (InterpolationAlgorithm) String

func (interpolationAlg InterpolationAlgorithm) String() string

String returns the name of the interpolation algorithm.

type Media

type Media struct {
	// contains filtered or unexported fields
}

Media is a media file containing audio, video and other types of streams.

func NewMedia

func NewMedia(filename string) (*Media, error)

NewMedia returns a new media container analyzer for the specified media file.

func (*Media) AudioStreams

func (media *Media) AudioStreams() []*AudioStream

AudioStreams returns all the audio streams of the media file.

func (*Media) Close

func (media *Media) Close()

Close closes the media container.

func (*Media) CloseDecode

func (media *Media) CloseDecode() error

CloseDecode closes the media container for decoding.

func (*Media) Duration

func (media *Media) Duration() (time.Duration, error)

Duration returns the overall duration of the media file.

func (*Media) FormatLongName

func (media *Media) FormatLongName() string

FormatLongName returns the long name of the media container.

func (*Media) FormatMIMEType

func (media *Media) FormatMIMEType() string

FormatMIMEType returns the MIME type name of the media container.

func (*Media) FormatName

func (media *Media) FormatName() string

FormatName returns the name of the media format.

func (*Media) OpenDecode

func (media *Media) OpenDecode() error

OpenDecode opens the media container for decoding.

CloseDecode() should be called afterwards.

func (*Media) ReadPacket

func (media *Media) ReadPacket() (*Packet, bool, error)

ReadPacket reads the next packet from the media stream.

func (*Media) StreamCount

func (media *Media) StreamCount() int

StreamCount returns the number of streams.

func (*Media) Streams

func (media *Media) Streams() []Stream

Streams returns a slice of all the available media data streams.

func (*Media) VideoStreams

func (media *Media) VideoStreams() []*VideoStream

VideoStreams returns all the video streams of the media file.

type Packet

type Packet struct {
	// contains filtered or unexported fields
}

Packet is a piece of encoded data acquired from the media container.

It can be either a video frame or an audio frame.

func (*Packet) Data

func (pkt *Packet) Data() []byte

Data returns the data encoded in the packet.

func (*Packet) Size

func (pkt *Packet) Size() int

Returns the size of the packet data.

func (*Packet) StreamIndex

func (pkt *Packet) StreamIndex() int

StreamIndex returns the index of the stream the packet belongs to.

func (*Packet) Type

func (pkt *Packet) Type() StreamType

Type returns the type of the packet (video or audio).

type Stream

type Stream interface {

	// Index returns the index
	// number of the stream.
	Index() int
	// Type returns the type
	// identifier of the stream.
	//
	// It's either video or audio.
	Type() StreamType
	// CodecName returns the
	// shortened name of the stream codec.
	CodecName() string
	// CodecLongName returns the
	// long name of the stream codec.
	CodecLongName() string
	// BitRate returns the stream
	// bitrate (in bps).
	BitRate() int64
	// Duration returns the time
	// duration of the stream
	Duration() (time.Duration, error)
	// TimeBase returns the numerator
	// and the denominator of the stream
	// time base fraction to convert
	// time duration in time base units
	// of the stream.
	TimeBase() (int, int)
	// FrameRate returns the approximate
	// frame rate (FPS) of the stream.
	FrameRate() (int, int)
	// FrameCount returns the total number
	// of frames in the stream.
	FrameCount() int64
	// Open opens the stream for decoding.
	Open() error
	// Rewind rewinds the whole media to the
	// specified time location based on the stream.
	Rewind(time.Duration) error
	// ApplyFilter applies a filter defined
	// by the given string to the stream.
	ApplyFilter(string) error
	// Filter returns the name and arguments
	// of the filter currently applied to the
	// stream or "" if no filter applied.
	Filter() string
	// RemoveFilter removes the currently applied
	// filter from the stream and frees its memory.
	RemoveFilter() error
	// ReadFrame decodes the next frame from the stream.
	ReadFrame() (Frame, bool, error)
	// Closes the stream for decoding.
	Close() error
	// contains filtered or unexported methods
}

Stream is an abstract media data stream.

type StreamType

type StreamType int

StreamType is a type of a media stream.

const (
	// StreamVideo denotes the stream keeping video frames.
	StreamVideo StreamType = C.AVMEDIA_TYPE_VIDEO
	// StreamAudio denotes the stream keeping audio frames.
	StreamAudio StreamType = C.AVMEDIA_TYPE_AUDIO
)

func (StreamType) String

func (streamType StreamType) String() string

String returns the string representation of stream type identifier.

type UnknownStream

type UnknownStream struct {
	// contains filtered or unexported fields
}

UnknownStream is a stream containing frames consisting of unknown data.

func (*UnknownStream) ApplyFilter

func (stream *UnknownStream) ApplyFilter(args string) error

ApplyFilter applies a filter defined by the given string to the stream.

func (*UnknownStream) BitRate

func (stream *UnknownStream) BitRate() int64

BitRate returns the bit rate of the stream (in bps).

func (*UnknownStream) Close

func (unknown *UnknownStream) Close() error

Close is just a stub.

func (*UnknownStream) CodecLongName

func (stream *UnknownStream) CodecLongName() string

CodecName returns the long name of the codec that was used for encoding the stream.

func (*UnknownStream) CodecName

func (stream *UnknownStream) CodecName() string

CodecName returns the name of the codec that was used for encoding the stream.

func (*UnknownStream) Duration

func (stream *UnknownStream) Duration() (time.Duration, error)

Duration returns the duration of the stream.

func (*UnknownStream) Filter

func (stream *UnknownStream) Filter() string

Filter returns the name and arguments of the filter currently applied to the stream or "" if no filter applied.

func (*UnknownStream) FrameCount

func (stream *UnknownStream) FrameCount() int64

FrameCount returns the total number of frames in the stream.

func (*UnknownStream) FrameRate

func (stream *UnknownStream) FrameRate() (int, int)

FrameRate returns the frame rate of the stream as a fraction with a numerator and a denominator.

func (*UnknownStream) Index

func (stream *UnknownStream) Index() int

Index returns the index of the stream.

func (*UnknownStream) Open

func (unknown *UnknownStream) Open() error

Open is just a stub.

func (*UnknownStream) Opened

func (stream *UnknownStream) Opened() bool

Opened returns 'true' if the stream is opened for decoding, and 'false' otherwise.

func (*UnknownStream) ReadFrame

func (unknown *UnknownStream) ReadFrame() (Frame, bool, error)

ReadFrame is just a stub.

func (*UnknownStream) RemoveFilter

func (stream *UnknownStream) RemoveFilter() error

RemoveFilter removes the currently applied filter from the stream and frees its memory.

func (*UnknownStream) Rewind

func (stream *UnknownStream) Rewind(t time.Duration) error

Rewind rewinds the stream to the specified time position.

Can be used on all the types of streams. However, it's better to use it on the video stream of the media file if you don't want the streams of the playback to desynchronyze.

func (*UnknownStream) TimeBase

func (stream *UnknownStream) TimeBase() (int, int)

TimeBase the numerator and the denominator of the stream time base factor fraction.

All the duration values of the stream are multiplied by this factor to get duration in seconds.

func (*UnknownStream) Type

func (stream *UnknownStream) Type() StreamType

Type returns the stream media data type.

type VideoFrame

type VideoFrame struct {
	// contains filtered or unexported fields
}

VideoFrame is a single frame of a video stream.

func (*VideoFrame) Data

func (frame *VideoFrame) Data() []byte

Data returns a byte slice of RGBA pixels of the frame image.

func (*VideoFrame) Image

func (frame *VideoFrame) Image() *image.RGBA

Image returns the RGBA image of the frame.

func (*VideoFrame) IndexCoded

func (frame *VideoFrame) IndexCoded() int

IndexCoded returns the index of the frame in the bitstream order.

func (*VideoFrame) IndexDisplay

func (frame *VideoFrame) IndexDisplay() int

IndexDisplay returns the index of the frame in the display order.

func (*VideoFrame) PresentationOffset

func (frame *VideoFrame) PresentationOffset() (time.Duration, error)

PresentationOffset returns the duration offset since the start of the media at which the frame should be played.

type VideoStream

type VideoStream struct {
	// contains filtered or unexported fields
}

VideoStream is a streaming holding video frames.

func (*VideoStream) ApplyFilter

func (stream *VideoStream) ApplyFilter(args string) error

ApplyFilter applies a filter defined by the given string to the stream.

func (*VideoStream) AspectRatio

func (video *VideoStream) AspectRatio() (int, int)

AspectRatio returns the fraction of the video stream frame aspect ratio (1/0 if unknown).

func (*VideoStream) BitRate

func (stream *VideoStream) BitRate() int64

BitRate returns the bit rate of the stream (in bps).

func (*VideoStream) Close

func (video *VideoStream) Close() error

Close closes the video stream for decoding.

func (*VideoStream) CodecLongName

func (stream *VideoStream) CodecLongName() string

CodecName returns the long name of the codec that was used for encoding the stream.

func (*VideoStream) CodecName

func (stream *VideoStream) CodecName() string

CodecName returns the name of the codec that was used for encoding the stream.

func (*VideoStream) Duration

func (stream *VideoStream) Duration() (time.Duration, error)

Duration returns the duration of the stream.

func (*VideoStream) Filter

func (stream *VideoStream) Filter() string

Filter returns the name and arguments of the filter currently applied to the stream or "" if no filter applied.

func (*VideoStream) FrameCount

func (stream *VideoStream) FrameCount() int64

FrameCount returns the total number of frames in the stream.

func (*VideoStream) FrameRate

func (stream *VideoStream) FrameRate() (int, int)

FrameRate returns the frame rate of the stream as a fraction with a numerator and a denominator.

func (*VideoStream) Height

func (video *VideoStream) Height() int

Height returns the height of the video stream frame.

func (*VideoStream) Index

func (stream *VideoStream) Index() int

Index returns the index of the stream.

func (*VideoStream) Open

func (video *VideoStream) Open() error

OpenDecode opens the video stream for decoding with default parameters.

func (*VideoStream) OpenDecode

func (video *VideoStream) OpenDecode(width, height int, alg InterpolationAlgorithm) error

OpenDecode opens the video stream for decoding with the specified parameters.

func (*VideoStream) Opened

func (stream *VideoStream) Opened() bool

Opened returns 'true' if the stream is opened for decoding, and 'false' otherwise.

func (*VideoStream) ReadFrame

func (video *VideoStream) ReadFrame() (Frame, bool, error)

ReadFrame reads the next frame from the stream.

func (*VideoStream) ReadVideoFrame

func (video *VideoStream) ReadVideoFrame() (*VideoFrame, bool, error)

ReadVideoFrame reads the next video frame from the video stream.

func (*VideoStream) RemoveFilter

func (stream *VideoStream) RemoveFilter() error

RemoveFilter removes the currently applied filter from the stream and frees its memory.

func (*VideoStream) Rewind

func (stream *VideoStream) Rewind(t time.Duration) error

Rewind rewinds the stream to the specified time position.

Can be used on all the types of streams. However, it's better to use it on the video stream of the media file if you don't want the streams of the playback to desynchronyze.

func (*VideoStream) TimeBase

func (stream *VideoStream) TimeBase() (int, int)

TimeBase the numerator and the denominator of the stream time base factor fraction.

All the duration values of the stream are multiplied by this factor to get duration in seconds.

func (*VideoStream) Type

func (stream *VideoStream) Type() StreamType

Type returns the stream media data type.

func (*VideoStream) Width

func (video *VideoStream) Width() int

Width returns the width of the video stream frame.

Directories

Path Synopsis
examples

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL