gostream

package
v0.21.0-rc0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 15, 2024 License: AGPL-3.0 Imports: 39 Imported by: 7

README

gostream

gostream is a library to simplify the streaming of images as video and audio chunks to audio to a series of WebRTC peers. The impetus for this existing was for doing simple GUI / audio/video streaming to a browser all within go with as little cgo as possible. The package will likely be refactored over time to support some more generalized use cases and as such will be in version 0 for the time being. Many parameters are hard coded and need to be configurable over time. Use at your own risk, and please file issues!

TODO

  • Support multiple codecs (e.g. Firefox macos-arm does not support h264 by default yet)
  • Verify Windows Logitech StreamCam working
  • Reconnect on server restart
  • Check closes and frees
  • Address code TODOs (including context.TODO)
  • Documentation (inner func docs, package docs, example docs)
  • Version 0.1.0
  • Support removal of streams
  • Synchronize audio with video

With NixOS (Experimental)

nix-shell --pure

Examples

  • Stream current desktop: go run gostream/cmd/stream_video/main.go
  • Stream camera: go run gostream/cmd/stream_video/main.go -camera
  • Stream microphone: go run gostream/cmd/stream_audio/main.go -playback
  • Stream microphone and camera: go run gostream/cmd/stream_av/main.go -camera
  • Playback microphone from browser: go run gostream/cmd/stream_audio/main.go -playback

Documentation

Overview

Package gostream implements a simple server for serving video streams over WebRTC.

Index

Constants

This section is empty.

Variables

View Source 
var Debug = false

Debug is helpful to turn on when the library isn't working quite right.

View Source 
var DefaultConstraints = mediadevices.MediaStreamConstraints{
	Video: func(constraint *mediadevices.MediaTrackConstraints) {
		constraint.Width = prop.IntRanged{640, 4096, 1920}
		constraint.Height = prop.IntRanged{400, 2160, 1080}
		constraint.FrameRate = prop.FloatRanged{0, 200, 60}
		constraint.FrameFormat = prop.FrameFormatOneOf{
			frame.FormatI420,
			frame.FormatI444,
			frame.FormatYUY2,
			frame.FormatYUYV,
			frame.FormatUYVY,
			frame.FormatRGBA,
			frame.FormatMJPEG,
			frame.FormatNV12,
			frame.FormatZ16,
			frame.FormatNV21,
		}
	},
}

DefaultConstraints are suitable for finding any available device.

View Source 
var ErrNotFound = errors.New("failed to find the best driver that fits the constraints")

ErrNotFound happens when there is no driver found in a query.

View Source 
var ErrServerAlreadyStarted = errors.New("already started")

ErrServerAlreadyStarted happens when the server has already been started.

Functions

func DriverFromMediaSource 

func DriverFromMediaSource[T, U any](src MediaSource[T]) (driver.Driver, error)

DriverFromMediaSource returns the underlying driver from the MediaSource.

func LabelsFromMediaSource 

func LabelsFromMediaSource[T, U any](src MediaSource[T]) ([]string, error)

LabelsFromMediaSource returns the labels from the underlying driver in the MediaSource.

func MIMETypeHint 

func MIMETypeHint(ctx context.Context, defaultType string) string

MIMETypeHint gets the hint of what MIME type to use in encoding; if nothing is set, the default provided is used.

func PropertiesFromMediaSource 

func PropertiesFromMediaSource[T, U any](src MediaSource[T]) ([]prop.Media, error)

PropertiesFromMediaSource returns properties from underlying driver in the given MediaSource.

func QueryAudioDeviceLabels 

func QueryAudioDeviceLabels() []string

QueryAudioDeviceLabels lists all known audio devices.

func QueryScreenDevicesLabels 

func QueryScreenDevicesLabels() []string

QueryScreenDevicesLabels lists all known screen devices.

func QueryVideoDeviceLabels 

func QueryVideoDeviceLabels() []string

QueryVideoDeviceLabels lists all known video devices (not a screen).

func ReadAudio 

func ReadAudio(ctx context.Context, source AudioSource) (wave.Audio, func(), error)

ReadAudio gets a single audio wave from an audio source. Using this has less of a guarantee than AudioSource.Stream that the Nth wave follows the N-1th wave.

func ReadImage 

func ReadImage(ctx context.Context, source VideoSource) (image.Image, func(), error)

ReadImage gets a single image from a video source. Using this has less of a guarantee than VideoSource.Stream that the Nth image follows the N-1th image.

func ReadMedia 

func ReadMedia[T any](ctx context.Context, source MediaSource[T]) (T, func(), error)

ReadMedia gets a single media from a source. Using this has less of a guarantee than MediaSource.Stream that the Nth media element follows the N-1th media element.

func StreamAudioSource 

func StreamAudioSource(ctx context.Context, as AudioSource, stream Stream) error

StreamAudioSource streams the given video source to the stream forever until context signals cancellation.

func StreamAudioSourceWithErrorHandler 

func StreamAudioSourceWithErrorHandler(
	ctx context.Context, as AudioSource, stream Stream, errHandler ErrorHandler,
) error

StreamAudioSourceWithErrorHandler streams the given audio source to the stream forever until context signals cancellation, audio errors are sent via the error handler.

func StreamVideoSource 

func StreamVideoSource(ctx context.Context, vs VideoSource, stream Stream) error

StreamVideoSource streams the given video source to the stream forever until context signals cancellation.

func StreamVideoSourceWithErrorHandler 

func StreamVideoSourceWithErrorHandler(
	ctx context.Context, vs VideoSource, stream Stream, errHandler ErrorHandler,
) error

StreamVideoSourceWithErrorHandler streams the given video source to the stream forever until context signals cancellation, frame errors are sent via the error handler.

func WithMIMETypeHint 

func WithMIMETypeHint(ctx context.Context, mimeType string) context.Context

WithMIMETypeHint provides a hint to readers that media should be encoded to this type.

Types

type AudioPropertyProvider 

type AudioPropertyProvider = MediaPropertyProvider[prop.Audio]

AudioPropertyProvider providers information about an audio source.

type AudioReader 

type AudioReader = MediaReader[wave.Audio]

An AudioReader is anything that can read and recycle audio data.

type AudioReaderFunc 

type AudioReaderFunc = MediaReaderFunc[wave.Audio]

An AudioReaderFunc is a helper to turn a function into an AudioReader.

type AudioSource 

type AudioSource = MediaSource[wave.Audio]

An AudioSource is responsible for producing audio chunks when requested. A source should produce the chunk as quickly as possible and introduce no rate limiting of its own as that is handled internally.

func NewAudioSource 

func NewAudioSource(r AudioReader, p prop.Audio) AudioSource

NewAudioSource instantiates a new audio read closer.

func NewAudioSourceForDriver 

func NewAudioSourceForDriver(d driver.Driver, r AudioReader, p prop.Audio) AudioSource

NewAudioSourceForDriver instantiates a new audio read closer and references the given driver.

type AudioStream 

type AudioStream = MediaStream[wave.Audio]

An AudioStream streams audio forever until closed.

func NewEmbeddedAudioStream 

func NewEmbeddedAudioStream(src AudioSource) AudioStream

NewEmbeddedAudioStream returns an audio stream from an audio source that is intended to be embedded/composed by another source. It defers the creation of its stream.

func NewEmbeddedAudioStreamFromReader 

func NewEmbeddedAudioStreamFromReader(reader AudioReader) AudioStream

NewEmbeddedAudioStreamFromReader returns an audio stream from an audio reader that is intended to be embedded/composed by another source. It defers the creation of its stream.

type DeviceInfo 

type DeviceInfo struct {
	ID         string
	Labels     []string
	Properties []prop.Media
	Priority   driver.Priority
	Error      error
}

DeviceInfo describes a driver.

func QueryAudioDevices 

func QueryAudioDevices() []DeviceInfo

QueryAudioDevices lists all known audio devices.

func QueryScreenDevices 

func QueryScreenDevices() []DeviceInfo

QueryScreenDevices lists all known screen devices.

func QueryVideoDevices 

func QueryVideoDevices() []DeviceInfo

QueryVideoDevices lists all known video devices (not a screen).

type DriverInUseError 

type DriverInUseError struct {
	// contains filtered or unexported fields
}

DriverInUseError is returned when closing drivers that still being read from.

func (*DriverInUseError) Error 

func (err *DriverInUseError) Error() string

type ErrorHandler 

type ErrorHandler func(ctx context.Context, mediaErr error)

ErrorHandler receives the error returned by a TSource.Next regardless of whether or not the error is nil (This allows for error handling logic based on consecutively retrieved errors).

type HotSwappableAudioSource 

type HotSwappableAudioSource = HotSwappableMediaSource[wave.Audio, prop.Audio]

A HotSwappableAudioSource allows for continuous streaming of audio of swappable underlying audio sources.

func NewHotSwappableAudioSource 

func NewHotSwappableAudioSource(src AudioSource) HotSwappableAudioSource

NewHotSwappableAudioSource returns a hot swappable audio source.

type HotSwappableMediaSource 

type HotSwappableMediaSource[T, U any] interface {
	MediaSource[T]
	MediaPropertyProvider[U]
	Swap(src MediaSource[T])
}

A HotSwappableMediaSource allows for continuous streaming of media of swappable underlying media sources.

func NewHotSwappableMediaSource 

func NewHotSwappableMediaSource[T, U any](src MediaSource[T]) HotSwappableMediaSource[T, U]

NewHotSwappableMediaSource returns a hot swappable media source.

type HotSwappableVideoSource 

type HotSwappableVideoSource = HotSwappableMediaSource[image.Image, prop.Video]

A HotSwappableVideoSource allows for continuous streaming of video of swappable underlying video sources.

func NewHotSwappableVideoSource 

func NewHotSwappableVideoSource(src VideoSource) HotSwappableVideoSource

NewHotSwappableVideoSource returns a hot swappable video source.

type MediaPropertyProvider 

type MediaPropertyProvider[U any] interface {
	MediaProperties(ctx context.Context) (U, error)
}

MediaPropertyProvider providers information about a source.

type MediaReader 

type MediaReader[T any] interface {
	Read(ctx context.Context) (data T, release func(), err error)
	Close(ctx context.Context) error
}

A MediaReader is anything that can read and recycle data. It is expected that reader can handle multiple reads at the same time. This would ideally only happen during streaming when a specific MIME type is requested. In the future, we may be able to notice multiple MIME types and either do deferred encode/decode or have the reader do it for us.

type MediaReaderFunc 

type MediaReaderFunc[T any] func(ctx context.Context) (T, func(), error)

A MediaReaderFunc is a helper to turn a function into a MediaReader.

func (MediaReaderFunc[T]) Close 

func (mrf MediaReaderFunc[T]) Close(ctx context.Context) error

Close does nothing.

func (MediaReaderFunc[T]) Read 

func (mrf MediaReaderFunc[T]) Read(ctx context.Context) (T, func(), error)

Read calls the underlying function to get a media.

type MediaReleasePair 

type MediaReleasePair[T any] struct {
	Media   T
	Release func()
}

MediaReleasePair associates a media with a corresponding function to release its resources once the receiver of a pair is finished with the media.

type MediaReleasePairWithError 

type MediaReleasePairWithError[T any] struct {
	Media   T
	Release func()
	Err     error
}

MediaReleasePairWithError contains the result of fetching media.

type MediaSource 

type MediaSource[T any] interface {
	// Stream returns a stream that makes a best effort to return consecutive media elements
	// that may have a MIME type hint dictated in the context via WithMIMETypeHint.
	Stream(ctx context.Context, errHandlers ...ErrorHandler) (MediaStream[T], error)

	// Close cleans up any associated resources with the Source (e.g. a Driver).
	Close(ctx context.Context) error
}

A MediaSource can produce Streams of Ts.

func GetAnyAudioSource 

func GetAnyAudioSource(
	constraints mediadevices.MediaStreamConstraints,
	logger golog.Logger,
) (MediaSource[wave.Audio], error)

GetAnyAudioSource attempts to find any suitable audio device.

func GetAnyScreenSource 

func GetAnyScreenSource(
	constraints mediadevices.MediaStreamConstraints,
	logger golog.Logger,
) (MediaSource[image.Image], error)

GetAnyScreenSource attempts to find any suitable screen device.

func GetAnyVideoSource 

func GetAnyVideoSource(
	constraints mediadevices.MediaStreamConstraints,
	logger golog.Logger,
) (MediaSource[image.Image], error)

GetAnyVideoSource attempts to find any suitable video device (not a screen).

func GetNamedAudioSource 

func GetNamedAudioSource(
	name string,
	constraints mediadevices.MediaStreamConstraints,
	logger golog.Logger,
) (MediaSource[wave.Audio], error)

GetNamedAudioSource attempts to find an audio device by the given name.

func GetNamedScreenSource 

func GetNamedScreenSource(
	name string,
	constraints mediadevices.MediaStreamConstraints,
	logger golog.Logger,
) (MediaSource[image.Image], error)

GetNamedScreenSource attempts to find a screen device by the given name.

func GetNamedVideoSource 

func GetNamedVideoSource(
	name string,
	constraints mediadevices.MediaStreamConstraints,
	logger golog.Logger,
) (MediaSource[image.Image], error)

GetNamedVideoSource attempts to find a video device (not a screen) by the given name.

func GetPatternedAudioSource 

func GetPatternedAudioSource(
	labelPattern *regexp.Regexp,
	constraints mediadevices.MediaStreamConstraints,
	logger golog.Logger,
) (MediaSource[wave.Audio], error)

GetPatternedAudioSource attempts to find an audio device by the given label pattern.

func GetPatternedScreenSource 

func GetPatternedScreenSource(
	labelPattern *regexp.Regexp,
	constraints mediadevices.MediaStreamConstraints,
	logger golog.Logger,
) (MediaSource[image.Image], error)

GetPatternedScreenSource attempts to find a screen device by the given label pattern.

func GetPatternedVideoSource 

func GetPatternedVideoSource(
	labelPattern *regexp.Regexp,
	constraints mediadevices.MediaStreamConstraints,
	logger golog.Logger,
) (MediaSource[image.Image], error)

GetPatternedVideoSource attempts to find a video device (not a screen) by the given label pattern.

type MediaStream 

type MediaStream[T any] interface {
	// Next returns the next media element in the sequence (best effort).
	// Note: This element is mutable and shared globally; it MUST be copied
	// before it is mutated.
	Next(ctx context.Context) (T, func(), error)

	// Close signals this stream is no longer needed and releases associated
	// resources.
	Close(ctx context.Context) error
}

A MediaStream streams media forever until closed.

func NewEmbeddedMediaStream 

func NewEmbeddedMediaStream[T, U any](src MediaSource[T]) MediaStream[T]

NewEmbeddedMediaStream returns a media stream from a media source that is intended to be embedded/composed by another source. It defers the creation of its media stream.

func NewEmbeddedMediaStreamFromReader 

func NewEmbeddedMediaStreamFromReader[T, U any](reader MediaReader[T], p U) MediaStream[T]

NewEmbeddedMediaStreamFromReader returns a media stream from a media reader that is intended to be embedded/composed by another source. It defers the creation of its media stream.

func NewMediaStreamForChannel 

func NewMediaStreamForChannel[T any](ctx context.Context) (context.Context, MediaStream[T], chan<- MediaReleasePairWithError[T])

NewMediaStreamForChannel returns a MediaStream backed by a channel.

type StandaloneStreamServer 

type StandaloneStreamServer interface {
	// AddStream adds the given stream for new connections to see.
	AddStream(stream Stream) error
	// Start starts the server and waits for new connections.
	Start(ctx context.Context) error
	// Stop stops the server and stops the underlying streams.
	Stop(ctx context.Context) error
}

A StandaloneStreamServer is a convenience helper for solely streaming a series streams. Streams can be added over time for future new connections.

func NewStandaloneStreamServer 

func NewStandaloneStreamServer(
	port int,
	logger golog.Logger,
	opts []StandaloneStreamServerOption,
	streams ...Stream,
) (StandaloneStreamServer, error)

NewStandaloneStreamServer returns a server that will run on the given port and initially starts with the given streams.

type StandaloneStreamServerOption 

type StandaloneStreamServerOption interface {
	// contains filtered or unexported methods
}

StandaloneStreamServerOption configures how we set up the server. Cribbed from https://github.com/grpc/grpc-go/blob/aff571cc86e6e7e740130dbbb32a9741558db805/dialoptions.go#L41

func WithStandaloneAllowReceive 

func WithStandaloneAllowReceive(allowReceive bool) StandaloneStreamServerOption

WithStandaloneAllowReceive returns an Option which sets whether or not this stream server wants to receive media.

func WithStandaloneOnPeerAdded 

func WithStandaloneOnPeerAdded(f func(pc *webrtc.PeerConnection)) StandaloneStreamServerOption

WithStandaloneOnPeerAdded returns an Option which sets a function to call when a new peer connection is added.

func WithStandaloneOnPeerRemoved 

func WithStandaloneOnPeerRemoved(f func(pc *webrtc.PeerConnection)) StandaloneStreamServerOption

WithStandaloneOnPeerRemoved returns an Option which sets a function to call when an existing peer connection is remvoed.

type StandaloneStreamServerOptions 

type StandaloneStreamServerOptions struct {
	// contains filtered or unexported fields
}

StandaloneStreamServerOptions configures a StandaloneStreamServer.

type Stream 

type Stream interface {
	Name() string

	// Start starts processing frames.
	Start()

	// Ready signals that there is at least one client connected and that
	// streams are ready for input. The returned context should be used for
	// signaling that streaming is no longer ready.
	StreamingReady() (<-chan struct{}, context.Context)

	InputVideoFrames(props prop.Video) (chan<- MediaReleasePair[image.Image], error)

	InputAudioChunks(props prop.Audio) (chan<- MediaReleasePair[wave.Audio], error)

	// Stop stops further processing of frames.
	Stop()
	// contains filtered or unexported methods
}

A Stream is sink that accepts any image frames for the purpose of displaying in a WebRTC video track.

func NewStream 

func NewStream(config StreamConfig) (Stream, error)

NewStream returns a newly configured stream that can begin to handle new connections.

type StreamAlreadyRegisteredError 

type StreamAlreadyRegisteredError struct {
	// contains filtered or unexported fields
}

StreamAlreadyRegisteredError indicates that a stream has a name that is already registered on the stream server.

func (*StreamAlreadyRegisteredError) Error 

func (e *StreamAlreadyRegisteredError) Error() string

type StreamConfig 

type StreamConfig struct {
	Name                string
	VideoEncoderFactory codec.VideoEncoderFactory
	AudioEncoderFactory codec.AudioEncoderFactory

	// TargetFrameRate will hint to the stream to try to maintain this frame rate.
	TargetFrameRate int

	Logger golog.Logger
}

A StreamConfig describes how a Stream should be managed.

type StreamServer 

type StreamServer interface {
	// ServiceServer returns a service server for gRPC.
	ServiceServer() streampb.StreamServiceServer

	// NewStream creates a new stream from config and adds it for new connections to see.
	// Returns the added stream if it is successfully added to the server.
	NewStream(config StreamConfig) (Stream, error)

	// AddStream adds the given stream for new connections to see.
	AddStream(stream Stream) error

	// Close closes the server.
	Close() error
}

A StreamServer manages a collection of streams. Streams can be added over time for future new connections.

func NewStreamServer 

func NewStreamServer(streams ...Stream) (StreamServer, error)

NewStreamServer returns a server that will run on the given port and initially starts with the given stream.

type VideoPropertyProvider 

type VideoPropertyProvider = MediaPropertyProvider[prop.Video]

VideoPropertyProvider providers information about a video source.

type VideoReader 

type VideoReader = MediaReader[image.Image]

An VideoReader is anything that can read and recycle video data.

type VideoReaderFunc 

type VideoReaderFunc = MediaReaderFunc[image.Image]

An VideoReaderFunc is a helper to turn a function into an VideoReader.

type VideoSource 

type VideoSource = MediaSource[image.Image]

A VideoSource is responsible for producing images when requested. A source should produce the image as quickly as possible and introduce no rate limiting of its own as that is handled internally.

func NewResizeVideoSource 

func NewResizeVideoSource(src VideoSource, width, height int) VideoSource

NewResizeVideoSource returns a source that resizes images to the set dimensions.

func NewVideoSource 

func NewVideoSource(r VideoReader, p prop.Video) VideoSource

NewVideoSource instantiates a new video source.

func NewVideoSourceForDriver 

func NewVideoSourceForDriver(d driver.Driver, r VideoReader, p prop.Video) VideoSource

NewVideoSourceForDriver instantiates a new video source and references the given driver.

type VideoStream 

type VideoStream = MediaStream[image.Image]

An VideoStream streams video forever until closed.

func NewEmbeddedVideoStream 

func NewEmbeddedVideoStream(src VideoSource) VideoStream

NewEmbeddedVideoStream returns a video stream from a video source that is intended to be embedded/composed by another source. It defers the creation of its stream.

func NewEmbeddedVideoStreamFromReader 

func NewEmbeddedVideoStreamFromReader(reader VideoReader) VideoStream

NewEmbeddedVideoStreamFromReader returns a video stream from a video reader that is intended to be embedded/composed by another source. It defers the creation of its stream.

Source Files

View all Source files

Directories

Path Synopsis
cmd
stream_audio
Package main streams audio.
 Package main streams audio.
stream_av
Package main streams audio and video.
 Package main streams audio and video.
stream_video
Package main streams video.
 Package main streams video.
Package codec defines the encoder and factory interfaces for encoding video frames and audio chunks.
 Package codec defines the encoder and factory interfaces for encoding video frames and audio chunks.
h264
Package h264 uses a V4L2-compatible h.264 hardware encoder (h264_v4l2m2m) to encode images.
 Package h264 uses a V4L2-compatible h.264 hardware encoder (h264_v4l2m2m) to encode images.
opus
Package opus contains the opus video codec.
 Package opus contains the opus video codec.
vpx
Package vpx contains the vpx video codec.
 Package vpx contains the vpx video codec.
x264
Package x264 contains the x264 video codec.
 Package x264 contains the x264 video codec.
ffmpeg
avcodec
Package avcodec is a wrapper around FFmpeg/release6.1.
 Package avcodec is a wrapper around FFmpeg/release6.1.
avlog
Package avlog is libav's logging facilities See https://ffmpeg.org/doxygen/3.0/log_8h.html
 Package avlog is libav's logging facilities See https://ffmpeg.org/doxygen/3.0/log_8h.html
avutil
Package avutil is a wrapper around FFmpeg/release6.1.
 Package avutil is a wrapper around FFmpeg/release6.1.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.