skyeye

module
v0.13.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 6, 2024 License: MIT

README

SkyEye: AI Powered GCI Bot for DCS

SkyEye is a Ground Controlled Intercept (GCI) bot for the flight simulator Digital Combat Simulator (DCS). A GCI bot allows players to request information about the airspace in English using either voice commands or text entry, and to receive answers via verbal speech and text messages.

SkyEye uses Speech-To-Text and Text-To-Speech technology which runs locally on the same computer as SkyEye. No cloud APIs are required. It works with any DCS mission, singleplayer or multiplayer. No special scripting or mission editor setup is required. You can run it for less than a nickel per hour on a cloud server, or run it on a PC in your home.

SkyEye is under active development. All of the radio calls I planned to support have been implemented - but there is still lots of work to do on performance, quality, accessibility, and additional features. To see what I'm working on, check out the dev blog!

Getting Started

Technology

SkyEye would not be possible without these people and projects, for whom I am deeply appreciative:

  • DCS-SRS by @ciribob. Ciribob also patiently answered many of my questions on SRS internals and provided helpful debugging tips whenever I ran into a block in the SRS integration.
  • Tacview - specifically, ACMI real time telemetry - provides the data feed from DCS World.
  • @rurounijones's OverlordBot was a useful reference against SkyEye during early development, and Jones himself was also patient with my questions on Discord.
  • @ggerganov's whisper.cpp models provides speech-to-text.
  • @rodaine's numwords module is invaluable for parsing numeric quantities from voice input.
  • Piper by the Rhasspy voice assistant project is used for speech-to-text.
  • The Jenny dataset by Dioco provides the feminine voice for SkyEye.
  • @popey's dataset provides the masculine voice for SkyEye.
  • @amitybell's embedded Piper module makes distribution and implementation of Piper a breeze. @nabbl improved this module by adding support for macOS and variable speeds.
  • The Opus codec and the hraban/opus module provides audio compression for the SRS protocol.
  • @hbollon's go-edlib module provides algorithms to help SkyEye understand when it slightly mishears/the user slightly misspeaks a callsign or command over the radio.
  • @lithammer's shortuuid module provides a GUID implementation compatible with the SRS protocols.
  • @zaf's resample module helps with audio format conversion between Piper and SRS.
  • @martinlindhe's unit module provides easy angular, length, speed and frequency unit conversion.
  • @paulmach's orb module provides a simple, flexible GIS library for analyzing the geometric relationships between aircraft.
  • @proway's go-igrf module implements the International Geomagnetic Reference Field used to correct for magnetic declination.
  • Cobra is used for the CLI frontend, including configuration flags, help and examples. Viper is used to load configuration from a file/environment variables.
  • MSYS2 provides a Windows build environment.
  • Oto was helpful for debugging audio format conversion problems.
  • zerolog is helpful for general logging and printf debugging.
  • testify is used in unit tests.
  • Multiple DCS communities provide invaluable feedback and morale-booster energy:
  • The Ace Combat series by PROJECT ACES/Bandai Namco and Project Wingman by Sector D2 are massive influences on my interest in GCI/AWACS, and aviation in general. This project would not exist without the impact of Ace Combat 04: Shattered Skies.
  • And of course, DCS World is produced by Eagle Dynamics.

FAQ

Is this ready?

This project is currently available in Limited Availability. Anyone can download the software and try it out, but it may contain bugs or have performance or quality issues. I am currently only providing support for a limited number of personal friends.

A General Availability release is expected during winter 2024-2025. At that point, I expect the software to be stable with few or no issues, and support will be provided to the general audience.

You can check current progress here!

What kind of hardware does it require?

See the Hardware section of the admin guide.

Can I train the speech recognition on my voice/accent?

Since the software runs 100% locally, the speech recognition model is a local file. Server operators can provide a trained model as an alternative to the off-the-shelf model. See this blog post for an example.

I don't plan to provide a mechanism for players to submit their voice recordings to the main repository due to data privacy concerns.

Does this use Line-Of-Sight restrictions?

No. Excluding this feature was an explicit choice in order to avoid the complexity demon.

If this is a critical feature for you, consider using MOOSE's AWACS module instead. It supports Line-Of-Sight and datalink simulation, at the tradeoff of requiring some special setup in the Mission Editor.

OverlordBot also optionally supports this feature, although less than 1% of users used it.

Will this work with DCS' built-in VoIP?

Hopefully in the future Eagle Dynamics will add support for external GCI bots. If anyone at ED is reading this, access to any relevant preview builds would be really helpful!

Could this use a Large Language Model? (llama, mistral, etc.)

This deserves a longer answer, for now see this issue

TL;DR most of the controller logic is simple geometry that completes in about a millisecond. An LLM is several orders of magnitude slower, less accurate and a more difficult user experience.

We use AI for the "squishy" problems - understanding human speech, and synthesizing human-like speech. We use traditional code for the algorithmic problems.

Could this provide ATC services?

This deserves a longer answer, for now see this issue

TL;DR I have no plans to attempt an ATC bot.

When is SkyEye's birthday?

October 12th. At some point I'll put an Ace Combat 04 easter egg in there.

Directories

Path Synopsis
cmd
skyeye
skyeye-scaler
internal
application
package application is the main package for the SkyEye application.
 package application is the main package for the SkyEye application.
cli
conf
pkg
bearings
Package bearings contains functions for working with absolute and magnetic compass bearings.
 Package bearings contains functions for working with absolute and magnetic compass bearings.
brevity
Package brevity contains types and models for air combat communication brevity.
 Package brevity contains types and models for air combat communication brevity.
coalitions
Package coalitions defines the coalitions in DCS World.
 Package coalitions defines the coalitions in DCS World.
commands
Package commands provides GCI command entrypoints other than voice commands.
 Package commands provides GCI command entrypoints other than voice commands.
composer
Package composer converts brevity responses from structured forms into natural language.
 Package composer converts brevity responses from structured forms into natural language.
controller
Package controller implements high-level logic for Ground-Controlled Interception (GCI).
 Package controller implements high-level logic for Ground-Controlled Interception (GCI).
encyclopedia
Package encyclopedia is a database of aircraft data.
 Package encyclopedia is a database of aircraft data.
parser
Package parser converts brevity requests from natural language into structured forms.
 Package parser converts brevity requests from natural language into structured forms.
pcm
Package pcm converts between different representations of PCM audio data.
 Package pcm converts between different representations of PCM audio data.
radar
Package radar implements mid-level logic for Ground-Controlled Interception (GCI).
 Package radar implements mid-level logic for Ground-Controlled Interception (GCI).
recognizer
Package recognizer recognizes text from speech.
 Package recognizer recognizes text from speech.
sim
Package sim contains message types for simulation state changes.
 Package sim contains message types for simulation state changes.
simpleradio
Package simpleradio contains a SimpleRadio-Standalone client.
 Package simpleradio contains a SimpleRadio-Standalone client.
simpleradio/types
Package types contains types used by the SRS client.
 Package types contains types used by the SRS client.
simpleradio/voice
Package voice contains the types used by the SRS audio protocol to send and receive audio data over the network.
 Package voice contains the types used by the SRS audio protocol to send and receive audio data over the network.
spatial
Package spatial contains geospatial functions.
 Package spatial contains geospatial functions.
synthesizer
Package synthesizer contains text-to-speech synthesizers.
 Package synthesizer contains text-to-speech synthesizers.
synthesizer/speakers
Package speakers contains interfaces and implementations for text-to-speech speakers.
 Package speakers contains interfaces and implementations for text-to-speech speakers.
synthesizer/voices
Package voices defines the available voices for the synthesizer package.
 Package voices defines the available voices for the synthesizer package.
telemetry
Package telemetry contains clients for reading ACMI data from files or real-time telemetry.
 Package telemetry contains clients for reading ACMI data from files or real-time telemetry.
traces
Package traces defines types and interfaces for collecting and publishing GCI workflow traces.
 Package traces defines types and interfaces for collecting and publishing GCI workflow traces.
trackfiles
Package trackfiles records aircraft movement over time.
 Package trackfiles records aircraft movement over time.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.