jotbot

README

JotBot - AI-powered code documentation

JotBot auto-generates missing documentation for your Go and TypeScript files. The documentation of this repository was entirely generated by JotBot, so feel free to check out the results in the source code (Go code is currently missing documentation).

The documentation for this repository was created using the gpt-3.5-turbo model. In my personal tests, I found that text-davinci-003 consistently produces better results. However, due to its 10x higher cost, the default setting remains gpt-3.5-turbo.

I'm in the process of re-generating the documentation using GPT-4. Will update when done.

Features

• Generate documentation for Go and TypeScript codebases
• Customize glob patterns for included and excluded files
• Filter code symbols by matching regular expressions
• Limit the number of files to generate documentation for
• Run in dry mode to preview changes without applying them
• Control the AI model and token limits used for generating documentation
• Optionally commit changes to a Git branch

To-Do

• Configurable OpenAI settings (temperature, top_p etc.)
• Any ideas? open an issue or start a discussion

Installation

Via go install

If you have Go installed, you can simply install JotBot using go install:

go install github.com/modernice/jotbot/cmd/jotbot@latest

Standalone Binary

TODO: Setup GitHub Action (GoReleaser)

TypeScript Support

To enable TypeScript (and JavaScript) support, you also need to install the jotbot-ts npm package.

npm install -g jotbot-ts
pnpm install -g jotbot-ts

Usage

To generate missing documentation for your codebase, run the following command:

jotbot generate [options]

By default, this command will find all Go and TypeScript (and JavaScript) files in the current and nested directories and generate documentation for them. Excluded from the search are by default:

• **/.*/**
• **/dist/**
• **/node_modules/**
• **/vendor/**
• **/testdata/**
• **/test/**
• **/tests/**

CLI options

jotbot --help

Option	Description	Default
--root	Root directory of the repository	"."
--include, -i	Glob pattern(s) to include files
--include-tests, -T	Include TestXXX() functions (Go-specific)
--exclude, -e	Glob pattern(s) to exclude files
--exclude-internal, -E	Exclude 'internal' directories (Go-specific)	true
--match, -m	Regular expression(s) to match identifiers
--symbol, -s	Symbol(s) to search for in code (TS/JS-specific)
--branch	Branch name to commit changes to (leave empty to not commit)
--limit	Limit the number of files to generate documentation for	0
--dry	Print the changes without applying them	false
--model	OpenAI model used to generate documentation	"gpt-3.5-turbo"
--maxTokens	Maximum number of tokens to generate for a single documentation	512
--parallel, -p	Number of files to handle concurrently	4
--workers	Number of workers to use per file	2
--override, -o	Override existing documentation
--key	OpenAI API key
--verbose, -v	Enable verbose logging	false

License

MIT

Documentation

Index

• type Finding
  • func (f Finding) String() string
• type JotBot
  • func New(root string, opts ...Option) *JotBot
  • func (bot *JotBot) ConfigureLanguage(name string, lang Language)
  • func (bot *JotBot) Extensions() []string
  • func (bot *JotBot) Find(ctx context.Context, opts ...find.Option) ([]Finding, error)
  • func (bot *JotBot) Generate(ctx context.Context, findings []Finding, svc generate.Service, ...) (*Patch, error)
• type Language
• type Option
  • func Match(filters ...*regexp.Regexp) Option
  • func WithLanguage(name string, lang Language) Option
  • func WithLogger(h slog.Handler) Option
• type Patch
  • func (p *Patch) Apply(ctx context.Context, root string) error
  • func (p *Patch) DryRun(ctx context.Context, root string) (map[string][]byte, error)

Types

type Finding

type Finding struct {
	Identifier string
	File       string
	Language   string
}

Finding represents a single code identifier found within a file with its associated language.

func (Finding) String

func (f Finding) String() string

String returns a formatted string representation of the Finding, including the file path and identifier.

type JotBot

type JotBot struct {
	// contains filtered or unexported fields
}

JotBot is a utility for finding and generating code snippets in various programming languages within a project. It supports configurable languages, filters, and file systems, and offers patching capabilities for applying generated changes.

func New

func New(root string, opts ...Option) *JotBot

New creates a new JotBot instance with the given root directory and options. The returned JotBot is configured with the provided languages, filters, and logger.

func (*JotBot) ConfigureLanguage

func (bot *JotBot) ConfigureLanguage(name string, lang Language)

ConfigureLanguage configures a Language for the given name, associating it with the provided extensions. The Language is used by JotBot to find and generate code for identifiers in files with the associated extensions.

func (*JotBot) Extensions

func (bot *JotBot) Extensions() []string

Extensions returns a slice of file extensions that are associated with configured languages in the JotBot instance.

func (*JotBot) Find

func (bot *JotBot) Find(ctx context.Context, opts ...find.Option) ([]Finding, error)

Find searches the JotBot's root directory for files matching the provided options, extracts identifiers from those files using the configured languages, and returns a slice of Findings containing the extracted identifiers, file paths, and language names. It filters the extracted identifiers based on the JotBot's configured filters. If any error occurs during the process, it returns the error.

func (*JotBot) Generate

func (bot *JotBot) Generate(ctx context.Context, findings []Finding, svc generate.Service, opts ...generate.Option) (*Patch, error)

Generate creates a Patch by generating code snippets for the given Findings using the provided generate.Service and options. It returns a Patch that can be applied to the project or used in a dry run.

type Language

type Language interface {
	patch.Language
	generate.Language

	// Extensions returns a slice of file extensions that are supported by the
	// Language. Each extension is associated with a specific language
	// implementation for code generation and patching purposes.
	Extensions() []string

	// Find searches the provided byte slice for identifiers in the language and
	// returns a slice of found identifiers along with any error encountered during
	// the search.
	Find([]byte) ([]string, error)
}

Language is an interface that combines the patch.Language and generate.Language interfaces, adding methods to retrieve supported file extensions and find identifiers within source code. It is used by JotBot to work with different programming languages, allowing for language-specific operations such as finding identifiers and generating patches.

type Option

type Option func(*JotBot)

Option is a function that takes a pointer to a JotBot and modifies its configuration. It can be used to set custom languages, loggers, or filters for the JotBot instance.

func Match

func Match(filters ...*regexp.Regexp) Option

Match appends the provided regular expression filters to the JotBot's filters. These filters are used to determine which findings should be included in the final result when searching for identifiers in files.

func WithLanguage

func WithLanguage(name string, lang Language) Option

WithLanguage configures a JotBot instance to use the specified language for processing files with the given name. It associates the language with its supported file extensions.

func WithLogger

func WithLogger(h slog.Handler) Option

WithLogger configures a JotBot to use the provided slog.Handler for logging.

type Patch

type Patch struct {
	*patch.Patch
	// contains filtered or unexported fields
}

Patch is a wrapper around a patch.Patch that provides additional functionality for applying and dry-running patches using Language implementations. It is used to generate, apply, and preview changes to source code based on the provided findings.

func (*Patch) Apply

func (p *Patch) Apply(ctx context.Context, root string) error

Apply applies the patch to the filesystem rooted at the given root path, using the configured languages for applying changes. The operation can be canceled through the provided context. It returns an error if any issues are encountered during the application process.

func (*Patch) DryRun

func (p *Patch) DryRun(ctx context.Context, root string) (map[string][]byte, error)

DryRun applies the patch to a virtual filesystem rooted at the provided root path, returning a map of file paths to their new contents without modifying the actual filesystem. The operation can be canceled through the provided context.