github

package
v0.0.0-...-0bec3fc Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 26, 2024 License: BSD-3-Clause Imports: 30 Imported by: 0

Documentation

Overview

Package github implements sync mechanism to mirror GitHub issue state into a storage.DB as well as code to inspect that state and to make issue changes on GitHub. All the functionality is provided by the Client, created by New.

Index

Constants

View Source 
const DocWatcherID = "githubdocs"

Variables

This section is empty.

Functions

func CleanBody 

func CleanBody(body string) string

CleanBody should clean the body for indexing. For now we assume the LLM is good enough at Markdown not to bother. In the future we may want to make various changes like inlining the programs associated with playground URLs, and we may also want to remove any HTML tags from the Markdown.

func CleanTitle 

func CleanTitle(title string) string

CleanTitle should clean the title for indexing. For now we assume the LLM is good enough at Markdown not to bother.

func ParseIssueURL 

func ParseIssueURL(u string) (project string, number int64, err error)

ParseIssueURL expects a GitHUB API URL for an issue (for example "https://api.github.com/repos/org/r/issues/123") and returns the project (for example "org/r") and issue number.

func Scrub 

func Scrub(req *http.Request) error

Scrub is a scrubber for use with rsc.io/httprr when writing tests that access GitHub through an httprr.RecordReplay. It removes auth credentials from the request.

func Token 

func Token(sdb secret.DB) string

Token returns the secret for "api.github.com".

func ValidWebhookTestdata 

func ValidWebhookTestdata(t *testing.T, event WebhookEventType, payload any) (*http.Request, secret.DB)

ValidWebhookTestdata returns an HTTP request and a secret DB (inputs to ValidateWebhookRequest) that will pass validation. payload is marshaled into JSON as the body of the returned request.

For testing.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

A Client is a connection to GitHub state in a database and on GitHub itself.

func New 

func New(lg *slog.Logger, db storage.DB, sdb secret.DB, hc *http.Client) *Client

New returns a new client that uses the given logger, databases, and HTTP client.

The secret database is expected to have a secret named "api.github.com" of the form "user:pass" where user is a user-name (ignored by GitHub) and pass is an API token ("ghp_...").

func (*Client) Add 

func (c *Client) Add(project string) error

Add adds a GitHub project of the form "owner/repo" (for example "golang/go") to the database. It only adds the project sync metadata. The initial data fetch does not happen until [Sync] or [SyncProject] is called. If the project is already present, Add does nothing and returns nil.

func (*Client) DocWatcher 

func (c *Client) DocWatcher() *timed.Watcher[*Event]

DocWatcher returns the event watcher with name "githubdocs". Implements docs.Source.DocWatcher.

func (*Client) DownloadIssue 

func (c *Client) DownloadIssue(ctx context.Context, url string) (*Issue, error)

DownloadIssue downloads the current issue JSON from the given URL and decodes it into an issue. Given an issue, c.DownloadIssue(issue.URL) fetches the very latest state for the issue.

func (*Client) DownloadIssueComment 

func (c *Client) DownloadIssueComment(ctx context.Context, url string) (*IssueComment, error)

DownloadIssueComment downloads the current comment JSON from the given URL and decodes it into an IssueComment. Given a comment, c.DownloadIssueComment(comment.URL) fetches the very latest state for the comment.

func (*Client) EditIssue 

func (c *Client) EditIssue(ctx context.Context, issue *Issue, changes *IssueChanges) error

EditIssue applies the changes to issue on GitHub.

func (*Client) EditIssueComment 

func (c *Client) EditIssueComment(ctx context.Context, comment *IssueComment, changes *IssueCommentChanges) error

EditIssueComment changes the comment on GitHub to have the new body. It is typically a good idea to use c.DownloadIssueComment first and check that the live comment body matches the one obtained from the database, to minimize race windows.

func (*Client) EnableTesting 

func (c *Client) EnableTesting()

EnableTesting enables testing mode, in which edits are diverted and a TestingClient is available. If the program is itself a test binary (built or run using “go test”), testing mode is enabled automatically. EnableTesting can be useful in experimental programs to make sure that no edits are applied to GitHub.

func (*Client) EventWatcher 

func (c *Client) EventWatcher(name string) *timed.Watcher[*Event]

EventWatcher returns a new timed.Watcher with the given name. It picks up where any previous Watcher of the same name left off.

func (*Client) Events 

func (c *Client) Events(project string, issueMin, issueMax int64) iter.Seq[*Event]

Events returns an iterator over issue events for the given project, limited to issues in the range issueMin ≤ issue ≤ issueMax. If issueMax < 0, there is no upper limit. The events are iterated over in (Project, Issue, API, ID) order, so "/issues" events come first, then "/issues/comments", then "/issues/events". Within a specific API, the events are ordered by increasing ID, which corresponds to increasing event time on GitHub.

func (*Client) EventsAfter 

func (c *Client) EventsAfter(t timed.DBTime, project string) iter.Seq[*Event]

EventsAfter returns an iterator over events in the given project after DBTime t, which should be e.DBTime from the most recent processed event. The events are iterated over in DBTime order, so the DBTime of the last successfully processed event can be used in a future call to EventsAfter. If project is the empty string, then events from all projects are returned.

func (*Client) LookupIssueURL 

func (c *Client) LookupIssueURL(url string) (*Issue, error)

LookupIssueURL looks up an issue by URL (for example "https://github.com/golang/go/issues/12345"), only consulting the database (not actual GitHub).

func (*Client) PostIssueComment 

func (c *Client) PostIssueComment(ctx context.Context, issue *Issue, changes *IssueCommentChanges) (id, url string, err error)

PostIssueComment posts a new comment with the given body (written in Markdown) on issue. It returns an API URL for the new comment, and a URL suitable for display.

func (*Client) Sync 

func (c *Client) Sync(ctx context.Context) error

Sync syncs all projects.

func (*Client) SyncProject 

func (c *Client) SyncProject(ctx context.Context, project string) (err error)

SyncProject syncs a single project.

func (*Client) Testing 

func (c *Client) Testing() *TestingClient

Testing returns a TestingClient, which provides access to Client functionality intended for testing. Testing only returns a non-nil TestingClient in testing mode, which is active if the current program is a test binary (that is, testing.Testing returns true) or if Client.EnableTesting has been called. Otherwise, Testing returns nil.

Each Client has only one TestingClient associated with it. Every call to Testing returns the same TestingClient.

func (*Client) ToDocs 

func (*Client) ToDocs(e *Event) (iter.Seq[*docs.Doc], bool)

ToDocs converts an event containing an issue to an embeddable document. It returns (nil, false) if the event is not an issue. Implements docs.Source.ToDocs.

type Event 

type Event struct {
	DBTime  timed.DBTime // when event was last written
	Project string       // project ("golang/go")
	Issue   int64        // issue number
	API     string       // API endpoint for event: "/issues", "/issues/comments", or "/issues/events"
	ID      int64        // ID of event; each API has a different ID space. (Project, Issue, API, ID) is assumed unique
	JSON    []byte       // JSON for the event data
	Typed   any          // Typed unmarshaling of the event data, of type *Issue, *IssueComment, or *IssueEvent
}

An Event is a single GitHub issue event stored in the database.

func (*Event) LastWritten 

func (e *Event) LastWritten() timed.DBTime

LastWritten implements docs.Entry.LastWritten.

type Issue 

type Issue struct {
	URL              string    `json:"url"`
	HTMLURL          string    `json:"html_url"`
	Number           int64     `json:"number"`
	User             User      `json:"user"`
	Title            string    `json:"title"`
	CreatedAt        string    `json:"created_at"`
	UpdatedAt        string    `json:"updated_at"`
	ClosedAt         string    `json:"closed_at"`
	Body             string    `json:"body"`
	Assignees        []User    `json:"assignees"`
	Milestone        Milestone `json:"milestone"`
	State            string    `json:"state"`
	PullRequest      *struct{} `json:"pull_request"`
	Locked           bool      `json:"locked"`
	ActiveLockReason string    `json:"active_lock_reason"`
	Labels           []Label   `json:"labels"`
}

Issue is the GitHub JSON structure for an issue creation event.

func LookupIssue 

func LookupIssue(db storage.DB, project string, issue int64) (*Issue, error)

LookupIssue looks up an issue by project and issue number (for example "golang/go", 12345), only consulting the database (not actual GitHub).

func (*Issue) Author 

func (x *Issue) Author() *model.Identity

func (*Issue) Body_ 

func (x *Issue) Body_() string

func (*Issue) CanEdit 

func (x *Issue) CanEdit() bool

func (*Issue) CanHaveChildren 

func (x *Issue) CanHaveChildren() bool

func (*Issue) CreatedAt_ 

func (x *Issue) CreatedAt_() time.Time

func (*Issue) DocID 

func (i *Issue) DocID() string

DocID returns the ID of this issue for storage in a docs.Corpus or a storage.VectorDB.

func (*Issue) ID 

func (x *Issue) ID() string

Methods implementing model.Post.

func (*Issue) ParentID 

func (x *Issue) ParentID() string

func (*Issue) Project 

func (x *Issue) Project() string

Project returns the issue's GitHub project (for example, "golang/go").

func (*Issue) Properties 

func (x *Issue) Properties() map[string]any

func (*Issue) Title_ 

func (x *Issue) Title_() string

func (*Issue) UpdatedAt_ 

func (x *Issue) UpdatedAt_() time.Time

func (*Issue) Updates 

func (x *Issue) Updates() model.PostUpdates

type IssueChanges 

type IssueChanges struct {
	Title  string    `json:"title,omitempty"`
	Body   string    `json:"body,omitempty"`
	State  string    `json:"state,omitempty"`
	Labels *[]string `json:"labels,omitempty"`
}

An IssueChanges specifies changes to make to an issue. Fields that are the empty string or a nil pointer are ignored.

Note that Labels is the new set of all labels for the issue, not labels to add. If you are adding a single label, you need to include all the existing labels as well. Labels is a *[]string so that it can be set to new([]string) to clear the labels.

func (*IssueChanges) SetBody 

func (ch *IssueChanges) SetBody(s string) error

func (*IssueChanges) SetTitle 

func (ch *IssueChanges) SetTitle(s string) error

type IssueComment 

type IssueComment struct {
	URL       string `json:"url"`
	IssueURL  string `json:"issue_url"`
	HTMLURL   string `json:"html_url"`
	User      User   `json:"user"`
	CreatedAt string `json:"created_at"`
	UpdatedAt string `json:"updated_at"`
	Body      string `json:"body"`
}

IssueComment is the GitHub JSON structure for an issue comment event.

func (*IssueComment) Author 

func (x *IssueComment) Author() *model.Identity

func (*IssueComment) Body_ 

func (x *IssueComment) Body_() string

func (*IssueComment) CanEdit 

func (x *IssueComment) CanEdit() bool

func (*IssueComment) CanHaveChildren 

func (x *IssueComment) CanHaveChildren() bool

func (*IssueComment) CommentID 

func (x *IssueComment) CommentID() int64

CommentID returns the issue comment's numeric ID. The ID appears to be unique across all comments on GitHub, but we only assume it is unique within a single issue.

func (*IssueComment) CreatedAt_ 

func (x *IssueComment) CreatedAt_() time.Time

func (*IssueComment) ID 

func (x *IssueComment) ID() string

Methods implementing model.Post.

func (*IssueComment) Issue 

func (x *IssueComment) Issue() int64

Issue returns the issue comment's issue number.

func (*IssueComment) ParentID 

func (x *IssueComment) ParentID() string

func (*IssueComment) Project 

func (x *IssueComment) Project() string

Project returns the issue comment's GitHub project (for example, "golang/go").

func (*IssueComment) Properties 

func (x *IssueComment) Properties() map[string]any

func (*IssueComment) Title_ 

func (x *IssueComment) Title_() string

func (*IssueComment) UpdatedAt_ 

func (x *IssueComment) UpdatedAt_() time.Time

func (*IssueComment) Updates 

func (x *IssueComment) Updates() model.PostUpdates

type IssueCommentChanges 

type IssueCommentChanges struct {
	Body string `json:"body,omitempty"`
}

func (*IssueCommentChanges) SetBody 

func (ch *IssueCommentChanges) SetBody(s string) error

func (*IssueCommentChanges) SetTitle 

func (ch *IssueCommentChanges) SetTitle(string) error

type IssueEvent 

type IssueEvent struct {
	// NOTE: Issue field is not present when downloading for a specific issue,
	// only in the master feed for the whole repo. So do not add it here.
	ID         int64     `json:"id"`
	URL        string    `json:"url"`
	Actor      User      `json:"actor"`
	Event      string    `json:"event"`
	Labels     []Label   `json:"labels"`
	LockReason string    `json:"lock_reason"`
	CreatedAt  string    `json:"created_at"`
	CommitID   string    `json:"commit_id"`
	Assigner   User      `json:"assigner"`
	Assignees  []User    `json:"assignees"`
	Milestone  Milestone `json:"milestone"`
	Rename     Rename    `json:"rename"`
}

IssueEvent is the GitHub JSON structure for an issue metadata event.

type IssueOverviewResult 

type IssueOverviewResult struct {
	*Issue            // the issue itself
	NewComments   int // number of new comments for this issue (for [CommentsAfterOverview])
	TotalComments int // number of comments for this issue

	Overview *llmapp.OverviewResult // the LLM-generated issue and comment summary
}

IssueOverviewResult is the result of IssueOverview or UpdateOverview. It contains the generated overview and metadata about the issue.

func IssueOverview 

func IssueOverview(ctx context.Context, lc *llmapp.Client, db storage.DB, project string, issue int64) (*IssueOverviewResult, error)

IssueOverview returns an LLM-generated overview of the issue and its comments. It does not make any requests to GitHub; the issue and comment data must already be stored in db.

func UpdateOverview 

func UpdateOverview(ctx context.Context, lc *llmapp.Client, db storage.DB,
	project string, issue int64, lastRead int64) (*IssueOverviewResult, error)

UpdateOverview returns an LLM-generated overview of the issue and its comments, separating the comments into "old" and "new" groups broken by the specifed lastRead comment id. (The lastRead comment itself is considered "old", and must be present in the database). UpdateOverview does not make any requests to GitHub; the issue and comment data must already be stored in db.

type Label 

type Label struct {
	Name string `json:"name"`
}

A Label represents a project issue tracker label in GitHub JSON.

type Milestone 

type Milestone struct {
	Title string `json:"title"`
}

A Milestone represents a project issue milestone in GitHub JSON.

type Rename 

type Rename struct {
	From string `json:"from"`
	To   string `json:"to"`
}

A Rename describes an issue title renaming in GitHub JSON.

type Repository 

type Repository struct {
	Project string `json:"full_name"`
}

Repository is the repository in which an event occurred. https://docs.github.com/en/rest/repos/repos?apiVersion=2022-11-28#get-a-repository

type TestingClient 

type TestingClient struct {
	// contains filtered or unexported fields
}

A TestingClient provides access to Client functionality intended for testing.

See Client.Testing for a description of testing mode.

func (*TestingClient) AddIssue 

func (tc *TestingClient) AddIssue(project string, issue *Issue)

AddIssue adds the given issue to the identified project, assigning it a new issue number starting at 10⁹. AddIssue creates a new entry in the associated Client's underlying database, so other Client's using the same database will see the issue too.

NOTE: Only one TestingClient should be adding issues, since they do not coordinate in the database about ID assignment. Perhaps they should, but normally there is just one Client.

func (*TestingClient) AddIssueComment 

func (tc *TestingClient) AddIssueComment(project string, issue int64, comment *IssueComment) int64

AddIssueComment adds the given issue comment to the identified project issue, assigning it a new comment ID starting at 10¹⁰. AddIssueComment creates a new entry in the associated Client's underlying database, so other Client's using the same database will see the issue comment too.

NOTE: Only one TestingClient should be adding issues, since they do not coordinate in the database about ID assignment. Perhaps they should, but normally there is just one Client.

func (*TestingClient) AddIssueEvent 

func (tc *TestingClient) AddIssueEvent(project string, issue int64, event *IssueEvent)

AddIssueEvent adds the given issue event to the identified project issue, assigning it a new comment ID starting at 10¹¹. AddIssueEvent creates a new entry in the associated Client's underlying database, so other Client's using the same database will see the issue event too.

NOTE: Only one TestingClient should be adding issues, since they do not coordinate in the database about ID assignment. Perhaps they should, but normally there is just one Client.

func (*TestingClient) ClearEdits 

func (tc *TestingClient) ClearEdits()

ClearEdits clears the list of edits that are meant to be applied

func (*TestingClient) Edits 

func (tc *TestingClient) Edits() []*TestingEdit

Edits returns a list of all the edits that have been applied using Client methods (for example Client.EditIssue, Client.EditIssueComment, Client.PostIssueComment). These edits have not been applied on GitHub, only diverted into the TestingClient.

See Client.Testing for a description of testing mode.

NOTE: These edits are not applied to the underlying database, since they are also not applied to the underlying database when using a real connection to GitHub; instead we wait for the next sync to download GitHub's view of the edits. See Client.EditIssue.

func (*TestingClient) LoadTxtar 

func (tc *TestingClient) LoadTxtar(file string) error

LoadTxtar loads issue histories from the named txtar file, writing them to the database using TestingClient.AddIssue, TestingClient.AddIssueComment, and TestingClient.AddIssueEvent.

The file should contain a txtar archive (see golang.org/x/tools/txtar). Each file in the archive should be named “project#n” (for example “golang/go#123”) and contain an issue history in the format printed by the rsc.io/github/issue command. See the file ../testdata/rsctmp.txt for an example.

To download a specific set of issues into a new file, you can use a script like: 

go install rsc.io/github/issue@latest
project=golang/go
(for i in 1 2 3 4 5
do
	echo "-- $project#$i --"
	issue -p $project $i
done) > testdata/proj.txt

func (*TestingClient) LoadTxtarData 

func (tc *TestingClient) LoadTxtarData(data []byte) error

LoadTxtarData loads issue histories from the txtar file content data. See [LoadTxtar] for a description of the format.

type TestingEdit 

type TestingEdit struct {
	Project             string
	Issue               int64
	Comment             int64
	IssueChanges        *IssueChanges
	IssueCommentChanges *IssueCommentChanges
}

A TestingEdit is a diverted edit, which was logged instead of actually applied on GitHub.

func (*TestingEdit) String 

func (e *TestingEdit) String() string

String returns a basic string representation of the edit.

type User 

type User struct {
	Login string `json:"login"`
}

A User represents a user or organization account in GitHub JSON.

type WebhookEvent 

type WebhookEvent struct {
	// Type specifies the type of event's Payload.
	Type WebhookEventType
	// Payload is the unmarshaled JSON payload of the webhook event,
	// of the Go type specified by Type.
	//
	// Event types that are not implemented use Go type [json.RawMessage].
	Payload any
}

WebhookEvent contains the data sent in a GitHub webhook request that is relevant for responding to the event.

func ValidateWebhookRequest 

func ValidateWebhookRequest(r *http.Request, db secret.DB) (*WebhookEvent, error)

ValidateWebhookRequest verifies that the request's payload matches the HMAC tag in the header and returns a WebhookEvent containing the unmarshaled payload.

The function is intended to validate authenticated POST requests received from GitHub webhooks.

It expects:

  • a POST request
  • an "X-GitHub-Event" header entry with a non-empty event type (see WebhookEventType)
  • a non-empty request body containing valid JSON representing an event of the specified event type
  • an "X-Hub-Signature-256" header entry of the form "sha256=HMAC", where HMAC is a valid hex-encoded HMAC tag of the request body computed with the key in db named "github-webhook"

The function returns an error if any of these conditions is not met.

func (*WebhookEvent) Project 

func (e *WebhookEvent) Project() string

Project returns the GitHub project (e.g., "golang/go") for the event, or an empty string if the project cannot be determined.

type WebhookEventType 

type WebhookEventType string

WebhookEventType is the name GitHub uses to refer to a type of GitHub webhook event.

Event types that are not implemented use Go type json.RawMessage.

See https://docs.github.com/en/webhooks/webhook-events-and-payloads for all possible event types. 

const (
	// An issue event.
	// Corresponds to Go type [*WebhookIssueEvent].
	WebhookEventTypeIssue WebhookEventType = "issues"
	// An issue comment event.
	// Corresponds to Go type [*WebhookIssueCommentEvent].
	WebhookEventTypeIssueComment WebhookEventType = "issue_comment"
)

type WebhookIssueAction 

type WebhookIssueAction string
const (
	WebhookIssueActionOpened WebhookIssueAction = "opened"
)

type WebhookIssueCommentAction 

type WebhookIssueCommentAction string
const (
	WebhookIssueCommentActionCreated WebhookIssueCommentAction = "created"
)

type WebhookIssueCommentEvent 

type WebhookIssueCommentEvent struct {
	Action     WebhookIssueCommentAction `json:"action"`
	Issue      Issue                     `json:"issue"`
	Repository Repository                `json:"repository"`
}

WebhookIssueEvent is the structure of the JSON payload for a GitHub "issue_comment" event (for example, a new comment posted). https://docs.github.com/en/webhooks/webhook-events-and-payloads#issue_comment

type WebhookIssueEvent 

type WebhookIssueEvent struct {
	Action     WebhookIssueAction `json:"action"`
	Issue      Issue              `json:"issue"`
	Repository Repository         `json:"repository"`
}

WebhookIssueEvent is the structure of the JSON payload for a GitHub "issues" event (for example, a new issue created). https://docs.github.com/en/webhooks/webhook-events-and-payloads#issues

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.