git

package module
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 6, 2024 License: MIT Imports: 46 Imported by: 0

README

git-pr-logo

pico/git-pr a self-hosted git collaboration server

We are trying to build the simplest git collaboration tool. The goal is to make self-hosting a git server as simple as running an SSH server -- all without sacrificing external collaborators time and energy.

git format-patch isn't the problem and pull requests aren't the solution.

We are combining mailing list and pull request workflows. In order to build the simplest collaboration tool, we needed something as simple as generating patches but the ease-of-use of pull requests.

The goal is not to create another code forge here. The goal is to create a very simple self-hosted git solution with the ability to collaborate with external contributors. All the code owner needs to setup a running git server:

  • A single golang binary

All an external contributor needs is:

  • An SSH keypair
  • An SSH client

demo video

https://youtu.be/d28Dih-BBUw

the problem

Email is great as a decentralized system to send and receive changes (patchsets) to a git repo. However, onboarding a new user to a mailing list, properly setting up their email client, and then finally submitting the code contribution is enough to make many developers give up. Further, because we are leveraging the email protocol for collaboration, we are limited by its feature-set. For example, it is not possible to make edits to emails, everyone has a different client, those clients have different limitations around plain text email and downloading patches from it.

Github pull requests are easy to use, easy to edit, and easy to manage. The downside is it forces the user to be inside their website to perform reviews. For quick changes, this is great, but when you start reading code within a web browser, there are quite a few downsides. At a certain point, it makes more sense to review code inside your local development environment, IDE, etc. There are tools and plugins that allow users to review PRs inside their IDE, but it requires a herculean effort to make it usable.

Further, self-hosted solutions that mimic a pull request require a lot of infrastructure in order to manage it. A database, a web site connected to git, admin management, and services to manage it all. Another big point of friction: before an external user submits a code change, they first need to create an account and then login. This adds quite a bit of friction for a self-hosted solution, not only for an external contributor, but also for the code owner who has to provision the infra. Often times they also have to fork the repo within the code forge before submitting a PR. Then they never make a contribution ever again and a forked repo lingers. That seems silly.

introducing patch requests (PR)

Instead, we want to create a self-hosted git "server" that can handle sending and receiving patches without the cumbersome nature of setting up email or the limitations imposed by the email protocol. Further, we want the primary workflow to surround the local development environment. Github is bringing the IDE to the browser in order to support their workflow, we want to flip that idea on its head by making code reviews a first-class citizen inside your local development environment. This has an interesting side-effect: the owner is placed in a more collaborative role because they must create at least one patch to submit a review. They are already in their local editor, they are already creating a git commit and "pushing" it, so naturally it is easier to make code changes during the review itself.

We see this as a hybrid between the github workflow of a pull request and sending and receiving patches over email.

The basic idea is to leverage an SSH app to handle most of the interaction between contributor and owner of a project. Everything can be done completely within the terminal, in a way that is ergonomic and fully featured.

The web view is mostly for discovery.

Notifications would happen with RSS and all state mutations would result in the generation of static web assets so the web views can be hosted using a simple web file server.

format-patch workflow

# Owner hosts repo `test.git` using github

# Contributor clones repo
git clone git@github.com:picosh/test.git

# Contributor wants to make a change
# Contributor makes changes via commits
git add -A && git commit -m "fix: some bugs"

# Contributor runs:
git format-patch origin/main --stdout | ssh pr.pico.sh pr create test
# > Patch Request has been created (ID: 1)

# Owner can checkout patch:
ssh pr.pico.sh pr print 1 | git am -3
# Owner can comment (IN CODE), commit, then send another format-patch
# on top of the PR:
git format-patch origin/main --stdout | ssh pr.pico.sh pr add --review 1
# UI clearly marks patch as a review

# Contributor can checkout reviews
ssh pr.pico.sh pr print 1 | git am -3

# Owner can reject a pr:
ssh pr.pico.sh pr close 1

# Owner can accept a pr:
ssh pr.pico.sh pr accept 1

# Owner can prep PR for upstream:
git rebase -i origin/main

# Then push to upstream
git push origin main

# Done!

The fundamental collaboration tool here is format-patch. Whether you are submitting code changes or reviewing them, it all happens in code. Both contributor and owner are simply creating new commits and generating patches on top of each other. This obviates the need to have a web viewer where the reviewer can "comment" on a line of code block. There's no need, apply the contributor's patches, write comments or code changes, generate a new patch, send the patch to the git server as a "review." This flow also works the exact same if two users are collaborating on a set of changes.

This also solves the problem of sending multiple patchsets for the same code change. There's a single, central Patch Request where all changes and collaboration happens.

We could figure out a way to leverage git notes for reviews / comments, but honestly, that solution feels brutal and outside the comfort level of most git users. Just send reviews as code and write comments in the programming language you are using. It's the job of the contributor to "address" those comments and then remove them in subsequent patches. This is the forcing function to address all comments: the patch won't be merged if there are comment unaddressed in code; they cannot be ignored or else they will be upstreamed erroneously.

installation and setup

setup

Copy or create a git-pr.toml file inside ./data directory:

mkdir data
vim ./data/git-pr.toml
# configure file

docker

Run the ssh app image:

docker run -d -v ./data:/app/data ghcr.io/picosh/pico/git-ssh:latest

Run the web app image:

docker run -d -v ./data:/app/data ghcr.io/picosh/pico/git-web:latest

golang

Clone this repo and then build the go binaries:

make build

./build/ssh --config ./data/git-pr.toml

./build/web --config ./data/git-pr.toml

done!

Access the ssh app:

ssh -p 2222 localhost help

Access the web app:

curl localhost:3000

roadmap

[!IMPORTANT]
This project is being actively developed and we have not reached alpha status yet.

  1. User-provided template files
  2. Better diff algo between patchsets
  3. Generate event log summary as a cover letter?
  4. Support a diff workflow (convert git diff into mbox patch format)
  5. More robust ACL rules (OR integrate with self-hosted git repos like gitolite)
  6. Pubsub system to send events
  7. Adapter to statically generate web view

ideas

  1. TUI?
  2. Officially support git remotes?
  3. PR build steps? (e.g. ci/cd, status checks, merge checks)
  4. Bulk modify PRs? (rsync, sftp, sshfs)

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var COST_MAX = 65536
View Source 
var ErrPatchExists = errors.New("patch already exists for patch request")
View Source 
var ErrRepoNoNamespace = fmt.Errorf("repo must be namespaced by username")
View Source 
var RANGE_DIFF_CREATION_FACTOR_DEFAULT = 60

Functions

func DoDiff added in v0.3.0 

func DoDiff(src, dst string) []diffmatchpatch.Diff

func GetAuthorizedKeys added in v0.3.0 

func GetAuthorizedKeys(pubkeys []string) ([]ssh.PublicKey, error)

func GitPatchRequestMiddleware 

func GitPatchRequestMiddleware(be *Backend, pr GitPatchRequest) wish.Middleware

func GitSshServer 

func GitSshServer(cfg *GitCfg, killCh chan error)

func LoadConfigFile added in v0.3.0 

func LoadConfigFile(fpath string, logger *slog.Logger)

func NewCli 

func NewCli(sesh ssh.Session, be *Backend, pr GitPatchRequest) *cli.App

func NewTabWriter 

func NewTabWriter(out io.Writer) *tabwriter.Writer

func ParsePatch added in v0.3.0 

func ParsePatch(patchRaw string) ([]*gitdiff.File, string, error)

func RangeDiffToStr added in v0.3.0 

func RangeDiffToStr(diffs []*RangeDiffOutput) string

func SqliteOpen added in v0.3.0 

func SqliteOpen(dsn string, logger *slog.Logger) (*sqlx.DB, error)

Open opens a database connection.

func StartWebServer 

func StartWebServer(cfg *GitCfg)

Types

type Acl 

type Acl struct {
	ID         int64          `db:"id"`
	Pubkey     sql.NullString `db:"pubkey"`
	IpAddress  sql.NullString `db:"ip_address"`
	Permission string         `db:"permission"`
	CreatedAt  time.Time      `db:"created_at"`
}

Acl is a db model for access control.

type Backend 

type Backend struct {
	Logger *slog.Logger
	DB     *sqlx.DB
	Cfg    *GitCfg
}

func (*Backend) CanCreateRepo added in v0.3.0 

func (be *Backend) CanCreateRepo(repo *Repo, requester *User) error

func (*Backend) CreateRepoNs added in v0.3.0 

func (be *Backend) CreateRepoNs(userName, repoName string) string

Repo Namespace.

func (*Backend) GetPatchRequestAcl added in v0.3.0 

func (be *Backend) GetPatchRequestAcl(repo *Repo, prq *PatchRequest, requester *User) *PrAcl

func (*Backend) IsAdmin 

func (be *Backend) IsAdmin(pk ssh.PublicKey) bool

func (*Backend) IsPrOwner 

func (be *Backend) IsPrOwner(pka, pkb int64) bool

func (*Backend) KeyForFingerprint 

func (be *Backend) KeyForFingerprint(pk ssh.PublicKey) string

func (*Backend) KeyForKeyText 

func (be *Backend) KeyForKeyText(pk ssh.PublicKey) string

func (*Backend) KeysEqual 

func (be *Backend) KeysEqual(pka, pkb string) bool

func (*Backend) Pubkey 

func (be *Backend) Pubkey(pk ssh.PublicKey) string

func (*Backend) PubkeyToPublicKey 

func (be *Backend) PubkeyToPublicKey(pubkey string) (ssh.PublicKey, error)

func (*Backend) SplitRepoNs added in v0.3.0 

func (be *Backend) SplitRepoNs(repoNs string) (string, string)

func (*Backend) ValidateRepoNs added in v0.3.0 

func (be *Backend) ValidateRepoNs(repoNs string) error

type EventLog 

type EventLog struct {
	ID             int64         `db:"id"`
	UserID         int64         `db:"user_id"`
	RepoID         sql.NullInt64 `db:"repo_id"`
	PatchRequestID sql.NullInt64 `db:"patch_request_id"`
	PatchsetID     sql.NullInt64 `db:"patchset_id"`
	Event          string        `db:"event"`
	Data           string        `db:"data"`
	CreatedAt      time.Time     `db:"created_at"`
}

EventLog is a event log for RSS or other notification systems.

type EventLogData 

type EventLogData struct {
	*EventLog
	UserData
	*Patchset
	FormattedPatchsetID string
	Date                string
}

type GitCfg 

type GitCfg struct {
	DataDir    string          `koanf:"data_dir"`
	Url        string          `koanf:"url"`
	Host       string          `koanf:"host"`
	SshPort    string          `koanf:"ssh_port"`
	WebPort    string          `koanf:"web_port"`
	AdminsStr  []string        `koanf:"admins"`
	Admins     []ssh.PublicKey `koanf:"admins_pk"`
	CreateRepo string          `koanf:"create_repo"`
	Theme      string          `koanf:"theme"`
	TimeFormat string          `koanf:"time_format"`
	Logger     *slog.Logger
}

func NewGitCfg 

func NewGitCfg(logger *slog.Logger) *GitCfg

type GitPatchRequest 

type GitPatchRequest interface {
	GetUsers() ([]*User, error)
	GetUserByID(userID int64) (*User, error)
	GetUserByName(name string) (*User, error)
	GetUserByPubkey(pubkey string) (*User, error)
	GetRepos() ([]*Repo, error)
	GetRepoByID(repoID int64) (*Repo, error)
	GetRepoByName(user *User, repoName string) (*Repo, error)
	CreateRepo(user *User, repoName string) (*Repo, error)
	UpsertUser(pubkey, name string) (*User, error)
	IsBanned(pubkey, ipAddress string) error
	SubmitPatchRequest(repoID int64, userID int64, patchset io.Reader) (*PatchRequest, error)
	SubmitPatchset(prID, userID int64, op PatchsetOp, patchset io.Reader) ([]*Patch, error)
	GetPatchRequestByID(prID int64) (*PatchRequest, error)
	GetPatchRequests() ([]*PatchRequest, error)
	GetPatchRequestsByRepoID(repoID int64) ([]*PatchRequest, error)
	GetPatchsetsByPrID(prID int64) ([]*Patchset, error)
	GetPatchsetByID(patchsetID int64) (*Patchset, error)
	GetLatestPatchsetByPrID(prID int64) (*Patchset, error)
	GetPatchesByPatchsetID(prID int64) ([]*Patch, error)
	UpdatePatchRequestStatus(prID, userID int64, status string) error
	UpdatePatchRequestName(prID, userID int64, name string) error
	DeletePatchsetByID(userID, prID int64, patchsetID int64) error
	CreateEventLog(tx *sqlx.Tx, eventLog EventLog) error
	GetEventLogs() ([]*EventLog, error)
	GetEventLogsByRepoName(user *User, repoName string) ([]*EventLog, error)
	GetEventLogsByPrID(prID int64) ([]*EventLog, error)
	GetEventLogsByUserID(userID int64) ([]*EventLog, error)
	DiffPatchsets(aset *Patchset, bset *Patchset) ([]*RangeDiffOutput, error)
}

type LinkData 

type LinkData struct {
	Url  template.URL
	Text string
}

type MetaData 

type MetaData struct {
	URL string
}

type Patch 

type Patch struct {
	ID            int64          `db:"id"`
	UserID        int64          `db:"user_id"`
	PatchsetID    int64          `db:"patchset_id"`
	AuthorName    string         `db:"author_name"`
	AuthorEmail   string         `db:"author_email"`
	AuthorDate    time.Time      `db:"author_date"`
	Title         string         `db:"title"`
	Body          string         `db:"body"`
	BodyAppendix  string         `db:"body_appendix"`
	CommitSha     string         `db:"commit_sha"`
	ContentSha    string         `db:"content_sha"`
	BaseCommitSha sql.NullString `db:"base_commit_sha"`
	RawText       string         `db:"raw_text"`
	CreatedAt     time.Time      `db:"created_at"`
	Files         []*gitdiff.File
}

Patch is a database model for a single entry in a patchset. This usually corresponds to a git commit.

func ParsePatchset added in v0.3.0 

func ParsePatchset(patchset io.Reader) ([]*Patch, error)

func (*Patch) CalcDiff added in v0.3.0 

func (p *Patch) CalcDiff() string

type PatchData 

type PatchData struct {
	*Patch
	PatchFiles          []*PatchFile
	PatchHeader         *gitdiff.PatchHeader
	Url                 template.URL
	Review              bool
	FormattedAuthorDate string
}

type PatchFile added in v0.3.0 

type PatchFile struct {
	*gitdiff.File
	Adds     int64
	Dels     int64
	DiffText template.HTML
}

type PatchRange added in v0.3.0 

type PatchRange struct {
	*Patch
	Matching int
	Diff     string
	DiffSize int
	Shown    bool
}

func NewPatchRange added in v0.3.0 

func NewPatchRange(patch *Patch) *PatchRange

type PatchRequest 

type PatchRequest struct {
	ID        int64     `db:"id"`
	UserID    int64     `db:"user_id"`
	RepoID    int64     `db:"repo_id"`
	Name      string    `db:"name"`
	Text      string    `db:"text"`
	Status    string    `db:"status"`
	CreatedAt time.Time `db:"created_at"`
	UpdatedAt time.Time `db:"updated_at"`
	// only used for aggregate queries
	LastUpdated string `db:"last_updated"`
}

PatchRequest is a database model for patches submitted to a Repo.

type Patchset 

type Patchset struct {
	ID             int64     `db:"id"`
	UserID         int64     `db:"user_id"`
	PatchRequestID int64     `db:"patch_request_id"`
	Review         bool      `db:"review"`
	CreatedAt      time.Time `db:"created_at"`
}

type PatchsetData 

type PatchsetData struct {
	*Patchset
	UserData
	FormattedID string
	Date        string
	RangeDiff   []*RangeDiffOutput
}

type PatchsetOp 

type PatchsetOp int
const (
	OpNormal PatchsetOp = iota
	OpReview
	OpAccept
	OpClose
)

type PrAcl added in v0.3.0 

type PrAcl struct {
	CanModify bool
	CanReview bool
	CanDelete bool
}

type PrCmd 

type PrCmd struct {
	Backend *Backend
}

func (PrCmd) CreateEventLog 

func (cmd PrCmd) CreateEventLog(tx *sqlx.Tx, eventLog EventLog) error

func (PrCmd) CreateRepo added in v0.3.0 

func (pr PrCmd) CreateRepo(user *User, repoName string) (*Repo, error)

func (PrCmd) DeletePatchsetByID 

func (cmd PrCmd) DeletePatchsetByID(userID int64, prID int64, patchsetID int64) error

func (PrCmd) DiffPatchsets 

func (cmd PrCmd) DiffPatchsets(prev *Patchset, next *Patchset) ([]*RangeDiffOutput, error)

func (PrCmd) GetEventLogs 

func (cmd PrCmd) GetEventLogs() ([]*EventLog, error)

func (PrCmd) GetEventLogsByPrID 

func (cmd PrCmd) GetEventLogsByPrID(prID int64) ([]*EventLog, error)

func (PrCmd) GetEventLogsByRepoName added in v0.3.0 

func (cmd PrCmd) GetEventLogsByRepoName(user *User, repoName string) ([]*EventLog, error)

func (PrCmd) GetEventLogsByUserID 

func (cmd PrCmd) GetEventLogsByUserID(userID int64) ([]*EventLog, error)

func (PrCmd) GetLatestPatchsetByPrID 

func (pr PrCmd) GetLatestPatchsetByPrID(prID int64) (*Patchset, error)

func (PrCmd) GetPatchRequestByID 

func (cmd PrCmd) GetPatchRequestByID(prID int64) (*PatchRequest, error)

func (PrCmd) GetPatchRequests 

func (cmd PrCmd) GetPatchRequests() ([]*PatchRequest, error)

func (PrCmd) GetPatchRequestsByRepoID 

func (cmd PrCmd) GetPatchRequestsByRepoID(repoID int64) ([]*PatchRequest, error)

func (PrCmd) GetPatchesByPatchsetID 

func (pr PrCmd) GetPatchesByPatchsetID(patchsetID int64) ([]*Patch, error)

func (PrCmd) GetPatchsetByID added in v0.3.0 

func (pr PrCmd) GetPatchsetByID(patchsetID int64) (*Patchset, error)

func (PrCmd) GetPatchsetsByPrID 

func (pr PrCmd) GetPatchsetsByPrID(prID int64) ([]*Patchset, error)

func (PrCmd) GetRepoByID 

func (pr PrCmd) GetRepoByID(repoID int64) (*Repo, error)

func (PrCmd) GetRepoByName added in v0.3.0 

func (pr PrCmd) GetRepoByName(user *User, repoName string) (*Repo, error)

func (PrCmd) GetRepos 

func (pr PrCmd) GetRepos() (repos []*Repo, err error)

func (PrCmd) GetUserByID 

func (pr PrCmd) GetUserByID(id int64) (*User, error)

func (PrCmd) GetUserByName added in v0.3.0 

func (pr PrCmd) GetUserByName(name string) (*User, error)

func (PrCmd) GetUserByPubkey 

func (pr PrCmd) GetUserByPubkey(pubkey string) (*User, error)

func (PrCmd) GetUsers 

func (pr PrCmd) GetUsers() ([]*User, error)

func (PrCmd) IsBanned 

func (pr PrCmd) IsBanned(pubkey, ipAddress string) error

func (PrCmd) SubmitPatchRequest 

func (cmd PrCmd) SubmitPatchRequest(repoID int64, userID int64, patchset io.Reader) (*PatchRequest, error)

func (PrCmd) SubmitPatchset 

func (cmd PrCmd) SubmitPatchset(prID int64, userID int64, op PatchsetOp, patchset io.Reader) ([]*Patch, error)

func (PrCmd) UpdatePatchRequestName 

func (cmd PrCmd) UpdatePatchRequestName(prID int64, userID int64, name string) error

func (PrCmd) UpdatePatchRequestStatus 

func (cmd PrCmd) UpdatePatchRequestStatus(prID int64, userID int64, status string) error

Status types: open, closed, accepted, reviewed.

func (PrCmd) UpsertUser 

func (pr PrCmd) UpsertUser(pubkey, name string) (*User, error)

type PrData 

type PrData struct {
	UserData
	ID     int64
	Title  string
	Date   string
	Status string
}

type PrDetailData 

type PrDetailData struct {
	Page      string
	Repo      LinkData
	Pr        PrData
	Patchset  *Patchset
	Patches   []PatchData
	Branch    string
	Logs      []EventLogData
	Patchsets []PatchsetData
	MetaData
}

type PrListData 

type PrListData struct {
	UserData
	RepoNs   string
	RepoLink LinkData
	PrLink   LinkData
	Title    string
	ID       int64
	DateOrig time.Time
	Date     string
	Status   string
}

type PrTableData added in v0.3.0 

type PrTableData struct {
	Prs []*PrListData
	MetaData
}

type RangeDiffOutput added in v0.3.0 

type RangeDiffOutput struct {
	Header string
	Diff   []diffmatchpatch.Diff
	Type   string
}

func RangeDiff added in v0.3.0 

func RangeDiff(a []*Patch, b []*Patch) []*RangeDiffOutput

type Repo 

type Repo struct {
	ID        int64     `db:"id"`
	Name      string    `db:"name"`
	UserID    int64     `db:"user_id"`
	CreatedAt time.Time `db:"created_at"`
	UpdatedAt time.Time `db:"updated_at"`
}

Repo is a container for patch requests.

type RepoDetailData 

type RepoDetailData struct {
	Name     string
	UserID   int64
	Username string
	Branch   string
	Prs      []*PrListData
	MetaData
}

type User 

type User struct {
	ID        int64     `db:"id"`
	Pubkey    string    `db:"pubkey"`
	Name      string    `db:"name"`
	CreatedAt time.Time `db:"created_at"`
	UpdatedAt time.Time `db:"updated_at"`
}

User is a db model for users.

type UserData added in v0.3.0 

type UserData struct {
	UserID    int64
	Name      string
	IsAdmin   bool
	Pubkey    string
	CreatedAt string
}

type UserDetailData added in v0.3.0 

type UserDetailData struct {
	Prs      []*PrListData
	UserData UserData
	MetaData
}

type WebCtx 

type WebCtx struct {
	Pr        *PrCmd
	Backend   *Backend
	Formatter *formatterHtml.Formatter
	Logger    *slog.Logger
	Theme     *chroma.Style
}

Source Files

View all Source files

Directories

Path Synopsis
cmd
ssh
web
contrib
dev

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.