git

package module
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 18, 2024 License: MIT Imports: 41 Imported by: 0

README

git-pr-logo

pico/git-pr a self-hosted git collaboration server

We are trying to build the simplest git collaboration tool. The goal is to make self-hosting a git server as simple as running an SSH server -- all without sacrificing external collaborators time and energy.

git format-patch isn't the problem and pull requests aren't the solution.

We are combining mailing list and pull request workflows. In order to build the simplest collaboration tool, we needed something as simple as generating patches but the ease-of-use of pull requests.

The goal is not to create another code forge here. The goal is to create a very simple self-hosted git solution with the ability to collaborate with external contributors. All the code owner needs to setup a running git server:

  • A single golang binary

All an external contributor needs is:

  • An SSH keypair
  • An SSH client

demo video

https://youtu.be/d28Dih-BBUw

the problem

Email is great as a decentralized system to send and receive changes (patchsets) to a git repo. However, onboarding a new user to a mailing list, properly setting up their email client, and then finally submitting the code contribution is enough to make many developers give up. Further, because we are leveraging the email protocol for collaboration, we are limited by its feature-set. For example, it is not possible to make edits to emails, everyone has a different client, those clients have different limitations around plain text email and downloading patches from it.

Github pull requests are easy to use, easy to edit, and easy to manage. The downside is it forces the user to be inside their website to perform reviews. For quick changes, this is great, but when you start reading code within a web browser, there are quite a few downsides. At a certain point, it makes more sense to review code inside your local development environment, IDE, etc. There are tools and plugins that allow users to review PRs inside their IDE, but it requires a herculean effort to make it usable.

Further, self-hosted solutions that mimic a pull request require a lot of infrastructure in order to manage it. A database, a web site connected to git, admin management, and services to manage it all. Another big point of friction: before an external user submits a code change, they first need to create an account and then login. This adds quite a bit of friction for a self-hosted solution, not only for an external contributor, but also for the code owner who has to provision the infra. Often times they also have to fork the repo within the code forge before submitting a PR. Then they never make a contribution ever again and a forked repo lingers. That seems silly.

introducing patch requests (PR)

Instead, we want to create a self-hosted git "server" that can handle sending and receiving patches without the cumbersome nature of setting up email or the limitations imposed by the email protocol. Further, we want the primary workflow to surround the local development environment. Github is bringing the IDE to the browser in order to support their workflow, we want to flip that idea on its head by making code reviews a first-class citizen inside your local development environment. This has an interesting side-effect: the owner is placed in a more collaborative role because they must create at least one patch to submit a review. They are already in their local editor, they are already creating a git commit and "pushing" it, so naturally it is easier to make code changes during the review itself.

We see this as a hybrid between the github workflow of a pull request and sending and receiving patches over email.

The basic idea is to leverage an SSH app to handle most of the interaction between contributor and owner of a project. Everything can be done completely within the terminal, in a way that is ergonomic and fully featured.

The web view is mostly for discovery.

Notifications would happen with RSS and all state mutations would result in the generation of static web assets so the web views can be hosted using a simple web file server.

format-patch workflow

# Owner hosts repo `test.git` using github

# Contributor clones repo
git clone git@github.com:picosh/test.git

# Contributor wants to make a change
# Contributor makes changes via commits
git add -A && git commit -m "fix: some bugs"

# Contributor runs:
git format-patch origin/main --stdout | ssh pr.pico.sh pr create test
# > Patch Request has been created (ID: 1)

# Owner can checkout patch:
ssh pr.pico.sh pr print 1 | git am -3
# Owner can comment (IN CODE), commit, then send another format-patch
# on top of the PR:
git format-patch origin/main --stdout | ssh pr.pico.sh pr add --review 1
# UI clearly marks patch as a review

# Contributor can checkout reviews
ssh pr.pico.sh pr print 1 | git am -3

# Owner can reject a pr:
ssh pr.pico.sh pr close 1

# Owner can accept a pr:
ssh pr.pico.sh pr accept 1

# Owner can prep PR for upstream:
git rebase -i origin/main

# Then push to upstream
git push origin main

# Done!

The fundamental collaboration tool here is format-patch. Whether you a submitting code changes or you are reviewing code changes, it all happens in code. Both contributor and owner are simply creating new commits and generating patches on top of each other. This obviates the need to have a web viewer where the reviewer can "comment" on a line of code block. There's no need, apply the contributor's patches, write comments or code changes, generate a new patch, send the patch to the git server as a "review." This flow also works the exact same if two users are collaborating on a set of changes.

This also solves the problem of sending multiple patchsets for the same code change. There's a single, central Patch Request where all changes and collaboration happens.

We could figure out a way to leverage git notes for reviews / comments, but honestly, that solution feels brutal and outside the comfort level of most git users. Just send reviews as code and write comments in the programming language you are using. It's the job of the contributor to "address" those comments and then remove them in subsequent patches. This is the forcing function to address all comments: the patch won't be merged if there are comment unaddressed in code; they cannot be ignored or else they will be upstreamed erroneously.

installation and setup

setup

Copy or create a git-pr.toml file inside ./data directory:

mkdir data
vim ./data/git-pr.toml
# configure file

docker

Run the ssh app image:

docker run -d -v ./data:/app/data ghcr.io/picosh/pico/git-ssh:latest

Run the web app image:

docker run -d -v ./data:/app/data ghcr.io/picosh/pico/git-web:latest

golang

Clone this repo and then build the go binaries:

make build

./build/ssh --config ./data/git-pr.toml

./build/web --config ./data/git-pr.toml

done!

Access the ssh app:

ssh -p 2222 localhost help

Access the web app:

curl localhost:3000

roadmap

[!IMPORTANT]
This project is being actively developed and we have not reached alpha status yet.

  1. Better --cover-letter support
  2. Support git range-diff workflow (in web view?)
  3. More robust ACL rules (OR integrate with self-hosted git repos like gitolite)
  4. Adapter to statically generate web view
  5. Bulk modify PRs (rsync, sftp, sshfs)
  6. Generate event log summary as a cover letter?
  7. Support a diff workflow (convert git diff into mbox patch format)
  8. Pubsub system to send events

ideas

  1. Officially support git remotes?
  2. PR build steps? (e.g. ci/cd, status checks, merge checks)
  3. TUI?

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrPatchExists = errors.New("patch already exists for patch request")

Functions

func AuthorDateToTime 

func AuthorDateToTime(date string, logger *slog.Logger) time.Time

func GitPatchRequestMiddleware 

func GitPatchRequestMiddleware(be *Backend, pr GitPatchRequest) wish.Middleware

func GitSshServer 

func GitSshServer(cfg *GitCfg)

func NewCli 

func NewCli(sesh ssh.Session, be *Backend, pr GitPatchRequest) *cli.App

func NewTabWriter 

func NewTabWriter(out io.Writer) *tabwriter.Writer

func StartWebServer 

func StartWebServer(cfg *GitCfg)

Types

type Acl 

type Acl struct {
	ID         int64          `db:"id"`
	Pubkey     sql.NullString `db:"pubkey"`
	IpAddress  sql.NullString `db:"ip_address"`
	Permission string         `db:"permission"`
	CreatedAt  time.Time      `db:"created_at"`
}

Acl is a db model for access control.

type Backend 

type Backend struct {
	Logger *slog.Logger
	DB     *DB
	Cfg    *GitCfg
}

func (*Backend) IsAdmin 

func (be *Backend) IsAdmin(pk ssh.PublicKey) bool

func (*Backend) IsPrOwner 

func (be *Backend) IsPrOwner(pka, pkb int64) bool

func (*Backend) KeyForFingerprint 

func (be *Backend) KeyForFingerprint(pk ssh.PublicKey) string

func (*Backend) KeyForKeyText 

func (be *Backend) KeyForKeyText(pk ssh.PublicKey) string

func (*Backend) KeysEqual 

func (be *Backend) KeysEqual(pka, pkb string) bool

func (*Backend) Pubkey 

func (be *Backend) Pubkey(pk ssh.PublicKey) string

func (*Backend) PubkeyToPublicKey 

func (be *Backend) PubkeyToPublicKey(pubkey string) (ssh.PublicKey, error)

func (*Backend) RepoID 

func (be *Backend) RepoID(name string) string

func (*Backend) RepoName 

func (be *Backend) RepoName(id string) string

func (*Backend) ReposDir 

func (be *Backend) ReposDir() string

type DB 

type DB struct {
	*sqlx.DB
	// contains filtered or unexported fields
}

DB is the interface for a pico/git database.

func Open 

func Open(dsn string, logger *slog.Logger) (*DB, error)

Open opens a database connection.

func (*DB) Close 

func (d *DB) Close() error

Close implements db.DB.

type EventLog 

type EventLog struct {
	ID             int64         `db:"id"`
	UserID         int64         `db:"user_id"`
	RepoID         string        `db:"repo_id"`
	PatchRequestID sql.NullInt64 `db:"patch_request_id"`
	PatchsetID     sql.NullInt64 `db:"patchset_id"`
	Event          string        `db:"event"`
	Data           string        `db:"data"`
	CreatedAt      time.Time     `db:"created_at"`
}

EventLog is a event log for RSS or other notification systems.

type EventLogData 

type EventLogData struct {
	*EventLog
	FormattedPatchsetID string
	UserName            string
	Pubkey              string
	Date                string
}

type GitCfg 

type GitCfg struct {
	DataDir    string          `koanf:"data_dir"`
	Repos      []*Repo         `koanf:"repo"`
	Url        string          `koanf:"url"`
	Host       string          `koanf:"host"`
	SshPort    string          `koanf:"ssh_port"`
	WebPort    string          `koanf:"web_port"`
	AdminsStr  []string        `koanf:"admins"`
	Admins     []ssh.PublicKey `koanf:"admins_pk"`
	Theme      string          `koanf:"theme"`
	TimeFormat string          `koanf:"time_format"`
	Logger     *slog.Logger
}

func NewGitCfg 

func NewGitCfg(fpath string, logger *slog.Logger) *GitCfg

type GitPatchRequest 

type GitPatchRequest interface {
	GetUsers() ([]*User, error)
	GetUserByID(userID int64) (*User, error)
	GetUserByPubkey(pubkey string) (*User, error)
	UpsertUser(pubkey, name string) (*User, error)
	IsBanned(pubkey, ipAddress string) error
	GetRepos() ([]*Repo, error)
	GetReposWithLatestPr() ([]RepoWithLatestPr, error)
	GetRepoByID(repoID string) (*Repo, error)
	SubmitPatchRequest(repoID string, userID int64, patchset io.Reader) (*PatchRequest, error)
	SubmitPatchset(prID, userID int64, op PatchsetOp, patchset io.Reader) ([]*Patch, error)
	GetPatchRequestByID(prID int64) (*PatchRequest, error)
	GetPatchRequests() ([]*PatchRequest, error)
	GetPatchRequestsByRepoID(repoID string) ([]*PatchRequest, error)
	GetPatchsetsByPrID(prID int64) ([]*Patchset, error)
	GetLatestPatchsetByPrID(prID int64) (*Patchset, error)
	GetPatchesByPatchsetID(prID int64) ([]*Patch, error)
	UpdatePatchRequestStatus(prID, userID int64, status string) error
	UpdatePatchRequestName(prID, userID int64, name string) error
	DeletePatchsetByID(patchsetID int64) error
	CreateEventLog(eventLog EventLog) error
	GetEventLogs() ([]*EventLog, error)
	GetEventLogsByRepoID(repoID string) ([]*EventLog, error)
	GetEventLogsByPrID(prID int64) ([]*EventLog, error)
	GetEventLogsByUserID(userID int64) ([]*EventLog, error)
	DiffPatchsets(aset *Patchset, bset *Patchset) ([]*Patch, error)
}

type LinkData 

type LinkData struct {
	Url  template.URL
	Text string
}

type MetaData 

type MetaData struct {
	URL string
}

type Patch 

type Patch struct {
	ID            int64          `db:"id"`
	UserID        int64          `db:"user_id"`
	PatchsetID    int64          `db:"patchset_id"`
	AuthorName    string         `db:"author_name"`
	AuthorEmail   string         `db:"author_email"`
	AuthorDate    string         `db:"author_date"`
	Title         string         `db:"title"`
	Body          string         `db:"body"`
	BodyAppendix  string         `db:"body_appendix"`
	CommitSha     string         `db:"commit_sha"`
	ContentSha    string         `db:"content_sha"`
	BaseCommitSha sql.NullString `db:"base_commit_sha"`
	RawText       string         `db:"raw_text"`
	CreatedAt     time.Time      `db:"created_at"`
}

Patch is a database model for a single entry in a patchset. This usually corresponds to a git commit.

type PatchData 

type PatchData struct {
	*Patch
	Url                 template.URL
	DiffStr             template.HTML
	Review              bool
	FormattedAuthorDate string
}

type PatchRequest 

type PatchRequest struct {
	ID        int64     `db:"id"`
	UserID    int64     `db:"user_id"`
	RepoID    string    `db:"repo_id"`
	Name      string    `db:"name"`
	Text      string    `db:"text"`
	Status    string    `db:"status"`
	CreatedAt time.Time `db:"created_at"`
	UpdatedAt time.Time `db:"updated_at"`
	// only used for aggregate queries
	LastUpdated string `db:"last_updated"`
}

PatchRequest is a database model for patches submitted to a Repo.

type Patchset 

type Patchset struct {
	ID             int64     `db:"id"`
	UserID         int64     `db:"user_id"`
	PatchRequestID int64     `db:"patch_request_id"`
	Review         bool      `db:"review"`
	CreatedAt      time.Time `db:"created_at"`
}

type PatchsetData 

type PatchsetData struct {
	*Patchset
	FormattedID string
	UserName    string
	Pubkey      string
	Date        string
	DiffPatches []PatchData
}

type PatchsetOp 

type PatchsetOp int
const (
	OpNormal PatchsetOp = iota
	OpReview
)

type PrCmd 

type PrCmd struct {
	Backend *Backend
}

func (PrCmd) CreateEventLog 

func (cmd PrCmd) CreateEventLog(eventLog EventLog) error

func (PrCmd) DeletePatchsetByID 

func (cmd PrCmd) DeletePatchsetByID(patchsetID int64) error

func (PrCmd) DiffPatchsets 

func (cmd PrCmd) DiffPatchsets(prev *Patchset, next *Patchset) ([]*Patch, error)

func (PrCmd) GetEventLogs 

func (cmd PrCmd) GetEventLogs() ([]*EventLog, error)

func (PrCmd) GetEventLogsByPrID 

func (cmd PrCmd) GetEventLogsByPrID(prID int64) ([]*EventLog, error)

func (PrCmd) GetEventLogsByRepoID 

func (cmd PrCmd) GetEventLogsByRepoID(repoID string) ([]*EventLog, error)

func (PrCmd) GetEventLogsByUserID 

func (cmd PrCmd) GetEventLogsByUserID(userID int64) ([]*EventLog, error)

func (PrCmd) GetLatestPatchsetByPrID 

func (pr PrCmd) GetLatestPatchsetByPrID(prID int64) (*Patchset, error)

func (PrCmd) GetPatchRequestByID 

func (cmd PrCmd) GetPatchRequestByID(prID int64) (*PatchRequest, error)

func (PrCmd) GetPatchRequests 

func (cmd PrCmd) GetPatchRequests() ([]*PatchRequest, error)

func (PrCmd) GetPatchRequestsByRepoID 

func (cmd PrCmd) GetPatchRequestsByRepoID(repoID string) ([]*PatchRequest, error)

func (PrCmd) GetPatchesByPatchsetID 

func (pr PrCmd) GetPatchesByPatchsetID(patchsetID int64) ([]*Patch, error)

func (PrCmd) GetPatchsetsByPrID 

func (pr PrCmd) GetPatchsetsByPrID(prID int64) ([]*Patchset, error)

func (PrCmd) GetRepoByID 

func (pr PrCmd) GetRepoByID(repoID string) (*Repo, error)

func (PrCmd) GetRepos 

func (pr PrCmd) GetRepos() ([]*Repo, error)

func (PrCmd) GetReposWithLatestPr 

func (pr PrCmd) GetReposWithLatestPr() ([]RepoWithLatestPr, error)

func (PrCmd) GetUserByID 

func (pr PrCmd) GetUserByID(id int64) (*User, error)

func (PrCmd) GetUserByPubkey 

func (pr PrCmd) GetUserByPubkey(pubkey string) (*User, error)

func (PrCmd) GetUsers 

func (pr PrCmd) GetUsers() ([]*User, error)

func (PrCmd) IsBanned 

func (pr PrCmd) IsBanned(pubkey, ipAddress string) error

func (PrCmd) SubmitPatchRequest 

func (cmd PrCmd) SubmitPatchRequest(repoID string, userID int64, patchset io.Reader) (*PatchRequest, error)

func (PrCmd) SubmitPatchset 

func (cmd PrCmd) SubmitPatchset(prID int64, userID int64, op PatchsetOp, patchset io.Reader) ([]*Patch, error)

func (PrCmd) UpdatePatchRequestName 

func (cmd PrCmd) UpdatePatchRequestName(prID int64, userID int64, name string) error

func (PrCmd) UpdatePatchRequestStatus 

func (cmd PrCmd) UpdatePatchRequestStatus(prID int64, userID int64, status string) error

Status types: open, closed, accepted, reviewed.

func (PrCmd) UpsertUser 

func (pr PrCmd) UpsertUser(pubkey, name string) (*User, error)

type PrData 

type PrData struct {
	ID       int64
	IsAdmin  bool
	Title    string
	Date     string
	UserName string
	Pubkey   string
	Status   string
}

type PrDetailData 

type PrDetailData struct {
	Page      string
	Repo      LinkData
	Pr        PrData
	Patches   []PatchData
	Branch    string
	Logs      []EventLogData
	Patchsets []PatchsetData
	MetaData
}

type PrListData 

type PrListData struct {
	LinkData
	ID       int64
	IsAdmin  bool
	UserName string
	Pubkey   string
	Date     string
	Status   string
}

type PrWithRepo 

type PrWithRepo struct {
	LastUpdatedPrID int64
	RepoID          string
}

type Repo 

type Repo struct {
	ID            string `koanf:"id"`
	Desc          string `koanf:"desc"`
	CloneAddr     string `koanf:"clone_addr"`
	DefaultBranch string `koanf:"default_branch"`
}

type RepoData 

type RepoData struct {
	LinkData
	Desc     string
	LatestPr *PrListData
}

type RepoDetailData 

type RepoDetailData struct {
	ID          string
	CloneAddr   string
	Branch      string
	OpenPrs     []PrListData
	AcceptedPrs []PrListData
	ClosedPrs   []PrListData
	ReviewedPrs []PrListData
	MetaData
}

type RepoListData 

type RepoListData struct {
	Repos []RepoData
	MetaData
}

type RepoWithLatestPr 

type RepoWithLatestPr struct {
	*Repo
	User         *User
	PatchRequest *PatchRequest
}

type User 

type User struct {
	ID        int64     `db:"id"`
	Pubkey    string    `db:"pubkey"`
	Name      string    `db:"name"`
	CreatedAt time.Time `db:"created_at"`
	UpdatedAt time.Time `db:"updated_at"`
}

User is a db model for users.

type WebCtx 

type WebCtx struct {
	Pr        *PrCmd
	Backend   *Backend
	Formatter *formatterHtml.Formatter
	Logger    *slog.Logger
	Theme     *chroma.Style
}

Source Files

View all Source files

Directories

Path Synopsis
cmd
ssh
web

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.