admin

package
v0.37.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 16, 2023 License: Apache-2.0 Imports: 34 Imported by: 0

README

admin

This directory contains the control-plane for multi-user, hosted deployments of Rill.

Running in development

  1. Create a .env file at the root of the repo containing:
RILL_ADMIN_DATABASE_DRIVER=postgres
RILL_ADMIN_DATABASE_URL=postgres://postgres:postgres@localhost:5432/postgres
RILL_ADMIN_HTTP_PORT=8080
RILL_ADMIN_GRPC_PORT=9090
RILL_ADMIN_METRICS_EXPORTER="prometheus"
RILL_ADMIN_TRACES_EXPORTER=""
RILL_ADMIN_EXTERNAL_URL=http://localhost:8080
RILL_ADMIN_FRONTEND_URL=http://localhost:3000
RILL_ADMIN_ALLOWED_ORIGINS=*
# Hex-encoded comma-separated list of key pairs. To generate, run "go run ./scripts/generate_keypairs/main.go"
# For details: https://pkg.go.dev/github.com/gorilla/sessions#NewCookieStore
RILL_ADMIN_SESSION_KEY_PAIRS=7938b8c95ac90b3731c353076daeae8a,90c22a5a6c6b442afdb46855f95eb7d6
# JWKS details for signing JWTs. The JWKS must contain *private* keys. To generate, run "go run ./scripts/generate_jwks/main.go"
RILL_ADMIN_SIGNING_JWKS=
RILL_ADMIN_SIGNING_KEY_ID=
# Get these from https://auth0.com/ (or ask a team member)
RILL_ADMIN_AUTH_DOMAIN=gorillio-stage.auth0.com
RILL_ADMIN_AUTH_CLIENT_ID=
RILL_ADMIN_AUTH_CLIENT_SECRET=
# Get these from https://github.com/ (or ask a team member)
RILL_ADMIN_GITHUB_APP_ID=302634
RILL_ADMIN_GITHUB_APP_NAME=rill-cloud-dev
RILL_ADMIN_GITHUB_APP_PRIVATE_KEY=
RILL_ADMIN_GITHUB_APP_WEBHOOK_SECRET=
RILL_ADMIN_GITHUB_CLIENT_ID=
RILL_ADMIN_GITHUB_CLIENT_SECRET=
# For email client
RILL_ADMIN_EMAIL_SMTP_HOST=
RILL_ADMIN_EMAIL_SMTP_PORT=
RILL_ADMIN_EMAIL_SMTP_USERNAME=
RILL_ADMIN_EMAIL_SMTP_PASSWORD=
RILL_ADMIN_EMAIL_SENDER_EMAIL=
RILL_ADMIN_EMAIL_SENDER_NAME=
RILL_ADMIN_EMAIL_BCC=
  1. In a separate terminal, run Postgres in the background:
docker-compose -f admin/docker-compose.yml up 
# Data is persisted. To clear, run: docker-compose -f admin/docker-compose.yml down --volumes
  1. Run the server:
go run ./cli admin start
  1. Ping the server:
go run ./cli admin ping --url http://localhost:9090

You can now call the local admin server from the CLI by overriding the admin API URL. For example:

go run ./cli org create --api-url http://localhost:9090

Adding endpoints

We define our APIs using gRPC and use gRPC-Gateway to map the RPCs to a RESTful API. See proto/README.md for details.

To add a new endpoint:

  1. Describe the endpoint in proto/rill/admin/v1/api.proto
  2. Re-generate gRPC and OpenAPI interfaces by running make proto.generate
  3. Copy the new handler signature from the AdminServiceServer interface in proto/gen/rill/admin/v1/api_grpc_pb.go
  4. Paste the handler signature and implement it in a relevant file in admin/server/

Using the Github App in development

We use a Github App to listen to pushes on repositories connected to Rill to do automated deployments. The app has access to read contents and receives webhooks on git push.

Github relies on webhooks to deliver information about new connections, pushes, etc. In development, in order for webhooks to be received on localhost, we use this proxy service: https://github.com/probot/smee.io.

Setup instructions:

  1. Install Smee
npm install --global smee-client
  1. Run it (get IDENTIFIER from the Github App info or a team member):
smee --port 8080 --path /github/webhook --url https://smee.io/IDENTIFIER

CLI login/logout

For trying out CLI login add api-url parameter to point to local admin server like this:

go run ./cli login --api-url http://localhost:9090/

For trying out CLI logout add api-url parameter to point to local admin server like this:

go run ./cli logout --api-url http://localhost:9090/

Adding a new field preferences

To add a new preference field for the user, follow these steps:

  1. Include a new column named preference_<name> in the users table. This can be accomplished by appending an appropriate ALTER TABLE query to a newly created .sql file located within the postgres/migrations folder.
  2. In the admin api.proto file, incorporate the optional preference field within the message UserPreferences definition.
  3. Revise the method definition for UpdateUserPreferences to encompass the handling of the new preference in the respective service.
  4. Adjust the UpdateUser SQL query to encompass the new preference field, ensuring that it is included during the update operation.
  5. Identify all instances where the UpdateUser method is called and update them to include the new preference value.

By meticulously following these steps, the new preference field can be successfully incorporated for the user. Remember to update the database schema, proto file, service method, SQL query, and method invocations to properly accommodate the new preference field.

Documentation

Index

Constants

View Source 
const DeviceAuthCodeTTL = 10 * time.Minute

Variables

View Source 
var (
	ErrUserIsNotCollaborator      = fmt.Errorf("user is not a collaborator for the repository")
	ErrGithubInstallationNotFound = fmt.Errorf("github installation not found")
)

Functions

This section is empty.

Types

type AuthToken added in v0.23.0 

type AuthToken interface {
	Token() *authtoken.Token
	OwnerID() string
}

AuthToken is the interface package admin uses to provide a consolidated view of a token string and its DB model.

type Github added in v0.25.0 

type Github interface {
	AppClient() *github.Client
	InstallationClient(installationID int64) (*github.Client, error)
	InstallationToken(ctx context.Context, installationID int64) (string, error)
}

Github exposes the features we require from the Github API.

func NewGithub added in v0.25.0 

func NewGithub(appID int64, appPrivateKey string) (Github, error)

NewGithub returns a new client for connecting to Github.

type Options 

type Options struct {
	DatabaseDriver  string
	DatabaseDSN     string
	ProvisionerSpec string
	ExternalURL     string
}

type Service 

type Service struct {
	DB          database.DB
	Provisioner *provisioner.StaticProvisioner
	Email       *email.Client
	Used        *usedFlusher
	Github      Github
	Logger      *zap.Logger
	// contains filtered or unexported fields
}

func New 

func New(ctx context.Context, opts *Options, logger *zap.Logger, issuer *auth.Issuer, emailClient *email.Client, github Github) (*Service, error)

func (*Service) Close 

func (s *Service) Close() error

func (*Service) CreateOrUpdateUser added in v0.23.0 

func (s *Service) CreateOrUpdateUser(ctx context.Context, email, name, photoURL string) (*database.User, error)

func (*Service) CreateOrganizationForUser added in v0.24.0 

func (s *Service) CreateOrganizationForUser(ctx context.Context, userID, orgName, description string) (*database.Organization, error)

func (*Service) CreateProject added in v0.23.0 

func (s *Service) CreateProject(ctx context.Context, org *database.Organization, userID string, opts *database.InsertProjectOptions) (*database.Project, error)

CreateProject creates a new project and provisions and reconciles a prod deployment for it.

func (*Service) GetGithubInstallation added in v0.24.0 

func (s *Service) GetGithubInstallation(ctx context.Context, githubURL string) (int64, error)

GetGithubInstallation returns a non zero Github installation ID if the Github App is installed on the repository and is not in suspended state The githubURL should be a HTTPS URL for a Github repository without the .git suffix.

func (*Service) HibernateDeployments added in v0.29.1 

func (s *Service) HibernateDeployments(ctx context.Context) error

HibernateDeployments tears down unused deployments

func (*Service) IssueDeploymentAuthToken added in v0.36.0 

func (s *Service) IssueDeploymentAuthToken(ctx context.Context, deploymentID string, ttl *time.Duration) (AuthToken, error)

IssueDeploymentAuthToken generates and persists a new auth token for a deployment.

func (*Service) IssueDeviceAuthCode added in v0.24.0 

func (s *Service) IssueDeviceAuthCode(ctx context.Context, clientID string) (*database.DeviceAuthCode, error)

func (*Service) IssueServiceAuthToken added in v0.31.0 

func (s *Service) IssueServiceAuthToken(ctx context.Context, serviceID string, ttl *time.Duration) (AuthToken, error)

IssueServiceAuthToken generates and persists a new auth token for a service.

func (*Service) IssueUserAuthToken added in v0.23.0 

func (s *Service) IssueUserAuthToken(ctx context.Context, userID, clientID, displayName string, representingUserID *string, ttl *time.Duration) (AuthToken, error)

IssueUserAuthToken generates and persists a new auth token for a user.

func (*Service) LookupGithubRepoForUser added in v0.24.0 

func (s *Service) LookupGithubRepoForUser(ctx context.Context, installationID int64, githubURL, gitUsername string) (*github.Repository, error)

LookupGithubRepoForUser returns a Github repository iff the Github App is installed on the repository and user is a collaborator of the project. The githubURL should be a HTTPS URL for a Github repository without the .git suffix.

func (*Service) LookupReport added in v0.37.0 

func (s *Service) LookupReport(ctx context.Context, depl *database.Deployment, reportName string) (*runtimev1.ReportSpec, error)

LookupReport fetches a report's spec from a runtime deployment.

func (*Service) OrganizationPermissionsForDeployment added in v0.36.0 

func (s *Service) OrganizationPermissionsForDeployment(ctx context.Context, orgID, deploymentID string) (*adminv1.OrganizationPermissions, error)

OrganizationPermissionsForDeployment resolves organization permissions for a deployment. A deployment does not get any permissions on the org it belongs to. It only has permissions on the project it belongs to.

func (*Service) OrganizationPermissionsForService added in v0.33.2 

func (s *Service) OrganizationPermissionsForService(ctx context.Context, orgID, serviceID string) (*adminv1.OrganizationPermissions, error)

OrganizationPermissionsForService resolves organization permissions for a service. A service currently gets full permissions on the org they belong to.

func (*Service) OrganizationPermissionsForUser added in v0.33.2 

func (s *Service) OrganizationPermissionsForUser(ctx context.Context, orgID, userID string) (*adminv1.OrganizationPermissions, error)

OrganizationPermissionsForUser resolves organization permissions for a user.

func (*Service) ProcessGithubEvent added in v0.23.0 

func (s *Service) ProcessGithubEvent(ctx context.Context, rawEvent any) error

ProcessGithubEvent processes a Github event (usually received over webhooks). After validating that the event is a valid Github event, it moves further processing to the background and returns a nil error.

func (*Service) ProjectPermissionsForDeployment added in v0.36.0 

func (s *Service) ProjectPermissionsForDeployment(ctx context.Context, projectID, deploymentID string, orgPerms *adminv1.OrganizationPermissions) (*adminv1.ProjectPermissions, error)

ProjectPermissionsForDeployment resolves project permissions for a deployment. A deployment currently gets full read and no write permissions on the project it belongs to.

func (*Service) ProjectPermissionsForService added in v0.33.2 

func (s *Service) ProjectPermissionsForService(ctx context.Context, projectID, serviceID string, orgPerms *adminv1.OrganizationPermissions) (*adminv1.ProjectPermissions, error)

ProjectPermissionsService resolves project permissions for a service. A service currently gets full permissions on all projects in the org they belong to.

func (*Service) ProjectPermissionsForUser added in v0.33.2 

func (s *Service) ProjectPermissionsForUser(ctx context.Context, projectID, userID string, orgPerms *adminv1.OrganizationPermissions) (*adminv1.ProjectPermissions, error)

ProjectPermissionsForUser resolves project permissions for a user.

func (*Service) RevokeAuthToken added in v0.23.0 

func (s *Service) RevokeAuthToken(ctx context.Context, token string) error

RevokeAuthToken removes an auth token from persistent storage.

func (*Service) TeardownProject added in v0.23.0 

func (s *Service) TeardownProject(ctx context.Context, p *database.Project) error

TeardownProject tears down a project and all its deployments.

func (*Service) TriggerReconcile added in v0.23.0 

func (s *Service) TriggerReconcile(ctx context.Context, depl *database.Deployment) (err error)

TriggerReconcile triggers a reconcile for a deployment.

func (*Service) TriggerReconcileAndAwaitReport added in v0.37.0 

func (s *Service) TriggerReconcileAndAwaitReport(ctx context.Context, depl *database.Deployment, reportName string) error

TriggerReconcileAndAwaitReport triggers a reconcile and polls the runtime until the given report's spec version has been updated (or ctx is canceled).

func (*Service) TriggerRedeploy added in v0.24.3 

func (s *Service) TriggerRedeploy(ctx context.Context, proj *database.Project, prevDepl *database.Deployment) (*database.Project, error)

TriggerRedeploy de-provisions and re-provisions a project's prod deployment.

func (*Service) TriggerRefreshSources added in v0.24.3 

func (s *Service) TriggerRefreshSources(ctx context.Context, depl *database.Deployment, sources []string) (err error)

TriggerRefreshSource triggers refresh of a deployment's sources. If the sources slice is nil, it will refresh all sources.

func (*Service) TriggerReport added in v0.37.0 

func (s *Service) TriggerReport(ctx context.Context, depl *database.Deployment, report string) (err error)

TriggerReport triggers an ad-hoc run of a report

func (*Service) UpdateOrgDeploymentAnnotations added in v0.32.0 

func (s *Service) UpdateOrgDeploymentAnnotations(ctx context.Context, org *database.Organization) error

UpdateOrgDeploymentAnnotations iterates over projects of the given org and updates annotations of corresponding deployments with the new organization name NOTE : this does not trigger reconcile.

func (*Service) UpdateProject added in v0.23.0 

func (s *Service) UpdateProject(ctx context.Context, proj *database.Project, opts *database.UpdateProjectOptions) (*database.Project, error)

UpdateProject updates a project and any impacted deployments. It runs a reconcile if deployment parameters (like branch or variables) have been changed and reconcileDeployment is set.

func (*Service) ValidateAuthToken added in v0.23.0 

func (s *Service) ValidateAuthToken(ctx context.Context, token string) (AuthToken, error)

ValidateAuthToken validates an auth token against persistent storage.

Source Files

View all Source files

Directories

Path Synopsis
postgres
pkg
authtoken
gitutil
nameseeds
pgtestcontainer
publicemail
urlutil
auth
cookies

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.