README
¶
admin
This directory contains the control-plane for the managed, multi-user version of Rill (currently available on ui.rilldata.com
).
Running in development
Run the following command from the repository root to start a full development environment except the admin service:
rill devtool start cloud --except admin # optional: --reset
For as long as the devtool is running, rill
commands will target your local development environment instead of rilldata.com
(you can manually switch environments using rill devtool switch-env
.)
Then separately start the admin service (and start/stop it when you make code changes):
go run ./cli admin start
Using Github webhooks in development
The local development environment is not capable of receiving Github webhooks. In most cases, you can just run rill project reconcile
to manually trigger a reconcile after pushing changes to Github.
Continue reading only if you are making changes to the Github webhooks code and need to these changes specifically.
We use a Github App to listen to pushes on repositories connected to Rill to do automated deployments. The app has access to read contents
and receives webhooks on git push
.
Github relies on webhooks to deliver information about new connections, pushes, etc. In development, in order for webhooks to be received on localhost
, we use this proxy service: https://github.com/probot/smee.io.
Setup instructions:
- Install Smee
npm install --global smee-client
- Run it (get
IDENTIFIER
from the Github App info or a team member):
smee --port 8080 --path /github/webhook --url https://smee.io/IDENTIFIER
Adding endpoints
We define our APIs using gRPC and use gRPC-Gateway to map the RPCs to a RESTful API. See proto/README.md
for details.
To add a new endpoint:
- Describe the endpoint in
proto/rill/admin/v1/api.proto
- Re-generate gRPC and OpenAPI interfaces by running
make proto.generate
- Copy the new handler signature from the
AdminServiceServer
interface inproto/gen/rill/admin/v1/api_grpc_pb.go
- Paste the handler signature and implement it in a relevant file in
admin/server/
Adding a new user preferences field
To add a new preference field for the user, follow these steps:
- Include a new column named
preference_<name>
in theusers
table. This can be accomplished by appending an appropriateALTER TABLE
query to a newly created.sql
file located within thepostgres/migrations
folder. - In the admin
api.proto
file, incorporate the optional preference field within themessage UserPreferences
definition. - Revise the method definition for UpdateUserPreferences to encompass the handling of the new preference in the respective service.
- Adjust the
UpdateUser
SQL query to encompass the new preference field, ensuring that it is included during the update operation. - Identify all instances where the
UpdateUser
method is called and update them to include the new preference value.
By meticulously following these steps, the new preference field can be successfully incorporated for the user. Remember to update the database schema, proto file, service method, SQL query, and method invocations to properly accommodate the new preference field.
Documentation
¶
Index ¶
- Constants
- Variables
- type AuthToken
- type DeploymentAnnotations
- type Github
- type Options
- type Service
- func (s *Service) Close() error
- func (s *Service) CreateOrUpdateUser(ctx context.Context, email, name, photoURL string) (*database.User, error)
- func (s *Service) CreateOrganizationForUser(ctx context.Context, userID, orgName, description string) (*database.Organization, error)
- func (s *Service) CreateProject(ctx context.Context, org *database.Organization, ...) (*database.Project, error)
- func (s *Service) GetGithubInstallation(ctx context.Context, githubURL string) (int64, error)
- func (s *Service) HibernateDeployments(ctx context.Context) error
- func (s *Service) IssueDeploymentAuthToken(ctx context.Context, deploymentID string, ttl *time.Duration) (AuthToken, error)
- func (s *Service) IssueDeviceAuthCode(ctx context.Context, clientID string) (*database.DeviceAuthCode, error)
- func (s *Service) IssueServiceAuthToken(ctx context.Context, serviceID string, ttl *time.Duration) (AuthToken, error)
- func (s *Service) IssueUserAuthToken(ctx context.Context, userID, clientID, displayName string, ...) (AuthToken, error)
- func (s *Service) LookupAlert(ctx context.Context, depl *database.Deployment, alertName string) (*runtimev1.AlertSpec, error)
- func (s *Service) LookupGithubRepoForUser(ctx context.Context, installationID int64, githubURL, gitUsername string) (*github.Repository, error)
- func (s *Service) LookupReport(ctx context.Context, depl *database.Deployment, reportName string) (*runtimev1.ReportSpec, error)
- func (s *Service) NewDeploymentAnnotations(org *database.Organization, proj *database.Project) DeploymentAnnotations
- func (s *Service) OpenMetricsProject(ctx context.Context) (*metrics.Client, bool, error)
- func (s *Service) OrganizationPermissionsForDeployment(ctx context.Context, orgID, deploymentID string) (*adminv1.OrganizationPermissions, error)
- func (s *Service) OrganizationPermissionsForService(ctx context.Context, orgID, serviceID string) (*adminv1.OrganizationPermissions, error)
- func (s *Service) OrganizationPermissionsForUser(ctx context.Context, orgID, userID string) (*adminv1.OrganizationPermissions, error)
- func (s *Service) ProcessGithubEvent(ctx context.Context, rawEvent any) error
- func (s *Service) ProjectPermissionsForDeployment(ctx context.Context, projectID, deploymentID string, ...) (*adminv1.ProjectPermissions, error)
- func (s *Service) ProjectPermissionsForService(ctx context.Context, projectID, serviceID string, ...) (*adminv1.ProjectPermissions, error)
- func (s *Service) ProjectPermissionsForUser(ctx context.Context, projectID, userID string, ...) (*adminv1.ProjectPermissions, error)
- func (s *Service) ResolveLatestRuntimeVersion() string
- func (s *Service) RevokeAuthToken(ctx context.Context, token string) error
- func (s *Service) TeardownProject(ctx context.Context, p *database.Project) error
- func (s *Service) TriggerReconcile(ctx context.Context, depl *database.Deployment) (err error)
- func (s *Service) TriggerReconcileAndAwaitResource(ctx context.Context, depl *database.Deployment, name, kind string) error
- func (s *Service) TriggerRedeploy(ctx context.Context, proj *database.Project, prevDepl *database.Deployment) (*database.Project, error)
- func (s *Service) TriggerRefreshSources(ctx context.Context, depl *database.Deployment, sources []string) (err error)
- func (s *Service) TriggerReport(ctx context.Context, depl *database.Deployment, report string) (err error)
- func (s *Service) UpdateDeployment(ctx context.Context, depl *database.Deployment, opts *UpdateDeploymentOptions) error
- func (s *Service) UpdateOrgDeploymentAnnotations(ctx context.Context, org *database.Organization) error
- func (s *Service) UpdateProject(ctx context.Context, proj *database.Project, ...) (*database.Project, error)
- func (s *Service) ValidateAuthToken(ctx context.Context, token string) (AuthToken, error)
- func (s *Service) ValidateRuntimeVersion(ver string) error
- type UpdateDeploymentOptions
Constants ¶
const DeviceAuthCodeTTL = 10 * time.Minute
Variables ¶
var ( ErrUserIsNotCollaborator = fmt.Errorf("user is not a collaborator for the repository") ErrGithubInstallationNotFound = fmt.Errorf("github installation not found") )
Functions ¶
This section is empty.
Types ¶
type AuthToken ¶ added in v0.23.0
AuthToken is the interface package admin uses to provide a consolidated view of a token string and its DB model.
type DeploymentAnnotations ¶ added in v0.43.0
type DeploymentAnnotations struct {
// contains filtered or unexported fields
}
type Github ¶ added in v0.25.0
type Github interface { AppClient() *github.Client InstallationClient(installationID int64) (*github.Client, error) InstallationToken(ctx context.Context, installationID int64) (string, error) }
Github exposes the features we require from the Github API.
type Service ¶
type Service struct { DB database.DB ProvisionerSet map[string]provisioner.Provisioner Email *email.Client Github Github AI ai.Client Used *usedFlusher Logger *zap.Logger VersionNumber string VersionCommit string AutoscalerCron string // contains filtered or unexported fields }
func (*Service) CreateOrUpdateUser ¶ added in v0.23.0
func (*Service) CreateOrganizationForUser ¶ added in v0.24.0
func (*Service) CreateProject ¶ added in v0.23.0
func (s *Service) CreateProject(ctx context.Context, org *database.Organization, opts *database.InsertProjectOptions) (*database.Project, error)
CreateProject creates a new project and provisions and reconciles a prod deployment for it.
func (*Service) GetGithubInstallation ¶ added in v0.24.0
GetGithubInstallation returns a non zero Github installation ID if the Github App is installed on the repository and is not in suspended state The githubURL should be a HTTPS URL for a Github repository without the .git suffix.
func (*Service) HibernateDeployments ¶ added in v0.29.1
HibernateDeployments tears down unused deployments
func (*Service) IssueDeploymentAuthToken ¶ added in v0.36.0
func (s *Service) IssueDeploymentAuthToken(ctx context.Context, deploymentID string, ttl *time.Duration) (AuthToken, error)
IssueDeploymentAuthToken generates and persists a new auth token for a deployment.
func (*Service) IssueDeviceAuthCode ¶ added in v0.24.0
func (*Service) IssueServiceAuthToken ¶ added in v0.31.0
func (s *Service) IssueServiceAuthToken(ctx context.Context, serviceID string, ttl *time.Duration) (AuthToken, error)
IssueServiceAuthToken generates and persists a new auth token for a service.
func (*Service) IssueUserAuthToken ¶ added in v0.23.0
func (s *Service) IssueUserAuthToken(ctx context.Context, userID, clientID, displayName string, representingUserID *string, ttl *time.Duration) (AuthToken, error)
IssueUserAuthToken generates and persists a new auth token for a user.
func (*Service) LookupAlert ¶ added in v0.41.0
func (s *Service) LookupAlert(ctx context.Context, depl *database.Deployment, alertName string) (*runtimev1.AlertSpec, error)
LookupAlert fetches a alert's spec from a runtime deployment.
func (*Service) LookupGithubRepoForUser ¶ added in v0.24.0
func (s *Service) LookupGithubRepoForUser(ctx context.Context, installationID int64, githubURL, gitUsername string) (*github.Repository, error)
LookupGithubRepoForUser returns a Github repository iff the Github App is installed on the repository and user is a collaborator of the project. The githubURL should be a HTTPS URL for a Github repository without the .git suffix.
func (*Service) LookupReport ¶ added in v0.37.0
func (s *Service) LookupReport(ctx context.Context, depl *database.Deployment, reportName string) (*runtimev1.ReportSpec, error)
LookupReport fetches a report's spec from a runtime deployment.
func (*Service) NewDeploymentAnnotations ¶ added in v0.43.0
func (s *Service) NewDeploymentAnnotations(org *database.Organization, proj *database.Project) DeploymentAnnotations
func (*Service) OpenMetricsProject ¶ added in v0.43.0
OpenMetricsProject opens a client for accessing the metrics project. If a metrics project is not configured, it returns false for the second return value. The returned client has a TTL of 30 minutes. TODO: Encapsulate token refresh logic in the metrics client.
func (*Service) OrganizationPermissionsForDeployment ¶ added in v0.36.0
func (s *Service) OrganizationPermissionsForDeployment(ctx context.Context, orgID, deploymentID string) (*adminv1.OrganizationPermissions, error)
OrganizationPermissionsForDeployment resolves organization permissions for a deployment. A deployment does not get any permissions on the org it belongs to. It only has permissions on the project it belongs to.
func (*Service) OrganizationPermissionsForService ¶ added in v0.33.2
func (s *Service) OrganizationPermissionsForService(ctx context.Context, orgID, serviceID string) (*adminv1.OrganizationPermissions, error)
OrganizationPermissionsForService resolves organization permissions for a service. A service currently gets full permissions on the org they belong to.
func (*Service) OrganizationPermissionsForUser ¶ added in v0.33.2
func (s *Service) OrganizationPermissionsForUser(ctx context.Context, orgID, userID string) (*adminv1.OrganizationPermissions, error)
OrganizationPermissionsForUser resolves organization permissions for a user.
func (*Service) ProcessGithubEvent ¶ added in v0.23.0
ProcessGithubEvent processes a Github event (usually received over webhooks). After validating that the event is a valid Github event, it moves further processing to the background and returns a nil error.
func (*Service) ProjectPermissionsForDeployment ¶ added in v0.36.0
func (s *Service) ProjectPermissionsForDeployment(ctx context.Context, projectID, deploymentID string, orgPerms *adminv1.OrganizationPermissions) (*adminv1.ProjectPermissions, error)
ProjectPermissionsForDeployment resolves project permissions for a deployment. A deployment currently gets full read and no write permissions on the project it belongs to.
func (*Service) ProjectPermissionsForService ¶ added in v0.33.2
func (s *Service) ProjectPermissionsForService(ctx context.Context, projectID, serviceID string, orgPerms *adminv1.OrganizationPermissions) (*adminv1.ProjectPermissions, error)
ProjectPermissionsService resolves project permissions for a service. A service currently gets full permissions on all projects in the org they belong to.
func (*Service) ProjectPermissionsForUser ¶ added in v0.33.2
func (s *Service) ProjectPermissionsForUser(ctx context.Context, projectID, userID string, orgPerms *adminv1.OrganizationPermissions) (*adminv1.ProjectPermissions, error)
ProjectPermissionsForUser resolves project permissions for a user.
func (*Service) ResolveLatestRuntimeVersion ¶ added in v0.44.0
func (*Service) RevokeAuthToken ¶ added in v0.23.0
RevokeAuthToken removes an auth token from persistent storage.
func (*Service) TeardownProject ¶ added in v0.23.0
TeardownProject tears down a project and all its deployments.
func (*Service) TriggerReconcile ¶ added in v0.23.0
TriggerReconcile triggers a reconcile for a deployment.
func (*Service) TriggerReconcileAndAwaitResource ¶ added in v0.41.0
func (s *Service) TriggerReconcileAndAwaitResource(ctx context.Context, depl *database.Deployment, name, kind string) error
TriggerReconcileAndAwaitResource triggers a reconcile and polls the runtime until the given resource's spec version has been updated (or ctx is canceled).
func (*Service) TriggerRedeploy ¶ added in v0.24.3
func (s *Service) TriggerRedeploy(ctx context.Context, proj *database.Project, prevDepl *database.Deployment) (*database.Project, error)
TriggerRedeploy de-provisions and re-provisions a project's prod deployment.
func (*Service) TriggerRefreshSources ¶ added in v0.24.3
func (s *Service) TriggerRefreshSources(ctx context.Context, depl *database.Deployment, sources []string) (err error)
TriggerRefreshSource triggers refresh of a deployment's sources. If the sources slice is nil, it will refresh all sources.
func (*Service) TriggerReport ¶ added in v0.37.0
func (s *Service) TriggerReport(ctx context.Context, depl *database.Deployment, report string) (err error)
TriggerReport triggers an ad-hoc run of a report
func (*Service) UpdateDeployment ¶ added in v0.43.0
func (s *Service) UpdateDeployment(ctx context.Context, depl *database.Deployment, opts *UpdateDeploymentOptions) error
func (*Service) UpdateOrgDeploymentAnnotations ¶ added in v0.32.0
func (s *Service) UpdateOrgDeploymentAnnotations(ctx context.Context, org *database.Organization) error
UpdateOrgDeploymentAnnotations iterates over projects of the given org and updates annotations of corresponding deployments with the new organization name NOTE : this does not trigger reconcile.
func (*Service) UpdateProject ¶ added in v0.23.0
func (s *Service) UpdateProject(ctx context.Context, proj *database.Project, opts *database.UpdateProjectOptions) (*database.Project, error)
UpdateProject updates a project and any impacted deployments. It runs a reconcile if deployment parameters (like branch or variables) have been changed and reconcileDeployment is set.
func (*Service) ValidateAuthToken ¶ added in v0.23.0
ValidateAuthToken validates an auth token against persistent storage.