syncer

Documentation

Overview

Copyright (C) 2022, Rob Lyon <rob@ctxswitch.com>

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Index

• type Repo
• type Syncer
  • func New(ctx context.Context, options SyncerOptions) *Syncer
  • func (rs *Syncer) Start(ctx context.Context) error
• type SyncerOptions
• Bugs

Types

type Repo

type Repo struct {
	Owner     string
	Name      string
	CloneURL  string
	CommitSHA string
	LocalPath string
}

Repo defines the attributes of a github repository that will be required for the Syncer service.

type Syncer

type Syncer struct {
	// contains filtered or unexported fields
}

Syncer is a service that polls Github looking for repositories that have been tagged with a specific topic as defined for GithubTopic. The list of repositories is returned and the latest commit sha is gathered. If a repo does not exist locally, it is cloned using the username and token and if the repo exists and has been updated as seen by comparing the commit sha, the changes are pulled in.

func New

func New(ctx context.Context, options SyncerOptions) *Syncer

New intializes a the github sync service and performs the initial sync.

func (*Syncer) Start

func (rs *Syncer) Start(ctx context.Context) error

Start runs the synchronization process. The process is repeated at an interval equal to the configured poll interval.

type SyncerOptions

type SyncerOptions struct {
	// A personal access token with permissions to access and list the
	// repositories.  Initially set in the config.
	GithubToken string
	// The user who the token belongs to.  Defaults to the Github user.
	// Initially set in the config.
	GithubTokenUser string
	// The Github user or organization that will be scraped.  Only single
	// values are currently supported.  Initially set in the config.
	GithubUser string
	// The topic that will be used as a filter to identify repositories
	// that will be synchronized.  Initially set in the config.
	GithubTopic string
	// The interval to check for changes on Github.  Takes a duration string
	// for the value.  The string is an unsigned decimal number(s), with
	// optional fraction and a unit suffix, such as "300ms", "-1.5h" or
	// "2h45m". Valid time units are "ns", "us" (or "µs"), "ms", "s", "m",
	// "h".  Initially set in the config.
	GithubPollInterval string
	// Changes the verbosity of the logging system.  Initially set in the config.
	GodocRoot string
	// The logger used by the godoc service. Initially set in the
	// config.
	Logger *zap.Logger
}

SyncerOptions defines the options available for running the Syncer service.

Notes

Bugs

• Negative values are not checked before the poll interval is passed to the ParseDuration function.

• Small values should not be allowed. We need to set a minimun value of potentially 1 minute. If not, we run the risk of exceeding limits which could cause the service to behave poorly.