libpages

package
v0.0.0-...-f10651d Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 7, 2024 License: BSD-3-Clause, BSD-3-Clause Imports: 38 Imported by: 4

Documentation

Index

Constants

View Source 
const CtxKBPOpID = "KBP"

CtxKBPOpID is the display name for unique operations in kbp and libpages.

Variables

This section is empty.

Functions

func ListenAndServe 

func ListenAndServe(ctx context.Context,
	config *ServerConfig, kbfsConfig libkbfs.Config) (err error)

ListenAndServe listens on 443 and 80 ports of all addresses, and serve Keybase Pages based on config and kbfsConfig. HTTPs setup is handled with ACME.

func MigrateActivityStatsStorerFromFileBasedToMySQL 

func MigrateActivityStatsStorerFromFileBasedToMySQL(
	logger *zap.Logger, fbRootDir string, mysqlDSN string)

MigrateActivityStatsStorerFromFileBasedToMySQL should only be used as part of a commandline tool.

Types

type ActivesGetter 

type ActivesGetter interface {
	// GetActives returns the number of active TLFs and active hosts in the
	// past dur.
	GetActives(dur time.Duration) (activeTlfs int, activeHosts int, err error)
}

ActivesGetter holds necessary data to generate number of active TLFs or hosts.

type ActivityStatsEnabler 

type ActivityStatsEnabler struct {
	// Storer specifies a backend storer that a StatsReporter should use to
	// store data necessary for generating activity-based stats.
	Storer ActivityStatsStorer
	// Durations specifies a slice of durations that activity-based stats
	// should be about. For example, [1h, 1d, 1week] makes the StatsReporter
	// should report hourly, daily, and weekly active stats.
	Durations []NameableDuration
	// Interval specifies how often the activity-based stats should be
	// reported.
	Interval time.Duration
}

ActivityStatsEnabler describes what backend storer a StatsReporter should use for activity-based stats, and how the stats should be generated.

type ActivityStatsStorer 

type ActivityStatsStorer interface {
	// RecordActives records that tlfID and host has just been active.
	RecordActives(tlfID tlf.ID, host string)

	// GetActivesGetter returns a ActivesGetter from current state of the
	// ActivityStatsStorer.
	GetActivesGetter() (ActivesGetter, error)
}

ActivityStatsStorer defines a set of methods to record activities based on TLF ID and host names.

func NewFileBasedActivityStatsStorer 

func NewFileBasedActivityStatsStorer(
	rootPath string, logger *zap.Logger) (ActivityStatsStorer, error)

NewFileBasedActivityStatsStorer creates an ActivityStatsStorer that stores activities on a local filesystem.

NOTE that this is meant to be for development and testing only and does not scale well.

func NewMySQLActivityStatsStorer 

func NewMySQLActivityStatsStorer(
	db *sql.DB, logger *zap.Logger) ActivityStatsStorer

NewMySQLActivityStatsStorer creates an ActivityStatsStorer that stores activities on a MySQL database.

type CacheableFS 

type CacheableFS struct {
	// contains filtered or unexported fields
}

CacheableFS is a wrapper around a *libfs.FS and a subdir. Use Use() to get a *libfs.FS that roots at subdir. This essentially delays "cd"ing into subdir, and is useful for caching a *libfs.FS object without the downside of caching a libkbfs.Node that can be obsolete when it's renamed or removed.

func (CacheableFS) EnsureNoSuchFileOutsideRoot 

func (fs CacheableFS) EnsureNoSuchFileOutsideRoot(name string) (err error)

EnsureNoSuchFileOutsideRoot walks from the sub dir that this FS is configured to use back all the way to the TLF root, and try to find a file named `name`. If the file is found, an error is returned.

For example, if a subdir /dir1/dir2/dir3 is configured as root dir, calling this function makes sure none of /a/b/{name}, /a/{name}, and /{name} exist. Though /a/b/c/{name} can exist.

func (CacheableFS) IsObsolete 

func (fs CacheableFS) IsObsolete() bool

IsObsolete returns true if fs has reached the end of life, because of a handle change for example. In this case user should not use this fs anymore, but instead make a new one.

func (CacheableFS) Use 

func (fs CacheableFS) Use() (*libfs.FS, error)

Use returns a *libfs.FS to use.

type CertStoreType 

type CertStoreType string

CertStoreType is a type for specifying if and what cert store should be used for acme/autocert. 

const (
	NoCertStore      CertStoreType = ""
	DiskCertStore    CertStoreType = "disk"
	KVStoreCertStore CertStoreType = "kvstore"
)

Possible cert store types.

type CtxKBPTagKey 

type CtxKBPTagKey int

CtxKBPTagKey is the type used for unique context tags within kbp and libpages. 

const (
	// CtxKBPKey is the tag key for unique operation IDs within kbp and
	// libpages.
	CtxKBPKey CtxKBPTagKey = iota
)

type ErrDomainBlockedInBlacklist 

type ErrDomainBlockedInBlacklist struct{}

ErrDomainBlockedInBlacklist is returned when the server is configured with a domain blacklist, and we receive a HTTP request that was sent to a domain that's in the blacklist.

func (ErrDomainBlockedInBlacklist) Error 

func (ErrDomainBlockedInBlacklist) Error() string

Error implements the error interface.

type ErrDomainNotAllowedInWhitelist 

type ErrDomainNotAllowedInWhitelist struct{}

ErrDomainNotAllowedInWhitelist is returned when the server is configured with a domain whitelist, and we receive a HTTP request that was sent to a domain that's not in the whitelist.

func (ErrDomainNotAllowedInWhitelist) Error 

func (ErrDomainNotAllowedInWhitelist) Error() string

Error implements the error interface.

type ErrInvalidKeybasePagesRecord 

type ErrInvalidKeybasePagesRecord struct{}

ErrInvalidKeybasePagesRecord is returned when the kbp= DNS record for a domain is invalid.

func (ErrInvalidKeybasePagesRecord) Error 

func (ErrInvalidKeybasePagesRecord) Error() string

Error returns the error interface.

type ErrKeybasePagesRecordNotFound 

type ErrKeybasePagesRecordNotFound struct{}

ErrKeybasePagesRecordNotFound is returned when a domain requested doesn't have a kbp= record configured.

func (ErrKeybasePagesRecordNotFound) Error 

func (ErrKeybasePagesRecordNotFound) Error() string

Error implements the error interface.

type ErrKeybasePagesRecordTooMany 

type ErrKeybasePagesRecordTooMany struct{}

ErrKeybasePagesRecordTooMany is returned when a domain requested has more than one kbp= record configured.

func (ErrKeybasePagesRecordTooMany) Error 

func (ErrKeybasePagesRecordTooMany) Error() string

Error implements the error interface.

type NameableDuration 

type NameableDuration struct {
	Duration time.Duration
	Name     string
}

NameableDuration is a wrapper around time.Duration that allows customized String() encoding.

func (NameableDuration) String 

func (d NameableDuration) String() string

String returns d.Name if it's not empty, or d.Duration.String().

type Root 

type Root struct {
	Type            RootType
	TlfType         tlf.Type
	TlfNameUnparsed string
	PathUnparsed    string
}

Root defines the root of a static site hosted by Keybase Pages. It is normally constructed from DNS records directly and is cheap to make.

func ParseRoot 

func ParseRoot(str string) (*Root, error)

ParseRoot parses a kbp= TXT record from a domain into a Root object.

func (*Root) MakeFS 

func (r *Root) MakeFS(
	ctx context.Context, log *zap.Logger, kbfsConfig libkbfs.Config) (
	fs CacheableFS, tlfID tlf.ID, shutdown func(), err error)

MakeFS makes a CacheableFS from *r, which can be adapted to a http.FileSystem (through ToHTTPFileSystem) to be used by http package to serve through HTTP. Caller must call Use() to get a usable FS.

type RootLoader 

type RootLoader interface {
	LoadRoot(domain string) (root Root, err error)
}

RootLoader is the interface for loading a site root.

func NewDNSRootLoader 

func NewDNSRootLoader(log *zap.Logger) RootLoader

NewDNSRootLoader makes a new RootLoader backed by DNS TXT record. It caches the root for a short period time. This is the RootLoader that should be used in all non-test scenarios.

When loading from DNS, it does so with following steps:

  1. Construct a domain name by prefixing the `domain` parameter with "_keybase_pages." or "_keybasepages". So for example, "static.keybase.io" turns into "_keybase_pages.static.keybase.io" or "_keybasepages.static.keybase.io".
  2. Load TXT record(s) from the domain constructed in step 1, and look for one starting with "kbp=". If exactly one exists, parse it into a `Root` and return it.

There must be exactly one "kbp=" TXT record configured for domain. If more than one exists, an ErrKeybasePagesRecordTooMany{} is returned. If none is found, an ErrKeybasePagesRecordNotFound{} is returned. In case the user has some configuration that requires other records that we can't foresee for now, other records (TXT or not) can co-exist with the "kbp=" record (as long as no CNAME record exists on the "_keybase_pages." or "_keybasepages." prefixed domain of course).

If the given domain is invalid, the domain name constructed in this step will be invalid too, which causes Go's DNS resolver to return a net.DNSError typed "no such host" error.

Examples for "static.keybase.io", "meatball.gao.io", "song.gao.io", "blah.strib.io", and "kbp.jzila.com" respectively: 

_keybase_pages.static.keybase.io TXT "kbp=/keybase/team/keybase.bots/static.keybase.io"
_keybase_pages.meatball.gao.io   TXT "kbp=/keybase/public/songgao/meatball/"
_keybase_pages.song.gao.io       TXT "kbp=/keybase/private/songgao,kb_bot/blah"
_keybase_pages.blah.strib.io     TXT "kbp=/keybase/private/strib#kb_bot/blahblahb" "lah/blah/"
_keybase_pages.kbp.jzila.com     TXT "kbp=git@keybase:private/jzila,kb_bot/kbp.git"

type RootType 

type RootType int

RootType specifies the type of a root. 

const (

	// KBFSRoot means the root is backed by a KBFS path.
	KBFSRoot RootType
	// GitRoot means the root is backed by a git repo stored in KBFS.
	GitRoot
)

func (RootType) String 

func (t RootType) String() string

String implements the fmt.Stringer interface

type ServedRequestInfo 

type ServedRequestInfo struct {
	// Host is the `Host` field of http.Request.
	Host string
	// Proto is the `Proto` field of http.Request.
	Proto string
	// Authenticated means the client set WWW-Authenticate in this request and
	// authentication using the given credentials has succeeded. It doesn't
	// necessarily indicate that the authentication is required for this
	// particular request.
	Authenticated bool
	// TlfID is the TLF ID associated with the site.
	TlfID tlf.ID
	// TlfType is the TLF type of the root that's used to serve the request.
	TlfType tlf.Type
	// RootType is the type of the root that's used to serve the request.
	RootType RootType
	// HTTPStatus is the HTTP status code that we have written for the request
	// in the response header.
	HTTPStatus int
	// CloningShown is set to true if a "CLONING" page instead of the real site
	// was served to the request.
	CloningShown bool
	// InvalidConfig is set to true if user has a config for the site being
	// requested, but it's invalid.
	InvalidConfig bool
}

ServedRequestInfo holds information regarding to an incoming request that might be useful for stats.

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server handles incoming HTTP requests by creating a Root for each host and serving content from it.

func (*Server) ServeHTTP 

func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP implements the http.Handler interface.

type ServerConfig 

type ServerConfig struct {
	// If DomainWhitelist is non-nil and non-empty, only domains in the
	// whitelist are served and others are blocked.
	DomainWhitelist []string
	// If DomainBlacklist is non-nil and non-empty, domains in the blacklist
	// and all subdomains under them are blocked. When a domain is present in
	// both blacklist and whitelist, the domain is blocked.
	DomainBlacklist []string
	UseStaging      bool
	Logger          *zap.Logger
	CertStore       CertStoreType
	StatsReporter   StatsReporter
	// contains filtered or unexported fields
}

ServerConfig holds configuration parameters for Server.

type StatsReporter 

type StatsReporter interface {
	// ReportServedRequest is called by libpages whenever a request comes in.
	ReportServedRequest(r *ServedRequestInfo)
}

StatsReporter defines a collection of methods for stats reporting.

func NewMultiStatReporter 

func NewMultiStatReporter(reporters ...StatsReporter) StatsReporter

NewMultiStatReporter creates a StatsReporter that reports through all passed in reporters.

func NewStathatReporter 

func NewStathatReporter(logger *zap.Logger, prefix string, ezKey string,
	enableActivityBasedStats *ActivityStatsEnabler) StatsReporter

NewStathatReporter create a new StatsReporter that reports stats to stathat. If enableActivityBasedStats, if set to non-nil, causes the reporter to generate activity-based stats. Caller should not modify enableActivityBasedStats passed into this function.

Source Files

View all Source files

Directories

Path Synopsis
tools
migrateActivityStatsStorerFromFileBasedToMySQL

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.