minimal

package
v1.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 3, 2020 License: Apache-2.0 Imports: 42 Imported by: 0

README

Minimal Gossip

This directory holds experimental and unofficial code for an implementation of minimal gossip for CT logs.

Background

The base Certicate Transparency specification does not include any explicit protection against an attack where a Log presents different views of its contents to different users (a "split view").

The general idea of gossip for CT logs is that each log's signed values (SCTs and STHs) are exchanged amongst gossip participants, so that a split view can be detected.

The CT spec recommends that clients of CT logs gossip amongst themselves; however, this has two problems:

  • It requires the details of the gossip mechanism to be defined, agreed upon and implemented by the clients.
  • It opens up the possibility of fingerprinting individual clients, based on the SCTs (or even STHs, for a log that issues them frequently) that they gossip.

Design

The minimal gossip approach allows the STH values for CT logs to be gossiped to other CT logs and stored in those logs. If log X's STH is periodically retrieved and stored in the Merkle trees of (independent) logs A, B and C, then it becomes difficult for X to present a split tree in general -- particularly as the web of cross-logged STHs expands.

Note, however, that this minimal gossip approach does not protect against the scenario where a malevolent CT log presents a split view to a single specific client (unless that client happens to be an auditor that is cross-checking other logs).

This approach does not require any new actors in the CT ecosystem, thus reducing impact. The impact is reduced further by embedding the gossiped STH values inside synthetic X.509 certificates (using a new extension); this allows the existing CT mechanisms to be used unmodified.

The only code change required by this approach is that the Log's certificate chain validation must not reject the STH extension, which is marked as critical to reduce the chances of the synthetic certificates being treated as valid web PKI X.509 certificates.

Operation

An entity that chooses to perform minimal gossip first chooses the set of source logs whose STHs will be tracked. The gossip operator then generates a new private key, and builds a self-signed root certificate associated to that key; this then needs to be added to each destination log's set of acceptable roots.

By accepting the gossiper's root, the destination log operator is implicitly trusting that the gossiper will not overwhelm the log with an excess of synthetic certificates. This is obviously the case when the gossiper is the same entity as the destination log operator.

The minimal gossip implementation is configured with this private key and set of source logs; it then periodically:

  • retrieves STHs from the source logs
  • converts each STH to a leaf certificate that chains to the gossip root, with the STH embedded in the leaf certificate
  • adds the leaf to each configured destination log.

X.509 Certificate Details

The synthetic certificate chains produced for minimal gossip consist of two certificates.

  • The root certificate is a self-signed certificate that the receiving log has been configured to accept as a root. The main feature of note is that:
    • This root is configured with an extended key usage that indicates minimal gossip use; the OID used for this is currently 1.3.6.1.4.1.11129.2.4.6.
  • The leaf certificate has:
    • A common name that includes STH information for convenience, currently of the form: 
      STH-for-Pilot <http://ct.googleapis.com/pilot> @1519488735960: size=7834486 hash=1823b948e9bf8e34188810bd3d41455f4a48852f3ebcfd98cc05167a388b5712
    • A validity period of 24 hours starting from the STH's timestamp.
    • A minimal gossip EKU.
    • A critical X.509 extension identified by the OID 1.3.6.1.4.1.11129.2.4.5; the corresponding OCTET STRING holds the TLS-encoding of the structure defined below. This extension is marked as critical to reduce the chances of this synthetic leaf certificate being treated as a valid X.509 certificate.

NOTE: OID values are unassigned and subject to change.

The contents of the STH extension are defined as the following TLS structure:

enum { v1(0), (255) } Version;  /* From RFC6962 s3.2 */

struct {
    uint8 log_url<0..255>;
    Version version;
    uint64 tree_size;
    uint64 timestamp;
    opaque sha256_root_hash[32];
    digitally-signed struct {
        Version version;
        SignatureType signature_type = tree_hash;
        uint64 timestamp;
        uint64 tree_size;
        opaque sha256_root_hash[32];
    } TreeHeadSignature;  /* From RFC6962 s3.5 */
} LogSTHInfo;

The combination of the contents of this structure with the source log's public key allows for STH consistency verification.

Verification

The STH entries stored in the destination log can be checked with the goshawk tool. This tool takes a log configuration file similar to the gossiper's configuration, describing:

  • A single destination log to be scanned for STH-holding certificates.
  • A set of source logs that will be checked for STH consistency.

The tool scans the destination log and processes certificates that have the LogSTHInfo extension identified by OID 1.3.6.1.4.1.11129.2.4.5. If the source URL for the STH is one that has been configured, then an STH consistency check is performed against a recent STH from that log.

Documentation

Overview

Package minimal provides a minimal gossip implementation for CT which uses X.509 certificate extensions to hold gossiped STH values for logs. This allows STH values to be exchanged between participating logs without any changes to the log software (although participating logs will need to add additional trusted roots for the gossip sources).

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type FetchOptions 

type FetchOptions struct {
	// Number of entries to request in one batch from the Log.
	BatchSize int
	// Number of concurrent fetcher workers to run.
	ParallelFetch int
	// Manage hub retrieval state persistence.
	State         ScanStateManager
	FlushInterval time.Duration
	// Mechanism for reporting compliance incidents.  If unset, a
	// LoggingReporter will be used.
	Reporter incident.Reporter
}

FetchOptions governs the overall hub retrieval behaviour.

type FileStateManager 

type FileStateManager struct {
	ScanState
	// contains filtered or unexported fields
}

FileStateManager is an implementation of the ScanStateManager interface that stores the scan state in a writable flat file.

func NewFileStateManager 

func NewFileStateManager(filename string) (*FileStateManager, error)

NewFileStateManager creates a FileStateManager from the contents of the given filename, which can be be empty but should be over-writable.

func (*FileStateManager) Flush 

func (f *FileStateManager) Flush(ctx context.Context) error

Flush stores the current contents to disk.

type Goshawk 

type Goshawk struct {
	// contains filtered or unexported fields
}

Goshawk is an agent that retrieves STHs from a Gossip Hub, either in the form of synthetic certificates or more directly as signed blobs. Each STH is then checked for consistency against the source log.

func NewBoundaryGoshawk 

func NewBoundaryGoshawk(_ context.Context, cfg *configpb.GoshawkConfig, hcLog, hcHub *http.Client, fetchOpts FetchOptions) (*Goshawk, error)

NewBoundaryGoshawk creates a Goshawk from the given configuration protobuf and a pair of http.Client instances for source logs and destination hubs, to allow (for example) gossip checking across (some kinds of) network boundaries.

func NewBoundaryGoshawkFromFile 

func NewBoundaryGoshawkFromFile(ctx context.Context, filename string, hcLog, hcHub *http.Client, fetchOpts FetchOptions) (*Goshawk, error)

NewBoundaryGoshawkFromFile creates a Goshawk that uses different http.Client instances for source logs and destination hubs, for example to allow gossip checking across (some kinds of) network boundaries.

func NewGoshawk 

func NewGoshawk(ctx context.Context, cfg *configpb.GoshawkConfig, hc *http.Client, fetchOpts FetchOptions) (*Goshawk, error)

NewGoshawk creates a Goshawk from the given configuration protobuf and optional http client.

func NewGoshawkFromFile 

func NewGoshawkFromFile(ctx context.Context, filename string, hc *http.Client, fetchOpts FetchOptions) (*Goshawk, error)

NewGoshawkFromFile creates a Goshawk from the given filename, which should contain text-protobuf encoded configuration data, together with an optional http Client.

func (*Goshawk) Fly 

func (hawk *Goshawk) Fly(ctx context.Context)

Fly starts a collection of goroutines to perform log scanning and STH consistency checking. It should be terminated by cancelling the passed-in context.

type Gossiper 

type Gossiper struct {
	// contains filtered or unexported fields
}

Gossiper is an agent that retrieves STH values from a set of source logs and distributes it to a destination log in the form of an X.509 certificate with the STH value embedded in it.

func NewBoundaryGossiper 

func NewBoundaryGossiper(ctx context.Context, cfg *configpb.GossipConfig, hcLog, hcHub *http.Client, mf monitoring.MetricFactory) (*Gossiper, error)

NewBoundaryGossiper creates a gossiper from the given configuration protobuf and a pair of http.Client instances for source logs and destination hubs, to allow (for example) gossiping across (some kinds of) network boundaries.

func NewBoundaryGossiperFromFile 

func NewBoundaryGossiperFromFile(ctx context.Context, filename string, hcLog, hcHub *http.Client, mf monitoring.MetricFactory) (*Gossiper, error)

NewBoundaryGossiperFromFile creates a gossiper that uses different http.Client instances for source logs and destination hubs, for example to allow gossiping across (some kinds of) network boundaries.

func NewGossiper 

func NewGossiper(ctx context.Context, cfg *configpb.GossipConfig, hc *http.Client, mf monitoring.MetricFactory) (*Gossiper, error)

NewGossiper creates a gossiper from the given configuration protobuf and optional http client.

func NewGossiperFromFile 

func NewGossiperFromFile(ctx context.Context, filename string, hc *http.Client, mf monitoring.MetricFactory) (*Gossiper, error)

NewGossiperFromFile creates a gossiper from the given filename, which should contain text-protobuf encoded configuration data, together with an optional http Client.

func (*Gossiper) CertForSTH 

func (g *Gossiper) CertForSTH(name, url string, sth *ct.SignedTreeHead) (*ct.ASN1Cert, error)

CertForSTH creates an X.509 certificate with the given STH embedded in it.

func (*Gossiper) CheckCanSubmit 

func (g *Gossiper) CheckCanSubmit(ctx context.Context) error

CheckCanSubmit checks whether the gossiper can submit STHs to all destination hubs.

func (*Gossiper) Run 

func (g *Gossiper) Run(ctx context.Context)

Run starts a gossiper set of goroutines. It should be terminated by cancelling the passed-in context.

func (*Gossiper) Submitter 

func (g *Gossiper) Submitter(ctx context.Context, s <-chan sthInfo)

Submitter periodically services the provided channel and submits the certificates received on it to the destination logs.

type ScanState 

type ScanState struct {
	Mu   sync.Mutex
	Next map[string]int64
}

ScanState holds a (mutex-protected) map of hub URL to next index to scan from.

func (*ScanState) GetHubIndex 

func (s *ScanState) GetHubIndex(hubURL string) int64

GetHubIndex returns the index at which scanning should commence for the given hub URL.

func (*ScanState) UpdateHubIndex 

func (s *ScanState) UpdateHubIndex(hubURL string, nextIndex int64)

UpdateHubIndex indicates that future scanning of the given hub should start from at least nextIndex.

type ScanStateManager 

type ScanStateManager interface {
	// GetHubIndex returns the index at which scanning should commence for the given hub URL.
	GetHubIndex(hubURL string) int64
	// UpdateHubIndex indicates that future scanning of the given hub should start from at least nextIndex.
	UpdateHubIndex(hubURL string, nextIndex int64)
	// Flush ensures state is stored.
	Flush(ctx context.Context) error
}

ScanStateManager controls hub scanning state, with the intention of allowing scanning to resume where it was left off across restarts etc.

Source Files

View all Source files

Directories

Path Synopsis
Package configpb holds a config protobuf for minimal gossip binaries.
 Package configpb holds a config protobuf for minimal gossip binaries.
The goshawk binary scans a destination log for gossiped STH values and checks consistency against the source logs.
 The goshawk binary scans a destination log for gossiped STH values and checks consistency against the source logs.
The gosmin binary runs a minimal gossip implementation.
 The gosmin binary runs a minimal gossip implementation.
Package mysql provides a MySQL based implementation of persistent state management for the goshawk tool.
 Package mysql provides a MySQL based implementation of persistent state management for the goshawk tool.
Package x509ext holds extensions types and values for minimal gossip.
 Package x509ext holds extensions types and values for minimal gossip.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.