auditlog

README

auditlog

As part of the development of SoftKSM, I've been thinking about how to include audit logs. Normal logs are used to convey information about a program's operation; audit logs attempt to provide an accountable audit trail for security events. In SoftKSM, for example, an audit log might record when an administrative action was taken, or when a key was generated, or that a cryptographic operation was performed. Audit logs attempt to provide a security snapshot of events for accounting and incident investigations.

Audit logs have several characteristics:

1. The logs must be immutable: the audit logger should not provide the capability to remove or tamper with events.
2. Audit logs are a chain of events: they are ordered. Just as in any other investigation, it's important to establish an order of events.
3. The provenance of a given event must be established. Provenance means that some proof of the event's origin must be presented.
4. The audit logger must provide a means of showing and certifying a chain of events. That is, the investigator must be able to view the chain of events and have the audit logger provide a digital signature on the events.

The model taken by auditlog is based on events defined as

 type Attribute struct {
     Name  string
     Value string
 }

type Event struct {
    When       int64
    Received   int64
    Serial     int64
    Level      string
    Actor      string
    Event      string
    Attributes []Attribute
    Signature  []byte
}

The When field records when the event was reported, and the Received field records when the event was entered into the chain. All timestamps used have a nanosecond resolution. The Serial field records which entry number the event is. The Level field is user-defined, but the proof-of-concept implementation uses the standard "DEBUG", "INFO", "WARNING", "ERROR", and "CRITICAL" fields. The Actor field is used to record who the event belongs to, and the Event field contains a description of the event. Attributes provide additional details, and the Signature field stores the ECDSA signature on the event.

The signature is generated from the SHA-256 digest of each of these fields, as well as the signature of the previous event. The chain is a Merkle tree of events, each dependent on the previous event. Each signature is dependent on every field in the event, including the Received and Serial fields; this makes tampering with the order of the logs immediately evident.

The auditor produces Certifications on request, which are defined as

type Certification struct {
    When   int64
    Chain  []*Event
    Errors []*ErrorEvent
    Public []byte
}

Example Usage

A new logger is created with a database file and a signature key. If the database file exists, the audit chain is verified.

const auditLogPath = "/var/lib/ksm/audit.db"

signer, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
    // Handle the error appropriately.
}

logger, err := auditlog.New(auditLogPath, signer)
if err != nil {
    // Handle the error appropriately.
}

There are seven functions for writing logs:

• Info
• InfoSync
• Warning
• WarningSync
• Error
• ErrorSync
• CriticalSync

They all take the same functions; their names indicate what log level they use. The Sync suffix indicates that the function will wait for the event to be recorded in the database (this takes on the order of tens to hundreds of milliseconds, on average).

The following example might be used in an authentication system, noting that a user logged in:

    attr := auditlog.Attribute{"username", "jqp"}
    logger.Info("auth", "login", []auditlog.Attribute{attr})

Certifications

A Certification contains a list of audit records. A formatted example certification produced (by default, certifications are not pretty-printed) looks like

{
    "chain": [
        {
            "Actor": "logger_test",
            "Attributes": [
                {
                    "Name": "test",
                    "Value": "123"
                },
                {
                    "Name": "foo",
                    "Value": "bar"
                },
                {
                    "Name": "baz",
                    "Value": "quux"
                }
            ],
            "Event": "generic",
            "Level": "INFO",
            "Received": 1412594956023501655,
            "Serial": 0,
            "Signature": "MEUCIFwNIaM7Hck6uyFStvgi2zolZgemxXdHVW/YshkZJhaJAiEAzuCDjJUa0JlPcI0IUcwFhiYSNy+2jeWtAYGXKfVV2n8=",
            "When": 1412594956023495772
        },
        {
            "Actor": "logger_test",
            "Attributes": [
                {
                    "Name": "test",
                    "Value": "123"
                },
                {
                    "Name": "foo",
                    "Value": "bar"
                },
                {
                    "Name": "baz",
                    "Value": "quux"
                }
            ],
            "Event": "warning",
            "Level": "WARNING",
            "Received": 1412594956041156294,
            "Serial": 1,
            "Signature": "MEUCIE7wA94TvIZrcNmQO3QoNrn9rvsjTsAguE581zXNwyq1AiEAvrqXRvCNsZCUm49QVxG3OBlnKWru9emzizgN1Qm8/zM=",
            "When": 1412594956026100241
        },
        {
            "Actor": "actor0",
            "Attributes": null,
            "Event": "ping",
            "Level": "INFO",
            "Received": 1412594956068389191,
            "Serial": 2,
            "Signature": "MEUCIQDf02F9xwimcmlKv0fZAznJkJxetd80H8kZgQYdZyOR+QIgfG3MoWV45IzOq7FZoxOTb32WPZnaa90dikKj70PSxzo=",
            "When": 1412594956056844823
        },
        {
            "Actor": "actor1",
            "Attributes": null,
            "Event": "ping",
            "Level": "INFO",
            "Received": 1412594956086931409,
            "Serial": 3,
            "Signature": "MEUCIBAr0HxDOu9T3bk/e6rCKls6zqILk+8N5vNVjmtm6L3iAiEAhRvG4fm5VgofJJwJuUhiJdgXAVb4To1wOONn64My6h0=",
            "When": 1412594956056848561
        }
    ],
    "errors": [
        {
            "event": {
                "Actor": "auditlog_test",
                "Attributes": null,
                "Event": "PRNG failure",
                "Level": "INFO",
                "Received": 1412594956056174667,
                "Serial": 2,
                "Signature": null,
                "When": 1412594956041239087
            },
            "message": "signature: EOF",
            "when": 1412594956056326920
        }
    ],
    "when": 1412594956803267916
}

The public key used to generate this certification is

-----BEGIN EC PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEtySFtEw1pvr1F8SngKxAoIwlUmzf
AS20/9IH1u/+jNEQT8rw2e84Oytrces8p49bcv/3jkmNG/VZDmpj7FlxuA==
-----END EC PUBLIC KEY-----

The verify_audit_chain tool will verify the chain, and save a formatted, verified chain:

$ verify_audit_chain certified.json
Verifying certified.json
OK: writing logs to verified_logs_0.json

verify_audit_chain can be installed with

go get github.com/kisom/auditlog/verify_audit_log

Database

auditlog uses Postgres as the backend. The SQL file containing the schema can be found in auditlog.sql.

License

auditlog is released under the ISC license.

Documentation

Overview

Package auditlog implements auditable logs for recording security events. The logs are currently backed by SQLite3. They are designed to form a chain of auditable, tamper-evident logs. The chain is a tree of signatures where the signature on each event is computed over both the event and the previous event's signature.

The audit logger is concerned with events. For example, an event might be recorded when a user logs in, or an administrative action is carried out.

Index

• type Attribute
• type Certification
  • func VerifyCertification(in []byte, signer *ecdsa.PublicKey) (*Certification, bool)
• type DBConnDetails
  • func (cd DBConnDetails) String() string
• type ECDSASignature
• type ErrorEvent
• type Event
  • func (ev *Event) String() string
  • func (ev *Event) Verify(signer *ecdsa.PublicKey, prev []byte) bool
• type Logger
  • func New(cd *DBConnDetails, signer *ecdsa.PrivateKey) (*Logger, error)
  • func (l *Logger) Certify(start, end uint64) ([]byte, error)
  • func (l *Logger) Count() uint64
  • func (l *Logger) CriticalSync(actor, event string, attributes []Attribute)
  • func (l *Logger) Debug(actor, event string, attributes []Attribute)
  • func (l *Logger) Error(actor, event string, attributes []Attribute)
  • func (l *Logger) ErrorSync(actor, event string, attributes []Attribute)
  • func (l *Logger) Info(actor, event string, attributes []Attribute)
  • func (l *Logger) InfoSync(actor, event string, attributes []Attribute)
  • func (l *Logger) Public() ([]byte, error)
  • func (l *Logger) RootSignature() ([]byte, error)
  • func (l *Logger) Start() error
  • func (l *Logger) Stop()
  • func (l *Logger) Warning(actor, event string, attributes []Attribute)
  • func (l *Logger) WarningSync(actor, event string, attributes []Attribute)

Types

type Attribute

type Attribute struct {
	Name  string
	Value string
}

An Attribute is used to encode additional details about an event. An example attribute might be

Attribute{
        Name: "user",
        Value: "root",
}

type Certification

type Certification struct {
	When   int64         `json:"when"`
	Chain  []*Event      `json:"chain"`
	Errors []*ErrorEvent `json:"errors"`
}

A Certification contains a snapshot an audit chain, errors that occurred in the range of events, and a nanosecond-resolution timestamp of when the certification was built.

func VerifyCertification

func VerifyCertification(in []byte, signer *ecdsa.PublicKey) (*Certification, bool)

VerifyCertification verifies a JSON-encoded certification against the signer's public key.

type DBConnDetails

type DBConnDetails struct {
	Name, User, Password, Host, Port string
	SSL                              bool
}

DBConnDetails contains the connection parameters for the database.

func (DBConnDetails) String

func (cd DBConnDetails) String() string

type ECDSASignature

type ECDSASignature struct {
	R, S *big.Int
}

An ECDSASignature is the structure into which an ECDSA signature is packed.

type ErrorEvent

type ErrorEvent struct {
	When    int64  `json:"when"`
	Message string `json:"message"`
	Event   *Event `json:"event"`
}

An ErrorEvent is stored in the error log; these are used to record a failure of the auditor to sign and store an event. The event contained in the ErrorEvent stores the serial number the event would have been assigned, which will be reused by future events. These are recorded on the following failures: database failures (failure to begin or commit a transaction, or when the database returns a failure), and failure to compute a signature.

type Event

type Event struct {
	// Serial is the event's position in the audit chain.
	Serial uint64

	// When is a nanosecond-resolution timestamp recording when
	// the event was logged.
	When int64

	// Received is a nanosecond-resolution timestamp recording
	// when the event was processed by the audit logger.
	Received int64

	// Level contains a text description inidicating the log
	// level; this is currently defined as one of the strings
	// "DEBUG", "INFO", "WARNING", "ERROR", or "CRITICAL".
	Level string

	// Actor indicates the component that reported the event.
	Actor string

	// Event contains a text description of the event that
	// occurred.
	Event string

	// Attributes is an (optional) list of additional details that
	// may be relevant to the event.
	Attributes []Attribute

	// Signature contains the audit logger's ECDSA signature on
	// the event. This signature is computed on the SHA-256 digest
	// of all the other fields in the event and the previous event
	// in the chain's signature.
	Signature []byte
	// contains filtered or unexported fields
}

An Event captures information about an event.

func (*Event) String

func (ev *Event) String() string

String returns a string for the event. The timestamp is formatted to second-resolution RFC3339 format.

func (*Event) Verify

func (ev *Event) Verify(signer *ecdsa.PublicKey, prev []byte) bool

Verify checks the signature on the event. The prev argument should be the previous event's signature.

type Logger

type Logger struct {
	// contains filtered or unexported fields
}

A Logger is responsible for recording security events.

func New

func New(cd *DBConnDetails, signer *ecdsa.PrivateKey) (*Logger, error)

New sets up a new logger, using the signer for signatures and backed by the database at the specified file. If the database exists, the audit chain will be verified.

func (*Logger) Certify

func (l *Logger) Certify(start, end uint64) ([]byte, error)

Certify returns a certification for the requested range of events; start and end are event serial numbers. The certification is returned in JSON.

func (*Logger) Count

func (l *Logger) Count() uint64

Count returns the number of recorded events.

func (*Logger) CriticalSync

func (l *Logger) CriticalSync(actor, event string, attributes []Attribute)

CriticalSync records a critical failure of this system. This is almost always followed by a shutdown, and therefore only a synchronous version that waits for the event to be recorded is provided.

func (*Logger) Debug

func (l *Logger) Debug(actor, event string, attributes []Attribute)

Debug records a debug event. In practice, this should not be used; it is intended only for debugging the audit logger. This does not wait for the audit logger to finish recording the event.

func (*Logger) Error

func (l *Logger) Error(actor, event string, attributes []Attribute)

Error records an error event. An example might be an authentication failure. This does not wait for the audit logger to finish recording the event.

func (*Logger) ErrorSync

func (l *Logger) ErrorSync(actor, event string, attributes []Attribute)

ErrorSync performs the same function as error, except it waits for the event to be recorded.

func (*Logger) Info

func (l *Logger) Info(actor, event string, attributes []Attribute)

Info records an informational event. This probably includes events that are expected normally. This does not wait for the audit logger to finish recording the event.

func (*Logger) InfoSync

func (l *Logger) InfoSync(actor, event string, attributes []Attribute)

InfoSync performs the same function as Info, except it waits for the event to be recorded.

func (*Logger) Public

func (l *Logger) Public() ([]byte, error)

Public returns the public signature key packed as in DER-encoded PKIX format.

func (*Logger) RootSignature

func (l *Logger) RootSignature() ([]byte, error)

RootSignature returns the signature of the root event (i.e. the event with serial = 0). The user can store a copy of this, and use it to ensure the root of the chain has not been tampered with.

func (*Logger) Start

func (l *Logger) Start() error

Start starts up the audit logger. This must be called prior to logging events.

func (*Logger) Stop

func (l *Logger) Stop()

Stop halts the logger and cleanly shuts down the database connection.

func (*Logger) Warning

func (l *Logger) Warning(actor, event string, attributes []Attribute)

Warning records an event that isn't an error, but it is a more urgent event. Examples of warning events might be users selecting a deprecated cipher. This does not wait for the audit logger to finish recording the event.

func (*Logger) WarningSync

func (l *Logger) WarningSync(actor, event string, attributes []Attribute)

WarningSync performs the same function as Warning, except it waits for the event to be recorded.