vault-audit-filter

README

Vault Audit Filter

vault-audit-filter is a Go-based tool designed to filter and log HashiCorp Vault audit logs based on configurable rules. It provides fine-grained control over how Vault audit events are processed and categorized, allowing you to capture critical events while reducing noise from routine operations.

Features

• Configurable Rule-Based Filtering: Define rules to match specific audit events, such as read, write, delete, or specific paths in Vault.
• Multiple Rule Groups: Organize rules into groups and log them to separate files.
• Dynamic Logging: Log audit events to specified files with log rotation and size limits.
• Supports Multiple Operations: Filters common Vault operations, including KV operations, metadata updates, and deletion events.
• Performance-Oriented: Built with gnet to handle high concurrency.

Table of Contents

• Getting Started
• Installation
• Configuration
• Usage
• Testing
• Contributing
• License

Getting Started

These instructions will help you set up and run vault-audit-filter on your local machine.

Prerequisites

• Go: Ensure you have Go 1.21 or later installed. You can download it here: https://golang.org/dl/
• Vault: You should have HashiCorp Vault installed and configured. Instructions can be found here: https://www.vaultproject.io/docs/install

Installation

Clone the repository:

git clone https://github.com/ncode/vault-audit-filter.git
cd vault-audit-filter

Build the Project

To build the binary:

go build -o vault-audit-filter .

Running the Application

Once you have built the project, you can run the vault-audit-filter executable:

./vault-audit-filter --config config.yaml

Configuration

vault-audit-filter uses a YAML-based configuration file that allows you to define rule groups, specify logging files, and configure Vault settings.

Sample Configuration (config.yaml)

vault:
  address: "http://127.0.0.1:8200"
  token: "your-vault-token"
  audit_path: "/vault-audit-filter"
  audit_address: "127.0.0.1:1269"
  audit_description: "Vault Audit Filter Device"

rule_groups:
  - name: "normal_operations"
    rules:
      - 'Request.Operation in ["read", "update"] && Request.Path startsWith "secret/data/" && Auth.PolicyResults.Allowed == true'
    log_file:
      file_path: "/var/log/vault_normal_operations.log"
      max_size: 100      # Max size in MB
      max_backups: 5     # Max number of backup files
      max_age: 30        # Max age in days
      compress: true     # Compress rotated files

  - name: "critical_events"
    rules:
      - 'Request.Operation == "delete" && Auth.PolicyResults.Allowed == true'
      - 'Request.Path startsWith "secret/metadata/" && Auth.PolicyResults.Allowed == true'
    log_file:
      file_path: "/var/log/vault_critical_events.log"
      max_size: 100
      max_backups: 5
      max_age: 30
      compress: true

Configuration Parameters

• Vault Settings:

  • vault.address: The address of your Vault instance.
  • vault.token: Vault token for authentication.
  • vault.audit_path: The path for Vault's audit device.
  • vault.audit_address: The address for receiving audit logs.
  • vault.audit_description: Description for the Vault audit device.

• Rule Groups:

  • rule_groups.name: The name of the rule group.
  • rule_groups.rules: A list of expressions using expr to define rules for audit log filtering.
  • log_file.file_path: The file path where matching logs will be written.
  • log_file.max_size: The maximum size of the log file in MB before rotation.
  • log_file.max_backups: The number of backup logs to keep.
  • log_file.max_age: The maximum number of days to retain logs.
  • log_file.compress: Whether to compress the old log files.

Rule Syntax

Rules are written using the expr language, a simple and safe expression language for Go. Rules can be based on the following properties of audit logs:

• Request.Operation: The type of operation (read, update, delete, etc.).
• Request.Path: The Vault path being accessed.
• Auth.PolicyResults.Allowed: Whether the operation was allowed.

Example Rule:

'Request.Operation == "update" && Request.Path startsWith "secret/data/" && Auth.PolicyResults.Allowed == true'

Usage

To run vault-audit-filter with your configuration file, use:

$ ./vault-audit-filter --config config.yaml

Command-Line Options

• --config: Specify the path to the configuration file (default is config.yaml).
• --log-level: Set the logging level (debug, info, warn, error).

Environment Variables

You can also define environment variables to override configuration file values. For example:

$ export VAULT_ADDRESS="http://127.0.0.1:8200"
$ export VAULT_TOKEN="your-vault-token"

Development

For development purposes, you can use the provided Makefile located at configs/development/Makefile to build and run the project using Docker and Docker Compose. This is how I test my changes and have a playground of sorts.

Contributing

We welcome contributions from the community! Before submitting a pull request, ensure that:

• The code compiles without errors.
• All tests pass.
• Your changes are well-documented.

License

This project is licensed under the Apache License, Version 2.0. See the LICENSE file for details.

Documentation

Overview

Copyright © 2024 Juliano Martinez

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.