pillager

README

Pillager

Pillage filesystems for sensitive information with Go.

Table of Contents

1. Summary
2. Installation
3. Usage
4. Documentation

Summary

Pillager is designed to provide a simple means of leveraging Go's strong concurrency model to recursively search directories for sensitive information in files. Pillager does this by standing on the shoulders of a few giants. Once pillager finds files that match the specified pattern, the file is scanned using a series of concurrent workers that each take a line of the file from the job queue and hunt for sensitive pattern matches. The available pattern filters can be defined in a pillager.toml file or you can use the default ruleset.

Installation

Go

If you have Go setup on your system, you can install Pillager with go install

go install github.com/brittonhayes/pillager@latest

Scoop (Windows)

scoop bucket add pillager https://github.com/brittonhayes/pillager-scoop.git
scoop install pillager

Homebrew (OSX/Linux)

brew tap brittonhayes/homebrew-pillager
brew install pillager

Docker Image

docker run --rm -it ghcr.io/brittonhayes/pillager:latest hunt .

If you're looking for a binary, check the latest releases for the executable that matches your system

Usage

To see all the commands available with pillager

# To see instructions for the entire application
pillager

# From any subcommand
pillager [cmd] --help

User Interface

Pillager provides a terminal user interface built with bubbletea if you'd like to scan for secrets interactively.

Configuration

Gitleaks Rules

Pillager provides full support for Gitleaks^2 rules. This can either be passed in with a rules[^1] section in your pillager.toml file, or you can use the default ruleset by leaving the config flag blank.

[^1]: Gitleaks Rules Reference

# pillager.toml
# Basic configuration
verbose = false 
path = "."
workers = 4
redact = false 
reporter = "json-pretty"

# Rules for secret detection
[[rules]]
description = "AWS Access Key"
id = "aws-access-key"
regex = '''(A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}'''
tags = ["aws", "credentials"]

[[rules]]
description = "AWS Secret Key"
id = "aws-secret-key"
regex = '''(?i)aws(.{0,20})?(?-i)['\"][0-9a-zA-Z\/+]{40}['\"]'''
tags = ["aws", "credentials"]

[[rules]]
description = "GitHub Token"
id = "github-token"
regex = '''ghp_[0-9a-zA-Z]{36}'''
tags = ["github", "token"]

[[rules]]
description = "Private Key"
id = "private-key"
regex = '''-----BEGIN (?:RSA|OPENSSH|DSA|EC|PGP) PRIVATE KEY( BLOCK)?-----'''
tags = ["key", "private"]

# Allowlist configuration
[allowlist]
paths = [
    ".*/_test\\.go$",
    ".*/testdata/.*",
    ".*\\.md$",
    ".*/vendor/.*"
]
regexes = [
    "EXAMPLE_KEY",
    "DUMMY_SECRET"
] 

Built-in Output Formats

Pillager has a series of built-in output formats available. Pick your flavor!

Basic

pillager hunt .

JSON

pillager hunt ./example -f json | jq

Wordlist

# Use pillager to generate a new-line delimited wordlist from findings
pillager hunt . -f wordlist 

# Use pillager to append a wordlist and then use your favorite hashcat attack mode
pillager hunt ./ -f wordlist >> rockyou.txt && hashcat -a 0 hash.txt rockyou.txt

JSON output is designed to work seamlessly with the amazing jq utility for easy parsing.

Click to view more output formats

JSON Pretty

pillager hunt . -f json-pretty

HTML

pillager hunt . -f html > results.html

Markdown

pillager hunt . -f markdown > results.md

Markdown Table

pillager hunt . -f table > results.md

Custom Go Template

pillager hunt . --template "{{ range .}}Secret: {{.Secret}}{{end}}"

Custom Go Template from File

pillager hunt . -t "$(cat internal/templates/simple.tmpl)"

Custom Templates

Pillager allows you to use powerful go text/template and sprig functions to customize the output format. Here are a few template examples.

Basic

{{ range . -}}
    File: {{ .File }}
    Secret: {{ .Secret}}
    Description: {{ quote .Description }}
{{ end -}}

Markdown Styling

# Results

{{ range . -}}
    ## {{ .File }}
    - Location: {{.StartLine}}
{{end}}

More template examples can be found in the templates directory.

Documentation

GoDoc documentation is available on pkg.go.dev for pillager.

Development

To get involved developing features and fixes for Pillager, get started with the following:

• Install Go
• Install Taskfile.dev
• Read the CONTRIBUTING.MD

---

Shoulders of Giants ⭐

spf13's Cobra

What is Cobra?

Cobra is a library providing a simple interface to create powerful modern CLI interfaces similar to git & go tools. Cobra is also an application that will generate your application scaffolding to rapidly develop a Cobra-based application.

If you've seen a CLI written in Go before, there's a pretty high chance it was built with Cobra. I can't recommend this library enough. It empowers developers to make consistent, dynamic, and self-documenting command line tools with ease. Some examples include kubectl, hugo, and Github's gh CLI.

Gitleaks

What is Gitleaks?

Gitleaks^2 is a SAST tool for detecting hardcoded secrets like passwords, api keys, and tokens in git repos.

Gitleaks is an amazing tool for secret leak prevention. If you haven't implemented Gitleaks as a pre-commit checker, it's worth your time to check it out.

Why is Gitleaks relevant to Pillager?

Pillager implements the powerful rules functionality of Gitleaks while taking a different approach to presenting and handling the secrets found. While I have provided a baseline set of default rules, Pillager becomes much more powerful if you allow users to create rules for their own use-cases.

Check out the included rules[^1] for a baseline ruleset.

---

This goes without saying but I'm going to say it anyways: I am not responsible for any repercussions caused by your use of pillager. This tool is intended for defensive use, educational use, and security researcher use with the consent of all involved parties. Malicious behavior with pillager is in no way condoned, nor encouraged. Please use this tool responsibly and ensure you have permission to scan for secrets on any systems before doing so.

At it's core, Pillager is designed to assist you in determining if a system is affected by common sources of credential leakage as documented by the MITRE ATT&CK[^3] framework.

[^3]: MITRE ATT&CK Website

MITRE ATT&CK Technique - T1552,003 - Unsecured Credentials: Bash History

MITRE ATT&CK Technique - T1552,001 - Unsecured Credentials: Credentials In Files

Documentation

Overview

Package pillager is a tool for hunting through filesystems for sensitive information.

Installation

Go

go install github.com/brittonhayes/pillager@latest

Windows

scoop bucket add pillager https://github.com/brittonhayes/pillager-scoop.git
scoop install pillager

OSX/Linux

brew tap brittonhayes/homebrew-pillager
brew install pillager

Index

• type Allowlist
• type Finding
• type Options
• type Rule

Types

type Allowlist

type Allowlist struct {
	Paths   []string `toml:"paths"`
	Regexes []string `toml:"regexes"`
}

Allowlist represents paths and patterns to ignore

type Finding

type Finding struct {
	Description string
	StartLine   int
	EndLine     int
	StartColumn int
	EndColumn   int

	// Match is the full content of what is matched by the scanner.
	Match string

	// Secret contains the full content of what is matched in
	// the scanner query.
	Secret string

	// File is the name of the file containing the finding
	File string

	// Entropy is the shannon entropy of Value
	Entropy float32

	// Rule is the name of the rule that was matched
	RuleID string
}

Finding contains information about strings that have been captured by a scanner query.

type Options

type Options struct {
	Path      string    `toml:"path"`
	Template  string    `toml:"template"`
	Workers   int       `toml:"workers"`
	Verbose   bool      `toml:"verbose"`
	Redact    bool      `toml:"redact"`
	Reporter  string    `toml:"reporter"`
	Rules     []Rule    `toml:"rules"`
	Allowlist Allowlist `toml:"allowlist"`
}

Options holds configuration for scanners

type Rule

type Rule struct {
	ID          string   `toml:"id"`
	Description string   `toml:"description"`
	Path        string   `toml:"path"`
	Regex       string   `toml:"regex"`
	Keywords    []string `toml:"keywords"`
	Tags        []string `toml:"tags"`
	Allowlist   Allowlist
}

Rule represents a scanning rule