diffence

README

diffence

• Checks a git diff for passwords/secret keys accidentally committed
• Golang 1.7+

---

Check the entire history of current branch for passwords/keys committed

$ git log -p | diffence

Example

$ git log -p --full-diff | diffence

------------------
Violation 1
Commit: 4cc087a1b4731d1017844cc86323df43068b0409
File: web/src/db/seed.sql
Reason: "SQL dump file"

------------------
Violation 2
Commit: 142e6019248c0d53a5240242ed1a75c0cc110a0b
File: config/passwords.ini
Reason: "Contains word: password"

---

Add false positives to .secignore

$ cat .secignore
file/that/is/not/really/a/secret/but/looks/like/one/to/diffence
these/pems/are/ok/*.pem

See example in this repo.

---

Install

Binary

Download the latest stable release.

CLI

$ go get -u github.com/techjacker/diffence/cmd/diffence

Library

$ go get -u github.com/techjacker/diffence

---

CLI tool

Example Usage

$ touch key.pem

$ git add -N key.pem

$ git diff --stat HEAD
gds HEAD
 key.pem | 0
 1 file changed, 0 insertions(+), 0 deletions(-)

$ git diff HEAD |diffence
File key.pem violates 1 rules:

Caption: Potential cryptographic private key
Description: <nil>
Part: extension
Pattern: pem
Type: match

---

Rules

• Analyse fPaths with gitrob rules
• Analyse added lines - need to find/create ruleset that can analyse file contents
• Add option to use your own rules again file path/contents

---

Tests

$ go test ./...

---

Local Development

Build & Run Locally

$ go install -race ./cmd/diffence

OR

$ go build -race ./cmd/diffence

Check for race conditions

$ go run -race ./cmd/diffence/main.go

Documentation

Index

• Constants
• func LoadDefaultRules() (*[]Rule, error)
• func LoadRulesJSON(fPath string) (*[]Rule, error)
• func LoadRulesJSONFromPwd(rulesPath string) *[]Rule
• func ScanDiffs(data []byte, atEOF bool) (advance int, token []byte, err error)
• func SplitDiffHashKey(s string) (string, string)
• func SplitDiffs(r io.Reader, l List) error
• type Checker
• type Diff
  • func (d *Diff) Push(s string)
• type DiffChecker
  • func (dc DiffChecker) Check(r io.Reader) (Result, error)
• type DiffItem
  • func (d *DiffItem) GetHashKey() string
• type Ignorer
  • func NewIgnorer(r io.Reader) *Ignorer
  • func NewIgnorerFromFile(fPath string) *Ignorer
  • func (i Ignorer) Match(in string) bool
• type List
• type Logger
• type MatchedRules
• type Matcher
• type Result
  • func (r Result) Log(l Logger)
  • func (r Result) Matches() int
• type Results
  • func (r Results) Matches() int
• type Rule
  • func (r *Rule) Match(in string) bool
  • func (r *Rule) String() string

Constants

const (
	// RuleTypeRegex is the regex type for pattern matching
	RuleTypeRegex = "regex"

	// RuleTypeMatch is the string match type for pattern matching
	RuleTypeMatch = "match"
)

https://github.com/michenriksen/gitrob#signature-keys

const (
	// RulePartPath checks the whole path of the file
	RulePartPath = "path"

	// RulePartFilename checks the name of the file
	RulePartFilename = "filename"

	// RulePartExtension checks the extension of the file
	RulePartExtension = "extension"
)

Functions

func LoadDefaultRules

func LoadDefaultRules() (*[]Rule, error)

LoadDefaultRules unmarshalls the go generated byte slice of the gitrob JSON rules file

func LoadRulesJSON

func LoadRulesJSON(fPath string) (*[]Rule, error)

LoadRulesJSON reads a file of JSON rules from the local filesystem

func LoadRulesJSONFromPwd

func LoadRulesJSONFromPwd(rulesPath string) *[]Rule

LoadRulesJSONFromPwd reads a rules JSON from a path relative to the process's pwd

func ScanDiffs

func ScanDiffs(data []byte, atEOF bool) (advance int, token []byte, err error)

ScanDiffs splits on the diff of an inidividual file

func SplitDiffHashKey

func SplitDiffHashKey(s string) (string, string)

SplitDiffHashKey splits a DiffItem's hash key

func SplitDiffs

func SplitDiffs(r io.Reader, l List) error

SplitDiffs splits a single diff txt into an individual DiffItem for each file changed

Types

type Checker

type Checker interface {
	Check(io.Reader) (Result, error)
}

Checker checks diffs for rule violations

type Diff

type Diff struct {
	Items []DiffItem
	Error error
	// contains filtered or unexported fields
}

Diff is a list of split diffs

func (*Diff) Push

func (d *Diff) Push(s string)

Push a diff on to the list

type DiffChecker

type DiffChecker struct {
	Rules   *[]Rule
	Ignorer Matcher
}

DiffChecker checks an io.Reader for matches against the supplied ruleset

func (DiffChecker) Check

func (dc DiffChecker) Check(r io.Reader) (Result, error)

Check is a clean syntax but memory inefficient method for finding diffs that match the supplied rules (use an array instead of a map for better performance)

type DiffItem

type DiffItem struct {
	// contains filtered or unexported fields
}

DiffItem is a diff struct for an inidividual file

func (*DiffItem) GetHashKey

func (d *DiffItem) GetHashKey() string

GetHashKey returns the hash key identifier for the diff

type Ignorer

type Ignorer struct {
	// contains filtered or unexported fields
}

Ignorer is used to exclude content in .secignore files

func NewIgnorer

func NewIgnorer(r io.Reader) *Ignorer

NewIgnorer will create an Ignorer from a read stream

func NewIgnorerFromFile

func NewIgnorerFromFile(fPath string) *Ignorer

NewIgnorerFromFile will safely create an Ignorer whether the file exists or not

func (Ignorer) Match

func (i Ignorer) Match(in string) bool

Match reports whether a filepath is listed in Ignorer.patterns[]string

type List

type List interface {
	Push(string)
}

List is an interface for adding items to a list

type Logger

type Logger interface {
	Print(v ...interface{})
	Printf(format string, v ...interface{})
}

Logger is the logger interface

type MatchedRules

type MatchedRules map[string][]Rule

MatchedRules is slice of matched rules for each file in diff [fPath] => Rule{rule1, rule2}

type Matcher

type Matcher interface {
	Match(string) bool
}

Matcher is an interface for matching against string inputs

type Result

type Result struct {
	// Have any of the files matches against the rules?
	Matched      bool
	MatchedRules MatchedRules
}

Result compiles the results of matched rules for a diff

func (Result) Log

func (r Result) Log(l Logger)

Log prints the results of ma

func (Result) Matches

func (r Result) Matches() int

Matches returns the number of files in the diff that matched against any of the rules

type Results

type Results []Result

Results is a slice of Result structs

func (Results) Matches

func (r Results) Matches() int

Matches returns the number of diffs which had at least one file match against any rules

type Rule

type Rule struct {
	Caption     string      `'Rule':"caption"`
	Description interface{} `'Rule':"description"`
	Part        string      `'Rule':"part"`
	Pattern     string      `'Rule':"pattern"`
	Type        string      `'Rule':"type"`
}

Rule defines a pattern to match against a diff

func (*Rule) Match

func (r *Rule) Match(in string) bool

Match runs rules against input strings

func (*Rule) String

func (r *Rule) String() string

String returns a string representation of the rule