password

README

password

Dictionary Password Validation for Go.

Protect your bcrypt/scrypt/PBKDF encrypted passwords against dictionary attacks.

Motivated by Password Requirements Done Better - or why password requirements help hackers

This library will help you import a password dictionary and will allow you to validate new/changed passwords against the dictionary.

You are able to use your own database and password dictionary. Currently the package supports importing dictionaries similar to CrackStation's Password Cracking Dictionary, and has "drivers" for MongoDB, BoltDB, MySQL and PostgreSQL. For a feasible in-memory database see the Bloom filter driver

installation

As always, the package is installed with go get github.com/klauspost/password.

usage

With this library you can:

1. Import a password dictionary into your database
2. Check new passords against the dictionary
3. Sanitize passwords before authenticating a user

All of the 3 functionality parts can be used or replaced as it suits your application. In particular you probably do not want to import dictionaries on your webserver, so you can separate that functionality into a separate command.

setting up a database

To use the built-in drivers, see the documentation for them. But here is an example of how to set up a Bolt database:

import(
	"github.com/boltdb/bolt"
	"github.com/klauspost/password"
	"github.com/klauspost/password/drivers/boltpw"
)

	// Open the database using the Bolt driver
	// You probably have this elsewhere if you already use Bolt
  	db, err := bolt.Open("password.db", 0666, nil)
	if err != nil {
		panic(err)
	}
	defer db.Close()

So far pretty standard. We open the database as we always would. This is used by the driver in github.com/klauspost/password/drivers/boltpw to write and check passwords.

	// Use the driver to read/write to the bucket "commonpwd"
	chk, err := boltpw.New(db, "commonpwd")
	if err != nil {
		t.Fatal(err)
	} 

The object we get back can then be used to check passwords, assuming you have imported a database.

	err = password.Check(chk, "SecretPassword", nil)
	if err != nil {
		// Password failed sanitazion or was in database.
		panic(err)
	}

importing a dictionary

Example that will import the crackstation into memory. Replace testdb.NewMemDBBulk() with a constructor to the database you want to use.

import (
	"os"
	
	"github.com/klauspost/password"
	"github.com/klauspost/password/drivers/testdb"
	"github.com/klauspost/password/tokenizer"
)

func Import() {
	r, err := os.Open("crackstation-human-only.txt.gz")
	if err != nil {
	  panic(err)
	}
	mem := testdb.NewMemDBBulk()
	in, err := tokenizer.NewGzLine(r)
	if err != nil {
		panic(err)
	}
	err = password.Import(in, mem, nil)
	if err != nil {
		panic(err)
	}
}

checking a password

This is an example of checking and preparing a password to be stored in the database. Passwords allowed to be full UTF8, and are compared case insensitively.

func PreparePassword(db password.DB, toCheck string)  (string, error) {
	err := password.Check(db, toCheck, nil)
	if err != nil {
		// Password failed sanitazion or was in database.
		return "", err
	}
	
	// We use the default sanitizer to sanitize/normalize the password
	toStore, _ := password.Sanitize(toCheck, nil)
	if err != nil {
		// Shouldn't happen, since we already passed sanitaztion in the check once
		// File a bug if it does.
		panic(err)
	}

	// bcrypt the result and return it
	return bcrypt.GenerateFromPassword([]byte(toStore), 12)
}

sanitizers

You can replace the sanitizer with your own when checking passwords. This can be used to reject passwords that match username, email, you site name and similar information you might have on the user. For an example of that, see the Sanitizer interface.

You can use different sanitizers for importing a dictionaries and checking individual passwords. You should run the sanitizer on all passwords before checking or encrypting them for storage, as proposed in the "checking a password" above.

dictionaries

• CrackStation's Password Cracking Dictionary

Contains a very good dictionary. Their "Human Passwords Only" is very good at catching common bad passwords, and is a good base dicitonary. Can be opened with tokenizer.NewGzLine.

Here is a HTTP download provided by me, please use only if you cannot download torrents. I have recompressed them for a smaller download size:

• Human Passwords Only - 209MB.
• Full Dictionary - 3.5GB

License is CC-by-SA. This license allows you to use the data commercially.

• SkullSecurity Passwords

Mostly small and varying quality. Can be opened with tokenizer.NewBz2Line.

• g0tmi1k Dictionaries + Wordlists.

Hard to download. 18-in-1 has a lot of sameword1; sameword2, etc. Mostly ascii passwords. Needs to be uncompressed or recompressed.

• klauspost "paranoid passwords" dictionary

I have created a dictionary by combining 'Crackstation', 'g0tmi1k' and 'WPA-PSK WORDLIST 3 Final'. The passwords are all in lower-case, unicode KD-normalized, unique and sorted.

• Download dictionary. 1123 Million entries, 3.1GB gzipped.

Note: This dictionary cannot be used for password retrival. Released as CC-by-SA.

compatibility

Unless security related issues should show up, the interfaces and functions should not change in this package. If it is impossible to remain compatible, it will always be shown by a compiler error. So if the library compiles after an update it will remain compatible.

license

This code is published under an MIT license. See LICENSE file for more information.

Documentation

Overview

Dictionary Password Validation for Go

For usage and examples see: https://github.com/klauspost/password (or open the README.md)

This library will help you import a password dictionary and will allow you to validate new/changed passwords against the dictionary.

You are able to use your own database and password dictionary. Currently the package supports importing dictionaries similar to CrackStation's Password Cracking Dictionary: https://crackstation.net/buy-crackstation-wordlist-password-cracking-dictionary.htm

It and has "drivers" for various backends, see the "drivers" directory, where there are implementations and a test framework that will help you test your own drivers.

Index

• Variables
• func Check(password string, db DB, san Sanitizer) error
• func Import(in Tokenizer, out DbWriter, san Sanitizer) (err error)
• func Sanitize(password string, san Sanitizer) (string, error)
• func SanitizeOK(password string, san Sanitizer) error
• type BulkWriter
• type DB
• type DbWriter
• type Sanitizer
• type Tokenizer

Examples

• Import
• Import (Xz)
• Sanitizer

Variables

var BulkMax = 1000

BulkMax is the maximum number of passwords sent at once to the writer. You can change this before starting an import.

var ErrInvalidString = errors.New("invalid utf8 sequence")

ErrInvalidString is returned by the default sanitizer if the string contains an invalid utf8 character sequence.

var ErrPasswordInDB = errors.New("password found in database")

ErrPasswordInDB is returedn by Check, if the password is in the database.

var ErrSanitizeTooShort = errors.New("password too short")

ErrSanitizeTooShort is returned by the default sanitizer, if the input password is less than 8 runes.

var Logger = log.New(os.Stdout, "", log.LstdFlags)

Logger used for output during Import. This can be exchanged with your own.

Functions

func Check

func Check(password string, db DB, san Sanitizer) error

Check a password against the database. It will return an error if:

• Sanitazition fails.
• DB lookup returns an error
• Password is in database (ErrPasswordInDB)

If nil is passed as Sanitizer, DefaultSanitizer will be used.

func Import

func Import(in Tokenizer, out DbWriter, san Sanitizer) (err error)

Import will populate a database with common passwords.

You must supply a Tokenizer (see tokenizer package for default tokenizers) that will deliver the passwords, a DbWriter, where the passwords will be sent, and finally a Sanitizer to clean up the passwords - - if you send nil DefaultSanitizer will be used.

Example

r, err := os.Open("./testdata/testdata.txt.gz")
if err != nil {
	panic("cannot open file")
}
// Create a database to write to
mem := testdb.NewMemDBBulk()

// The input is gzipped text file with
// one input per line, so we choose a tokenizer
// that matches.
in, err := tokenizer.NewGzLine(r)
if err != nil {
	panic(err)
}
// Import using the default sanitizer
err = Import(in, mem, nil)
if err != nil {
	panic(err)
}
// Data is now imported, let's do a check
// Check a password that is in the sample data
err = Check("tl1992rell", mem, nil)
fmt.Println(err)

Output:

password found in database

Example (Xz)

Open a xz compressed archive and import it. Uses the "xi2.org/x/xz" package to read xz files.

r, err := os.Open("rockyou.txt.xz")
if err != nil {
	// Fake it
	fmt.Println("Imported", 9341543, "items")
	return
}
xzr, err := xz.NewReader(r, 0)
if err != nil {
	panic(err)
}
mem := testdb.NewMemDBBulk()
in := tokenizer.NewLine(xzr)
err = Import(in, mem, nil)
if err != nil {
	panic(err)
}
fmt.Println("Imported", len(*mem), "items")

Output:

Imported 9341543 items

func Sanitize

func Sanitize(password string, san Sanitizer) (string, error)

Sanitize will sanitize a password, useful before hashing and storing it.

If the sanitizer is nil, DefaultSanitizer will be used.

func SanitizeOK

func SanitizeOK(password string, san Sanitizer) error

SanitizeOK can be used to check if a password passes the sanitizer.

If the sanitizer is nil, DefaultSanitizer will be used.

Types

type BulkWriter

type BulkWriter interface {
	AddMultiple([]string) error
}

If your DbWriter implements this, input will be sent in batches instead of using Add.

type DB

type DB interface {
	Has(string) (bool, error)
}

A DB should check the database for the supplied password. The password sent to the interface has always been sanitized.

type DbWriter

type DbWriter interface {
	Add(string) error
}

A DbWriter is used for adding passwords to a database. Items sent to Add has always been sanitized, however the same passwords can be sent multiple times.

type Sanitizer

type Sanitizer interface {
	Sanitize(string) (string, error)
}

A Sanitizer should prepare a password, and check the basic properties that should be satisfied. For an example, see DefaultSanitizer

Example

This example shows how to create a custom sanitizer that checks if the password matches the username or email.

CustomSanitizer is defined as:

type CustomSanitizer struct {
    email string
    username string
}

func (c CustomSanitizer) Sanitize(s string) (string, error) {
    s, err := DefaultSanitizer.Sanitize(s)
    if err != nil {
        return "", err
    }
    if strings.EqualFold(s, c.email) {
        return "", errors.New("password cannot be the same as email")
    }
    if strings.EqualFold(s, c.username) {
        return "", errors.New("password cannot be the same as user name")
    }
    return s, nil
}

// Create a custom sanitizer.
san := CustomSanitizer{email: "john@doe.com", username: "johndoe73"}

// Check some passwords
err := SanitizeOK("john@doe.com", san)
fmt.Println(err)

err = SanitizeOK("JohnDoe73", san)
fmt.Println(err)

err = SanitizeOK("MyP/|$$W0rd", san)
fmt.Println(err)

Output:

password cannot be the same as email
password cannot be the same as user name
<nil>

var DefaultSanitizer Sanitizer

DefaultSanitizer should be used for adding passwords to the database. Assumes input is UTF8.

DefaultSanitizer performs the following sanitazion:

• Trim space, tab and newlines from start+end of input
• Check that there is at least 8 runes. Return ErrSanitizeTooShort if not.
• Check that the input is valid utf8. Return ErrInvalidString if not.
• Normalize input using Unicode Normalization Form KD

If input is less than 8 runes ErrSanitizeTooShort is returned.

type Tokenizer

type Tokenizer interface {
	Next() (string, error)
}

Tokenizer delivers input tokens (passwords). Calling Next() should return the next password, and when finished io.EOF should be returned.

It is ok for the Tokenizer to send empty strings and duplicate values.