gomarkov

README

gomarkov

Go implementation of markov chains for textual data.

You can find out more about markov chains here and here

Usage

package main

import (
	"github.com/mb-14/gomarkov"
	"fmt"
	"strings"
	"io/ioutil"
	"encoding/json"
)

func main() {
	//Create a chain of order 2
	chain := gomarkov.NewChain(2)

	//Feed in training data
	chain.Add(strings.Split("I want a cheese burger", " "))
	chain.Add(strings.Split("I want a chilled sprite", " "))
	chain.Add(strings.Split("I want to go to the movies", " "))

	//Get transition probability of a sequence
	prob, _ := chain.TransitionProbability("a", []string{"I", "want"})
	fmt.Println(prob)
	//Output: 0.6666666666666666

	//You can even generate new text based on an initial seed
	chain.Add(strings.Split("Mother should I build the wall?", " "))
	chain.Add(strings.Split("Mother should I run for President?", " "))
	chain.Add(strings.Split("Mother should I trust the government?", " "))
	next, _ := chain.Generate([]string{"should", "I"})
	fmt.Println(next)

	//The chain is JSON serializable
	jsonObj, _ := json.Marshal(chain)
	err := ioutil.WriteFile("model.json", jsonObj, 0644)
	if err != nil {
		fmt.Println(err)
	}
}

Examples

• Gibberish username detector

• Fake Hackernews post generator

• Pokemon name generator

Documentation

Index

• Constants
• type Chain
  • func NewChain(order int) *Chain
  • func (chain *Chain) Add(input []string)
  • func (chain *Chain) Generate(current NGram) (string, error)
  • func (chain *Chain) GenerateDeterministic(current NGram, prng PRNG) (string, error)
  • func (chain Chain) MarshalJSON() ([]byte, error)
  • func (chain *Chain) TransitionProbability(next string, current NGram) (float64, error)
  • func (chain *Chain) UnmarshalJSON(b []byte) error
• type NGram
• type PRNG
• type Pair
  • func MakePairs(tokens []string, order int) []Pair

Constants

const (
	StartToken = "^"
	EndToken   = "$"
)

Tokens are wrapped around a sequence of words to maintain the start and end transition counts

Types

type Chain

type Chain struct {
	Order int
	// contains filtered or unexported fields
}

Chain is a markov chain instance

func NewChain

func NewChain(order int) *Chain

NewChain creates an instance of Chain

func (*Chain) Add

func (chain *Chain) Add(input []string)

Add adds the transition counts to the chain for a given sequence of words

func (*Chain) Generate

func (chain *Chain) Generate(current NGram) (string, error)

Generate generates new text based on an initial seed of words

func (*Chain) GenerateDeterministic

func (chain *Chain) GenerateDeterministic(current NGram, prng PRNG) (string, error)

GenerateDeterministic generates new text deterministically, based on an initial seed of words and using a specified PRNG. Use it for reproducibly pseudo-random results (i.e. pass the same PRNG and same state every time).

func (Chain) MarshalJSON

func (chain Chain) MarshalJSON() ([]byte, error)

MarshalJSON ...

func (*Chain) TransitionProbability

func (chain *Chain) TransitionProbability(next string, current NGram) (float64, error)

TransitionProbability returns the transition probability between two states

func (*Chain) UnmarshalJSON

func (chain *Chain) UnmarshalJSON(b []byte) error

UnmarshalJSON ...

type NGram

type NGram []string

NGram is a array of words

type PRNG

type PRNG interface {
	// Intn returns a number number in the half-open interval [0,n)
	Intn(int) int
}

PRNG is a pseudo-random number generator compatible with math/rand interfaces.

type Pair

type Pair struct {
	CurrentState NGram  // n = order of the chain
	NextState    string // n = 1
}

Pair is a pair of consecutive states in a sequece

func MakePairs

func MakePairs(tokens []string, order int) []Pair

MakePairs generates n-gram pairs of consecutive states in a sequence