tgspam

Documentation

Index

• type Config
• type Detector
  • func NewDetector(p Config) *Detector
  • func (d *Detector) AddApprovedUser(user approved.UserInfo) error
  • func (d *Detector) ApprovedUsers() (res []approved.UserInfo)
  • func (d *Detector) Check(req spamcheck.Request) (spam bool, cr []spamcheck.Response)
  • func (d *Detector) IsApprovedUser(userID string) bool
  • func (d *Detector) LoadSamples(exclReader io.Reader, spamReaders, hamReaders []io.Reader) (LoadResult, error)
  • func (d *Detector) LoadStopWords(readers ...io.Reader) (LoadResult, error)
  • func (d *Detector) RemoveApprovedUser(id string) error
  • func (d *Detector) Reset()
  • func (d *Detector) UpdateHam(msg string) error
  • func (d *Detector) UpdateSpam(msg string) error
  • func (d *Detector) WithHamUpdater(s SampleUpdater)
  • func (d *Detector) WithMetaChecks(mc ...MetaCheck)
  • func (d *Detector) WithOpenAIChecker(client openAIClient, config OpenAIConfig)
  • func (d *Detector) WithSpamUpdater(s SampleUpdater)
  • func (d *Detector) WithUserStorage(storage UserStorage) (count int, err error)
• type HTTPClient
• type LoadResult
• type MetaCheck
  • func ImagesCheck() MetaCheck
  • func LinkOnlyCheck() MetaCheck
  • func LinksCheck(limit int) MetaCheck
  • func VideosCheck() MetaCheck
• type OpenAIConfig
• type SampleUpdater
• type UserStorage

Types

type Config

type Config struct {
	SimilarityThreshold float64    // threshold for spam similarity, 0.0 - 1.0
	MinMsgLen           int        // minimum message length to check
	MaxAllowedEmoji     int        // maximum number of emojis allowed in a message
	CasAPI              string     // CAS API URL
	FirstMessageOnly    bool       // if true, only the first message from a user is checked
	FirstMessagesCount  int        // number of first messages to check for spam
	HTTPClient          HTTPClient // http client to use for requests
	MinSpamProbability  float64    // minimum spam probability to consider a message spam with classifier, if 0 - ignored
	OpenAIVeto          bool       // if true, openai will be used to veto spam messages, otherwise it will be used to veto ham messages
	MultiLangWords      int        // if true, check for number of multi-lingual words
}

Config is a set of parameters for Detector.

type Detector

type Detector struct {
	Config
	// contains filtered or unexported fields
}

Detector is a spam detector, thread-safe. It uses a set of checks to determine if a message is spam, and also keeps a list of approved users.

func NewDetector

func NewDetector(p Config) *Detector

NewDetector makes a new Detector with the given config.

func (*Detector) AddApprovedUser

func (d *Detector) AddApprovedUser(user approved.UserInfo) error

AddApprovedUser adds user IDs to the list of approved users.

func (*Detector) ApprovedUsers

func (d *Detector) ApprovedUsers() (res []approved.UserInfo)

ApprovedUsers returns a list of approved users.

func (*Detector) Check

func (d *Detector) Check(req spamcheck.Request) (spam bool, cr []spamcheck.Response)

Check checks if a given message is spam. Returns true if spam and also returns a list of check results.

func (*Detector) IsApprovedUser

func (d *Detector) IsApprovedUser(userID string) bool

IsApprovedUser checks if a given user ID is approved.

func (*Detector) LoadSamples

func (d *Detector) LoadSamples(exclReader io.Reader, spamReaders, hamReaders []io.Reader) (LoadResult, error)

LoadSamples loads spam samples from a reader and updates the classifier. Reset spam, ham samples/classifier, and excluded tokens.

func (*Detector) LoadStopWords

func (d *Detector) LoadStopWords(readers ...io.Reader) (LoadResult, error)

LoadStopWords loads stop words from a reader. Reset stop words list before loading.

func (*Detector) RemoveApprovedUser

func (d *Detector) RemoveApprovedUser(id string) error

RemoveApprovedUser removes approved user for given IDs

func (*Detector) Reset

func (d *Detector) Reset()

Reset resets spam samples/classifier, excluded tokens, stop words and approved users.

func (*Detector) UpdateHam

func (d *Detector) UpdateHam(msg string) error

UpdateHam appends a message to the ham samples file and updates the classifier

func (*Detector) UpdateSpam

func (d *Detector) UpdateSpam(msg string) error

UpdateSpam appends a message to the spam samples file and updates the classifier

func (*Detector) WithHamUpdater

func (d *Detector) WithHamUpdater(s SampleUpdater)

WithHamUpdater sets a SampleUpdater for ham samples.

func (*Detector) WithMetaChecks

func (d *Detector) WithMetaChecks(mc ...MetaCheck)

WithMetaChecks sets a list of meta-checkers.

func (*Detector) WithOpenAIChecker

func (d *Detector) WithOpenAIChecker(client openAIClient, config OpenAIConfig)

WithOpenAIChecker sets an openAIChecker for spam checking.

func (*Detector) WithSpamUpdater

func (d *Detector) WithSpamUpdater(s SampleUpdater)

WithSpamUpdater sets a SampleUpdater for spam samples.

func (*Detector) WithUserStorage

func (d *Detector) WithUserStorage(storage UserStorage) (count int, err error)

WithUserStorage sets a UserStorage for approved users and loads approved users from it.

type HTTPClient

type HTTPClient interface {
	Do(req *http.Request) (*http.Response, error)
}

HTTPClient is an interface for http client, satisfied by http.Client.

type LoadResult

type LoadResult struct {
	ExcludedTokens int // number of excluded tokens
	SpamSamples    int // number of spam samples
	HamSamples     int // number of ham samples
	StopWords      int // number of stop words (phrases)
}

LoadResult is a result of loading samples.

type MetaCheck

type MetaCheck func(req spamcheck.Request) spamcheck.Response

MetaCheck represents a function type that takes a `spamcheck.Request` as input and returns a boolean value and a `spamcheck.Response`. The boolean value indicates whether the check. It checks the message's meta.

func ImagesCheck

func ImagesCheck() MetaCheck

ImagesCheck is a function that returns a MetaCheck function. It checks if the number of images in the message is greater than zero and the message is empty (i.e. it contains only images).

func LinkOnlyCheck

func LinkOnlyCheck() MetaCheck

LinkOnlyCheck is a function that returns a MetaCheck function that checks if the req.Msg contains only links.

func LinksCheck

func LinksCheck(limit int) MetaCheck

LinksCheck is a function that returns a MetaCheck function that checks the number of links in the message. It uses custom meta-info if it is provided, otherwise it counts the number of links in the message.

func VideosCheck

func VideosCheck() MetaCheck

VideosCheck is a function that returns a MetaCheck function. It checks if the message has a video or video note and the message is empty (i.e. it contains only videos).

type OpenAIConfig

type OpenAIConfig struct {
	// https://platform.openai.com/docs/api-reference/chat/create#chat/create-max_tokens
	MaxTokensResponse int // Hard limit for the number of tokens in the response
	// The OpenAI has a limit for the number of tokens in the request + response (4097)
	MaxTokensRequest  int // Max request length in tokens
	MaxSymbolsRequest int // Fallback: Max request length in symbols, if tokenizer was failed
	Model             string
	SystemPrompt      string
	RetryCount        int
}

OpenAIConfig contains parameters for openAIChecker

type SampleUpdater

type SampleUpdater interface {
	Append(msg string) error        // append a message to the samples storage
	Reader() (io.ReadCloser, error) // return a reader for the samples storage
}

SampleUpdater is an interface for updating spam/ham samples on the fly.

type UserStorage

type UserStorage interface {
	Read() ([]approved.UserInfo, error) // read approved users from storage
	Write(au approved.UserInfo) error   // write approved user to storage
	Delete(id string) error             // delete approved user from storage
}

UserStorage is an interface for approved users storage.