agent

Documentation

Overview

Package passive provides capability for doing url enumeration

Index

• Variables
• func ContainsAny[T comparable](s1, s2 []T) bool
• type Agent
  • func New(sourceNames, excludedSourceNames []string, useAllSources bool) *Agent
  • func (a *Agent) EnumerateQueries(query string, proxy string, rateLimit int, timeout int, ...) chan source.Result
  • func (a *Agent) EnumerateUrlsWithCtx(ctx context.Context, query string, proxy string, rateLimit int, timeout int, ...) chan source.Result
  • func (a *Agent) GetStatistics() map[string]source.Statistics
• type CustomRateLimit
• type EnumerateOption
  • func WithCustomRateLimit(crl *CustomRateLimit) EnumerateOption
• type EnumerationOptions

Variables

var AllSources = map[string]source.Source{
	"alienvault":     &alienvault.Source{},
	"commoncrawl":    &commoncrawl.Source{},
	"urlscan":        &urlscan.Source{},
	"waybackarchive": &waybackarchive.Source{},
	"virustotal":     &virustotal.Source{},
}

Functions

func ContainsAny

func ContainsAny[T comparable](s1, s2 []T) bool

ContainsAny checks if any of the elements in s2 are in s1

Types

type Agent

type Agent struct {
	// contains filtered or unexported fields
}

Agent is a struct for running url enumeration against a given host. It wraps subscraping package and provides a layer to build upon.

func New

func New(sourceNames, excludedSourceNames []string, useAllSources bool) *Agent

New creates a new agent for url discovery

func (*Agent) EnumerateQueries

func (a *Agent) EnumerateQueries(query string, proxy string, rateLimit int, timeout int, maxEnumTime time.Duration, options ...EnumerateOption) chan source.Result

EnumerateQueries wraps EnumerateQueriesWithCtx with an empty context

func (*Agent) EnumerateUrlsWithCtx

func (a *Agent) EnumerateUrlsWithCtx(ctx context.Context, query string, proxy string, rateLimit int, timeout int, maxEnumTime time.Duration, options ...EnumerateOption) chan source.Result

EnumerateUrlsWithCtx enumerates all the urls for a given query

func (*Agent) GetStatistics

func (a *Agent) GetStatistics() map[string]source.Statistics

type CustomRateLimit

type CustomRateLimit struct {
	Custom mapsutil.SyncLockMap[string, uint]
}

type EnumerateOption

type EnumerateOption func(opts *EnumerationOptions)

func WithCustomRateLimit

func WithCustomRateLimit(crl *CustomRateLimit) EnumerateOption

type EnumerationOptions

type EnumerationOptions struct {
	// contains filtered or unexported fields
}