core

package
v0.0.0-...-6e861f0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 30, 2018 License: MIT Imports: 12 Imported by: 0

Documentation

Overview

Package core handles a majority of the main functionality of SubFinder. It's meant to be small and extensible.

Index

Examples

Constants

This section is empty.

Variables

View Source 
var HTTPClient = &http.Client{
	Transport: &http.Transport{
		Dial: func(network string, address string) (net.Conn, error) {
			separator := strings.LastIndex(address, ":")
			ip, err := dnsCache.FetchOneString(address[:separator])
			if err != nil {
				return nil, err
			}
			return net.Dial("tcp", ip+address[separator:])
		},

		MaxIdleConns:        1024,
		MaxIdleConnsPerHost: 2,

		TLSHandshakeTimeout:   10 * time.Second,
		IdleConnTimeout:       10 * time.Second,
		ResponseHeaderTimeout: 10 * time.Second,
		ExpectContinueTimeout: 1 * time.Second,
	},
}

HTTPClient is a reusable component that can be used in sources.

Functions

func AggregateCustomResults 

func AggregateCustomResults(in chan *Result, custom func(r *Result) bool) <-chan *Result

AggregateCustomResults takes a given results channel as input along with a custom filter function that will be executed with each Result.

Example 
fakeResults := []*Result{
	&Result{Type: "color", Success: "red"},
	&Result{Type: "color", Success: "green"},
	&Result{Type: "color", Success: "blue"},
	&Result{Type: "color", Failure: errors.New("no color")},
	&Result{Type: "wiggle", Failure: errors.New("wiggle")},
	&Result{Success: "example"},
	&Result{Failure: errors.New("example1")},
	&Result{Failure: errors.New("example2")},
}

// imagine a function doing something useful
fakeResultsChan := make(chan *Result)
go func(fakeResults []*Result, fakeResultsChan chan *Result) {
	defer close(fakeResultsChan)
	for _, result := range fakeResults {
		fakeResultsChan <- result
	}
}(fakeResults, fakeResultsChan)

// consume aggregated results
for result := range AggregateCustomResults(fakeResultsChan, func(r *Result) bool {
	return r.Type == "color" && r.IsSuccess() // only successful results of type "color"
}) {
	fmt.Println(result.Success)
}

Output:

red
green
blue

func AggregateFailedResults 

func AggregateFailedResults(in chan *Result) <-chan *Result

AggregateFailedResults takes a given results channel as input and only sends failed results down the returned output channel.

Example 
fakeResults := []*Result{
	&Result{Success: true},
	&Result{Success: 0},
	&Result{Success: "wiggle"},
	&Result{Failure: errors.New("example1")},
	&Result{Failure: errors.New("example2")},
}

fakeResultsChan := make(chan *Result)
go func(fakeResults []*Result, fakeResultsChan chan *Result) {
	defer close(fakeResultsChan)
	for _, result := range fakeResults {
		fakeResultsChan <- result
	}
}(fakeResults, fakeResultsChan)

counter := 0

for range AggregateFailedResults(fakeResultsChan) {
	counter++
}

fmt.Println(counter)

Output:

2

func AggregateSuccessfulResults 

func AggregateSuccessfulResults(in chan *Result) <-chan *Result

AggregateSuccessfulResults takes a given results channel as input and only sends successful results down the returned output channel.

Example 
fakeResults := []*Result{
	&Result{Success: true},
	&Result{Success: 0},
	&Result{Success: "wiggle"},
	&Result{Failure: errors.New("example1")},
	&Result{Failure: errors.New("example2")},
}

fakeResultsChan := make(chan *Result)
go func(fakeResults []*Result, fakeResultsChan chan *Result) {
	defer close(fakeResultsChan)
	for _, result := range fakeResults {
		fakeResultsChan <- result
	}
}(fakeResults, fakeResultsChan)

counter := 0

for range AggregateSuccessfulResults(fakeResultsChan) {
	counter++
}

fmt.Println(counter)

Output:

3

func EnumerateSubdomains 

func EnumerateSubdomains(ctx context.Context, domain string, options *EnumerationOptions) <-chan *Result

EnumerateSubdomains takes the given domain and with each Source from EnumerationOptions, it will spawn a go routine to start processing that Domain. The result channels from each source are merged into one results channel to be consumed. 

 ____________________________     Source1.ProcessDomain     ___________        _____
|                            | /                         \ |           |      |     |
| EnumerationOptions.Sources | -- Source2.ProcessDomain -- |  Results  | ---> |  ?  |
|____________________________| \                         / |___________|      |_____|
                                  Source3.ProcessDomain
Example 
domain := "google.com"

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

sources := []Source{
	&FakeSource1{},
	&FakeSource2{},
}

options := &EnumerationOptions{
	Sources: sources,
}

counter := 0

for result := range EnumerateSubdomains(ctx, domain, options) {
	if result.Failure == nil {
		counter++
	}
}

fmt.Println(counter)

Output:

6

func MergeResults 

func MergeResults(inputs ...<-chan *Result) <-chan *Result

MergeResults takes in N number of result channels and merges them into one resul channel.

func NewMultiSubdomainExtractor 

func NewMultiSubdomainExtractor(domain string) func([]byte) []string

NewMultiSubdomainExtractor creates a new extractor that looks for as many subdomains as it can find.

func NewSingleSubdomainExtractor 

func NewSingleSubdomainExtractor(domain string) func([]byte) string

NewSingleSubdomainExtractor creates a new extractor that looks for only one subdomain.

func UniqResults 

func UniqResults(input <-chan *Result) <-chan *Result

UniqResults filters a given input stream for uniq outputs. Note: this will only be a filter for successful results.

Types

type EnumerationOptions 

type EnumerationOptions struct {
	Sources   []Source
	Context   context.Context
	Recursive bool
	Debug     bool
	Uniq      bool
}

EnumerationOptions provides all the data needed for subdomain enumeration. This includes all the sources which will be queried to find them.

Example 
opts := EnumerationOptions{}
if opts.HasSources() {
	fmt.Println("sources found in options")
} else {
	fmt.Println("sources not found in options")
}

Output:

sources not found in options

func (*EnumerationOptions) HasSources 

func (opts *EnumerationOptions) HasSources() bool

HasSources checks if the EnumerationOptions have any source defined.

Example 
opts := EnumerationOptions{}
fmt.Println(opts.HasSources())

Output:

false

type GeneralOptions 

type GeneralOptions struct {
	Verbose        bool          // Show verbose information.
	ColorSupport   bool          // Whether to use color or not.
	AvailableCores int           // Number of logical CPUs usable by the current process.
	DefaultTimeout time.Duration // Timeout for requests to different sources.
	TargetDomains  []string      // The target domains.
	Recursive      bool          // Perform recursive subdomain discovery or not.
	PassiveOnly    bool          // Perform only passive subdomain discovery or not.
	IgnoreErrors   bool          // Ignore errors or not.
	OutputType     string        // Type of output wanted (json, plaintext, ect).
	Sources        []Source      // List of source types to use.
	OutputDir      string        // Directory to use for any output.
	Resolvers      []string      // List of DNS resolvers to use.
}

GeneralOptions represents a set of global options an application may be aware of (thinking of a command-line app).

Example 
opts := GeneralOptions{}
opts.Print()

Output:

Verbose:	 'false'
ColorSupport:	 'false'
AvailableCores:	 '0'
DefaultTimeout:	 '0s'
TargetDomains:	 '[]'
Recursive:	 'false'
PassiveOnly:	 'false'
IgnoreErrors:	 'false'
OutputType:	 ''
Sources:	 '[]'
OutputDir:	 ''
Resolvers:	 '[]'

func NewDefaultGeneralOptions 

func NewDefaultGeneralOptions() *GeneralOptions

NewDefaultGeneralOptions create a new GeneralOptions with some sane defaults.

Example 
opts := NewDefaultGeneralOptions()
opts.Print()

Output:

Verbose:	 'false'
ColorSupport:	 'true'
AvailableCores:	 '8'
DefaultTimeout:	 '5s'
TargetDomains:	 '[]'
Recursive:	 'false'
PassiveOnly:	 'false'
IgnoreErrors:	 'false'
OutputType:	 'plaintext'
Sources:	 '[]'
OutputDir:	 ''
Resolvers:	 '[1.1.1.1 1.0.0.1 8.8.8.8 8.8.4.4 9.9.9.9 149.112.112.112 208.67.222.222 208.67.220.220]'

func (*GeneralOptions) Print 

func (opts *GeneralOptions) Print()

Print takes the Printable form of the GeneralOptions and outputs it to STDOUT.

func (*GeneralOptions) Printable 

func (opts *GeneralOptions) Printable() string

Printable returns a formatted string of the GeneralOptions.

type Result 

type Result struct {
	sync.RWMutex
	Timestamp time.Time
	Type      string
	Success   interface{}
	Failure   error
}

Result contains the information from any given source. Upon success, a Source source should provide a string as the found subdomain. Upon Failure, the source should provide an error.

Example 
result := Result{Type: "example", Success: "info.bing.com"}
if result.Failure != nil {
	fmt.Println(result.Type, ":", result.Failure)
} else {
	fmt.Println(result.Type, ":", result.Success)
}

Output:

example : info.bing.com

func NewResult 

func NewResult(t string, s interface{}, f error) *Result

NewResult wraps up the creation of a new Result. This function will set the Timestamp value of the Result to the current time in UTC format, which is always preferred.

Example 
result := NewResult("example", "info.google.com", nil)

if result.IsFailure() {
	fmt.Println(result.Failure.Error())
} else {
	if result.Success.(string) == "info.google.com" {
		fmt.Println("found example in success")
	}
}

Output:

found example in success

func (*Result) GetFailure 

func (r *Result) GetFailure() error

GetFailure safely gets the error value from a Result which could be shared by multiple go routines.

Example 
McErr := errors.New("whoa there")
result := NewResult("", nil, McErr)
fmt.Println(result.Failure)

Output:

whoa there

func (*Result) GetSuccess 

func (r *Result) GetSuccess() interface{}

GetSuccess safely gets the Success value from a Result which could be shared by multiple go routines.

Example 
result := NewResult("bing", "info.bing.com", nil)
s := result.GetSuccess()
fmt.Println(s)

Output:

info.bing.com

func (*Result) GetTimestamp 

func (r *Result) GetTimestamp() time.Time

GetTimestamp safely gets the Timestamp value from a Result which could be shared by multiple go routines.

Example 
result := NewResult("bing", "info.bing.com", nil)
newTimestamp := time.Now().UTC()
result.Timestamp = newTimestamp
fmt.Println(result.GetTimestamp() == newTimestamp)

Output:

true

func (*Result) GetType 

func (r *Result) GetType() string

GetType safely gets the type value from a Result which could be shared by multiple go routines.

Example 
result := NewResult("bing", "info.bing.com", nil)
t := result.GetType()
fmt.Println(t)

Output:

bing

func (*Result) HasTimestamp 

func (r *Result) HasTimestamp() bool

HasTimestamp checks if the Result has a timestamp set.

Example 
result := Result{} // no Timestamp set
fmt.Println(result.HasTimestamp())

Output:

false

func (*Result) HasType 

func (r *Result) HasType() bool

HasType checks if the Result has a type value set.

Example 
result := Result{Type: "example"}
fmt.Println(result.HasType())

Output:

true

func (*Result) IsFailure 

func (r *Result) IsFailure() bool

IsFailure checks if the Result has any Failure before determining if the result failed.

Example 
result := Result{Failure: errors.New("failed to party")}
if result.IsFailure() {
	fmt.Println(result.Failure.Error())
}

Output:

failed to party

func (*Result) IsPrintable 

func (r *Result) IsPrintable() (bool, string)

IsPrintable checks if the underlying Result has any printable information.

Example 
result := NewResult("example", "ex.ample.com", nil)
ok, _ := result.IsPrintable()
fmt.Println(ok)

Output:

true

func (*Result) IsSuccess 

func (r *Result) IsSuccess() bool

IsSuccess checks if the Result has any Failure, or that the Success interface{} has actually been filled before determining if the result succeeded.

Example 
result := Result{Success: "wiggle.github.com"}
if result.IsSuccess() {
	fmt.Println(result.Success)
}

Output:

wiggle.github.com

func (*Result) JSON 

func (r *Result) JSON() ([]byte, error)

JSON returns the Result as a JSON object within a slice of bytes.

Example 
result := NewResult("example", "ex.ample.com", nil)
result.Timestamp = time.Time{} // set default timestamp
bytes, _ := result.JSON()
fmt.Println(string(bytes))

Output:

{"Timestamp":"0001-01-01T00:00:00Z","Type":"example","Success":"ex.ample.com","Failure":null}

func (*Result) Print 

func (r *Result) Print() error

Print will print the Printable version of the Result to the screen or return an error if the underlying Result has any printable information. Useful for debugging.

Example 
result := NewResult("example", "ex.ample.com", nil)
result.Timestamp = time.Time{} // set default timestamp
result.Print()

Output:

Type: example Success: ex.ample.com

func (*Result) Printable 

func (r *Result) Printable() string

Printable turns a Result's information into a printable format (for STDOUT).

Example 
result := NewResult("example", "ex.ample.com", nil)
result.Timestamp = time.Time{} // set default timestamp
printable := result.Printable()
fmt.Println(printable)

Output:

Type: example Success: ex.ample.com

func (*Result) SetFailure 

func (r *Result) SetFailure(err error)

SetFailure safely sets a new error value for a Result which could be shared by multiple go routines.

Example 
McErr := errors.New("whoa there")
result := NewResult("thisis", "totally.fine.com", nil)
result.SetFailure(McErr)
if result.IsFailure() {
	fmt.Println(result.Failure, "we found our failure!")
}
if result.IsSuccess() {
	fmt.Println("this will never print because the failure was set")
}

Output:

whoa there we found our failure!

func (*Result) SetSuccess 

func (r *Result) SetSuccess(success interface{})

SetSuccess safely sets a new Success value for a Result which could be shared by multiple go routines.

Example 
result := NewResult("example", "", nil)
// do work, possibly in multiple go routines
result.SetSuccess([]string{"a.com", "b.com", "c.com"})
// check if success
fmt.Println(result.IsSuccess())

Output:

true

func (*Result) SetTimestamp 

func (r *Result) SetTimestamp(t time.Time)

SetTimestamp safely sets a new Timestamp value for a Result which could be shared by multiple go routines.

Example 
result := NewResult("bing", "info.bing.com", nil)
newTimestamp := time.Now().UTC()
result.SetTimestamp(newTimestamp)
fmt.Println(result.Timestamp == newTimestamp)

Output:

true

func (*Result) SetType 

func (r *Result) SetType(t string)

SetType safely sets a new type value for a Result which could be shared by multiple go routines.

Example 
result := NewResult("bing", "info.bing.com", nil)
result.SetType("google")
fmt.Println(result.Type)

Output:

google

type Source 

type Source interface {
	ProcessDomain(context.Context, string) <-chan *Result
}

Source defines the minimum interface any subdomain enumeration module should follow.

Source Files

View all Source files

Directories

Path Synopsis
Package sources is a collection of pre-built core.Source(s) that can be used to enumerate subdomains passively.
 Package sources is a collection of pre-built core.Source(s) that can be used to enumerate subdomains passively.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.