templates

package
v3.0.0-...-e70001a Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 12, 2024 License: MIT Imports: 73 Imported by: 0

Documentation

Overview

Package templates contains the parser for a template for the engine.

This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of the MPL was not distributed with this file, You can obtain one at http://mozilla.org/MPL/2.0/. DO NOT EDIT: this file is automatically generated by docgen

Package templates nolint //do not lint as examples with no usage

Index

Constants

View Source 
const (
	SyntaxWarningStats           = "syntax-warnings"
	SyntaxErrorStats             = "syntax-errors"
	RuntimeWarningsStats         = "runtime-warnings"
	SkippedCodeTmplTamperedStats = "unsigned-warnings"
	ExcludedHeadlessTmplStats    = "headless-flag-missing-warnings"
	TemplatesExcludedStats       = "templates-executed"
	ExcludedCodeTmplStats        = "code-flag-missing-warnings"
	ExludedDastTmplStats         = "fuzz-flag-missing-warnings"
	SkippedUnsignedStats         = "skipped-unsigned-stats" // tracks loading of unsigned templates
	ExcludedSelfContainedStats   = "excluded-self-contained-stats"
	ExcludedFileStats            = "excluded-file-stats"
	SkippedRequestSignatureStats = "skipped-request-signature-stats"
)
View Source 
const (
	Unsigned = "unsigned"
)

Variables

View Source 
var (
	ErrCreateTemplateExecutor          = errors.New("cannot create template executer")
	ErrIncompatibleWithOfflineMatching = errors.New("template can't be used for offline matching")
	// track how many templates are verfied and by which signer
	SignatureStats = map[string]*atomic.Uint64{}
)
View Source 
var (
	Colorizer         aurora.Aurora
	SeverityColorizer func(severity.Severity) string
)
View Source 
var (
	ErrMandatoryFieldMissingFmt = errorutil.NewWithFmt("mandatory '%s' field is missing")
	ErrInvalidField             = errorutil.NewWithFmt("invalid field format for '%s' (allowed format is %s)")
	ErrWarningFieldMissing      = errorutil.NewWithFmt("field '%s' is missing")
	ErrCouldNotLoadTemplate     = errorutil.NewWithFmt("Could not load template %s: %s")
	ErrLoadedWithWarnings       = errorutil.NewWithFmt("Loaded template %s: with syntax warning : %s")
)
View Source 
var (
	TemplateDoc                   encoder.Doc
	MODELInfoDoc                  encoder.Doc
	STRINGSLICEStringSliceDoc     encoder.Doc
	STRINGSLICERawStringSliceDoc  encoder.Doc
	SEVERITYHolderDoc             encoder.Doc
	MODELClassificationDoc        encoder.Doc
	HTTPRequestDoc                encoder.Doc
	GENERATORSAttackTypeHolderDoc encoder.Doc
	HTTPMethodTypeHolderDoc       encoder.Doc
	FUZZRuleDoc                   encoder.Doc
	SliceOrMapSliceDoc            encoder.Doc
	ANALYZERSAnalyzerTemplateDoc  encoder.Doc
	SignatureTypeHolderDoc        encoder.Doc
	MATCHERSMatcherDoc            encoder.Doc
	MatcherTypeHolderDoc          encoder.Doc
	DNSRequestDoc                 encoder.Doc
	DNSRequestTypeHolderDoc       encoder.Doc
	FILERequestDoc                encoder.Doc
	NETWORKRequestDoc             encoder.Doc
	NETWORKInputDoc               encoder.Doc
	NetworkInputTypeHolderDoc     encoder.Doc
	HEADLESSRequestDoc            encoder.Doc
	ENGINEActionDoc               encoder.Doc
	ActionTypeHolderDoc           encoder.Doc
	USERAGENTUserAgentHolderDoc   encoder.Doc
	SSLRequestDoc                 encoder.Doc
	WEBSOCKETRequestDoc           encoder.Doc
	WEBSOCKETInputDoc             encoder.Doc
	WHOISRequestDoc               encoder.Doc
	CODERequestDoc                encoder.Doc
	JAVASCRIPTRequestDoc          encoder.Doc
	HTTPSignatureTypeHolderDoc    encoder.Doc
	VARIABLESVariableDoc          encoder.Doc
)
View Source 
var ErrExcluded = errors.New("the template was excluded")

ErrExcluded is returned for excluded templates

View Source 
var (
	ErrNotATemplate = errorutil.NewWithTag("signer", "given filePath is not a template")
)
View Source 
var (
	ReTemplateID = regexp.MustCompile(`^([a-zA-Z0-9]+[-_])*[a-zA-Z0-9]+$`)
)

Functions

func Cluster 

func Cluster(list []*Template) [][]*Template

Cluster clusters a list of templates into a lesser number if possible based on the similarity between the sent requests.

If the attributes match, multiple requests can be clustered into a single request which saves time and network resources during execution.

The clusterer goes through all the templates, looking for templates with a single HTTP/DNS/TLS request to an endpoint (multiple requests aren't clustered as of now).

All the templates are iterated and any templates with request that is identical to the first individual request is compared for equality. The equality check is performed as described below -

Cases where clustering is not performed (request is considered different)

  • If request contains payloads,raw,body,unsafe,req-condition,name attributes
  • If request methods,max-redirects,disable-cookie,redirects are not equal
  • If request paths aren't identical.
  • If request headers aren't identical
  • Similarly for DNS, only identical DNS requests are clustered to a target.
  • Similarly for TLS, only identical TLS requests are clustered to a target.

If multiple requests are identified as identical, they are appended to a slice. Finally, the engine creates a single executer with a clusteredexecuter for all templates in a cluster.

func ClusterID 

func ClusterID(templates []*Template) string

ClusterID transforms clusterization into a mathematical hash repeatable across executions with the same templates

func GetJsCompiler 

func GetJsCompiler() *compiler.Compiler

func GetTemplateDoc 

func GetTemplateDoc() *encoder.FileDoc

GetTemplateDoc returns documentation for the file templates_doc.go.

func PrintDeprecatedProtocolNameMsgIfApplicable 

func PrintDeprecatedProtocolNameMsgIfApplicable(isSilent bool, verbose bool)

PrintDeprecatedProtocolNameMsgIfApplicable prints a message if deprecated protocol names are used Unless mode is silent we print a message for deprecated protocol name

func SignTemplate 

func SignTemplate(templateSigner *signer.TemplateSigner, templatePath string) error

SignTemplate signs the tempalate using custom signer

func TemplateLogMessage 

func TemplateLogMessage(id, name string, authors []string, templateSeverity severity.Severity) string

TemplateLogMessage returns a beautified log string for a template

func TemplateSignerLFA 

func TemplateSignerLFA()

New Signer/Verification logic requires it to load content of file references and this is done respecting sandbox restrictions to avoid any security issues AllowLocalFileAccess is a function that allows local file access by disabling sandbox restrictions and **MUST** be called before signing / verifying any templates for intialization

func UseOptionsForSigner 

func UseOptionsForSigner(opts *types.Options)

UseOptionsForSigner sets the options to use for signing templates instead of default options

func VerifyTemplateSignature 

func VerifyTemplateSignature(templatePath string) (bool, error)

VerifyTemplateSignature verifies the signature of the template using default signers

Types

type Cache 

type Cache struct {
	// contains filtered or unexported fields
}

Templates is a cache for caching and storing templates for reuse.

func NewCache 

func NewCache() *Cache

New returns a new templates cache

func (*Cache) Has 

func (t *Cache) Has(template string) (*Template, []byte, error)

Has returns true if the cache has a template. The template is returned along with any errors if found.

func (*Cache) Purge 

func (t *Cache) Purge()

Purge the cache

func (*Cache) Store 

func (t *Cache) Store(id string, tpl *Template, raw []byte, err error)

Store stores a template with data and error

type ClusterExecuter 

type ClusterExecuter struct {
	// contains filtered or unexported fields
}

ClusterExecuter executes a group of requests for a protocol for a clustered request. It is different from normal executers since the original operators are all combined and post processed after making the request.

func NewClusterExecuter 

func NewClusterExecuter(requests []*Template, options *protocols.ExecutorOptions) *ClusterExecuter

NewClusterExecuter creates a new request executer for list of requests

func (*ClusterExecuter) Compile 

func (e *ClusterExecuter) Compile() error

Compile compiles the execution generators preparing any requests possible.

func (*ClusterExecuter) Execute 

func (e *ClusterExecuter) Execute(ctx *scan.ScanContext) (bool, error)

Execute executes the protocol group and returns true or false if results were found.

func (*ClusterExecuter) ExecuteWithResults 

func (e *ClusterExecuter) ExecuteWithResults(ctx *scan.ScanContext) ([]*output.ResultEvent, error)

ExecuteWithResults executes the protocol requests and returns results instead of writing them.

func (*ClusterExecuter) Requests 

func (e *ClusterExecuter) Requests() int

Requests returns the total number of requests the rule will perform

type Parser 

type Parser struct {
	ShouldValidate bool
	NoStrictSyntax bool
	// contains filtered or unexported fields
}

func NewParser 

func NewParser() *Parser

func (*Parser) Cache 

func (p *Parser) Cache() *Cache

Cache returns the parsed templates cache

func (*Parser) LoadTemplate 

func (p *Parser) LoadTemplate(templatePath string, t any, extraTags []string, catalog catalog.Catalog) (bool, error)

LoadTemplate returns true if the template is valid and matches the filtering criteria.

func (*Parser) LoadWorkflow 

func (p *Parser) LoadWorkflow(templatePath string, catalog catalog.Catalog) (bool, error)

LoadWorkflow returns true if the workflow is valid and matches the filtering criteria.

func (*Parser) ParseTemplate 

func (p *Parser) ParseTemplate(templatePath string, catalog catalog.Catalog) (any, error)

ParseTemplate parses a template and returns a *templates.Template structure

type Preprocessor 

type Preprocessor interface {
	// Process processes the data and returns the processed data.
	ProcessNReturnData(data []byte) ([]byte, map[string]interface{})
	// Exists check if the preprocessor exists in the data.
	Exists(data []byte) bool
}

type TagFilter 

type TagFilter struct {
	// contains filtered or unexported fields
}

TagFilter is used to filter nuclei templates for tag based execution

func NewTagFilter 

func NewTagFilter(config *TagFilterConfig) (*TagFilter, error)

New returns a tag filter for nuclei tag based execution

It takes into account Tags, Severities, ExcludeSeverities, Authors, IncludeTags, ExcludeTags, Conditions.

func (*TagFilter) Match 

func (tagFilter *TagFilter) Match(template *Template, extraTags []string) (bool, error)

Match filters templates based on user provided tags, authors, extraTags and severity. If the template contains tags specified in the deny-list, it will not be matched unless it is explicitly specified by user using the includeTags (matchAllows field). Matching rule: (tag1 OR tag2...) AND (author1 OR author2...) AND (severity1 OR severity2...) AND (extraTags1 OR extraTags2...) Returns true if the template matches the filter criteria, false otherwise.

type TagFilterConfig 

type TagFilterConfig struct {
	Tags              []string
	ExcludeTags       []string
	Authors           []string
	Severities        severity.Severities
	ExcludeSeverities severity.Severities
	IncludeTags       []string
	IncludeIds        []string
	ExcludeIds        []string
	Protocols         types.ProtocolTypes
	ExcludeProtocols  types.ProtocolTypes
	IncludeConditions []string
}

type Template 

type Template struct {
	// description: |
	//   ID is the unique id for the template.
	//
	//   #### Good IDs
	//
	//   A good ID uniquely identifies what the requests in the template
	//   are doing. Let's say you have a template that identifies a git-config
	//   file on the webservers, a good name would be `git-config-exposure`. Another
	//   example name is `azure-apps-nxdomain-takeover`.
	// examples:
	//   - name: ID Example
	//     value: "\"CVE-2021-19520\""
	ID string `` /* 174-byte string literal not displayed */
	// description: |
	//   Info contains metadata information about the template.
	// examples:
	//   - value: exampleInfoStructure
	Info model.Info `` /* 137-byte string literal not displayed */
	// description: |
	//   Flow contains the execution flow for the template.
	// examples:
	//   - flow: |
	// 		for region in regions {
	//		    http(0)
	//		 }
	//		 for vpc in vpcs {
	//		    http(1)
	//		 }
	//
	Flow string `` /* 223-byte string literal not displayed */
	// description: |
	//   Requests contains the http request to make in the template.
	//   WARNING: 'requests' will be deprecated and will be removed in a future release. Please use 'http' instead.
	// examples:
	//   - value: exampleNormalHTTPRequest
	RequestsHTTP []*http.Request `` /* 159-byte string literal not displayed */
	// description: |
	//   HTTP contains the http request to make in the template.
	// examples:
	//   - value: exampleNormalHTTPRequest
	// RequestsWithHTTP is placeholder(internal) only, and should not be used instead use RequestsHTTP
	// Deprecated: Use RequestsHTTP instead.
	RequestsWithHTTP []*http.Request `` /* 135-byte string literal not displayed */
	// description: |
	//   DNS contains the dns request to make in the template
	// examples:
	//   - value: exampleNormalDNSRequest
	RequestsDNS []*dns.Request `` /* 131-byte string literal not displayed */
	// description: |
	//   File contains the file request to make in the template
	// examples:
	//   - value: exampleNormalFileRequest
	RequestsFile []*file.Request `` /* 135-byte string literal not displayed */
	// description: |
	//   Network contains the network request to make in the template
	//   WARNING: 'network' will be deprecated and will be removed in a future release. Please use 'tcp' instead.
	// examples:
	//   - value: exampleNormalNetworkRequest
	RequestsNetwork []*network.Request `` /* 163-byte string literal not displayed */
	// description: |
	//   TCP contains the network request to make in the template
	// examples:
	//   - value: exampleNormalNetworkRequest
	// RequestsWithTCP is placeholder(internal) only, and should not be used instead use RequestsNetwork
	// Deprecated: Use RequestsNetwork instead.
	RequestsWithTCP []*network.Request `` /* 144-byte string literal not displayed */
	// description: |
	//   Headless contains the headless request to make in the template.
	RequestsHeadless []*headless.Request `` /* 151-byte string literal not displayed */
	// description: |
	//   SSL contains the SSL request to make in the template.
	RequestsSSL []*ssl.Request `` /* 131-byte string literal not displayed */
	// description: |
	//   Websocket contains the Websocket request to make in the template.
	RequestsWebsocket []*websocket.Request `` /* 155-byte string literal not displayed */
	// description: |
	//   WHOIS contains the WHOIS request to make in the template.
	RequestsWHOIS []*whois.Request `` /* 139-byte string literal not displayed */
	// description: |
	//   Code contains code snippets.
	RequestsCode []*code.Request `yaml:"code,omitempty" json:"code,omitempty" jsonschema:"title=code snippets to make,description=Code snippets"`
	// description: |
	//   Javascript contains the javascript request to make in the template.
	RequestsJavascript []*javascript.Request `` /* 159-byte string literal not displayed */

	// description: |
	//   Workflows is a yaml based workflow declaration code.
	workflows.Workflow `yaml:",inline,omitempty" jsonschema:"title=workflows to run,description=Workflows to run for the template"`
	CompiledWorkflow   *workflows.Workflow `yaml:"-" json:"-" jsonschema:"-"`

	// description: |
	//   Self Contained marks Requests for the template as self-contained
	SelfContained bool `` /* 175-byte string literal not displayed */
	// description: |
	//  Stop execution once first match is found
	StopAtFirstMatch bool `` /* 161-byte string literal not displayed */

	// description: |
	//   Signature is the request signature method
	//   WARNING: 'signature' will be deprecated and will be removed in a future release. Prefer using 'code' protocol for writing cloud checks
	// values:
	//   - "AWS"
	Signature http.SignatureTypeHolder `` /* 203-byte string literal not displayed */

	// description: |
	//   Variables contains any variables for the current request.
	Variables variables.Variable `` /* 184-byte string literal not displayed */

	// description: |
	//   Constants contains any scalar constant for the current template
	Constants map[string]interface{} `` /* 171-byte string literal not displayed */

	// TotalRequests is the total number of requests for the template.
	TotalRequests int `yaml:"-" json:"-"`
	// Executer is the actual template executor for running template requests
	Executer protocols.Executer `yaml:"-" json:"-"`

	Path string `yaml:"-" json:"-"`

	// Verified defines if the template signature is digitally verified
	Verified bool `yaml:"-" json:"-"`
	// TemplateVerifier is identifier verifier used to verify the template (default nuclei-templates have projectdiscovery/nuclei-templates)
	TemplateVerifier string `yaml:"-" json:"-"`
	// RequestsQueue contains all template requests in order (both protocol & request order)
	RequestsQueue []protocols.Request `yaml:"-" json:"-"`

	// ImportedFiles contains list of files whose contents are imported after template was compiled
	ImportedFiles []string `yaml:"-" json:"-"`
}

Template is a YAML input file which defines all the requests and other metadata for a template.

func ClusterTemplates 

func ClusterTemplates(templatesList []*Template, options protocols.ExecutorOptions) ([]*Template, int)

func Parse 

func Parse(filePath string, preprocessor Preprocessor, options protocols.ExecutorOptions) (*Template, error)

Parse parses a yaml request template file TODO make sure reading from the disk the template parsing happens once: see parsers.ParseTemplate vs templates.Parse

func ParseTemplateFromReader 

func ParseTemplateFromReader(reader io.Reader, preprocessor Preprocessor, options protocols.ExecutorOptions) (*Template, error)

ParseTemplateFromReader reads the template from reader returns the parsed template

func (*Template) GetFileImports 

func (template *Template) GetFileImports() []string

GetFileImports returns a list of files that are imported by the template

func (*Template) HasCodeProtocol 

func (template *Template) HasCodeProtocol() bool

HasCodeProtocol returns true if the template has a code protocol section

func (*Template) HasFileProtocol 

func (template *Template) HasFileProtocol() bool

HasFileProtocol returns true if the template has a file protocol section

func (*Template) ImportFileRefs 

func (template *Template) ImportFileRefs(options *protocols.ExecutorOptions) error

ImportFileRefs checks if sensitive fields like `flow` , `source` in code protocol are referencing files instead of actual javascript / engine code if so it loads the file contents and replaces the reference

func (*Template) IsFuzzing 

func (template *Template) IsFuzzing() bool

IsFuzzing returns true if the template is a fuzzing template

func (*Template) MarshalJSON 

func (template *Template) MarshalJSON() ([]byte, error)

MarshalJSON forces recursive struct validation during marshal operation

func (*Template) MarshalYAML 

func (template *Template) MarshalYAML() ([]byte, error)

MarshalYAML forces recursive struct validation during marshal operation

func (*Template) Requests 

func (template *Template) Requests() int

Requests returns the total request count for the template

func (*Template) Type 

func (template *Template) Type() types.ProtocolType

Type returns the type of the template

func (*Template) UnmarshalJSON 

func (template *Template) UnmarshalJSON(data []byte) error

UnmarshalJSON forces recursive struct validation after unmarshal operation

func (*Template) UnmarshalYAML 

func (template *Template) UnmarshalYAML(unmarshal func(interface{}) error) error

MarshalYAML forces recursive struct validation after unmarshal operation

func (*Template) UsesRequestSignature 

func (template *Template) UsesRequestSignature() bool

UsesRequestSignature returns true if the template uses a request signature like AWS

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.