Documentation
¶
Overview ¶
Package ht provides functions for easy testing of HTTP based protocols.
Testing is done by constructing a request, executing the request and performing various checks on the returned response. The type Test captures this idea. Each Test may contain an arbitrary list of Checks which perform the actual validation work. Tests can be grouped into Suites which may provide a common cookie jar for their tests and may execute setup and teardown actions.
All elements like Checks, Request, Tests and Suites are organized in a way to allow easy deserialisation from a text format. This allows to load and execute whole Suites at runtime.
Checks ¶
A typical check validates a certain property of the received response. E.g.
StatusCode{Expect: 302} Body{Contains: "foobar"} Body{Contains: "foobar", Count: 2} Body{Contains: "illegal", Count: -1}
The last three examples show how zero values of optional fields are commonly used: The zero value of Count means "any number of occurences". Forbidding the occurenc of "foobar" thus requires a negative Count.
The following checks are provided
- Body checks text in the response body
- ContentType checks Content-Type header
- DeleteCookie checks for proper deletion of cookies
- FinalURL checks final URL after a redirect chain
- Header checks presence and values of received HTTP header
- HTMLContains checks text content of CSS-selected elements
- HTMLTag checks occurence HTML elements choosen via CSS-selectors
- Identity checks the SHA1 hash of the HTTP body
- Image checks image format, size and content
- JSON checks structure and content of a JSON body
- Links checks accesability of hrefs and srcs in HTML
- Logfile checks data written to a logfile
- Redirect checks for redirections
- ResponseTime checks lower and higer bounds on the response time
- SetCookie checks properties of received cookies
- StatusCode checks the received HTTP status code
- UTF8Encoded checks that the HTTP body is UTF-8 encoded
- W3CValidHTML checks if body parses as valid HTML5
- XML checks elements of a XML body
Requests ¶
Requests allow to specify a HTTP request in a declarative manner. A wide varity of request can be generated from a purly textual representation of the Request.
All the ugly stuff like parameter encoding, generation of multipart bodies, etc. are hidden from the user.
Parametrisations ¶
Hardcoding e.g. the hostname in a test has obvious drawbacks. To overcome these the Request and Checks may be parametrised. This is done by a simple variable expansion in which occurences of variables are replaced by their values. Variables may occur in all (exported) string fields of Checks and all suitable string fields of Request in the form:
{{VARNAME}}
The variable substitution is performed during compilation of a Test which includes compilation of the embeded Checks.
The current time with an optional offset can be substituted by a special construct:
{{NOW}} --> Wed, 01 Oct 2014 12:22:36 CEST {{NOW + 15s}} --> Wed, 01 Oct 2014 12:22:51 CEST {{NOW + 25m | "15:04"}} --> 12:47 {{NOW + 3d | "2006-Jan-02"}} --> 2014-Oct-04
Formating the time is done with the usual reference time of package time and defaults to RFC1123. Offset can be negative, the known units are "s" for seconds, "m" for minutes, "h" for hours and "d" for days.
Some random values can be include by the following syntax:
{{RANDOM NUMBER 99}} --> 22 {{RANDOM NUMBER 32-99}} --> 45 {{RANDOM NUMBER 99 %04x}} --> 002d {{RANDOM TEXT 8}} --> que la victoire Accoure à tes mâles {{RANDOM TEXT 2-5}} --> Accoure à tes {{RANDOM TEXT de 5}} --> Denn die fromme Seele {{RANDOM EMAIL}} --> Leon.Schneider@gmail.com {{RANDOM EMAIL web.de}} --> Meier.Anna@web.de
Tests ¶
A Test is basically just a Request combined with a list of Checks. Running a Test is executing the request and validating the response according to the Checks. Before a test can be run the variable substitution in the Request and the Checks have to happen, a real HTTP request has to be crafted and checks have to be set up. This is done by compiling the test, a step wich may fail: a) if the Request is malformed (e.g. uses a malformed URL) or b) if the checks are malformed (e.g. uses a malformed regexp). Such Tests/Checks are labeled Bogus.
There are three ways in which a Tests may fail:
- The test setup is malformed, such tests are called Bogus.
- The request itself fails. This is called an Error
- Any of the checks fail. This is called a Failure
Unrolling a Test ¶
A common szenario is to do a test/check combination several times with tiny changes, e.g. a search with different queries. To facilliate writing these repeated tests it is possible to treat a Test as a template which is instantiated with different parametrizations. This process is called unrolling. The field UnrollWith of a test controlls this unrolling: It is a map of variabe names to variable values. The simplest definition is
UnrollWith: map[string][]string{"query": {"foo", "bar", "wuz"}}
with the test and probably the checks too containing references to the query variabel like "{{query}}". Unrolling such a test produces three different, new test, one with all occurences of "{{query}}" replaced by "foo", one with "bar" as the replacement and a third with "wuz". The unrolled tests do no longer contain the "{{query}}" variabel. If more than one variable is used during unrolling the situation is simple if both value sets have the same size: Variable substitution will use the first values first, then the second values and so on. If the variable have different length value sets e.g.
UnrollWith: map[string][]string{ "a": {"1", "2", "3"}, "b": {"x", "y"}, }
one would get 6 = 3*2 = the least common multiple of all value set length tests wit the first test having (a=1 b=x),the second one (a=2, b=y), the third one (a=3 b=x), and so on until the last one which has (a=3 b=y).
It is important to understand that Unrolling a Test produces several distinct Tests.
Suites of tests ¶
Normaly tests are not run individually but grouped into suites. Such suite may share a common cookie jar (and a common logger) and may contain setup and teardown actions. Any action on a Suite normaly requires its setup tests to pass, afterwards the main tests are executed and finaly the teardown tests are run (but no errors or failures are reported for teardown tests).
Index ¶
- Variables
- func AllWrongPenaltyFunc(s Status, c Criticality) float64
- func BoldBox(title string, prefix string) string
- func Box(title string, prefix string) string
- func DefaultPenaltyFunc(s Status, c Criticality) float64
- func JustBadPenaltyFunc(s Status, c Criticality) float64
- func NameOf(inst interface{}) string
- func RTHistogram(title string, data map[string][]int, dodge bool, filename string) error
- func RegisterCheck(check Check)
- func RegisterExtractor(ex Extractor)
- func TextContent(n *html.Node, raw bool) string
- func ToUpper(s string) string
- func Underline(title string, c string, prefix string) string
- type Body
- type BodyExtractor
- type CantCheck
- type Check
- type CheckList
- type CheckResult
- type Condition
- type ContentType
- type Cookie
- type Criticality
- type DeleteCookie
- type Duration
- type Extractor
- type ExtractorMap
- type FinalURL
- type HTMLContains
- type HTMLExtractor
- type HTMLTag
- type Header
- type Identity
- type Image
- type JSON
- type JSONExtractor
- type Latency
- type Links
- type LoadTestOptions
- type LoadtestResult
- type Logfile
- type MalformedCheck
- type NoServerError
- type PenaltyFunc
- type Poll
- type Redirect
- type Request
- type Resilience
- type Response
- type ResponseTime
- type SetCookie
- type Sorted
- type Status
- type StatusCode
- type Suite
- func (s Suite) AllTests() []*Test
- func (s *Suite) Execute()
- func (s *Suite) ExecuteConcurrent(maxConcurrent int) error
- func (s *Suite) ExecuteSetup()
- func (s *Suite) ExecuteTeardown()
- func (s *Suite) ExecuteTests()
- func (s Suite) HTMLReport(dir string) error
- func (s *Suite) JUnit4XML() (string, error)
- func (s *Suite) Prepare() error
- func (r Suite) PrintReport(w io.Writer) error
- func (s *Suite) Stats() (notRun int, skipped int, passed int, failed int, errored int, bogus int)
- type SuiteResult
- type Test
- func (t *Test) AsJSON5() ([]byte, error)
- func (t *Test) Benchmark(variables map[string]string, warmup int, count int, pause time.Duration, ...) []Test
- func (t *Test) Extract() map[string]string
- func (t *Test) PopulateCookies(jar http.CookieJar, u *url.URL)
- func (t Test) PrintReport(w io.Writer) error
- func (t *Test) Run(variables map[string]string) error
- type URLValues
- type UTF8Encoded
- type ValidHTML
- type ValidationIssue
- type W3CValidHTML
- type WrongCount
- type XML
Constants ¶
This section is empty.
Variables ¶
var ( // ErrBadBody is returned from checks if the request body is // not available (e.g. due to a failed request). ErrBadBody = errors.New("skipped due to bad body") // ErrNotFound is returned by checks if some expected value was // not found. ErrNotFound = errors.New("not found") // ErrFoundFirbidden is returned by checks if a forbidden value // is found. ErrFoundForbidden = errors.New("found forbidden") // ErrFailed is returned by a checks failing unspecificly. ErrFailed = errors.New("failed") )
var ( // DefaultUserAgent is the user agent string to send in http requests // if no user agent header is specified explicitely. DefaultUserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36" // DefaultAccept is the accept header to be sent if no accept header // is set explicitely in the test. DefaultAccept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" // DefaultClientTimeout is the timeout used by the http clients. DefaultClientTimeout = Duration(10 * time.Second) )
var ( TestTmpl *template.Template SuiteTmpl *template.Template HtmlSuiteTmpl *htmltemplate.Template )
var CheckRegistry = make(map[string]reflect.Type)
CheckRegistry keeps track of all known Checks.
var DefaultCriticality = CritError
DefaultCriticality is the criticality assigned to tests loaded from JSON5 which do not explicitely set the criticality.
var ExtractorRegistry = make(map[string]reflect.Type)
ExtractorRegistry keeps track of all known Extractors.
var Random *rand.Rand
Random is the source for all randmoness used in ht.
var Transport = &http.Transport{ Proxy: http.ProxyFromEnvironment, Dial: (&net.Dialer{ Timeout: 30 * time.Second, KeepAlive: 30 * time.Second, }).Dial, TLSClientConfig: &tls.Config{ InsecureSkipVerify: false, }, TLSHandshakeTimeout: 10 * time.Second, }
Transport is the http Transport used while making requests. It is exposed to allow different Timeouts or laxer TLS settings.
Functions ¶
func AllWrongPenaltyFunc ¶ added in v0.4.0
func AllWrongPenaltyFunc(s Status, c Criticality) float64
AllWrongPenaltyFunc ignores the criticality and returns 1 for all tests with status NotRun (as this happens only if the test wasn't executed due to failing setup tests), Fail, Error or Bogus.
func BoldBox ¶
BoldBox around title, indented by prefix.
################# ## ## ## Title ## ## ## #################
func DefaultPenaltyFunc ¶ added in v0.4.0
func DefaultPenaltyFunc(s Status, c Criticality) float64
DefaultPenaltyFunc penalises higher criticalities more than lower ones.
func JustBadPenaltyFunc ¶ added in v0.4.0
func JustBadPenaltyFunc(s Status, c Criticality) float64
JustBadPenaltyFunc returns 1 for the (status,criticality)-pairs in {NotRun, Fail, Error} X {CritError, CritFatal}. Note that Bogus tests yield 0.
func RTHistogram ¶
RTHistogram produces a png file containing a histogram of the int values in data. Several histograms can be plotted, the keys of data are the labels. If dodge is true the histogram bars are not stacked.
func RegisterCheck ¶
func RegisterCheck(check Check)
RegisterCheck registers the check. Once a check is registered it may be unmarshaled from its name and marshaled data.
func RegisterExtractor ¶ added in v0.4.0
func RegisterExtractor(ex Extractor)
RegisterExtractor registers the extratcor type. Once an extractor is registered it may be unmarshaled from its name and marshaled data.
func TextContent ¶ added in v0.7.0
TextContent returns the full text content of n. With raw processing the unprocessed content is returned. If raw==false then whitespace is normalized.
Types ¶
type Body ¶
type Body Condition
Body provides simple condition checks on the response body.
type BodyExtractor ¶ added in v0.4.0
type BodyExtractor struct { // Regexp is the regular expression to look for in the body. Regexp string // SubMatch selects which submatch (capturing group) of Regexp shall // be returned. A 0 value indicates the whole match. Submatch int `json:",omitempty"` }
BodyExtractor extracts a value from the uninterpreted response body.
type CantCheck ¶
type CantCheck struct {
// contains filtered or unexported fields
}
CantCheck is the error type returned by checks whose preconditions are not fulfilled, e.g. malformed HTML or XML.
type Check ¶
type Check interface { // Prepare is called to prepare the check, e.g. to compile // regular expressions or that like. Prepare() error // Execute executes the check. Execute(*Test) error }
Check is a single check performed on a Response.
func SubstituteVariables ¶
SubstituteVariables returns a deep copy of check with all exported string fields in check modified by applying r and all int and int64 fields modified by applying f. TODO: applying r is not "variable replacing"
type CheckList ¶
type CheckList []Check
CheckList is a slice of checks with the sole purpose of attaching JSON (un)marshaling methods.
func (CheckList) MarshalJSON5 ¶ added in v0.8.0
MarshalJSON produces a JSON arry of the checks in cl. Each check is serialized in the form
{ Check: "NameOfCheckAsRegistered", Field1OfCheck: Value1, Field2: Value2, ... }
func (*CheckList) UnmarshalJSON ¶
UnmarshalJSON unmarshals data to a slice of Checks.
type CheckResult ¶
type CheckResult struct { Name string // Name of the check as registered. JSON string // JSON serialization of check. Status Status // Outcome of check. All status but Error Duration Duration // How long the check took. Error error // For a Status of Bogus or Fail. }
CheckResult captures the outcom of a single check inside a test.
type Condition ¶
type Condition struct { // Equals is the exact value to be expected. // No other tests are performed if Equals is non-zero as these // other tests would be redundant. Equals string `json:",omitempty"` // Prefix is the required prefix Prefix string `json:",omitempty"` // Suffix is the required suffix. Suffix string `json:",omitempty"` // Contains must be contained in the string. Contains string `json:",omitempty"` // Regexp is a regular expression to look for. Regexp string `json:",omitempty"` // Count determines how many occurences of Contains or Regexp // are required for a match: // 0: Any positive number of matches is okay // > 0: Exactly that many matches required // < 0: No match allowed (invert the condition) Count int `json:",omitempty"` // Min and Max are the minimum and maximum length the string may // have. Two zero values disables this test. Min, Max int `json:",omitempty"` // contains filtered or unexported fields }
Condition is a conjunction of tests against a string. Note that Contains and Regexp conditions both use the same Count; most likely one would use either Contains or Regexp but not both.
func (Condition) Fullfilled ¶
Fullfilled returns whether s matches all requirements of c. A nil return value indicates that s matches the defined conditions. A non-nil return indicates missmatch.
func (Condition) FullfilledBytes ¶
FullfilledBytes provides a optimized version for Fullfilled(string(byteSlice)). TODO: Make this a non-lie.
type ContentType ¶ added in v0.5.0
type ContentType struct { // Is is the wanted content type. It may be abrevated, e.g. // "json" would match "application/json" Is string // Charset is an optional charset Charset string `json:",omitempty"` }
ContentType checks the Content-Type header.
func (ContentType) Execute ¶ added in v0.5.0
func (c ContentType) Execute(t *Test) error
Execute implements Check's Execute method.
func (ContentType) Prepare ¶ added in v0.5.0
func (ContentType) Prepare() error
Prepare implements Check's Prepare method.
type Criticality ¶ added in v0.4.0
type Criticality int
Criticality is the business criticality of this tests. Package ht does not interpret or use the business criticality of the tests.
const ( CritDefault Criticality = iota CritIgnore CritInfo CritWarn CritError CritFatal )
func (Criticality) MarshalJSON ¶ added in v0.4.0
func (c Criticality) MarshalJSON() ([]byte, error)
MarshalJSON produces a JSON representation of c.
func (Criticality) String ¶ added in v0.4.0
func (c Criticality) String() string
func (*Criticality) UnmarshalJSON ¶ added in v0.4.0
func (c *Criticality) UnmarshalJSON(data []byte) error
UnmarshalJSON allows to unmarshal the following JSON values to CritInfo:
"CritInfo" "Info" 1
type DeleteCookie ¶ added in v0.5.0
type DeleteCookie struct { Name string Path string `json:",omitempty"` Domain string `json:",omitempty"` }
DeleteCookie checks that the HTTP response properly deletes all cookies matching Name, Path and Domain. Path and Domain are optional in which case all cookies with the given Name are checkd for deletion.
func (DeleteCookie) Execute ¶ added in v0.5.0
func (c DeleteCookie) Execute(t *Test) error
Execute implements Check's Execute method.
func (*DeleteCookie) Prepare ¶ added in v0.5.0
func (c *DeleteCookie) Prepare() error
Prepare implements Check's Prepare method.
type Duration ¶
Duration is a time.Duration but has nicer JSON encoding.
func (Duration) MarshalJSON ¶
MarshalJSON provides JSON marshaling of d.
func (*Duration) UnmarshalJSON ¶
UnmarshalJSON provides JSON unmarshaling of data.
type ExtractorMap ¶ added in v0.4.0
ExtractorMap is a map of Extractors with the sole purpose of attaching JSON (un)marshaling methods.
func (ExtractorMap) MarshalJSON ¶ added in v0.4.0
func (em ExtractorMap) MarshalJSON() ([]byte, error)
MarshalJSON produces a JSON array of the Extractors in em. Each Extractor is serialized in the form
{ Extractor: "NameOfExtractorAsRegistered", Field1OfExtratcor: Value1, Field2: Value2, ... }
func (*ExtractorMap) UnmarshalJSON ¶ added in v0.4.0
func (em *ExtractorMap) UnmarshalJSON(data []byte) error
UnmarshalJSON unmarshals data to a map of Extractors.
type FinalURL ¶ added in v0.5.0
type FinalURL Condition
FinalURL checks the last URL after following all redirects. This check is useful only for tests with Request.FollowRedirects=true
type HTMLContains ¶
type HTMLContains struct { // Selector is the CSS selector of the HTML elements. Selector string // Text contains the expected plain text content of the HTL elements // selected through the given selector. Text []string `json:",omitempty"` // Raw turns of white space normalization and will check the unprocessed // text content. Raw bool `json:",omitempty"` // Complete makes sure that no excess HTML elements are found: // If true the len(Text) must be equal to the number of HTML elements // selected for the check to succeed. Complete bool `json:",omitempty"` // InOrder makes the check fail if the selected HTML elements have a // different order than given in Text. InOrder bool `json:",omitempty"` // contains filtered or unexported fields }
HTMLContains checks the text content (and optionally the order) of HTML elements selected by a CSS rule.
The text content found in the HTML document is normalized by roughly the following procedure:
- Newlines are inserted around HTML block elements (i.e. any non-inline element)
- Newlines and tabs are replaced by spaces.
- Multiple spaces are replaced by one space.
- Leading and trailing spaces are trimmed of.
As an example consider the following HTML:
<html><body> <ul class="fancy"><li>One</li><li>S<strong>econ</strong>d</li><li> Three </li></ul> </body></html>
The normalized text selected by a Selector of "ul.fancy" would be
"One Second Three"
func (*HTMLContains) Execute ¶
func (c *HTMLContains) Execute(t *Test) error
Execute implements Check's Execute method.
func (*HTMLContains) Prepare ¶
func (c *HTMLContains) Prepare() (err error)
Prepare implements Check's Prepare method.
type HTMLExtractor ¶ added in v0.4.0
type HTMLExtractor struct { // Selector is the CSS selector of an element, e.g. // head meta[name="_csrf"] or // form#login input[name="tok"] // div.token span Selector string // Attribute is the name of the attribute from which the // value should be extracted. The magic value "~text~" refers to the // normalized text content of the element and ~rawtext~ to the raw // text content. // E.g. in the examples above the following should be sensible: // content // value // ~text~ Attribute string }
HTMLExtractor allows to extract data from an executed Test. It supports extracting HTML attribute values and HTML text node values. Examples for CSRF token in the HTML:
<meta name="_csrf" content="18f0ca3f-a50a-437f-9bd1-15c0caa28413" /> <input type="hidden" name="_csrf" value="18f0ca3f-a50a-437f-9bd1-15c0caa28413"/>
type HTMLTag ¶ added in v0.3.0
type HTMLTag struct { // Selector is the CSS selector of the HTML elements. Selector string // Count determines the number of occurrences to check for: // < 0: no occurrence // == 0: one ore more occurrences // > 0: exactly that many occurrences Count int `json:",omitempty"` // contains filtered or unexported fields }
HTMLTag checks for the existens of HTML elements selected by CSS selectors.
type Header ¶
type Header struct { // Header is the HTTP header to check. Header string // Condition is applied to the first header value. A zero value checks // for the existence of the given Header only. Condition `json:",omitempty"` // Absent indicates that no header Header shall be part of the response. Absent bool `json:",omitempty"` }
Header provides a textual test of single-valued HTTP headers.
type Identity ¶
type Identity struct { // SHA1 is the expected hash as shown by sha1sum of the whole body. // E.g. 2ef7bde608ce5404e97d5f042f95f89f1c232871 for a "Hello World!" // body (no newline). SHA1 string }
Identity checks the value of the response body by comparing its SHA1 hash to the expected SHA1 value.
type Image ¶
type Image struct { // Format is the format of the image as registered in package image. Format string `json:",omitempty"` // If > 0 check width or height of image. Width, Height int `json:",omitempty"` // Fingerprint is either the 16 hex digit long Block Mean Value hash or // the 24 hex digit long Color Histogram hash of the image. Fingerprint string `json:",omitempty"` // Threshold is the limit up to which the received image may differ // from the given BMV or ColorHist fingerprint. Threshold float64 `json:",omitempty"` }
Image checks image format, size and fingerprint. As usual a zero value of a field skips the check of that property. Image fingerprinting is done via github.com/vdobler/ht/fingerprint. Only one of BMV or ColorHist should be used as there is just one threshold.
type JSON ¶
type JSON struct { // Expression is a boolean gojee expression which must evaluate // to true for the check to pass. Expression string `json:",omitempty"` // Path in the flattened JSON map to apply the Condition to. Path string `json:",omitempty"` // Condition to apply to the value selected by Path. // If Condition is the zero value then only the existence of // a JSON element selected by Path is checked. // Note that Condition s checked against the actual value in the // flattened JSON map which will contain the quotation marks for // string values. Condition `json:",omitempty"` // Sep is the seperator in Path when checking the Condition. // A zero value is equivalanet to "." Sep string `json:",omitempty"` // contains filtered or unexported fields }
JSON checking via github.com/nytlabs/gojee (Expression) and github.com/nytlabs/gojsonexplode (Path, Condition (+ Sep). Both, Expression and Path, may be empty in which case this check just makes sure the response bodyis wellformed JSON.
type JSONExtractor ¶ added in v0.6.0
type JSONExtractor struct { // Path in the flattened JSON map to extract. Path string `json:",omitempty"` // Sep is the seperator in Path. // A zero value is equivalent to "." Sep string `json:",omitempty"` }
JSONExtractor extracts a value from a JSON response body. It uses github.com/nytlabs/gojsonexplode to flatten the JSON file for easier access.
type Latency ¶ added in v0.7.0
type Latency struct { // N is the number if request to measure. It should be much larger // than Concurrent. Default is 50. N int `json:",omitempty"` // Concurrent is the number of concurrent requests in flight. // Defaults to 2. Concurrent int `json:",omitempty"` // Limits is a string of the following form: // "50% ≤ 150; 80% ≤ 200; 95% ≤ 250; 0.9995 ≤ 900" // The limits above would require the median of the response // times to be <= 150 ms and would allow only 1 request in 2000 to // exced 900ms. // Note that it must be the ≤ signe (U+2264), a plain < or a <= // is not recognized. Limits string `json:",omitempty"` // If SkipChecks is true no checks are performed i.e. only the // requests are executed. SkipChecks bool `json:",omitempty"` // DumpTo is the filename where the latencies are reported. // The special values "stdout" and "stderr" are recognised. DumpTo string `json:",omitempty"` // contains filtered or unexported fields }
type Links ¶
type Links struct { // Head triggers HEAD requests instead of GET requests. Head bool // Which links to test; a combination of "a", "img", "link" and "script". // E.g. use "a img" to check the href of all a tags and src of all img tags. Which string // Concurrency determines how many of the found links are checked // concurrently. A zero value indicates sequential checking. Concurrency int `json:",omitempty"` // Timeout is the client timeout if different from main test. Timeout Duration `json:",omitempty"` // OnlyLinks and IgnoredLinks can be used to select only a subset of // all links. OnlyLinks, IgnoredLinks []Condition `json:",omitempty"` // contains filtered or unexported fields }
Links checks links and references in HTML pages for availability.
type LoadTestOptions ¶
type LoadTestOptions struct { // Type determines wether a "throughput" or "concurrency" test is done. Type string // Count many request are made during the load test in total. Count int // Timout determines how long a test may run: The load test is // terminated if timeout is exeded, even if not Count many requests // heve been made yet. Timeout time.Duration // Rate is the average rate of requests in [request/sec]. Rate float64 // Uniform changes to uniform (equaly spaced) distribution of // requests. False uses an exponential distribution. Uniform bool }
LoadTestOptions controls details of a load test.
type LoadtestResult ¶
type LoadtestResult struct { Started time.Time Total int Passed int Failed int Errored int Skipped int Bogus int PassHist *loghist.Hist FailHist *loghist.Hist BothHist *loghist.Hist }
LoadtestResult captures aggregated values of a load test.
func AnalyseLoadtest ¶
func AnalyseLoadtest(results []Test) LoadtestResult
AnalyseLoadtest computes aggregate statistics of the given results.
func (LoadtestResult) String ¶
func (r LoadtestResult) String() string
String formats r in a useful way.
type Logfile ¶ added in v0.3.0
type Logfile struct { // Path is the file system path to the logfile." Path string // Condition the written stuff must fulfill. Condition `json:",omitempty"` // Disallow states what is forbidden in the written log. Disallow []string `json:",omitempty"` // contains filtered or unexported fields }
Logfile provides checks on files (i.e. it ignores the response). During preparation the current file size is determined and the checks are run against the bytes written after preparation.
type MalformedCheck ¶
type MalformedCheck struct {
Err error
}
MalformedCheck is the error type returned by checks who are badly parametrized, e.g. who try to check against a malformed regular expression.
func (MalformedCheck) Error ¶
func (m MalformedCheck) Error() string
type NoServerError ¶ added in v0.8.0
type NoServerError struct{}
NoServerError checks the HTTP status code for not beeing a 5xx server error and that the body could be read without errors or timeouts.
func (NoServerError) Execute ¶ added in v0.8.0
func (NoServerError) Execute(t *Test) error
Execute implements Check's Execute method.
func (NoServerError) Prepare ¶ added in v0.8.0
func (NoServerError) Prepare() error
Prepare implements Check's Prepare method.
type PenaltyFunc ¶ added in v0.4.0
type PenaltyFunc func(s Status, c Criticality) float64
PenaltyFunc is a function to calculate a penalty for a given test status and criticality combination.
type Poll ¶
type Poll struct { // Maximum number of redos. Both 0 and 1 mean: "Just one try. No redo." // Negative values indicate that the test should be skipped. Max int `json:",omitempty"` // Duration to sleep between redos. Sleep Duration `json:",omitempty"` }
Poll determines if and how to redo a test after a failure or if the test should be skipped alltogether. The zero value of Poll means "Just do the test once."
type Redirect ¶ added in v0.5.0
type Redirect struct { // To is matched against the Location header. It may begin with, // or end with contain three dots "..." which inicate that To should // match the end or the start or both ends of the Location header // value. (Note that only one occurence of "..." is supported." To string // If StatusCode is greater zero it is the required HTTP status code // expected in this response. If zero the valid status codes are // 301 (Moved Permanently), 302 (Found), 303 (See Other) and // 307 (Temporary Redirect) StatusCode int `json:",omitempty"` }
Redirect checks for HTTP redirections.
type Request ¶
type Request struct { // Method is the HTTP method to use. // A empty method is equivalent to "GET" Method string `json:",omitempty"` // URL ist the URL of the request. URL string // Params contains the parameters and their values to send in // the request. // // If the parameters are sent as multipart it is possible to include // files by letting the parameter values start with "@file:". Two // version are possible "@file:path/to/file" will send a file read // from the given filesystem path while "@file:@name:the-file-data" // will use the-file-data as the content. Params URLValues `json:",omitempty"` // ParamsAs determines how the parameters in the Param field are sent: // "URL" or "": append properly encoded to URL // "body" : send as application/x-www-form-urlencoded in body. // "multipart": send as multipart/form-data in body. // The two values "body" and "multipart" must not be used // on a GET or HEAD request. ParamsAs string `json:",omitempty"` // Header contains the specific http headers to be sent in this request. // User-Agent and Accept headers are set automaticaly to the global // default values if not set explicitely. Header http.Header `json:",omitempty"` // Cookies contains the cookies to send in the request. Cookies []Cookie `json:",omitempty"` // Body is the full body to send in the request. Body must be // empty if Params are sent as multipart or form-urlencoded. Body string `json:",omitempty"` // FollowRedirects determines if automatic following of // redirects should be done. FollowRedirects bool `json:",omitempty"` Request *http.Request `json:"-"` // the 'real' request SentBody string `json:"-"` // the 'real' body }
Request is a HTTP request.
type Resilience ¶ added in v0.8.0
type Resilience struct { // Methods is the space seperated list of HTTP methods to check, // e.g. "GET POST HEAD". The empty value will test the original // method of the test. Methods string `json:",omitempty"` // ModParam and ModHeader controll which modifications of parameter values // and header values are checked. // It is a space seprated string if the modifications explained above // e.g. "drop nonsense empty". ModParam, ModHeader string `json:",omitempty"` // ParamsAs controls how parameter values are transmitted, it // is a space seperated list of all transmission types like in // the Request.ParamsAs field, e.g. "URL body multipart" to check // URL query parameters, x-www-form-urlencoded and multipart/formdata. // The empty value will just check the type used in the original // test. ParamsAs string `json:",omitempty"` // SaveFailuresTo is the filename to which all failed checks shall // be logged. The data is appended to the file. SaveFailuresTo string }
Resilience checks the resilience of an URL against unexpected requests like different HTTP methods, changed or garbled parameters, different parameter transmission types and changed or garbled HTTP headers.
Parameters and Header values can undergo several different types of modifications
- drop: don't send at all
- double: send same value two times
- twice: send two different values
- change: change a single character (first, middle and last one)
- delete: drop single character (first, middle and last one)
- nonsense: "hubba%12bubba(!"
- malicious: "\000\ufeff<script>\r\r\n" (not suitable for headers)
- empty: ""
- type: change the type (if obvious)
- "1234" --> "xxxx"
- "i@you.me" --> "iXyouYme"
- "foobar " --> "12345"
- "08:45" --> "zz:zz"
- "#0c12ff --> "blün"
- large: produce much larger values
- "1234" --> "98765432123456789"
- "56.78" --> "8888888888888888.9999999999"
- "foo" --> "fooooooooooooooooooooooooooooooooooooooooooooooooooooo"
- "08:45" --> "25:65"
- negative produce negative values
- "1234" --> "-2"
- "56.78" --> "-3.0"
- tiny: produce 0 or short values
- "1234" --> "0"
- "12.3" --> "0"
- "foobar" --> "f"
- none: don't modify the individual parameters or header but don't send any parameters or headers
This check will make a wast amount of request to the given URL including the modifying and non-idempotent methods POST, PUT, and DELETE. Some care using this check is advisable.
func (Resilience) Execute ¶ added in v0.8.0
func (r Resilience) Execute(t *Test) error
Execute implements Check's Execute method.
func (Resilience) Prepare ¶ added in v0.8.0
func (r Resilience) Prepare() error
Prepare implements Check's Prepare method.
type Response ¶
type Response struct { // Response is the received HTTP response. Its body has bean read and // closed allready. Response *http.Response `json:",omitempty"` // Duration to receive response and read the whole body. Duration Duration // The received body and the error got while reading it. BodyStr string BodyErr error // Redirections records the URLs of automatic GET requests due to redirects. Redirections []string }
Response captures information about a http response.
type ResponseTime ¶
ResponseTime checks the response time.
func (ResponseTime) Execute ¶
func (c ResponseTime) Execute(t *Test) error
Execute implements Check's Execute method.
func (ResponseTime) Prepare ¶
func (ResponseTime) Prepare() error
Prepare implements Check's Prepare method.
type SetCookie ¶
type SetCookie struct { Name string `json:",omitempty"` // Name is the cookie name. Value Condition `json:",omitempty"` // Value is applied to the cookie value Path Condition `json:",omitempty"` // Path is applied to the path value Domain Condition `json:",omitempty"` // Domain is applied to the domain value // MinLifetime is the expectetd minimum lifetime of the cookie. // A positive value enforces a persistent cookie. // Negative values are illegal (use DelteCookie instead). MinLifetime Duration `json:",omitempty"` // Absent indicates that the cookie with the given Name must not be received. Absent bool `json:",omitempty"` // Type is the type of the cookie. It is a space seperated string of // the following (case-insensitive) keywords: // - "session": a session cookie // - "persistent": a persistent cookie // - "secure": a secure cookie, to be sont over https only // - "unsafe", aka insecure; to be sent also over http // - "httpOnly": not accesible from JavaScript // - "exposed": accesible from JavaScript, Flash, etc. Type string `json:",omitempty"` }
SetCookie checks for cookies being properly set. Note that the Path and Domain conditions are checked on the received Path and/or Domain and not on the interpreted values according to RFC 6265.
type Sorted ¶ added in v0.7.0
type Sorted struct { // Text is the list of text fragments to look for in the // response body or the normalized text content of the // HTML page. Text []string // AllowedMisses is the number of elements of Text which may // not be present in the response body. The default of 0 means // all elements of Text must be present. AllowedMisses int `json:",omitempty"` }
Sorted checks for an ordered occurence of items. The check Sorted could be replaced by a Regexp based Body test without loss of functionality; Sorted just makes the idea of "looking for a sorted occurence" clearer.
If the response has a Content-Type header indicating a HTML response the HTML will be parsed and the text content normalized as described in the HTMLContains check.
type Status ¶
type Status int
Status describes the status of a Test or a Check.
func (Status) MarshalText ¶
type StatusCode ¶
type StatusCode struct {
Expect int
}
StatusCode checks the HTTP statuscode.
func (StatusCode) Execute ¶
func (c StatusCode) Execute(t *Test) error
Execute implements Check's Execute method.
func (StatusCode) Prepare ¶
func (StatusCode) Prepare() error
Prepare implements Check's Prepare method.
type Suite ¶
type Suite struct { Name string Description string // Test contains the actual tests to execute. Tests []*Test // OmitChecks allows to omit all checks defined on the main tests. OmitChecks bool // Setup contain tests to be executed before the execution actual tests. // If one or more setup test fail, the main tets won't be executed. // Teardown tests are excuted after the main test. // Setup and Teardown share the cookie jar with the main tests. Setup, Teardown []*Test // Variables contains global variables to be used during this // execution Variables map[string]string // KeepCookies determines whether to use a cookie jar to keep // cookies between tests. KeepCookies bool // Log is the logger to be used by tests and checks. Log *log.Logger // Populated during execution Status Status Error error Started time.Time Duration Duration }
A Suite is a collection of tests which are run together. A Suite must be prepared before it can be executed or Executes concurrently.
func LoadSuite ¶
LoadSuite reads a suite from filename. Tests and mixins are read relative to the directory the suite lives in.
func (*Suite) ExecuteConcurrent ¶
ExecuteConcurrent executes all non-setup, non-teardown tests concurrently. But at most maxConcurrent tests of s are executed concurrently.
func (*Suite) ExecuteSetup ¶
func (s *Suite) ExecuteSetup()
ExecuteSetup runs the setup tests s. The tests are executed sequentialy, execution stops on the first error.
func (*Suite) ExecuteTeardown ¶
func (s *Suite) ExecuteTeardown()
ExecuteTeardown runs all teardown tests ignoring all errors.
func (*Suite) ExecuteTests ¶
func (s *Suite) ExecuteTests()
ExecuteTests run the non-setup, non-teardown tests of s sequentialy.
func (Suite) HTMLReport ¶
func (*Suite) JUnit4XML ¶ added in v0.2.0
JUnit4XML generates a JUnit 4 compatible XML result with each Check reported as an individual testcase. NotRun checks are reported as Skipped and Bogus checks are counted as Errored tests.
type SuiteResult ¶ added in v0.4.0
type SuiteResult struct { Started time.Time // Start time of earliest suite. Duration Duration // Cummulated duration of all accounted suites. // Count is the histogram of test results indexed by (status x criticality) // Row and column sums are provided. Count [][]int Suites []*Suite // Suites contains references to all accounted suites. }
SuiteResult allows to accumulate statistics of executed suites
func NewSuiteResult ¶ added in v0.4.0
func NewSuiteResult() *SuiteResult
NewSuiteResult returns an empty SuiteResult.
func (*SuiteResult) Account ¶ added in v0.4.0
func (r *SuiteResult) Account(s *Suite, setup bool, teardown bool)
Account updates the SuiteResult r with the results from s. Setup and teardown control whether the setup and teardown tests of s are included in the accounting.
func (*SuiteResult) KPI ¶ added in v0.4.0
func (r *SuiteResult) KPI(pf PenaltyFunc) float64
KPI condeses the results of the accumulated suites of r into one single float number by averaging the results after sending through the given penalty function.
func (*SuiteResult) Matrix ¶ added in v0.4.0
func (r *SuiteResult) Matrix() string
Matrix returns the histogram data as a formated string.
func (*SuiteResult) Merge ¶ added in v0.4.0
func (r *SuiteResult) Merge(o *SuiteResult)
func (*SuiteResult) Tests ¶ added in v0.4.0
func (r *SuiteResult) Tests() int
type Test ¶
type Test struct { Name string Description string `json:",omitempty"` // Request is the HTTP request. Request Request // Checks contains all checks to perform on the response to the HTTP request. Checks CheckList // VarEx may be used to popultate variables from the response. VarEx map[string]Extractor `json:",omitempty"` Poll Poll `json:",omitempty"` Timeout Duration `json:",omitempty"` // If zero use DefaultClientTimeout. Verbosity int `json:",omitempty"` // Verbosity level in logging. Criticality Criticality `json:",omitempty"` // Business criticality of this test // Pre-, Inter- and PostSleep are the sleep durations made // before the request, between request and the checks and // after the checks. PreSleep, InterSleep, PostSleep Duration `json:",omitempty"` // Jar is the cookie jar to use Jar http.CookieJar `json:"-"` Response Response `json:",omitempty"` // The following results are filled during Run. Status Status `json:"-"` Started time.Time `json:"-"` Error error `json:"-"` Duration Duration `json:"-"` FullDuration Duration `json:"-"` Tries int `json:"-"` CheckResults []CheckResult `json:"-"` // The individual checks. SeqNo string `json:"-"` Variables map[string]string // contains filtered or unexported fields }
Test is a single logical test which does one HTTP request and checks a number of Checks on the recieved Response.
func LoadTest ¶
LoadTest reads a test from filename. Tests and mixins are read relative to the directory the test lives in. Unrolling is performed. TODO
func Merge ¶
Merge merges all tests into one. The individual fields are merged in the following way.
Name Join all names Description Join all descriptions Request Method All nonempty must be the same URL Only one may be nonempty Params Merge by key ParamsAs All nonempty must be the same Header Merge by key Cookies Merge by cookie name Body Only one may be nonempty FollowRdr Last wins Checks Append all checks VarEx Merge, same keys must have same value Poll Max Use largest Sleep Use largest Timeout Use largets Verbosity Use largets PreSleep Summ of all; same for InterSleep and PostSleep ClientPool ignore Criticality Largest wins
func PerformanceLoadTest ¶
func PerformanceLoadTest(suites []*Suite, opts LoadTestOptions) ([]Test, error)
PerformanceLoadTest will perform a load test of the main tests of suites, the details of the load test is controlled by opts. Errors are reported if any suite's Setup failed.
func Repeat ¶
Repeat returns count copies of test with variables replaced based on vars. The keys of vars are the variable names. The values of a variable v are choosen from vars[v] by cycling through the list: In the n'th repetition is vars[v][n%N] with N=len(vars[v])).
func (*Test) AsJSON5 ¶ added in v0.8.0
AsJSON5 returns a JSON5 representation of the test. Executed tests can be serialised and will contain basically all information required to debug or re-run the test but note that several fields in the actual *http.Request and *http.Response structs are cleared during this serialisation.
func (*Test) Benchmark ¶
func (t *Test) Benchmark(variables map[string]string, warmup int, count int, pause time.Duration, conc int) []Test
Benchmark executes t count many times and reports the outcome. Before doing the measurements warmup many request are made and discarded. Conc determines the concurrency level. If conc==1 the given pause is made between request. A conc > 1 will execute conc many request in paralell (without pauses). TODO: move this into an BenmarkOptions
func (*Test) PopulateCookies ¶ added in v0.8.0
PopulateCookies populates t.Request.Cookies with the those cookies from jar which would be sent to u.
func (*Test) Run ¶
Run runs the test t. The actual HTTP request is crafted and executed and the checks are performed on the received response. This whole process is repeated on failure or skipped entirely according to t.Poll.
The given variables are subsitutet into the relevant parts of the reuestt and the checks.
Normally all checks in t.Checks are executed. If the first check in t.Checks is a StatusCode check against 200 and it fails, then the rest of the tests are skipped.
Run returns a non-nil error only if the test is bogus; a failing http request, problems reading the body or any failing checks do not trigger a non-nil return value.
type URLValues ¶
URLValues is a url.Values with a fancier JSON unmarshalling.
func (*URLValues) UnmarshalJSON ¶
UnmarshalJSON produces a url.Values (i.e. a map[string][]string) from various JSON5 representations. E.g.
{ a: 12, b: "foo", c: [ 23, "bar"] }
can be unmarshaled with the expected result.
type UTF8Encoded ¶
type UTF8Encoded struct{}
UTF8Encoded checks that the response body is valid UTF-8 without BOMs.
func (UTF8Encoded) Execute ¶
func (c UTF8Encoded) Execute(t *Test) error
Execute implements Check's Execute method.
func (UTF8Encoded) Prepare ¶
func (UTF8Encoded) Prepare() error
Prepare implements Check's Prepare method.
type ValidHTML ¶
type ValidHTML struct{}
ValidHTML checks for valid HTML 5. Kinda: It never fails. TODO: make it useful.
type ValidationIssue ¶
ValidationIssue contains extracted information from the output of a W3C validator run.
type W3CValidHTML ¶
type W3CValidHTML struct { // AllowedErrors is the number of allowed errors (after ignoring errors). AllowedErrors int `json:",omitempty"` // IgnoredErrros is a list of error messages to be ignored completely. IgnoredErrors []Condition `json:",omitempty"` }
W3CValidHTML checks for valid HTML but checking the response body via the online checker from W3C which is very strict.
func (W3CValidHTML) Execute ¶
func (w W3CValidHTML) Execute(t *Test) error
Execute implements Check's Execute method.
func (W3CValidHTML) Prepare ¶
func (W3CValidHTML) Prepare() error
Prepare implements Check's Prepare method.
type WrongCount ¶
type WrongCount struct {
Got, Want int
}
WrongCount is the error type returned by checks which require a certain number of matches.
func (WrongCount) Error ¶
func (m WrongCount) Error() string
type XML ¶
type XML struct { // Path is a XPath expression understood by launchpad.net/xmlpath. Path string // Condition the first element addressed by Path must fullfill. Condition // contains filtered or unexported fields }
XML allows to check XML request bodies.