uidetection

package
v0.0.0-...-9474aff Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 28, 2022 License: BSD-3-Clause Imports: 26 Imported by: 0

Documentation

Overview

Package uidetection provides image-based UI detections/interactions.

Index

Constants

View Source 
const (
	// KeyType represents the var name of the key type used for UI detection API.
	KeyType = "uidetection.key_type"
	// Key represents the var name of the key used for UI detection API.
	Key = "uidetection.key"
	// Server represents the var name of the server address used for UI detection API.
	Server = "uidetection.server"
)
View Source 
const (
	// ErrNotFound is the error when there is no matching elements found.
	ErrNotFound = "no matching elements found"
	// ErrMultipleMatch is the error when there are multiple matching elements found.
	ErrMultipleMatch = "there are multiple matches"
	// ErrNthNotFound is the error when the Nth element doesn't exist.
	ErrNthNotFound = "Nth element not found"
	// ErrEmptyBoundingBox is the error when the relative matchers create a
	// screenshot of zero size.
	ErrEmptyBoundingBox = "The element you're trying to screenshot has zero size"
)

Variables

View Source 
var UIDetectionVars = []string{
	KeyType,
	Key,
	Server,
}

UIDetectionVars contains a list of all variables used by the UI detection library.

Functions

This section is empty.

Types

type Context 

type Context struct {
	// contains filtered or unexported fields
}

Context provides functionalities for image-based UI automation.

func New 

func New(t *chrome.TestConn, keyType, key, server string) *Context

New returns a new UI Detection automation instance.

func NewDefault 

func NewDefault(t *chrome.TestConn) *Context

NewDefault returns a new UI Detection automation instance with default params.

func (*Context) Exists 

func (uda *Context) Exists(s *Finder) uiauto.Action

Exists returns an action that returns nil if the specified element exists.

func (*Context) Gone 

func (uda *Context) Gone(s *Finder) uiauto.Action

Gone returns an action that returns nil if the specified element doesn't exist.

func (*Context) LeftClick 

func (uda *Context) LeftClick(s *Finder) uiauto.Action

LeftClick returns an action that left-clicks a finder.

func (*Context) Location 

func (uda *Context) Location(ctx context.Context, s *Finder) (*Location, error)

Location finds the location of a finder in the screen.

func (*Context) RightClick 

func (uda *Context) RightClick(s *Finder) uiauto.Action

RightClick returns an action that right-clicks a finder.

func (*Context) Tap 

func (uda *Context) Tap(s *Finder) uiauto.Action

Tap performs a single touchscreen tap.

func (*Context) WaitUntilExists 

func (uda *Context) WaitUntilExists(s *Finder) uiauto.Action

WaitUntilExists returns an action that waits until the specified element exists.

func (*Context) WaitUntilGone 

func (uda *Context) WaitUntilGone(s *Finder) uiauto.Action

WaitUntilGone returns an action that waits until the specified element doesnt exist.

func (*Context) WithInterval 

func (uda *Context) WithInterval(interval time.Duration) *Context

WithInterval returns a new Context with the specified polling interval.

func (*Context) WithOptions 

func (uda *Context) WithOptions(optionList ...Option) *Context

WithOptions returns a new Context with the specified detection options.

func (*Context) WithPollOpts 

func (uda *Context) WithPollOpts(pollOpts testing.PollOptions) *Context

WithPollOpts returns a new Context with the specified polling options.

func (*Context) WithScreenshotStrategy 

func (uda *Context) WithScreenshotStrategy(s ScreenshotStrategy) *Context

WithScreenshotStrategy returns a new Context with the specified screenshot strategy.

func (*Context) WithTimeout 

func (uda *Context) WithTimeout(timeout time.Duration) *Context

WithTimeout returns a new Context with the specified timeout.

type CustomIconParam 

type CustomIconParam = func(*CustomIconParams)

CustomIconParam is a modifier to apply to CustomIconParams.

func MatchCount 

func MatchCount(matchCount int32) CustomIconParam

MatchCount controls the MatchCount param.

func MinConfidence 

func MinConfidence(minConfidence float64) CustomIconParam

MinConfidence controls the min confidence threshold.

type CustomIconParams 

type CustomIconParams struct {
	// MatchCount is the limit of number of matches.
	// Set to -1 to not limit the number of matches.
	MatchCount int32
	// MinConfidence is the threshold in the range [0.0, 1.0] below which
	// the matches will be considered as non-existent.
	MinConfidence float64
}

CustomIconParams provides all of the ways which you can configure a custom icon detection.

NOTE: The default values are the recommended values. The users are not expected to modify them if the test is passing with the defaults. If the defaults are not working, the users will need to find a proper value with the trial and error method.

See https://chromium.googlesource.com/chromiumos/platform/tast/+/HEAD/docs/using_uidetection.md for more details.

func DefaultCustomIconParams 

func DefaultCustomIconParams() *CustomIconParams

DefaultCustomIconParams return params with default values.

type Finder 

type Finder struct {
	// contains filtered or unexported fields
}

Finder represents a data structure that consists of arguments to find a UI element.

func CustomIcon 

func CustomIcon(iconFile string, paramList ...CustomIconParam) *Finder

CustomIcon returns a finder for a given icon.

func TextBlock 

func TextBlock(words []string, paramList ...TextParam) *Finder

TextBlock returns a finder for a given textblock. Use TextBlock if you want to find a group of nearby text, generally representing an entire text element. E.g. use uidetection.TextBlock([]string{"Save", "As"}) to find the "Save As" menu item.

func Word 

func Word(word string, paramList ...TextParam) *Finder

Word returns a finder for a given word.

func (*Finder) Above 

func (f *Finder) Above(other *Finder) *Finder

Above ensures that the element returned must be above the element returned by the finder on the screen.

func (*Finder) AboveA11yNode 

func (f *Finder) AboveA11yNode(other *nodewith.Finder) *Finder

AboveA11yNode ensures that the element returned must be above the element returned by the a11y node finder on the screen.

func (*Finder) AboveDp 

func (f *Finder) AboveDp(dp int) *Finder

AboveDp ensures that the element returned must be above dp display pixels on the screen.

func (*Finder) AbovePx 

func (f *Finder) AbovePx(px int) *Finder

AbovePx ensures that the element returned must be above px pixels on the screen.

func (*Finder) Below 

func (f *Finder) Below(other *Finder) *Finder

Below ensures that the element returned must be below the element returned by the finder on the screen.

func (*Finder) BelowA11yNode 

func (f *Finder) BelowA11yNode(other *nodewith.Finder) *Finder

BelowA11yNode ensures that the element returned must be below the element returned by the a11y node finder on the screen.

func (*Finder) BelowDp 

func (f *Finder) BelowDp(dp int) *Finder

BelowDp ensures that the element returned must be below dp display pixels on the screen.

func (*Finder) BelowPx 

func (f *Finder) BelowPx(px int) *Finder

BelowPx ensures that the element returned must be below px pixels on the screen.

func (*Finder) ExactMatch 

func (f *Finder) ExactMatch() *Finder

ExactMatch turns off the approximate match for the word finder. The results will be filtered by exact string matching. USE WITH CAUTION. Due to the performance of the OCR (optical character recognition) model, approximate match is the default behavior for error-tolerance. An example use case is when the matching word is short with two or three letters. TODO(b/211937254): Allow exact matches with max_edit_distance in new proto.

func (*Finder) First 

func (f *Finder) First() *Finder

First enables the finder to choose the first match of a UI element if there are multiple matches.

func (*Finder) LeftOf 

func (f *Finder) LeftOf(other *Finder) *Finder

LeftOf ensures that the element returned must be left of the element returned by the finder on the screen.

func (*Finder) LeftOfA11yNode 

func (f *Finder) LeftOfA11yNode(other *nodewith.Finder) *Finder

LeftOfA11yNode ensures that the element returned must be left of the element returned by the a11y node finder on the screen.

func (*Finder) LeftOfDp 

func (f *Finder) LeftOfDp(dp int) *Finder

LeftOfDp ensures that the element returned must be left of dp display pixels on the screen.

func (*Finder) LeftOfPx 

func (f *Finder) LeftOfPx(px int) *Finder

LeftOfPx ensures that the element returned must be left of px pixels on the screen.

func (*Finder) Nth 

func (f *Finder) Nth(nth int) *Finder

Nth enables the finder to choose the n-th match of a UI element. if there are multiple matches.

func (*Finder) RightOf 

func (f *Finder) RightOf(other *Finder) *Finder

RightOf ensures that the element returned must be right of the element returned by the finder on the screen.

func (*Finder) RightOfA11yNode 

func (f *Finder) RightOfA11yNode(other *nodewith.Finder) *Finder

RightOfA11yNode ensures that the element returned must be right of the element returned by the a11y node finder on the screen.

func (*Finder) RightOfDp 

func (f *Finder) RightOfDp(dp int) *Finder

RightOfDp ensures that the element returned must be right of dp display pixels on the screen.

func (*Finder) RightOfPx 

func (f *Finder) RightOfPx(px int) *Finder

RightOfPx ensures that the element returned must be right of px pixels on the screen.

func (*Finder) Within 

func (f *Finder) Within(other *Finder) *Finder

Within ensures that the element returned must be within the element returned by the finder on the screen.

func (*Finder) WithinA11yNode 

func (f *Finder) WithinA11yNode(other *nodewith.Finder) *Finder

WithinA11yNode ensures that the element returned must be above the element returned by the a11y node finder on the screen.

func (*Finder) WithinDp 

func (f *Finder) WithinDp(r coords.Rect) *Finder

WithinDp ensures that the element returned must be within the rectangle on the screen, measuren in dp.

func (*Finder) WithinPx 

func (f *Finder) WithinPx(r coords.Rect) *Finder

WithinPx ensures that the element returned must be within the rectangle on the screen, measured in pixels.

type Location 

type Location struct {
	// Rectangle of the location.
	coords.Rect
	// Text associated with the element, if any.
	Text string
}

Location represents the location of a matching UI element.

type Option 

type Option = func(*Options)

Option is a modifier to apply to Options.

func Retries 

func Retries(retries int) Option

Retries controls the number of retries.

func RetryInterval 

func RetryInterval(retryInterval time.Duration) Option

RetryInterval controls the interval between retries.

type Options 

type Options struct {
	// The number of times of retries.
	Retries int
	// The interval between retries.
	RetryInterval time.Duration
}

Options provides all of the ways which you can configure the UI detection API.

func DefaultOptions 

func DefaultOptions() *Options

DefaultOptions return options with default values.

type ScreenshotStrategy 

type ScreenshotStrategy int

ScreenshotStrategy holds the different screenshot strategies that can be used for image-based UI detection. 

const (
	StableScreenshot ScreenshotStrategy = iota
	ImmediateScreenshot
)

Holds all the screenshot types that can be used.

type TextParam 

type TextParam = func(*TextParams)

TextParam is a modifier to apply to TextParams.

func DisableApproxMatch 

func DisableApproxMatch(disableApproxMatch bool) TextParam

DisableApproxMatch controls the DisableApproxMatch param.

func MaxEditDistance 

func MaxEditDistance(maxEditDistance int32) TextParam

MaxEditDistance controls the MaxEditDistance param.

func RegexMode 

func RegexMode(regexMode bool) TextParam

RegexMode controls the RegexMode param.

type TextParams 

type TextParams struct {
	// RegexMode indicates whether the query string is a regex or not.
	RegexMode bool
	// DisableApproxMatch disables the approximate match.
	// approxmiate match is enabled by default to tolerate recognition errors,
	// so that some similar characters (i.e., "5" and "s") are treated the same.
	// Normally, you don't want to turn the approxmiate match off.
	DisableApproxMatch bool
	// MaxEditDistance is the Levenshtein distance threshold.
	// For example "string" and "sting" is the same match if MaxEditDistance=1.
	// NOTE: this param is applicable only if RegexMode is False.
	MaxEditDistance int32
}

TextParams provides all of the ways which you can configure a word detection or a textblock detection.

NOTE: The default values are the recommended values. The users are not expected to modify them if the test is passing with the defaults. If the defaults are not working, the users will need to find a proper value with the trial and error method.

See https://chromium.googlesource.com/chromiumos/platform/tast/+/HEAD/docs/using_uidetection.md for more details.

func DefaultTextParams 

func DefaultTextParams() *TextParams

DefaultTextParams return params with default values.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.