api3

README

Package api3

Purpose

This package extracts schemas metadata from OpenAPI files version 3. If your OpenAPI file is using older version 2 check out api2 package.

Description

Some connectors cannot serve metadata via APIs and do this via static files. Those files are a processed version of OpenAPI spec.

Loading File

var (
    // Static file containing openapi spec.
    //
    //go:embed specs.json
    apiFile []byte

	FileManager = api3.NewOpenapiFileManager(apiFile) // nolint:gochecknoglobals
)

Usage

Scripts that use this package are located under scripts/openapi/<connector_name>/main.go.

// Pseudo code, omitting err.
schemas = yourconnector.FileManager.GetExplorer().GetBasicReadObjects(
    ignoreEndpoints,
    objectEndpoints,
    displayNameOverride,
    objectArrayLocator,
)

Argument description:

• ignoreEndpoints - list of URL paths. This way you can hard code which paths to skip when processing file.

  • Full path string: /v1/order
  • Any path that has suffix batch: */batch
  • Any path that has prefix v2: /v2/*

• objectEndpoints - this is a mapping from URL path to the Object Name.

  • By default, last URI part is used as Object Name.

• objectArrayLocator - function that accepts JSON response field and Object name. Using both you can determine if this is the correct response field that will hold your schema. Some common implementations are provided.

  • api3.IdenticalObjectLocator - expects data to be stored under the same name as object name. Ex: {"contacts":[...]}
  • api3.DataObjectLocator - expects schema to be returned under data field. Ex: {"data":[...]}
  • Your implementation can have exception or do combination of the two based on different objects.

Additionally, Explorer can be configured to apply display name processing after data is extracted. For example, you can capitalize every word of display for better look. Edge cases should still be directly specified via displayNameOverride map.

Configuration

The OpenAPI schema explorer can be configured to tailor the handling of edge cases.

openapi.FileManager.GetExplorer(
    api3.WithDisplayNamePostProcessors(
        api3.CamelCaseToSpaceSeparated,
        api3.CapitalizeFirstLetterEveryWord,
    )
    api3.WithParameterFilterGetMethod(
        api3.OnlyOptionalQueryParameters        		
    )
)

Display Name. You can define a chained formatters of display name. For example: first, convert camel case into space-separated words; then, capitalize the first letters. Of course, a single method will suffice, but chained processors allow for better composition of out-of-the-box utility methods.

Parameter Filter. Some GET methods can be ignored based on the endpoint's input parameters. For example, retain endpoints that have exclusively optional query parameters.

Documentation

Index

• Variables
• func CamelCaseToSpaceSeparated(displayName string) string
• func CapitalizeFirstLetterEveryWord(displayName string) string
• func DataObjectLocator(objectName, fieldName string) bool
• func IdenticalObjectLocator(objectName, fieldName string) bool
• func OnlyOptionalQueryParameters(objectName string, operation *openapi3.Operation) bool
• func Pluralize(displayName string) string
• type DisplayNameProcessor
• type Document
  • func (s Document) GetPaths() map[string]*openapi3.PathItem
• type Explorer
  • func NewExplorer(data *openapi3.T, opts ...Option) *Explorer
  • func (e Explorer) GetPathItems(pathMatcher PathMatcher, endpointResources map[string]string) []PathItem
  • func (e Explorer) ReadObjects(operationName string, pathMatcher PathMatcher, ...) (Schemas, error)
  • func (e Explorer) ReadObjectsGet(pathMatcher PathMatcher, objectEndpoints map[string]string, ...) (Schemas, error)
  • func (e Explorer) ReadObjectsPost(pathMatcher PathMatcher, objectEndpoints map[string]string, ...) (Schemas, error)
• type ObjectArrayLocator
  • func CustomMappingObjectCheck(dict datautils.DefaultMap[string, string]) ObjectArrayLocator
• type OpenapiFileManager
  • func NewOpenapiFileManager(file []byte) *OpenapiFileManager
  • func (m OpenapiFileManager) GetExplorer(opts ...Option) (*Explorer, error)
• type Option
  • func WithDisplayNamePostProcessors(processors ...DisplayNameProcessor) Option
  • func WithMediaType(mediaType string) Option
  • func WithParameterFilterGetMethod(parameterFilter ReadOperationMethodFilter) Option
  • func WithPropertyFlattening(propertyFlattener PropertyFlattener) Option
• type PathItem
  • func (p PathItem) RetrieveSchemaOperation(operationName string, displayNameOverride map[string]string, ...) (*Schema, bool, error)
• type PathMatcher
• type PropertyFlattener
• type ReadOperationMethodFilter
• type Schema
  • func (s Schema) String() string
• type Schemas
  • func (s Schemas) Combine(others Schemas) Schemas
• type StarRulePathResolver
  • func NewAllowPathStrategy(paths []string) *StarRulePathResolver
  • func NewDenyPathStrategy(paths []string) *StarRulePathResolver
  • func (s StarRulePathResolver) IsPathMatching(path string) bool

Variables

var ErrUnprocessableObject = errors.New("don't know how to process schema")

Functions

func CamelCaseToSpaceSeparated

func CamelCaseToSpaceSeparated(displayName string) string

CamelCaseToSpaceSeparated converts camel case into lower case space separated string.

func CapitalizeFirstLetterEveryWord

func CapitalizeFirstLetterEveryWord(displayName string) string

CapitalizeFirstLetterEveryWord makes all words start with capital except some prepositions.

func DataObjectLocator

func DataObjectLocator(objectName, fieldName string) bool

DataObjectLocator item schema within response is always stored under the data field. Ex: requesting contacts or leads or users will return payload with {"data":[...]}.

func IdenticalObjectLocator

func IdenticalObjectLocator(objectName, fieldName string) bool

IdenticalObjectLocator item schema within response is stored under matching object name. Ex: requesting contacts will return payload with {"contacts":[...]}.

func OnlyOptionalQueryParameters

func OnlyOptionalQueryParameters(objectName string, operation *openapi3.Operation) bool

OnlyOptionalQueryParameters operation must include only optional query parameters.

func Pluralize

func Pluralize(displayName string) string

Pluralize will apply pluralization to the display name.

Types

type DisplayNameProcessor

type DisplayNameProcessor func(displayName string) string

DisplayNameProcessor allows to format Display Names.

type Document

type Document struct {
	// contains filtered or unexported fields
}

Document is a wrapper of openapi with null checks.

func (Document) GetPaths

func (s Document) GetPaths() map[string]*openapi3.PathItem

type Explorer

type Explorer struct {
	// contains filtered or unexported fields
}

Explorer allows to traverse schema in most common ways relevant for connectors metadata extraction.

func NewExplorer

func NewExplorer(data *openapi3.T, opts ...Option) *Explorer

NewExplorer creates explorer on openAPI v3 file. See Option to discover how explorer can be customized.

func (Explorer) GetPathItems

func (e Explorer) GetPathItems(
	pathMatcher PathMatcher, endpointResources map[string]string,
) []PathItem

GetPathItems returns path items where object name is a single word.

func (Explorer) ReadObjects

func (e Explorer) ReadObjects(
	operationName string,
	pathMatcher PathMatcher,
	objectEndpoints map[string]string,
	displayNameOverride map[string]string,
	locator ObjectArrayLocator,
) (Schemas, error)

ReadObjects will explore OpenAPI file returning list of Schemas. See every parameter for detailed customization.

operationName - under which REST operation the schema resides. Ex: GET - list reading, POST - search reading. pathMatcher - guides which URL paths to include in search or to ignore. objectEndpoints - URL path mapped to ObjectName. Ex: /customer/orders -> orders.

Note: deep connector would need to do the reverse mapping to reconstruct URL given orders objectName.

displayNameOverride - objectName mapped to custom Display name. locator - callback that returns true if fieldName matched the target location of Object in response. Ex: if (objectName == orders && fieldName == data) => true

Given response with fields {meta{}, data{}, pagination{}} for orders object,
the implementation indicates that schema will be located under `data`.

func (Explorer) ReadObjectsGet

func (e Explorer) ReadObjectsGet(
	pathMatcher PathMatcher,
	objectEndpoints map[string]string,
	displayNameOverride map[string]string,
	locator ObjectArrayLocator,
) (Schemas, error)

ReadObjectsGet is the same as ReadObjectsGet but retrieves schemas for endpoints that perform reading via GET. If you need schemas located under GET and POST operations, make 2 calls as they will have different arguments in particular PathMatchingStrategy, and then Combine two lists of schemas.

func (Explorer) ReadObjectsPost

func (e Explorer) ReadObjectsPost(
	pathMatcher PathMatcher,
	objectEndpoints map[string]string,
	displayNameOverride map[string]string,
	locator ObjectArrayLocator,
) (Schemas, error)

ReadObjectsPost is the same as ReadObjectsGet but retrieves schemas for endpoints that perform reading via POST.

type ObjectArrayLocator

type ObjectArrayLocator func(objectName, fieldName string) bool

ObjectArrayLocator is a procedure that decides if field name is related to the object name. Below you can find the common cases.

func CustomMappingObjectCheck

func CustomMappingObjectCheck(dict datautils.DefaultMap[string, string]) ObjectArrayLocator

CustomMappingObjectCheck builds ObjectArrayLocator using mapping, which knows exceptions and patterns to determine response field name.

Ex:

CustomMappingObjectCheck(datautils.NewDefaultMap(map[string]string{
		"orders":	"orders",
		"carts":	"carts",
		"coupons":	"coupons",
	}, func(key string) string { return "data" }))

This can be understood as follows: orders, carts, coupons REST resources will be found under JSON response field matching "it's name", while the rest will be located under "data" field.

type OpenapiFileManager

type OpenapiFileManager struct {
	// contains filtered or unexported fields
}

OpenapiFileManager locates openapi file. Allows to read data of interest. Use it when dealing with OpenAPI v3.

func NewOpenapiFileManager

func NewOpenapiFileManager(file []byte) *OpenapiFileManager

func (OpenapiFileManager) GetExplorer

func (m OpenapiFileManager) GetExplorer(opts ...Option) (*Explorer, error)

type Option

type Option = func(params *parameters)

func WithDisplayNamePostProcessors

func WithDisplayNamePostProcessors(processors ...DisplayNameProcessor) Option

WithDisplayNamePostProcessors will apply processors in the given order.

func WithMediaType

func WithMediaType(mediaType string) Option

WithMediaType picks which media type which should be used when searching schemas in API response. By default, schema is expected to be under "application/json" media response.

func WithParameterFilterGetMethod

func WithParameterFilterGetMethod(parameterFilter ReadOperationMethodFilter) Option

WithParameterFilterGetMethod adds custom callback to decide if GET operation should be included based on parameters definitions.

func WithPropertyFlattening

func WithPropertyFlattening(propertyFlattener PropertyFlattener) Option

WithPropertyFlattening allows nested fields to be moved to the top level. There are some APIs that hold fields of interest under grouping object, the nested object. This configuration flattens response schema fields. Please, have a look at PropertyFlattener documentation.

type PathItem

type PathItem struct {
	// contains filtered or unexported fields
}

func (PathItem) RetrieveSchemaOperation

func (p PathItem) RetrieveSchemaOperation(
	operationName string,
	displayNameOverride map[string]string,
	locator ObjectArrayLocator,
	displayProcessor DisplayNameProcessor,
	operationMethodFilter ReadOperationMethodFilter,
	propertyFlattener PropertyFlattener,
	mime string,
) (*Schema, bool, error)

type PathMatcher

type PathMatcher interface {
	IsPathMatching(path string) bool
}

type PropertyFlattener

type PropertyFlattener func(objectName, fieldName string) bool

PropertyFlattener is used to inherit fields from nested object moving them to the top level. Ex:

{
	"a":1,
	"b":2,
	"grouping": {
		"c":3,
		"d":4,
	},
	"e":5
}

If we return true on "grouping" fieldName then it will be flattened with the resulting list of fields becoming "a", "b", "c", "d", "e".

type ReadOperationMethodFilter

type ReadOperationMethodFilter func(objectName string, operation *openapi3.Operation) bool

ReadOperationMethodFilter callback that filters REST operations based on endpoint parameters.

type Schema

type Schema struct {
	ObjectName  string
	DisplayName string
	Fields      []string
	QueryParams []string
	URLPath     string
	ResponseKey string
	Problem     error
}

func (Schema) String

func (s Schema) String() string

type Schemas

type Schemas []Schema

func (Schemas) Combine

func (s Schemas) Combine(others Schemas) Schemas

type StarRulePathResolver

type StarRulePathResolver struct {
	// contains filtered or unexported fields
}

StarRulePathResolver will report if path matches endpoint rule. Match can occur in 3 different ways, * exact value is inside the registry * or using star rule for

• prefix matching,
• suffix matching.

func NewAllowPathStrategy

func NewAllowPathStrategy(paths []string) *StarRulePathResolver

NewAllowPathStrategy produces a path matching strategy that will accept only those paths that matched the list. Others will be denied. You can use star symbol to create a wild matcher. Ex: Basic: /v1/orders - matches exact path Suffix: */batch - matches paths ending with batch Prefix: /v2/* - matches paths starting with v2.

func NewDenyPathStrategy

func NewDenyPathStrategy(paths []string) *StarRulePathResolver

NewDenyPathStrategy produces a path matching strategy that will deny only those paths that matched the list. Others will be allowed. You can use star symbol to create a wild matcher. Ex: Basic: /v1/orders - deny exact path Suffix: */batch - deny paths ending with batch Prefix: /v2/* - deny paths starting with v2.

func (StarRulePathResolver) IsPathMatching

func (s StarRulePathResolver) IsPathMatching(path string) bool