vervet

package module
v8.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 18, 2024 License: Apache-2.0 Imports: 25 Imported by: 0

README

Vervet

Vervet is an HTTP API version lifecycle management tool, allowing APIs to be designed, developed, versioned and released from resources independently and concurrently.

In a large organization, there might be many teams involved in delivering a large API -- such as at Snyk where Vervet was developed.

Within a single small team, there is still often a need to simultaneously try new things in parts of an API while maintaining stability.

While Vervet was developed in the context of a RESTful API, Vervet can be used with any HTTP API expressed in OpenAPI 3 -- even if it does not adhere to strict REST principles.

API Versioning

To summarize the API versioning supported by Vervet:

What is versioned?

Resource versions are defined in OpenAPI 3, as if each resource were a standalone service.

How are resource version specs organized?

Resources are organized in a standard directory structure by release date, using OpenAPI extensions to define lifecycle concepts like stability.

How does versioning work?
  • Resources are versioned independently by date and stability, with a well-defined deprecation and sunsetting policy.
  • Additive, non-breaking changes can be made to released versions. Breaking changes trigger a new version.
  • New versions deprecate and sunset prior versions, on a timeline determined by the stability level.

Read more about API versioning.

Features

A brief tour of Vervet's features.

Building a service OpenAPI from resources

Vervet collects the OpenAPI specification of each resource version and merges them into a series of OpenAPI specifications that describe the entire application, at each distinct release version in its underlying parts.

Given a directory structure of resource versions, each defined by an OpenAPI specification as if it were an independent service:

tree resources

resources
├── _examples
│   └── hello-world
│       ├── 2021-06-01
│       │   └── spec.yaml
│       ├── 2021-06-07
│       │   └── spec.yaml
│       └── 2021-06-13
│           └── spec.yaml
└── projects
    └── 2021-06-04
        └── spec.yaml

and a Vervet project configuration that instructs how to put them together:

cat .vervet.yaml

apis:
  my-api:
    resources:
      - path: "resources"
    output:
      path: "versions"

vervet build aggregates these resources' individual OpenAPI specifications to describe the entire service API at each distinct version date and stability level from its component parts.

tree versions

versions/
├── 2021-06-01
│   ├── spec.json
│   └── spec.yaml
├── 2021-06-01~beta
│   ├── spec.json
│   └── spec.yaml
├── 2021-06-01~experimental
│   ├── spec.json
│   └── spec.yaml
├── 2021-06-04
│   ├── spec.json
│   └── spec.yaml
├── 2021-06-04~beta
│   ├── spec.json
│   └── spec.yaml
├── 2021-06-04~experimental
│   ├── spec.json
│   └── spec.yaml
├── 2021-06-07
│   ├── spec.json
│   └── spec.yaml
├── 2021-06-07~beta
│   ├── spec.json
│   └── spec.yaml
├── 2021-06-07~experimental
│   ├── spec.json
│   └── spec.yaml
├── 2021-06-13
│   ├── spec.json
│   └── spec.yaml
├── 2021-06-13~beta
│   ├── spec.json
│   └── spec.yaml
└── 2021-06-13~experimental
    ├── spec.json
    └── spec.yaml
Simplified Versioning (from 2024-10-15)

From 2024-10-15, Vervet introduced a new "simplified versioning" scheme.

The main differences introduced by simplified versioning are:

  • Stability dropped from the spec level to the individual path level: Each API path can now be labeled as either beta or GA (general availability). The experimental stability level has been removed. This means that paths within the same version of the spec can have different stability statuses, allowing for greater flexibility and precision.

  • No changes required from developers defining APIs: Developers still define their APIs in the same way as before, and Vervet handles the collation and versioning of specs in the new format automatically. The way specs are defined, updated, and maintained remains familiar, ensuring a smooth transition.

  • End-user API requests are simplified: API consumers no longer need to specify the stability when calling a particular version. They can simply provide the date of the version they want, and Vervet will resolve whether each path is GA or beta. If a path has been promoted to GA, subsequent versions cannot publish it as beta again.

This change is aimed at simplifying how users interact with versioned APIs while maintaining clear definitions and stabilities at the path level.

Examples for Simplified Versioning

To illustrate how simplified versioning works in practice, let’s consider some examples:

Example 1: Requesting a Version without Specifying Stability

Consider an API with paths /pets and /owners. Suppose the /pets path is GA and the /owners path is beta as of version date 2024-10-20.

  • Request:

    GET /api/2024-10-20/pets
    • Response: This request will be handled by the GA version of /pets.

  • Request:

    GET /api/2024-10-20/owners
    • Response: This request will be handled by the beta version of /owners.

The user does not need to specify the stability explicitly. Vervet determines the appropriate path stability (GA or beta) automatically.

Example 2: Path Promotion to GA

Let’s say that as of 2024-11-10, the /owners path is promoted to GA.

  • Request: 
    GET /api/2024-11-15/owners
    • Response: The request will be handled by the GA version of /owners.

Once a path is promoted to GA, any subsequent version cannot publish it as beta again. Therefore, users can be confident that they are always accessing a stable version of a path if it is marked as GA.

Example 3: Multiple Paths with Mixed Stability

Consider an API version dated 2024-12-01 with the following paths:

  • /pets: GA

  • /owners: GA

  • /appointments: beta

  • Request:

    GET /api/2024-12-01/appointments
    • Response: The request will be handled by the beta version of /appointments.

  • Request:

    GET /api/2024-12-01/pets
    • Response: The request will be handled by the GA version of /pets.

This approach allows different parts of an API to evolve at different paces, providing flexibility for developers and a clearer experience for end-users.

Code generation

Since Vervet models the composition, construction and versioning of an API, it is well positioned to coordinate code and artifact generation through the use of templates.

Generators may be defined in a YAML file, such as generators.yaml:

generators:
  version-readme:
    scope: version
    filename: "{{ .Path }}/README"
    template: "{{ .Here }}/templates/README.tmpl" # Located relative to the location of generators.yaml

The context of README.tmpl has full access to the resource version metadata and OpenAPI document object model.

Generated by vervet. DO NOT EDIT!

# My API

Files in this directory were generated by `@snyk/vervet`
for resource `{{ .ResourceVersion.Name }}` at version `{{ .ResourceVersion.Version.String }}`.

In a project with a .vervet.yaml configuration, execute the generators with

vervet generate -g generators.yaml

The simple generator above produces a README in each resource version directory.

tree resources

resources
└── thing
    └── 2021-10-21
        ├── README
        └── spec.yaml

Generators are defined using Go templates.

Template syntax may also be used to express a directory structure of many files. A more advanced example, an Express controller generated from each operation in a resource version OpenAPI spec:

generators:
  version-controller:
    scope: version
    # `files:` generates a collection of files -- which itself is expressed as a
    # YAML template.  Keys in this YAML are the paths of the files to generate,
    # whose values are the file contents.
    files: |-
      {{- $path := .Path -}}
      {{- $resource := .ResourceVersion -}}
      {{- $version := .ResourceVersion.Version -}}
      {{- range $path, $pathItem := .ResourceVersion.Document.Paths -}}
      {{- range $method, $operation := $pathItem -}}
      {{- $operationId := $operation.operationId -}}
      {{/* Construct a context object using the 'map' function */}}
      {{- $ctx := map "Context" . "OperationId" $operationId }}
      {{ $path }}/{{ $operationId }}.ts: |-
        {{/*
             Evaluate the template by including it with the necessary context.
             The generator's template (controller.ts.tmpl) is included as
             "contents" from within the `files:` template.
           */}}
        {{ include "contents" $ctx | indent 2 }}
      {{ end }}
      {{- end -}}
    template: "{{ .Here }}/templates/controller.ts.tmpl"

In this case, a template is being applied per operationId in the spec.yaml generated in the prior step. version-controller produces a collection of files, a controller module per resource, per version, per operation.

Finally, a note on scoping. Generators can be scoped to either a version or a resource.

scope: version generator templates execute with VersionScope. This maps 1:1 with a single resource version OpenAPI specification.

scope: resource generator templates execute with ResourceScope. This is a collection of resource versions, useful for building resource routers.

Installation

NPM

Within a project:

npm install @snyk/vervet

Or installed globally:

npm install -g @snyk/vervet

NPM packaging adapted from https://github.com/manifoldco/torus-cli.

Go

Go >= 1.16 required.

go install github.com/snyk/vervet/v6/cmd/vervet@latest

Building from source locally:

go build ./cmd/vervet

or

make build

Development

Vervet uses a reference set of OpenAPI documents in testdata/resources in tests. CLI tests compare runtime compiled output with pre-compiled, expected output in testdata/output to detect regressions.

When introducing changes that intentionally change the content of compiled output:

  • Run go generate ./testdata to update the contents of testdata/output
  • Verify that the compiled output is correct
  • Commit the changes to testdata/output in your proposed branch

Releasing a new version

A new version of vervet will automatically be generated for Github and npm when new features are introduced, i.e. when commits are merged that are marked with feat:.

Deprecating a version

After removing the endpoint version code and specs, you may see this issue:

ENOENT: no such file or directory, open '.../spec.yaml'

To solve this:

  1. Temporarily ignore the endpoint version code in .vervet.yaml
  2. Remove the endpoint versions from catalog-info.yaml
  3. Remove the old OpenAPI specs.

Example PR

Vervet Underground

What Vervet Underground does and why

In order to understand why Vervet Underground exists and the problem it solves, you should first become familiar with the API versioning scheme that Vervet supports. The main idea is, an API may be authored in parts, each of those parts may be versioned, and all the distinct versions are assembled to produce a cohesive timeline of versions for the entire service.

Just as Vervet compiles a timeline of OpenAPI versions for a single service from independently versioned parts, Vervet Underground (VU) compiles a timeline of OpenAPI spec versions for a SaaS from independently versioned microservices, each of which contributes parts of the SaaS API.

Service API aggregation by example

The pet store gets microservices

To illustrate how this works in practice, let's deconstruct a pet store into two services:

  • petfood.default.svc.cluster.local, which knows about pet food.
  • animals.default.svc.cluster.local, which knows about animals.

For sake of example, let's assume the following versions are published by each service:

petfood has:

  • 2021-07-04~experimental
  • 2021-08-09~beta
  • 2021-08-09~experimental (beta released, with some parts still experimental, so both are published)
  • 2021-09-14 . (first GA)
  • 2021-09-14~beta
  • 2021-09-14~experimental

animals has:

  • 2021-09-10~experimental
  • 2021-10-04~experimental
  • 2021-10-12~beta
  • 2021-10-12~experimental
  • 2021-11-05
  • 2021-11-05~beta
  • 2021-11-05~experimental

And, the OpenAPI spec for each version is available at /openapi. /openapi provides a JSON array of OpenAPI versions supported by the service, and /openapi/{version} fetches the OpenAPI spec for that particular {version}. For example, GET http://petfood.default.svc.cluster.local/openapi/2021-09-14~experimental.

There is some nuance to this to be aware of. You'll notice that some dates have multiple versions with different stabilities. This can happen because on that date, there is more than one API version available at different stability levels.

There are also some assumptions. These services are cooperatively contributing to the public pet store SaaS API. They cannot conflict with each other -- no overwriting each other's OpenAPI components, or publishing conflicting paths.

Pet store's public API

From these service versions, what versions of the pet store API are published to the public SaaS consumer? Well, the union of all of them! So we should be able to enumerate these versions from the public SaaS API:

GET https://api.petstore.example.com/openapi

200 OK
Content-Type: application/json
[
  `2021-07-04~experimental`,   // Contains only petfood so far...
  `2021-08-09~experimental`,
  `2021-08-09~beta`,
  `2021-09-10~beta`            // Petfood only (no beta version of animals yet...)
  `2021-09-10~experimental`,   // First animals (experimental version) + petfood
  `2021-09-14`,                // Petfood GA only, animals isn't GA yet
  `2021-09-14~beta`,
  `2021-09-14~experimental`,
  `2021-10-04`,
  `2021-10-04~beta`,
  `2021-10-04~experimental`,
  `2021-10-12`,
  `2021-10-12~beta`,
  `2021-10-12~experimental`,
  `2021-11-05`,                // First GA release of animals, also has petfood GA (from most recent 2021-09-14)
  `2021-11-05~beta`,
  `2021-11-05~experimental`,
]

Past API releases can change

The examples so far have been kept simple by assuming released API versions do not change. In practice, non-breaking changes are allowed to be made to existing versions of the API at any time. Non-breaking changes must be additive and optional. It is fine to add new HTTP methods, endpoints, request parameters or response fields, so long that the added parameters or fields are not required -- which existing generated client code would have no way of knowing about.

With VU, we should be able to request a version of the public SaaS API as it was on a given date, regardless of the version release date.

Tracking a non-breaking change by example

Let's assume that on 2021-11-08, the petfood service team adds a PATCH method to all their resources, to allow existing orders to be modified before they ship. It's a reasonable thing to do -- a new HTTP method doesn't break existing behavior! The team adds this method to every active (non-sunset) version retroactively -- why not? it was essentially the same backend code to implement it!

Vervet Underground not only compiles the initially published APIs from component services, it tracks changes in these APIs and updates its SaaS-level view of the API accordingly. So, VU scrapes the /openapi endpoints of its services periodically and detects the API changes on 2021-11-08, even though no new API was explicitly released that day, and now represents it as a new "discovered" version:

GET https://api.petstore.example.com/openapi

200 OK
Content-Type: application/json
[
  `2021-07-04~experimental`,
  ...
  `2021-11-05`,
  `2021-11-05~beta`,
  `2021-11-05~experimental`,
  `2021-11-08`,                // A wild new version appears!
  `2021-11-08~beta`,
  `2021-11-08~experimental`,
]

How VU tracks non-breaking changes (even our versions have versions!)

VU scrapes the /openapi endpoints of each service and tracks the changes in each version found. This may be stored in a directory structure, such as:

services/petfood
├── 2021-09-14~experimental
│   ├── 2021-09-14_11_23_24.spec.yaml
│   └── 2021-11-08_13_14_15.spec.yaml
...

where the scraped OpenAPI is compared against the most recent last snapshot: if they differ, a new snapshot is taken.

When the new 2021-11-08 snapshot is detected, this triggers a rebuild of the top-level SaaS OpenAPI specs with that new version added. The arrow of time eventually flows only one way and storage is cheap, so it's assumed that the compiled OpenAPI specs will be statically compiled up-front as service API changes are detected.

This snapshot version should not be taken to represent a breaking-change release, which has different deprecation and sunsetting implications. It is only used to represent what the API looked like at a given point in time.

What this means from a public API perspective

If a version date prior to 2021-11-08 resolves to 2021-09-14~experimental, you should see the 2021-09-14~experimental contributions to the API as it would have appeared at that time, in other words, you should see a view based on the 2021-07-04_11_23_24.spec.yaml snapshot of 2021-09-14~experimental.

If a version date after 2021-11-08 matches 2021-09-14~experimental, let's say a request for 2021-12-10~experimental, then you should see it as it would appear after the non-breaking change, 2021-11-08_13_14_15.spec.yaml.

Roadmap

Minimum Viable

VU aggregates OpenAPI specs from multiple services and serves them up in a single place for:

  • Docs
    • Docs will likely either render public /openapi directly client-side or periodically scrape & update
  • Routing public v3 /openapi to VU's aggregated OpenAPI versions
  • Add Akamai configuration to serve the blockstored specs initially for /openapi
    • Formal frontend presentation will be later

This unblocks docs for multi-service decomp.

Details
  • Could use block storage with a history of changes per service per version
  • Static config that tells VU where to scrape upstream OpenAPI from services (registry and friends)
  • Cron job to periodically scrape and update
    • Or we can try to make this push and set up a webhook...
Simplified Versioning Integration

From 2024-10-15, Vervet Underground also integrates the simplified versioning model. With simplified versioning:

  • API versions compiled by VU no longer require consumers to specify the stability level (beta, GA) in the request. Instead, consumers simply request a version by date, and the paths within that version automatically resolve to their correct stability levels (beta or GA).
  • Experimental stabilities are no longer supported; all paths are now either beta or GA.
  • Changes to a path's stability are reflected by date, ensuring consistent access to GA or beta versions without ambiguity.

This results in a more straightforward API usage for clients, as they no longer need to be concerned with explicitly requesting stabilities, which reduces friction and simplifies interaction with the aggregated service APIs.

Example: Pre and Post Pivot Date Behavior

To better understand how simplified versioning works in comparison to the previous versioning model, let’s consider an example both before and after the pivot date of 2024-10-15.

Pre Pivot Date Example (before 2024-10-15)

Consider a pet store API with services for animals and petfood. Assume the following versions:

  • animals has:

    • 2024-10-01~beta
    • 2024-10-01~experimental

  • petfood has:

    • 2024-09-20~GA
    • 2024-10-01~beta

If a client requested:

  • Request:

    GET /api/2024-10-01~beta/animals
    • Response: This request would be served by the beta version of /animals.

  • Request:

    GET /api/2024-10-01~experimental/animals
    • Response: This request would be served by the experimental version of /animals.

  • Request:

    GET /api/2024-09-20~GA/petfood
    • Response: This request would be served by the GA version of /petfood.

Here, users must explicitly specify the stability (~beta, ~experimental, or ~GA), which adds complexity to the request.

Post Pivot Date Example (after 2024-10-15)

After the pivot date of 2024-10-15, the simplified versioning model takes effect, where all paths are collated into a single spec with individual stabilities (beta or GA). Consider the following scenario:

  • animals has:

    • Path /animals marked as beta on 2024-10-20.

  • petfood has:

    • Path /petfood marked as GA on 2024-10-20.

If a client requested:

  • Request:

    GET /api/2024-10-20/animals
    • Response: This request will be handled by the beta version of /animals without specifying the stability explicitly.

  • Request:

    GET /api/2024-10-20/petfood
    • Response: This request will be handled by the GA version of /petfood.

With the simplified model, clients simply provide the date, and VU handles the rest, resolving the correct stability for each path automatically.

From 2024-10-15, Vervet Underground also integrates the simplified versioning model. With simplified versioning:

  • API versions compiled by VU no longer require consumers to specify the stability level (beta, GA) in the request. Instead, consumers simply request a version by date, and the paths within that version automatically resolve to their correct stability levels (beta or GA).
  • Experimental stabilities are no longer supported; all paths are now either beta or GA.
  • Changes to a path's stability are reflected by date, ensuring consistent access to GA or beta versions without ambiguity.

This results in a more straightforward API usage for clients, as they no longer need to be concerned with explicitly requesting stabilities, which reduces friction and simplifies interaction with the aggregated service APIs.

Documentation

Overview

Package vervet supports opinionated API versioning tools.

Index

Constants

View Source 
const (
	// ExtSnykApiStability is used to annotate a top-level resource version
	// spec with its API release stability level.
	ExtSnykApiStability = "x-snyk-api-stability"

	// ExtApiStability is used to annotate a path in a compiled OpenAPI spec
	// with its API release stability level.
	ExtApiStabilityLevel = "x-stability-level"

	// ExtSnykApiLifecycle is used to annotate compiled OpenAPI with lifecycle
	// stage: releases, deprecated or sunset. It is applied at the top-level as
	// well as per-operation.
	ExtSnykApiLifecycle = "x-snyk-api-lifecycle"

	// ExtSnykApiResource is used to annotate a path in a compiled OpenAPI spec
	// with its source resource name.
	ExtSnykApiResource = "x-snyk-api-resource"

	// ExtSnykApiVersion is used to annotate a path in a compiled OpenAPI spec
	// with its resolved release version. It is also used to identify the
	// overall version of the compiled spec at the document level.
	ExtSnykApiVersion = "x-snyk-api-version"

	// ExtSnykApiOwner is used to annotate an operation in a compiled OpenAPI spec
	// with the owners of the operation. This is useful to get to the owning github team.
	ExtSnykApiOwner = "x-snyk-api-owners"

	// ExtSnykApiReleases is used to annotate a path in a compiled OpenAPI spec
	// with all the release versions containing a change in the path info. This
	// is useful for navigating changes in a particular path across versions.
	ExtSnykApiReleases = "x-snyk-api-releases"

	// ExtSnykDeprecatedBy is used to annotate a path in a resource version
	// spec with the subsequent version that deprecates it. This may be used
	// by linters, service middleware and API documentation to indicate which
	// version deprecates a given version.
	ExtSnykDeprecatedBy = "x-snyk-deprecated-by"

	// ExtSnykSunsetEligible is used to annotate a path in a resource version
	// spec which is deprecated, with the sunset eligible date: the date after
	// which the resource version may be removed and no longer available.
	ExtSnykSunsetEligible = "x-snyk-sunset-eligible"
)
View Source 
const (
	// SunsetWIP is the duration past deprecation after which a work-in-progress version may be sunset.
	SunsetWIP = 0

	// SunsetExperimental is the duration past deprecation after which an experimental version may be sunset.
	SunsetExperimental = 24 * time.Hour

	// SunsetBeta is the duration past deprecation after which a beta version may be sunset.
	SunsetBeta = 91 * 24 * time.Hour

	// SunsetGA is the duration past deprecation after which a GA version may be sunset.
	SunsetGA = 181 * 24 * time.Hour
)
View Source 
const (
	// ExtSnykIncludeHeaders is used to annotate a response with a list of
	// headers. While OpenAPI supports header references, it does not yet
	// support including a collection of common headers. This extension is used
	// by vervet to include headers from a referenced document when compiling
	// OpenAPI specs.
	ExtSnykIncludeHeaders = "x-snyk-include-headers"
)
View Source 
const SpecGlobPattern = "**/[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]/spec.yaml"

SpecGlobPattern defines the expected directory structure for the versioned OpenAPI specs of a single resource: subdirectories by date, of the form YYYY-mm-dd, each containing a spec.yaml file.

Variables

View Source 
var DefaultPivotDate = MustParseVersion("2024-10-15")

DefaultPivotDate is the default pivot date after which the versioning strategy changes.

View Source 
var ErrNoMatchingVersion = fmt.Errorf("no matching version")

ErrNoMatchingVersion indicates the requested version cannot be satisfied by the declared versions that are available.

Functions

func ComponentsEqual added in v8.0.1 

func ComponentsEqual(x, y interface{}) bool

func ExtensionString 

func ExtensionString(extensions map[string]interface{}, key string) (string, error)

ExtensionString returns the string value of an OpenAPI extension.

func IncludeHeaders 

func IncludeHeaders(doc *Document) error

IncludeHeaders adds response headers included with the ExtSnykIncludeHeaders extension property.

func IsExtensionNotFound 

func IsExtensionNotFound(err error) bool

IsExtensionNotFound returns bool whether error from ExtensionString is not found versus unexpected.

func LoadVersions 

func LoadVersions(root fs.FS) ([]*openapi3.T, error)

LoadVersions loads all Vervet-compiled and versioned API specs from a filesystem root and returns them.

func Localize 

func Localize(ctx context.Context, doc *Document) error

Localize rewrites all references in an OpenAPI document to local references.

func Merge 

func Merge(dst, src *openapi3.T, replace bool) error

Merge adds the paths and components from a source OpenAPI document root, to a destination document root.

TODO: This is a naive implementation that should be improved to detect and resolve conflicts better. For example, distinct resources might have localized references with the same URIs but different content. Content-addressible resource versions may further facilitate governance; this also would facilitate detecting and relocating such conflicts.

TODO(next-release):

  • This function is suitable for overlay merging scenarios only.
  • Component merging should be removed. Use Collator for safe component merging.

func RemoveElements 

func RemoveElements(doc *openapi3.T, excludes ExcludePatterns) error

RemoveElements removes those elements from an OpenAPI document matching the given exclude patterns.

func RemoveRefs 

func RemoveRefs(target interface{}) error

RemoveRefs removes all $ref locations from an OpenAPI document object fragment. If the reference has already been resolved, this has the effect of "inlining" the formerly referenced object when serializing the OpenAPI document.

func ResolveRefsWithoutSourceName 

func ResolveRefsWithoutSourceName(t *openapi3.T, componentRef openapi3.ComponentRef) string

ResolveRefsWithoutSourceName resolves references without the source url/file name in ref background: this was the way kin-openapi used to resolve references, but it was changed in the recent versions(v0.127.0) to include the filename in the ref name. Although this method prevents conflicts, it causes existing specs to break.

func ToSpecJSON 

func ToSpecJSON(v interface{}) ([]byte, error)

ToSpecJSON renders an OpenAPI document object as JSON.

func ToSpecYAML 

func ToSpecYAML(v interface{}) ([]byte, error)

ToSpecYAML renders an OpenAPI document object as YAML.

func VersionDateStrings 

func VersionDateStrings(vs []Version) []string

VersionDateStrings returns a slice of distinct version date strings for a slice of Versions. Consecutive duplicate dates are removed.

func WithGeneratedComment 

func WithGeneratedComment(yamlBuf []byte) ([]byte, error)

WithGeneratedComment prepends a comment to YAML output indicating the file was generated.

Types

type Collator 

type Collator struct {
	// contains filtered or unexported fields
}

Collator merges resource versions into a single OpenAPI document.

func NewCollator 

func NewCollator(options ...CollatorOption) *Collator

NewCollator returns a new Collator instance.

func (*Collator) Collate 

func (c *Collator) Collate(rv *ResourceVersion) error

Collate merges a resource version into the current result.

func (*Collator) Result 

func (c *Collator) Result() *openapi3.T

Result returns the merged result. If no versions have been merged, returns nil.

type CollatorOption 

type CollatorOption func(*Collator)

CollatorOption defines an option when creating a Collator.

func StrictTags 

func StrictTags(strict bool) CollatorOption

StrictTags defines whether a collator should enforce a strict conflict check when merging tags.

func UseFirstRoute 

func UseFirstRoute(useFirstRoute bool) CollatorOption

UseFirstRoute determines whether a collator should use the first matching path in the result when merging paths. When true, the first matching path goes into the collated result, similar to how a routing table matches a path. When false, a conflicting path route will result in an error.

Path variable names do not differentiate path routes; /foo/{bar} and /foo/{baz} are regarded as the same route.

type Document 

type Document struct {
	*openapi3.T
	// contains filtered or unexported fields
}

Document is an OpenAPI 3 document object model.

func NewDocumentFile 

func NewDocumentFile(specFile string) (_ *Document, returnErr error)

NewDocumentFile loads an OpenAPI spec file from the given file path, returning a document object.

func NewResolvedDocument 

func NewResolvedDocument(t *openapi3.T, url *url.URL) *Document

NewResolvedDocument returns a Document that has already been loaded and references resolved from the given URL. The URL is provided to indicate the document's origin in logging and error messages.

func (*Document) Lifecycle 

func (d *Document) Lifecycle() (Lifecycle, error)

Lifecycle returns the lifecycle of the document.

func (*Document) LoadReference 

func (d *Document) LoadReference(relPath, refPath string, target interface{}) (_ string, returnErr error)

LoadReference loads a reference from refPath, relative to relPath, into target. The relative path of the reference is returned, so that references may be chain-loaded with successive calls.

func (*Document) Location 

func (d *Document) Location() *url.URL

Location returns the URL from where the document was loaded.

func (*Document) MarshalJSON 

func (d *Document) MarshalJSON() ([]byte, error)

MarshalJSON implements json.Marshaler.

func (*Document) RelativePath 

func (d *Document) RelativePath() string

RelativePath returns the relative path for resolving references from the file path location of the top-level document: the directory which contains the file from which the top-level document was loaded.

func (*Document) ResolveRefs 

func (d *Document) ResolveRefs() error

ResolveRefs resolves all Ref types in the document, causing the Value field of each Ref to be loaded and populated from its referenced location.

func (*Document) Version 

func (d *Document) Version() (Version, error)

Version returns the version of the document.

type ExcludePatterns 

type ExcludePatterns struct {
	ExtensionPatterns []string
	HeaderPatterns    []string
	Paths             []string
}

ExcludePatterns defines patterns matching elements to be removed from an OpenAPI document.

type Inliner 

type Inliner struct {
	// contains filtered or unexported fields
}

Inliner inlines the component.

func NewInliner 

func NewInliner() *Inliner

NewInliner returns a new Inliner instance.

func (*Inliner) AddRef 

func (in *Inliner) AddRef(ref string)

AddRef adds a JSON Reference URI to the set of references to be inlined.

func (*Inliner) Inline 

func (in *Inliner) Inline(doc *openapi3.T) error

Inline inlines all the JSON References previously indicated with AddRef in the given OpenAPI document.

func (*Inliner) ProcessCallbackRef 

func (in *Inliner) ProcessCallbackRef(ref *openapi3.CallbackRef) error

func (*Inliner) ProcessExampleRef 

func (in *Inliner) ProcessExampleRef(ref *openapi3.ExampleRef) error

func (*Inliner) ProcessHeaderRef 

func (in *Inliner) ProcessHeaderRef(ref *openapi3.HeaderRef) error

func (*Inliner) ProcessLinkRef 

func (in *Inliner) ProcessLinkRef(ref *openapi3.LinkRef) error

func (*Inliner) ProcessParameterRef 

func (in *Inliner) ProcessParameterRef(ref *openapi3.ParameterRef) error

func (*Inliner) ProcessRequestBodyRef 

func (in *Inliner) ProcessRequestBodyRef(ref *openapi3.RequestBodyRef) error

func (*Inliner) ProcessResponseRef 

func (in *Inliner) ProcessResponseRef(ref *openapi3.ResponseRef) error

func (*Inliner) ProcessSchemaRef 

func (in *Inliner) ProcessSchemaRef(ref *openapi3.SchemaRef) error

func (*Inliner) ProcessSecuritySchemeRef 

func (in *Inliner) ProcessSecuritySchemeRef(ref *openapi3.SecuritySchemeRef) error

type Lifecycle 

type Lifecycle int

Lifecycle defines the release lifecycle. 

const (

	// LifecycleUnreleased means the version has not been released yet.
	LifecycleUnreleased Lifecycle = iota

	// LifecycleReleased means the version is released.
	LifecycleReleased Lifecycle = iota

	// LifecycleDeprecated means the version is deprecated.
	LifecycleDeprecated Lifecycle = iota

	// LifecycleSunset means the version is eligible to be sunset.
	LifecycleSunset Lifecycle = iota

	// ExperimentalTTL is the duration after which experimental releases expire
	// and should be considered sunset.
	ExperimentalTTL = 90 * 24 * time.Hour
)

func ParseLifecycle 

func ParseLifecycle(s string) (Lifecycle, error)

ParseLifecycle parses a lifecycle string into a Lifecycle type, returning an error if the string is invalid.

func (Lifecycle) String 

func (l Lifecycle) String() string

String returns a string representation of the lifecycle stage. This method will panic if the value is empty.

func (Lifecycle) Valid 

func (l Lifecycle) Valid() bool

type RefIndex 

type RefIndex struct {
	// contains filtered or unexported fields
}

RefIndex indexes the distinct references used in an OpenAPI document.

func NewRefIndex 

func NewRefIndex(doc *openapi3.T) (*RefIndex, error)

NewRefIndex returns a new reference index on an OpenAPI document.

func (*RefIndex) HasRef 

func (ix *RefIndex) HasRef(ref string) bool

HasRef returns whether the indexed document contains the given ref.

type ResourceVersion 

type ResourceVersion struct {
	*Document
	Name    string
	Version Version
	// contains filtered or unexported fields
}

ResourceVersion defines a specific version of a resource, corresponding to a standalone OpenAPI specification document that defines its operations, schema, etc. While a resource spec may declare multiple paths, they should all describe operations on a single conceptual resource.

func (*ResourceVersion) Validate 

func (rv *ResourceVersion) Validate(ctx context.Context) error

Validate returns whether the ResourceVersion is valid. The OpenAPI specification must be valid, and must declare at least one path.

type ResourceVersions 

type ResourceVersions struct {
	// contains filtered or unexported fields
}

ResourceVersions defines a collection of multiple versions of a resource.

func LoadResourceVersions 

func LoadResourceVersions(epPath string) (*ResourceVersions, error)

LoadResourceVersions returns a ResourceVersions slice parsed from a directory structure of resource specs. This directory will be of the form: 

resource/
+- 2021-01-01
   +- spec.yaml
+- 2021-06-21
   +- spec.yaml
+- 2021-07-14
   +- spec.yaml

The resource version stability level is defined by the ExtSnykApiStability extension value at the top-level of the OpenAPI document.

func LoadResourceVersionsFileset 

func LoadResourceVersionsFileset(specYamls []string) (*ResourceVersions, error)

LoadResourceVersionFileset returns a ResourceVersions slice parsed from the directory structure described above for LoadResourceVersions.

func (*ResourceVersions) At 

func (rv *ResourceVersions) At(vs string) (*ResourceVersion, error)

At returns the ResourceVersion matching a version string. The version of the resource returned will be the latest available version with a stability equal to or greater than the requested version, or ErrNoMatchingVersion if no matching version is available.

func (*ResourceVersions) Name 

func (rv *ResourceVersions) Name() string

Name returns the resource name for a collection of resource versions.

func (*ResourceVersions) Versions 

func (rv *ResourceVersions) Versions() VersionSlice

Versions returns each Version defined for this resource.

type SpecVersions 

type SpecVersions struct {
	// contains filtered or unexported fields
}

SpecVersions stores a collection of versioned OpenAPI specs.

func LoadSpecVersions 

func LoadSpecVersions(root string) (*SpecVersions, error)

LoadSpecVersions returns SpecVersions loaded from a directory structure containing one or more Resource subdirectories.

func LoadSpecVersionsFileset 

func LoadSpecVersionsFileset(epPaths []string) (*SpecVersions, error)

LoadSpecVersionsFileset returns SpecVersions loaded from a set of spec files.

func (*SpecVersions) At 

func (sv *SpecVersions) At(v Version) (*openapi3.T, error)

At returns the OpenAPI document that matches the given version. If the version is not an exact match for an API release, the OpenAPI document effective on the given version date for the version stability level is returned. Returns ErrNoMatchingVersion if there is no release matching this version.

func (*SpecVersions) Versions 

func (sv *SpecVersions) Versions() VersionSlice

Versions returns the distinct API versions in this collection of OpenAPI documents.

type Stability 

type Stability int

Stability defines the stability level of the version. 

const (

	// StabilityWIP means the API is a work-in-progress and not yet ready.
	StabilityWIP Stability = iota

	// StabilityExperimental means the API is experimental and still subject to
	// drastic change.
	StabilityExperimental Stability = iota

	// StabilityBeta means the API is becoming more stable, but may undergo some
	// final changes before being released.
	StabilityBeta Stability = iota

	// StabilityGA means the API has been released and will not change.
	StabilityGA Stability = iota
)

func MustParseStability 

func MustParseStability(s string) Stability

MustParseStability parses a stability string into a Stability type, panicking if the string is invalid.

func ParseStability 

func ParseStability(s string) (Stability, error)

ParseStability parses a stability string into a Stability type, returning an error if the string is invalid.

func (Stability) Compare 

func (s Stability) Compare(sr Stability) int

Compare returns -1 if the given stability level is less than, 0 if equal to, and 1 if greater than the caller target stability level.

func (Stability) Resolvable 

func (s Stability) Resolvable() []Stability

func (Stability) String 

func (s Stability) String() string

String returns a string representation of the stability level. This method will panic if the value is empty.

type Version 

type Version struct {
	Date      time.Time
	Stability Stability
}

Version defines an API version. API versions may be dates of the form "YYYY-mm-dd", or stability tags "beta", "experimental".

func MustParseVersion 

func MustParseVersion(s string) Version

MustParseVersion parses a version string into a Version type, panicking if the string is invalid.

func ParseVersion 

func ParseVersion(s string) (Version, error)

ParseVersion parses a version string into a Version type, returning an error if the string is invalid.

func (Version) AddDays 

func (v Version) AddDays(days int) Version

AddDays returns the version corresponding to adding the given number of days to the version date.

func (Version) Compare 

func (v Version) Compare(vr Version) int

Compare returns -1 if the given version is less than, 0 if equal to, and 1 if greater than the caller target version.

func (Version) DateString 

func (v Version) DateString() string

DateString returns the string representation of the version date in YYYY-mm-dd form.

func (Version) DeprecatedBy 

func (v Version) DeprecatedBy(vr Version) bool

DeprecatedBy returns true if the given version deprecates the caller target version.

func (*Version) LifecycleAt 

func (v *Version) LifecycleAt(t time.Time) Lifecycle

LifecycleAt returns the Lifecycle of the version at the given time. If the time is the zero value (time.Time{}), then the following are used to determine the reference time:

If VERVET_LIFECYCLE_AT is set to an ISO date string of the form YYYY-mm-dd, this date is used as the reference time for deprecation, at midnight UTC.

Otherwise `time.Now().UTC()` is used for the reference time.

The current time is always used for determining whether a version is unreleased.

func (Version) String 

func (v Version) String() string

String returns the string representation of the version in YYYY-mm-dd~Stability form. This method will panic if the value is empty.

func (Version) Sunset 

func (v Version) Sunset(vr Version) (time.Time, bool)

Sunset returns, given a potentially deprecating version, the eligible sunset date and whether the caller target version would actually be deprecated and sunset by the given version.

type VersionIndex 

type VersionIndex struct {
	// contains filtered or unexported fields
}

VersionIndex provides a search over versions, resolving which version is in effect for a given date and stability level.

func NewVersionIndex 

func NewVersionIndex(vs VersionSlice) (vi VersionIndex)

NewVersionIndex returns a new VersionIndex of the given versions. The given VersionSlice will be sorted.

func (*VersionIndex) Deprecates 

func (vi *VersionIndex) Deprecates(q Version) (Version, bool)

Deprecates returns the version that deprecates the given version in the slice.

func (*VersionIndex) Resolve 

func (vi *VersionIndex) Resolve(query Version) (Version, error)

Resolve returns the released version effective on the query version date at the given version stability. Returns ErrNoMatchingVersion if no version matches.

Resolve should be used on a collection of already "compiled" or "collated" API versions.

func (*VersionIndex) ResolveForBuild 

func (vi *VersionIndex) ResolveForBuild(query Version) (Version, error)

ResolveForBuild returns the most stable version effective on the query version date with respect to the given version stability. Returns ErrNoMatchingVersion if no version matches.

Use ResolveForBuild when resolving version deprecation and effective releases _within a single resource_ during the "compilation" or "collation" process.

func (*VersionIndex) ResolveGAorBetaStability added in v8.7.5 

func (vi *VersionIndex) ResolveGAorBetaStability(query Version) (Version, error)

ResolveGAorBetaStability returns the GA or beta version effective on the query version date at the given version date. Returns ErrNoMatchingVersion if no version matches or if query stability is not GA.

func (*VersionIndex) Versions 

func (vi *VersionIndex) Versions() VersionSlice

Versions returns each Version defined.

type VersionSlice 

type VersionSlice []Version

VersionSlice is a sortable slice of Versions.

func (VersionSlice) Len 

func (vs VersionSlice) Len() int

Len implements sort.Interface.

func (VersionSlice) Less 

func (vs VersionSlice) Less(i, j int) bool

Less implements sort.Interface.

func (VersionSlice) Strings 

func (vs VersionSlice) Strings() []string

Strings returns a slice of string versions.

func (VersionSlice) Swap 

func (vs VersionSlice) Swap(i, j int)

Swap implements sort.Interface.

Source Files

View all Source files

Directories

Path Synopsis
cmd
vervet
vu-api
vu-scraper
Package config supports configuring the Vervet Underground service.
 Package config supports configuring the Vervet Underground service.
internal
backstage
Package backstage supports vervet's integration with Backstage to automatically populate API definitions in the catalog info from compiled versions.
 Package backstage supports vervet's integration with Backstage to automatically populate API definitions in the catalog info from compiled versions.
cmd
Package cmd provides subcommands for the vervet CLI.
 Package cmd provides subcommands for the vervet CLI.
compiler
files
generator
handler
Package handler contains the HTTP handlers that serve Vervet Underground responses.
 Package handler contains the HTTP handlers that serve Vervet Underground responses.
openapiwalker
scaffold
scraper
Package scraper provides support for scraping OpenAPI versions from services.
 Package scraper provides support for scraping OpenAPI versions from services.
simplebuild
storage
Package storage provides common functionality supporting Vervet Underground storage.
 Package storage provides common functionality supporting Vervet Underground storage.
storage/disk
Package disk provides an implementation of the storage used in Vervet Underground that uses a local filesystem.
 Package disk provides an implementation of the storage used in Vervet Underground that uses a local filesystem.
storage/gcs
storage/gcs/testing
storage/s3
Package s3 provides an implementation of Vervet Underground storage backed by Amazon S3.
 Package s3 provides an implementation of Vervet Underground storage backed by Amazon S3.
storage/s3/testing
testutil
Package versionware provides routing and middleware for building versioned HTTP services.
 Package versionware provides routing and middleware for building versioned HTTP services.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.