snapshot

README

Snapshot (PoC)

Tyk Operator includes a snapshot package which provides a CLI tool that allows users to export their existing Tyk APIs and Security Policies into CRD YAML format that can be used by Tyk Operator.

It can help you to migrate existing APIs and Policies to Kubernetes environment.

Notes: After the migration, please be reminded that you should stop using the Dashboard UI to manage the migrated APIs and Policies. Please see Using Tyk Operator to enable GitOps with Tyk for more information.

This tool is provided as PoC only and bear some limitations and restrictions. Please use with caution.

---

Prerequisites | Preparation | Installation | Usage | Limitations

---

Prerequisites

1. Docker or Tyk Operator binaries
2. Credentials to connect Tyk Dashboard or Gateway. Please visit Tyk Docs for details.

Preparation

Prepare the API metadata before exporting.

1. Specify the Kubernetes resource names in Config Data (Required)

Before exporting the APIs, you must specify the k8s metadata name to be used for each APIDefinition resource. It can be saved in config_data. You can optionally provide the namespace too.

NOTE: snapshot tool will skip exporting any APIs without "k8sName" specified in config_data.

Example config_data for your APIs:

{
  "k8sName": "httpbin-api-5",
  "k8sNamespace": "staging"
}

Corresponding resource will look like this:

apiVersion: tyk.tyk.io/v1alpha1
kind: ApiDefinition
metadata:
  name: httpbin-api-5
  namespace: staging

To edit Config Data through API Manager,

1. Go to API Manager
2. Select the API you want to export
3. Go to "Advanced Options" tab
4. Scroll down to "Config Data" and put in the k8s metadata name and namespace (optional).

1. Use API Category to group the APIs you want to export (Optional)

By default, snapshot tool will export all APIs that have proper config_data configured. If you do not want to export all APIs, you can use --category flag to select which API category you want to export.

You can categorize your APIs by teams, environments, or any way you want. In the example below, you can see that some APIs are categorized as #testing or #production.

To set API Category through API Manager,

1. Go to API Manager
2. Select the API you want to export
3. Under "Core Settings" tab, scroll down to "API Settings".
4. Type the Category name in the input box or select from dropdown list.
5. You will see the Categories of each API on APIs page.

Installation

From v0.14.0, snapshot tool is available via Docker and Tyk Operator binaries.

Docker

To install a particular version of tyk-operator via docker image please run the command bellow with the appropriate version you want to use. All available versions could be found on the Tyk Operator Docker Hub page here: https://hub.docker.com/r/tykio/tyk-operator/tags

docker pull tykio/tyk-operator:{tag}

Then the docker image tyk-operator can be used in the following way:

docker run -it --rm --env-file=.env -v "$(pwd)":/dist tykio/tyk-operator:{tag} [FLAGS]

where .env file includes Tyk credentials, as follows:

TYK_ORG=${TYK_ORG}
TYK_AUTH=${TYK_AUTH}
TYK_URL=${TYK_URL}
TYK_MODE=${TYK_MODE}

Tyk Operator binary

Please visit GitHub Releases of Tyk Operator and install Tyk Operator binary based on your platform.

Setting required environment variables

In order snapshot tool to connect your Tyk installation, store the Tyk Dashboard or Gateway connection parameters in environment variables before running snapshot.

• ${TYK_MODE}: ce for Tyk Open Source mode and pro for for Tyk Self Managed mode.
• ${TYK_URL}: Management URL of your Tyk Dashboard or Gateway.
• ${TYK_AUTH}: Operator user API Key.
• ${TYK_ORG}: Operator user ORG ID.

For more details on how to obtain the URL and credentials, please visit Tyk Docs.

Usage

tyk-operator connects to a Tyk installation, pull a snapshot of APIs and Security Policies from there and output
as Custom Resource that can be used with Tyk Operator.
  
Export API Definitions to a file:
  --apidef <output_file>
    	Pull a snapshot of ApiDefinitions from Tyk and output as CR. All exported ApiDefinitions are included in
       `output_file`.

  --category <category_name>
    	Only export APIs from specified category

Export Security Policies to a file:
  --policy <output_file>
    	Pull a snapshot of all SecurityPolicies from Tyk and output as CR. All exported SecurityPolicies are 
      included in `output_file`.

Export API Definitions and Security Policies into separate files:
  --separate 
        Pull a snapshot of ApiDefinitions and SecurityPolicies from Tyk and output as CR.
        Each ApiDefinition and SecurityPolicy CR will be output into separate files.
        It can be used with --category flag to export only APIs from a category.

Warning: All ApiDefinitions that SecurityPolicy access must exist in Kubernetes. Otherwise, when you try to apply the policies, SecurityPolicy controller logs an error since it cannot find corresponding ApiDefinition resource in the environment.

Applying the API and Policy Custom Resources

1. Apply the API Definition CRs first

2. After that, you can apply the Security Policies.

3. The APIs and Policies are now managed by Tyk Operator!

Limitations

• Tyk Operator doesn't work with OAS HTTP API at the moment. Support for it is coming soon.
• Not all API Definitions features are supported by Operator. Non-supported features would be lost during the conversion.

Please visit ApiDefinition and Policies documentations to see supported features.

• The CLI tool will include all fields from your API RAW Definitions (i.e. also the empty and default fields). You can manually clean up those fields if you're sure
• Please remember that this is a PoC for exporting ApiDefinitions to k8s resources. First, try on your testing environment.

Documentation

Index

• Constants
• Variables
• func PrintSnapshot(ctx context.Context, apiDefinitionsFile, policiesFile, category string, ...) error

Constants

const (
	SnapshotOutputDir = "./dist"
	NameKey           = "k8sName"
	NamespaceKey      = "k8sNamespace"
	DefaultName       = "REPLACE_ME"
	DefaultNs         = ""

	ApiDefinitionKind = "ApiDefinition"
	ApiVersion        = "tyk.tyk.io/v1alpha1"
)

Variables

var (

	// ErrNonExistentKey represents an error if the given key does not exist in the object.
	ErrNonExistentKey = errors.New("key does not exist in the Config Data")

	// ErrNonStringVal represents an error if the underlying value of interface{} is not string type.
	ErrNonStringVal = errors.New("failed to convert interface{} to string")

	// ErrNonExistentConfigData represents an error if the given ApiDefinition includes empty (nil) ConfigData field.
	ErrNonExistentConfigData = errors.New("failed to parse ConfigData: non existent")

	// ErrInvalidConfigData represents an error if the given ConfigData does not include the required key 'k8sName'.
	ErrInvalidConfigData = errors.New("failed to parse 'k8sName' field in ConfigData")
)

Functions

func PrintSnapshot

func PrintSnapshot(ctx context.Context, apiDefinitionsFile, policiesFile, category string, separate bool) error

PrintSnapshot outputs a snapshot of the Dashboard as a CR.