Devfile Library
About
The Devfile Parser library is a Golang module that:
- parses a devfile as specified by the api & schema.
- writes to the specified devfile with the updated data.
- generates Kubernetes objects for the various devfile resources.
- defines util functions for the devfile.
- downloads resources from a parent devfile if specified in the devfile.
Private repository support
Tokens are required to be set in the following cases:
- parsing a devfile from a private repository
- parsing a devfile containing a parent devfile from a private repository [1]
- parsing a devfile from a private repository containing a parent devfile from a public repository [2]
Set the token for the repository:
parser.ParserArgs{
...
// URL must point to a devfile.yaml
URL: <url-to-devfile-on-supported-git-provider-repo>/devfile.yaml
Token: <repo-personal-access-token>
...
}
Note: The url must also be set with a supported git provider repo url.
Minimum token scope required:
- GitHub: Read access to code
- GitLab: Read repository
- Bitbucket: Read repository
Note: To select token scopes for GitHub, a fine-grained token is required.
For more information about personal access tokens:
- GitHub docs
- GitLab docs
- Bitbucket docs
[1] Currently, this works under the assumption that the token can authenticate the devfile and the parent devfile; both devfiles are in the same repository.
[2] In this scenario, the token will be used to authenticate the main devfile.
Usage
The function documentation can be accessed via pkg.go.dev.
-
To parse a devfile, visit parse.go source file
// ParserArgs is the struct to pass into parser functions which contains required info for parsing devfile.
parserArgs := parser.ParserArgs{
Path: path,
FlattenedDevfile: &flattenedDevfile,
ConvertKubernetesContentInUri: &convertKubernetesContentInUri
RegistryURLs: registryURLs,
DefaultNamespace: defaultNamespace,
Context: context,
K8sClient: client,
ExternalVariables: externalVariables,
}
// Parses the devfile and validates the devfile data
// if top-level variables are not substituted successfully, the warnings can be logged by parsing variableWarning
devfile, variableWarning, err := devfilePkg.ParseDevfileAndValidate(parserArgs)
-
To override the HTTP request and response timeouts for a devfile with a parent reference from a registry URL, specify the HTTPTimeout value in the parser arguments
// specify the timeout in seconds
httpTimeout := 20
parserArgs := parser.ParserArgs{
HTTPTimeout: &httpTimeout
}
-
To get specific content from devfile
// To get all the components from the devfile
components, err := devfile.Data.GetComponents(DevfileOptions{})
// To get all the components from the devfile with attributes tagged - tool: console-import
// & import: {strategy: Dockerfile}
components, err := devfile.Data.GetComponents(DevfileOptions{
Filter: map[string]interface{}{
"tool": "console-import",
"import": map[string]interface{}{
"strategy": "Dockerfile",
},
},
})
// To get all the volume components
components, err := devfile.Data.GetComponents(DevfileOptions{
ComponentOptions: ComponentOptions{
ComponentType: v1.VolumeComponentType,
},
})
// To get all the exec commands that belong to the build group
commands, err := devfile.Data.GetCommands(DevfileOptions{
CommandOptions: CommandOptions{
CommandType: v1.ExecCommandType,
CommandGroupKind: v1.BuildCommandGroupKind,
},
})
-
To get the Kubernetes objects from the devfile, visit generators.go source file
// To get a slice of Kubernetes containers of type corev1.Container from the devfile component containers
containers, err := generator.GetContainers(devfile)
// To generate a Kubernetes deployment of type v1.Deployment
deployParams := generator.DeploymentParams{
TypeMeta: generator.GetTypeMeta(deploymentKind, deploymentAPIVersion),
ObjectMeta: generator.GetObjectMeta(name, namespace, labels, annotations),
InitContainers: initContainers,
Containers: containers,
Volumes: volumes,
PodSelectorLabels: labels,
}
deployment := generator.GetDeployment(deployParams)
-
To update devfile content
// To update an existing component in devfile object
err := devfile.Data.UpdateComponent(v1.Component{
Name: "component1",
ComponentUnion: v1.ComponentUnion{
Container: &v1.ContainerComponent{
Container: v1.Container{
Image: "image1",
},
},
},
})
// To add a new component to devfile object
err := devfile.Data.AddComponents([]v1.Component{
{
Name: "component2",
ComponentUnion: v1.ComponentUnion{
Container: &v1.ContainerComponent{
Container: v1.Container{
Image: "image2",
},
},
},
},
})
// To delete a component from the devfile object
err := devfile.Data.DeleteComponent(componentName)
-
To write to a devfile, visit writer.go source file
// If the devfile object has been created with devfile path already set, can simply call WriteYamlDevfile to writes the devfile
err := devfile.WriteYamlDevfile()
// To write to a devfile from scratch
// create a new DevfileData with a specific devfile version
devfileData, err := data.NewDevfileData(devfileVersion)
// set schema version
devfileData.SetSchemaVersion(devfileVersion)
// add devfile content use library APIs
devfileData.AddComponents([]v1.Component{...})
devfileData.AddCommands([]v1.Commands{...})
......
// create a new DevfileCtx
ctx := devfileCtx.NewDevfileCtx(devfilePath)
err = ctx.SetAbsPath()
// create devfile object with the new DevfileCtx and DevfileData
devfile := parser.DevfileObj{
Ctx: ctx,
Data: devfileData,
}
// write to the devfile on disk
err = devfile.WriteYamlDevfile()
-
To parse the outerloop Kubernetes/OpenShift component's uri or inline content, call the read and parse functions
// Read the YAML content
values, err := ReadKubernetesYaml(src, fs)
// Get the Kubernetes resources
resources, err := ParseKubernetesYaml(values)
-
By default, the parser will set all unset boolean properties to their spec defined default values. Clients can override this behaviour by specifiying the parser argument SetBooleanDefaults
to false
setDefaults := false
parserArgs := parser.ParserArgs{
SetBooleanDefaults: &setDefaults,
}
-
When parsing a devfile that contains a parent reference, if the parent uri is a supported git provider repo url with the correct personal access token, all resources from the parent git repo excluding the parent devfile.yaml will be downloaded to the location of the devfile being parsed. Note: The URL must point to a devfile.yaml
schemaVersion: 2.2.0
...
parent:
uri: <uri-to-parent-devfile>/devfile.yaml
...
-
By default, the library downloads the Git repository resources associated with the Git URL that is mentioned in a devfile uri field. To turn off the download, pass in the DownloadGitResources
property in the parser argument
downloadGitResources := false
parserArgs := parser.ParserArgs{
DownloadGitResources: &downloadGitResources,
}
- To download/access files from a private repository like a private GitHub use the
Token
property
parserArgs := parser.ParserArgs{
Token: "my-PAT",
}
src: YamlSrc{
URL: "http://github.com/my-private-repo",
Token: "my-PAT",
}
values, err := ReadKubernetesYaml(src, fs, nil)
If you would like to use the mock implementation for the DevfileUtils
interface method defined in pkg/devfile/parser/util/interface.go, then use
var devfileUtilsClient DevfileUtils
devfileUtilsClient = NewMockDevfileUtilsClient()
devfileUtilsClient.DownloadInMemory(params)
Projects using devfile/library
The following projects are consuming this library as a Golang dependency
Tests
To run unit tests and api tests. Visit library tests to find out more information on tests
make test
Issues
Issues are tracked in the devfile/api repo with the label area/library
Releases
The devfile/library releases are created annually or on demand. For existing devfile/library releases, please check the release page.
Create a New Release
The steps to create a new release are:
-
Create a separate branch for the particular release, for example, v2.0.x
-
Run this script to generate release changelog.
# generate a changelog for all the changes since release v2.0.0
./changelog-script.sh v2.0.0
- Create a new release here with a new tag (having the same name with the above branch - e.g.
v2.0.x
) and copy the generated changelog to the details
Contributing
Please see our contributing.md.