gen

README

show output schemas generation

These schemas are necessary to include SHOW output in every resource and datasource. The work is repetitive, so it's easier to just generate all the needed schemas and mappers.

Description

File generate.go invokes the generation logic from main.go. By default, all SDK show output struct are used (listed in sdk_show_result_structs.go). After successful generation all SDK objects will have:

• show output schema that can be used in the resource/datasource (e.g. warehouse_gen)
• mapper from the SDK object to the generated schema (e.g. warehouse_gen)

How it works

Invoking the generation

To generate all show outputs (with a cleanup first) run:

make clean-show-output-schemas generate-show-output-schemas

To generate only chosen subset of all objects run:

make clean-show-output-schemas generate-show-output-schemas SF_TF_GENERATOR_EXT_ALLOWED_OBJECT_NAMES="sdk.Warehouse,sdk.User"

Supported types

The following types are supported currently in the generator (schema and mappings):

• basic types (string, int, float64, bool)
• pointers to basic types (the same as above)
• time.Time (pointer too)
• enums based on string and int like sdk.WarehouseType or sdk.ResourceMonitorLevel (pointers too)
• identifiers (pointers too):
  • sdk.AccountIdentifier
  • sdk.ExternalObjectIdentifier
  • sdk.AccountObjectIdentifier
  • sdk.DatabaseObjectIdentifier
  • sdk.SchemaObjectIdentifier
  • sdk.TableColumnIdentifier
• sdk.ObjectIdentifier interface

To schema mappings

Given SDK struct field can be mapped to the generated schema depending on its type:

• no mapping (Identity) - used for string and other basic types
• string value mapping (ToString) - used e.g. for time.Time
• fully qualified name mapping (FullyQualifiedName) - used for all identifiers and sdk.ObjectIdentifier interface
• casting (CastToString and CastToInt) - used for enums with underlying type string or int

Changing the SDK object's show output

If you change the show output struct in the SDK:

1. Check if you don't introduce a type that is unsupported (check supported types and known limitations).
2. Run generation according to instructions.

Adding a new object to the SDK

1. Add the new show output struct to sdk_show_result_structs.go.
2. Check if you don't introduce a type that is unsupported (check supported types and known limitations).
3. Run generation according to instructions.

Next steps

Known limitations

• The following types (already existing in the SDK show output structs) are not yet supported (for all of them the schema will be generated with schema.TypeInvalid:
  • other basic types (e.g. int8, etc.)
  • slices of basic types ([]int, []string)
  • slices of identifiers ([]sdk.AccountIdentifier, []sdk.SchemaObjectIdentifier)
  • slices of enums ([]sdk.IntegrationType, []sdk.PluralObjectType)
  • structs (sdk.FileFormatTypeOptions)

Improvements

Functional improvements:

• handle the missing types (TODOs in schema_field_mapper.go)
• parametrize the generation, e.g.:
  • (optional) parametrize the output directory - currently, it's always written to schemas package
• discover a change and generate as part of a make pre-push

Implementation improvements:

• (optional) consider different implementations of Mapper (e.g. TODO in schema_field_mapper_test.go: ugly comparison of functions with the current implementation of mapper and not ideal implementation in the to_schema_mapper.tmpl: runMapper .Mapper $nameLowerCase "." .OriginalName)

Documentation

Index

• Variables
• type SchemaField
  • func MapToSchemaField(field genhelpers.Field) SchemaField
• type ShowResultSchemaModel
  • func ModelFromStructDetails(sdkStruct genhelpers.StructDetails) ShowResultSchemaModel
  • func (m ShowResultSchemaModel) SomeFunc()

Variables

var (
	PreambleTemplate, _ = template.New("preambleTemplate").Parse(preambleTemplateContent)

	SchemaTemplate, _ = template.New("schemaTemplate").Parse(schemaTemplateContent)

	ToSchemaMapperTemplate, _ = template.New("toSchemaMapperTemplate").Funcs(genhelpers.BuildTemplateFuncMap(
		genhelpers.FirstLetterLowercase,
		genhelpers.RunMapper,
	)).Parse(toSchemaMapperTemplateContent)

	AllTemplates = []*template.Template{PreambleTemplate, SchemaTemplate, ToSchemaMapperTemplate}
)

var AdditionalStructs = []any{
	sdk.SecurityIntegrationProperty{},
}

TODO [SNOW-1501905]: currently all this structs have the "Show" added to the schema, while these are not show outputs TODO [SNOW-1501905]: temporary struct, may be refactored with addition to generation of describe results; for now used to some structs needing a schema representation

var SdkShowResultStructs = []any{
	sdk.Account{},
	sdk.Alert{},
	sdk.ApiIntegration{},
	sdk.ApplicationPackage{},
	sdk.ApplicationRole{},
	sdk.Application{},
	sdk.DatabaseRole{},
	sdk.Database{},
	sdk.DynamicTable{},
	sdk.EventTable{},
	sdk.ExternalFunction{},
	sdk.ExternalTable{},
	sdk.FailoverGroup{},
	sdk.FileFormat{},
	sdk.Function{},
	sdk.Grant{},
	sdk.ManagedAccount{},
	sdk.MaskingPolicy{},
	sdk.MaterializedView{},
	sdk.NetworkPolicy{},
	sdk.NetworkRule{},
	sdk.NotificationIntegration{},
	sdk.Parameter{},
	sdk.PasswordPolicy{},
	sdk.Pipe{},
	sdk.PolicyReference{},
	sdk.Procedure{},
	sdk.ReplicationAccount{},
	sdk.ReplicationDatabase{},
	sdk.Region{},
	sdk.ResourceMonitor{},
	sdk.Role{},
	sdk.RowAccessPolicy{},
	sdk.Schema{},
	sdk.SecurityIntegration{},
	sdk.Sequence{},
	sdk.SessionPolicy{},
	sdk.Share{},
	sdk.Stage{},
	sdk.StorageIntegration{},
	sdk.Streamlit{},
	sdk.Stream{},
	sdk.Table{},
	sdk.Tag{},
	sdk.Task{},
	sdk.User{},
	sdk.View{},
	sdk.Warehouse{},
}

Types

type SchemaField

type SchemaField struct {
	Name                  string
	SchemaType            schema.ValueType
	OriginalName          string
	IsOriginalTypePointer bool
	Mapper                genhelpers.Mapper
}

func MapToSchemaField

func MapToSchemaField(field genhelpers.Field) SchemaField

TODO [SNOW-1501905]: handle other basic type variants TODO [SNOW-1501905]: handle any other interface (error) TODO [SNOW-1501905]: handle slices TODO [SNOW-1501905]: handle structs (chosen one or all)

type ShowResultSchemaModel

type ShowResultSchemaModel struct {
	Name         string
	SdkType      string
	SchemaFields []SchemaField
}

func ModelFromStructDetails

func ModelFromStructDetails(sdkStruct genhelpers.StructDetails) ShowResultSchemaModel

func (ShowResultSchemaModel) SomeFunc

func (m ShowResultSchemaModel) SomeFunc()