README
¶
Model Garage
Welcome to the Model Garage, a Golang toolkit for managing and working with DIMO models generated from vspec CSV schemas. Model Garage provides the following features:
Features
- Model Creation: Create models from vspec CSV schemas, represented as Go structs.
- JSON Conversion: Easily convert JSON data for your models with automatically generated and customizable conversion functions.
Migrations
To create a new migration, run the following command:
Make migration name=<migration_name>
This will create a new migration file the given name in the migrations directory.
this creation should be used over the goose binary to ensure expected behavior of embedded migrations.
Repo structure
Codegen
The codegen directory contains the code generation tool for creating models from vspec CSV schemas. The tool is a standalone application that can be run from the command line.
Example usage:
go run github.com/DIMO-Network/model-garage/cmd/codegen -output=pkg/vss -generators=all
Generation Info
The Model generation is handled by packages in internal/codegen. They are responsible for creating Go structs, Clickhouse tables, and conversion functions from the vspec CSV schema and definitions file. definitions file is a YAML file that specifies the conversions for each field in the vspec schema. The conversion functions are meant to be overridden with custom logic as needed. When generation is re-run, the conversion functions are not overwritten. Below is an example of a definitions file:
# vspecName: The name of the VSpec field in the VSS schema
- vspecName: DIMO.DefinitionID
# isArray: Whether the field is an array or not
# if null then the value is inferred from the vspec definition
isArray: null
# clickHouseType: The data type to use for ClickHouse Database.
#if empty then the type is inferred from the vspec definition
clickHouseType: ""
# goType: The data type to use for Golang struct.
# if empty then the type is inferred from the vspec definition
goType: ""
# gqlType: The data type to use for GraphQL schema.
# if empty then the type is inferred from the vspec definition
gqlType: ""
# conversion: The mapping of the original data to the VSpec field
conversion:
# originalName: The name of the field in the original data
originalName: data.definitionID
# originalType: The data type of the field in the original data
originalType: string
# isArray: Whether the field is an array or not
isArray: false
# requiredPrivileges: The list of privileges required to access the field
requiredPrivileges:
- VEHICLE_NON_LOCATION_DATA
- First, the vspec CSV schema and definitions file are parsed.
- Then a struct is created for each signal in the vpsec schema that is specified in the definitions file. With Clickhouse and JSON tags for each field. The CH and JSON names are the same as the vspec except
.are replaced with_. - Next, a Clickhouse table is created for the struct. The table name is the same as the package name. The table is created with the same fields as the struct with corresponding Clickhouse types.
- Finally, conversion functions are created for each struct. These functions convert the original data in the form of a JSON document to the struct.
Conversion Functions For each field, a conversion function is created. If a conversion is specified in the definitions file, the conversion function will use the specified conversion. If no conversion is specified, the conversion info function will assume a direct copy. The conversion functions are meant to be overridden with custom logic as needed. When generation is re-run, the conversion functions are not overwritten.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
clickhouse-container
Package main is a binary for creating a test ClickHouse container.
|
Package main is a binary for creating a test ClickHouse container. |
|
codegen
Package main provides the entrypoint for the code generation tool.
|
Package main provides the entrypoint for the code generation tool. |
|
internal
|
|
|
generator/convert
Package convert provides a function to generate conversion functions for a vehicle struct.
|
Package convert provides a function to generate conversion functions for a vehicle struct. |
|
generator/custom
Package custom provides the custom generation functionality that uses a provide template to generate a file.
|
Package custom provides the custom generation functionality that uses a provide template to generate a file. |
|
generator/migration
Package migration provides a function for migrating a clickhouse database to a schema.
|
Package migration provides a function for migrating a clickhouse database to a schema. |
|
generator/model
Package model provides the functionality to generate a Go struct file to represent the vehicle struct.
|
Package model provides the functionality to generate a Go struct file to represent the vehicle struct. |
|
pkg
|
|
|
clickhouseinfra
Package clickhouseinfra provides a set of functions to interact with ClickHouse containers.
|
Package clickhouseinfra provides a set of functions to interact with ClickHouse containers. |
|
codegen
Package codegen provides the code generation functionality for converting VSPEC signals to Go structs and ClickHouse tables.
|
Package codegen provides the code generation functionality for converting VSPEC signals to Go structs and ClickHouse tables. |
|
runner
Package runner is a package that provides a programmatic interface to the code generation tool.
|
Package runner is a package that provides a programmatic interface to the code generation tool. |
|
vss
Package vss holds the data structures and functions for working with signals from DIMOs VSS schema.
|
Package vss holds the data structures and functions for working with signals from DIMOs VSS schema. |
|
vss/convert
Package convert provides a function to generate conversion functions for a vehicle struct.
|
Package convert provides a function to generate conversion functions for a vehicle struct. |