README
¶
Model Garage
Welcome to the Model Garage, a Golang toolkit for managing and working with DIMO models generated from vspec CSV schemas. Model Garage provides the following features:
Features
- Model Creation: Create models from vspec CSV schemas, represented as Go structs.
- JSON Conversion: Easily convert JSON data for your models with automatically generated and customizable conversion functions.
Migrations
To create a new migration, run the following command:
Make migration name=<migration_name>
This will create a new migration file the given name in the migrations directory.
this creation should be used over the goose binary to ensure expected behavior of embedded migrations.
Repo structure
Codegen
The codegen directory contains the code generation tool for creating models from vspec CSV schemas. The tool is a standalone application that can be run from the command line.
Example usage:
go run github.com/DIMO-Network/model-garage/cmd/codegen -output=pkg/vss -generators=all
Generation Info
The Model generation is handled by packages in internal/codegen. They are responsible for creating Go structs, Clickhouse tables, and conversion functions from the vspec CSV schema and definitions file. definitions file is a YAML file that specifies the conversions for each field in the vspec schema. The conversion functions are meant to be overridden with custom logic as needed. When generation is re-run, the conversion functions are not overwritten. Below is an example of a definitions file:
# vspecName: The name of the VSpec field in the VSS schema
# required
- vspecName: DIMO.DefinitionID
# goType: The data type to use for Golang struct.
# available types: [float64, string]
# if empty then the type is inferred from the vspec definition
goType: ""
# conversion: The mapping of the original data to the VSpec field
conversion:
# originalName: The name of the field in the original data
# required
originalName: data.definitionID
# originalType: The data type of the field in the original data
originalType: string
# isArray: Whether the original field is an array or not
isArray: false
# requiredPrivileges: The list of privileges required to access the field
# required
requiredPrivileges:
- VEHICLE_NON_LOCATION_DATA
- First, the vspec CSV schema and definitions file are parsed.
- Then a struct is created for each signal in the vpsec schema that is specified in the definitions file. With Clickhouse and JSON tags for each field. The CH and JSON names are the same as the vspec except
.are replaced with_. - Next, a Clickhouse table is created for the struct. The table name is the same as the package name. The table is created with the same fields as the struct with corresponding Clickhouse types.
- Finally, conversion functions are created for each struct. These functions convert the original data in the form of a JSON document to the struct.
Conversion Functions For each field, a conversion function is created. If a conversion is specified in the definitions file, the conversion function will use the specified conversion. If no conversion is specified, the conversion info function will assume a direct copy. The conversion functions are meant to be overridden with custom logic as needed. When generation is re-run, the conversion functions are not overwritten.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
clickhouse-container
Package main is a binary for creating a test ClickHouse container.
|
Package main is a binary for creating a test ClickHouse container. |
|
codegen
Package main provides the entrypoint for the code generation tool.
|
Package main provides the entrypoint for the code generation tool. |
|
internal
|
|
|
generator/convert
Package convert provides a function to generate conversion functions for a vehicle struct.
|
Package convert provides a function to generate conversion functions for a vehicle struct. |
|
generator/custom
Package custom provides the custom generation functionality that uses a provide template to generate a file.
|
Package custom provides the custom generation functionality that uses a provide template to generate a file. |
|
generator/model
Package model provides the functionality to generate a Go struct file to represent the vehicle struct.
|
Package model provides the functionality to generate a Go struct file to represent the vehicle struct. |
|
pkg
|
|
|
codegen
Package codegen provides the code generation functionality for converting VSPEC signals to Go structs and ClickHouse tables.
|
Package codegen provides the code generation functionality for converting VSPEC signals to Go structs and ClickHouse tables. |
|
migrations
Package migrations provides the functionality for managing database migrations for the vss tables.
|
Package migrations provides the functionality for managing database migrations for the vss tables. |
|
runner
Package runner is a package that provides a programmatic interface to the code generation tool.
|
Package runner is a package that provides a programmatic interface to the code generation tool. |
|
schema
Package schema contains the types and functions for generating the schema from the spec and definition files.
|
Package schema contains the types and functions for generating the schema from the spec and definition files. |
|
version
Package version provides the version of the application.
|
Package version provides the version of the application. |
|
vss
Package vss holds the data structures and functions for working with signals from DIMOs VSS schema.
|
Package vss holds the data structures and functions for working with signals from DIMOs VSS schema. |
|
vss/convert
Package convert provides a function to generate conversion functions for a vehicle struct.
|
Package convert provides a function to generate conversion functions for a vehicle struct. |