README
¶
tm is used for defining the strucutre and Elasticsearch indexing rules for Messages
sent to Elasticsearch from rxtx through rtBeat with
the key rxtxMsg. tm Models define the properties of the rxtx payload.
The tm library defines a type of generic nested meta-data Model. The tm server creates a services for the storage, retrieval and searching of Models associated with a txn2/provision Account.
A Model consists of a record stored in the Elasticsearch index ACCOUNT-models and a corresponding Elasticsearch template (_template/ACCOUNT-data-MODEL) representing the index pattern ACCOUNT-data-MODEL-*.
Configuration
Configuration is inherited from txn2/micro. The following configuration is specific to tm:
| Flag | Environment Variable | Description |
|---|---|---|
| -esServer | ELASTIC_SERVER | Elasticsearch Server (default "http://elasticsearch:9200") |
| -mode | MODE | Protected or internal modes. ("internal" = token check bypass) |
Routes
| Method | Route Pattern | Description |
|---|---|---|
| POST | /model/:account | Upsert a model into an account. |
| GET | /model/:account/:id | Get a model by account and id. |
| POST | searchModels/:account | Search for models in an account with a Lucene query. |
Local Development
The project includes a Docker Compose file with Elasticsearch, Kibana and Cerebro:
docker-compose up
Run the source in token bypass mode and pointed to Elasticsearch exposed on localhost port 9200:
go run ./cmd/tm.go --mode=internal --esServer=http://localhost:9200
Examples
The following examples assume mode is set to internal and will not check a Bearer token for proper permissions.
Upsert Model
Upserting a Model will result in an Ack with a Result payload.
The following creates a model called test and will result in a record with the id test in the xorg-models index. A mapping template will be also be generated and stored at _template/xorg-data-test:
curl -X POST \
http://localhost:8080/model/xorg \
-H 'Content-Type: application/json' \
-d '{
"machine_name": "test",
"display_name": "",
"description_brief": "",
"description": "",
"data_type": "",
"format": "",
"parsers": null,
"type_class": "",
"group": "",
"parse": false,
"index": 0,
"fields": [
{
"machine_name": "event_type",
"display_name": "Event Type",
"description_brief": "",
"description": "",
"data_type": "keyword",
"format": "",
"parsers": null,
"type_class": "",
"group": "",
"parse": false,
"index": 0
},
{
"machine_name": "gps_utc_time",
"display_name": "GPS UTC Time",
"description_brief": "",
"description": "",
"data_type": "date",
"format": "yyyyMMddHHmmss",
"parsers": null,
"type_class": "",
"group": "",
"parse": false,
"index": 0
},
{
"machine_name": "location",
"display_name": "",
"description_brief": "",
"description": "",
"data_type": "nested",
"format": "",
"parsers": null,
"type_class": "",
"group": "",
"parse": false,
"index": 0,
"fields": [
{
"machine_name": "lat",
"display_name": "",
"description_brief": "",
"description": "",
"data_type": "float",
"format": "",
"parsers": null,
"type_class": "",
"group": "",
"parse": false,
"index": 0
},
{
"machine_name": "lon",
"display_name": "",
"description_brief": "",
"description": "",
"data_type": "float",
"format": "",
"parsers": null,
"type_class": "",
"group": "",
"parse": false,
"index": 0
}
]
}
]
}'
Get Model
Getting a Model will result in a ModelResultAck.
curl http://localhost:8080/model/xorg/test
Search Models
Searching for Models will result in a ModelSearchResultsAck.
curl -X POST \
http://localhost:8080/searchModels/xorg \
-d '{
"query": {
"match_all": {}
}
}'
Release Packaging
Build test release:
goreleaser --skip-publish --rm-dist --skip-validate
Build and release:
GITHUB_TOKEN=$GITHUB_TOKEN goreleaser --rm-dist
Documentation
¶
Overview ¶
Package tm implements Type Models for txn2 projects.
Index ¶
- Constants
- func GetModelsTemplateMapping() es.IndexTemplate
- func MakeModelTemplateMapping(account string, model *Model) es.IndexTemplate
- type Api
- func (a *Api) GetModel(account string, id string) (int, *ModelResult, error)
- func (a *Api) GetModelHandler(c *gin.Context)
- func (a *Api) SearchModels(account string, searchObj *es.Obj) (int, ModelSearchResults, *es.ErrorResponse, error)
- func (a *Api) SearchModelsHandler(c *gin.Context)
- func (a *Api) UpsertModel(account string, model *Model) (int, es.Result, *es.ErrorResponse, error)
- func (a *Api) UpsertModelHandler(c *gin.Context)
- type Config
- type Model
- type ModelResult
- type ModelResultAck
- type ModelSearchResults
- type ModelSearchResultsAck
Constants ¶
const IdxModel = "models"
Variables ¶
This section is empty.
Functions ¶
func GetModelsTemplateMapping ¶
func GetModelsTemplateMapping() es.IndexTemplate
GetModelsTemplateMapping
func MakeModelTemplateMapping ¶
MakeModelTemplateMapping creates a template for modeled data coming in from rxtx.
Types ¶
type Api ¶
type Api struct {
*Config
}
Api
func (*Api) SearchModels ¶ added in v0.0.4
func (a *Api) SearchModels(account string, searchObj *es.Obj) (int, ModelSearchResults, *es.ErrorResponse, error)
SearchModels
func (*Api) SearchModelsHandler ¶ added in v0.0.4
SearchAccountsHandler
func (*Api) UpsertModel ¶
UpsertModel
func (*Api) UpsertModelHandler ¶
UpsertModelHandler
type Config ¶
type Config struct {
Logger *zap.Logger
HttpClient *micro.Client
// used for communication with Elasticsearch
// if nil, HttpClient will be used.
Elastic *es.Client
ElasticServer string
}
Config
type Model ¶
type Model struct {
// MachineName is a lowercase under score delimited uniq id
MachineName string `json:"machine_name" mapstructure:"machine_name"`
// AliasOf is the machine name of a model this model is an alias of.
AliasOf string `json:"alias_of" mapstructure:"alias_of"`
// short human readable display name
DisplayName string `json:"display_name" mapstructure:"display_name"`
// a single sentence description
BriefDescription string `json:"description_brief" mapstructure:"description_brief"`
// full documentation in markdown
Description string `json:"description" mapstructure:"description"`
// default value expressed as a string
DefaultValue string `json:"default_value" mapstructure:"default_value"`
// integer, float, date, binary, text and keyword
DataType string `json:"data_type" mapstructure:"data_type"`
// used for data formats
Format string `json:"format" mapstructure:"format"`
// named parsers
Parsers []string `json:"parsers" mapstructure:"parsers"`
// belongs to a class of models
TypeClass string `json:"type_class" mapstructure:"type_class"`
// groups models
Group string `json:"group" mapstructure:"group"`
// false to ignore inbound parsing
Parse bool `json:"parse" mapstructure:"parse"`
// used by parsers of element ordered inbound data
Index int `json:"index" mapstructure:"index"`
// children of this model
Fields []Model `json:"fields" mapstructure:"fields"`
}
Model
type ModelResult ¶
type ModelResult struct {
es.Result
Source Model `json:"_source"`
}
ModelResult returned from Elastic
type ModelResultAck ¶ added in v0.0.11
type ModelResultAck struct {
ack.Ack
Payload ModelResult `json:"payload"`
}
ModelResultAck
type ModelSearchResults ¶ added in v0.0.4
type ModelSearchResults struct {
es.SearchResults
Hits struct {
Total int `json:"total"`
MaxScore float64 `json:"max_score"`
Hits []ModelResult `json:"hits"`
} `json:"hits"`
}
ModelSearchResults
type ModelSearchResultsAck ¶ added in v0.0.4
type ModelSearchResultsAck struct {
ack.Ack
Payload ModelSearchResults `json:"payload"`
}
AccountSearchResultsAck
Source Files
Directories