luagen

Documentation

Overview

Package luagen implements a lua code generator for proto code.

This is NOT a protoc-gen plugin, because its output format is intentionally quite different. Instead of generating a 1:1 proto:lua output for each .proto file, this generates a single, wholly standalone lua file, which includes all serialization logic for a transitive closure of proto messages and enums. This makes using the generated code substantially easier (just a single require), and also allows this to be used in contexts where you need protos defined in places where adding adjacent .lua files would be difficult (e.g. adding .lua files to upstream `google.protobuf` repos would be impossible, and forking those repos or creating a parallel proto hierarchy would be very annoying).

Use it by adding a file e.g. `luagen.go` which looks like:

// (See NOTE below for what this build tag means)
//go:build luagen

package main

import (
	"go.chromium.org/luci/common/proto/msgpackpb/luagen"

	yours "your/package/with/protos"
)

func main() {
	luagen.Main(
		// Here you can include any proto messages or enums you want to
		// specifically in the output.
		&yours.YourMessage{},
	  yours.ENUM_VALUE,
	)
}

You can then generate code for it in a go:generate comment in a normal package .go file like:

//go:generate go run luagen.go output.lua

NOTE: The `//go:build luagen` directive in the file allows it to exist as a `package main` in the same directory as other .go files. You can of course opt to instead put the generator snippet in a different directory, if you choose.

Index

• func Main(objects ...any)

Functions

func Main

func Main(objects ...any)

Main will generate a lua script with codec definitions for all of the given `objects` (and any messages/enums they refer to), then write the script to a file specified by `os.Args[1]`.

This function accepts a list of proto.Message instances as well as proto typed Enum values (i.e. which conform to the protoreflect.Enum interface). Order of these arguments does not matter.

Generated File Contents

The generated file will be a module which returns a single table `PB`. The Contents of PB are as follows:

E

The `PB.E` table stores enumeration values (both numeric and string) indexed by the full name of the Enum type.

{
  ["full.Path.To.Enum"] = {
    ["ENUM_VALUE_NAME"] = 10,
    [10] = "ENUM_VALUE_NAME",
  }
}

M

The `PB.M` table contains a mapping from full name of proto Message to it's "codec".

Each codec consists of a `keys` set (the string names of each field in the message), the `marshal` function (which takes a previously unmarshal'd table, and produces a new table which, when passed to cmsgpack.pack, will be msgpackpb encoded), and the `unmarshal` function (which takes the cmsgpack.unpack'd table, and transforms it into a message).

setInternTable

The generated lua has a special 'string internment' feature. Any string value in any decoded message can be replaced with a numeric index (1-based) into an internment table. When a message with a string field (containing a number on the wire) is encountered, the unmarshal routine will look the string up from the table instead.

This feature is used to allow re-use of the required Redis KEYS list for embedded lua scripts (i.e. Redis requires pre declaring all affected keys, and passing them as an unencoded list of strings; By calling PB.setInternTable(KEYS), the encoding side can encode any of these strings in the proto messages as indexes in KEYS instead).

This feature is ONLY used for unmarshaling; The generated marshal code cannot produce msgpackpb messages with interned strings.

marhsal

This is the main marshaling function; Called as `PB.marshal(obj)`, this takes a previously-unmarshaled message table, and returns it in msgpackpb encoding as a string.

unmarshal

This is the main unmarshaling function; Called as `PB.unmarshal(messageName, msgpackpb)`, this takes a fully qualified proto message name (i.e. key in the `M` table), and the msgpackpb encoded data, and returns a table which is the decoded message. Note that the returned table will have all unset fields populated with the following defaults:

string - ""
repeated X - {}
bool - false
all number types - 0
message types - nil (note! lua does not preserve these values!)
enum types - the zero-value enum string name

Unmarshaled messages also contain a "$type" field which is the fully qualified message name, and also an "$unknown" field, which is a table mapping from field number to raw decoded msgpackpb data, for unknown fields.

These unknown fields are restored verbatim on `marshal`.

new

New is a function to produce a brand new message, and is called like `PB.new(messageName, defaults)`, where `defaults` is an optional table of default field values to fill. `PB.new` will currently check the keys in `defaults` (to avoid typos), but will not check types (bad types will cause an error when calling `marshal` eventually.

Example Go usage

func main() {
	luagen.Main(
		&examplepb.Message{},
		examplepb.ZERO,   // just need one of the enum values
	)
}

Example lua usage

local PB = loadfile('generated.lua')()
local myObject = PB.new('full.name.of.Message', {
  field = "hello"
})
# myObject == {field = "hello", enum_field = "ZERO"}
myObject.enum_field = PB.E['full.name.of.Enum'][3] # "THREE"
# OR: myObject.enum_field = "THREE"
# OR: myObject.enum_field = 3
local encoded = PB.marshal(myObject)
# encoded is a msgpackpb string with two fields
local decoded = PB.unmarshal('full.name.of.Message', encoded)
# decoded is {field = "hello", enum_field = "THREE"}

NOTE: See `examplepb` subfolder, including its luagen.go and generated .lua output.