bindgen

package module
v0.0.0-...-432cd89 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 23, 2021 License: MIT Imports: 11 Imported by: 12

README

bindgen

package bindgen is a package that aids in the generation of bindings and idiomatic Go interfaces to C-libraries.

As it exists it's a collection of utility data structures and functions built on top of cznic/cc to simplify usage of the C parser. The translation has to be written manually still. Each package would have different translation needs, hence the minimal package.

Generation of bindings and interfaces is primarily an exercise syntactic parsing and translation. The semantics of the C language isn't as important, given the Go C pseudopackage obeys all the calling semantics of C anyway.

Much of the architecure was drawn directly from the gonum BLAS generation package.

Examples

The running example in this package will involve parsing a dummy C header that is defined like this:

typedef enum err_enum {
	SUCCESS = 0,
	FAILURE = 1,
} error;

typedef int foo; 
typedef int context;

void func1i(int* a);
void func1f(foo a); 
void func1fp(foo* a);

void func2i(int a, int b);
void func2f(foo a, int b);

error funcErr(const int* a, foo* retVal);
error funcCtx(const context* ctx, foo a, foo* retVal);

Pseudo-Interactive Conversion

This package provides pseudo-interactive conversions. It's NOT a wizard (remember those?), but provides wizard-like functionalities. The key idea is to have a main package that keeps changing, essentially treating this like a REPL.

Here's a sample process used. We'll start with a main package. Our directory looks like this:

.
└── main.go

First, use Explore() to find out if the header can be easily parsed.

After you're satisfied, use the GenIgnored and GenNameMap functions. These functions generate strings that are valid Go code. Write them to a file, which we will call mappings.go like so:

.
├── main.go
└── mappings.go

Now that the mappings are generated, you can now safely modify your main package to no longer generate the mappings. Instead, you can now work on the translation parts.

vs c-for-go

Should you use this or c-for-go? This package is not as full-featured. If you want an automatic solution, use c-for-go. Unfortunately, this author has found the results are not as idiomatic.

Consider the header in the example above again.

These are the idiomatic equivalents in Go of the last two:

// error funcErr(const int* a, foo* retVal);
func funcErr(a int) (retVal *foo, err error) {} 

// error funcCtx(const context* ctx, foo a, foo* retVal);
func (ctx *context) funcCtx(a foo) (retVal *foo, err error){}

With c-for-go, translating the above to those would entail a very complicated translation portion YAML file. A few attempts were made to use c-for-go to translate CUDA bindings to Go, but none were satisfactorily idiomatic.

But we're programmers, so why not program? This is the basis for this package - to provide easy-to-access features necessary to do the translation.

Granted, how you'd want to structure your code as "idiomatic" is up to you. Since this package is used quite a bit in generating the Go bindings for CUDA, there are a lot of repeated patterns like those two examples above.

The CUDA libraries also come with certain patterns that make translation easier - things like const parameters makes things clearer, and simple rules can be written in code to translate them. Sure, you end up not having something easily defined as YAML files, but you also end up with nicer (imo) code.

Conclusion: If you want something automatic, use the excellent (and well documented) c-for-go. If you want more control over the translation process, use this package.

Tips For Writing Bindings

Sometimes your includes directories can be a little wonky. Since the task at hand is writing/generating the bindings and Go API for said bindings, these are a few things I've found helpful:

  1. gcc -E -P FILE.h > CLEAN.h This expands all the macros that some files have.
  2. Mock data structures: some types that come from #include are complicated chains of even more #includes. Just replace them with dummy data structures.
  3. The same for typedef that are too complicated and involve too many other files.

Licence

This package is MIT licenced. It is derived from the Gonum generator package which is licenced under the _____(TODO).

Documentation

Overview

Example (ConvertingEnums)

This is an example of how to convert enums. 

package main

import (
	"bytes"
	"fmt"
	"strings"

	"github.com/gorgonia/bindgen"
	"modernc.org/cc"
)

// genEnums represents a list of enums we want to generate
var genEnums = map[bindgen.TypeKey]struct{}{
	{Kind: cc.Enum, Name: "error"}: {},
}

var enumMappings = map[bindgen.TypeKey]string{
	{Kind: cc.Enum, Name: "error"}: "Status",
}

// This is an example of how to convert enums.
func main() {
	t, err := bindgen.Parse(bindgen.Model(), "testdata/dummy.h")
	if err != nil {
		panic(err)
	}
	enums := func(decl *cc.Declarator) bool {
		name := bindgen.NameOf(decl)
		kind := decl.Type.Kind()
		tk := bindgen.TypeKey{Kind: kind, Name: name}
		if _, ok := genEnums[tk]; ok {
			return true
		}
		return false
	}
	decls, err := bindgen.Get(t, enums)
	if err != nil {
		panic(err)
	}

	var buf bytes.Buffer
	for _, d := range decls {
		// first write the type
		//	 type ___ int
		// This is possible because cznic/cc parses all enums as int.
		//
		// you are clearly free to add your own mapping.
		e := d.(*bindgen.Enum)
		tk := bindgen.TypeKey{Kind: cc.Enum, Name: e.Name}
		fmt.Fprintf(&buf, "type %v int\nconst (\n", enumMappings[tk])

		// then write the const definitions:
		// 	const(...)

		for _, a := range e.Type.EnumeratorList() {
			// this is a straightforwards mapping of the C defined name. The name is kept exactly the same, with a lowecase mapping
			// in real life, you might not want this, (for example, you may not want to export the names, which are typically in all caps),
			// or you might want different names

			enumName := string(a.DefTok.S())
			goName := strings.ToLower(enumName)
			fmt.Fprintf(&buf, "%v %v = C.%v\n", goName, enumMappings[tk], enumName)
		}
		buf.Write([]byte(")\n"))
	}
	fmt.Println(buf.String())

}

Output:

type Status int
const (
success Status = C.SUCCESS
failure Status = C.FAILURE
)
Example (Simple) 
package main

import (
	"fmt"
	"strings"

	"github.com/gorgonia/bindgen"
	"modernc.org/cc"
)

// GoSignature represents a Go signature
type GoSignature struct {
	Receiver Param
	Name     string
	Params   []Param
	Ret      []Param
}

func (sig *GoSignature) Format(f fmt.State, c rune) {
	f.Write([]byte("func "))
	if sig.Receiver.Name != "" {
		fmt.Fprintf(f, "(%v %v) ", sig.Receiver.Name, sig.Receiver.Type)
	}
	fmt.Fprintf(f, "%v(", sig.Name)
	for i, p := range sig.Params {
		fmt.Fprintf(f, "%v %v", p.Name, p.Type)
		if i < len(sig.Params)-1 {
			fmt.Fprint(f, ", ")
		}
	}
	fmt.Fprint(f, ")")

	switch len(sig.Ret) {
	case 0:
		return
	default:
		fmt.Fprint(f, " (")
		for i, p := range sig.Ret {
			fmt.Fprintf(f, "%v %v", p.Name, p.Type)
			if i < len(sig.Ret)-1 {
				fmt.Fprint(f, ", ")
			}
		}
		fmt.Fprint(f, ")")
	}
}

// Param represents the parameters in Go
type Param struct {
	Name, Type string
}

// functions say we only want functions declared
func functions(t *cc.TranslationUnit) ([]bindgen.Declaration, error) {
	filter := func(d *cc.Declarator) bool {
		if !strings.HasPrefix(bindgen.NameOf(d), "func") {
			return false
		}
		if d.Type.Kind() != cc.Function {
			return false
		}
		return true
	}
	return bindgen.Get(t, filter)
}

func decl2GoSig(d *bindgen.CSignature) *GoSignature {
	var params []Param
	sig := new(GoSignature)
outer:
	for _, p := range d.Parameters() {
		// check if its a receiver
		if ctxrec, ok := contextualFns[d.Name]; ok {
			if ctxrec.Name == p.Name() {
				sig.Receiver = ctxrec
				continue
			}
		}
		Type := cleanType(p.Type())
		if retP, ok := retVals[d.Name]; ok {
			for _, r := range retP {
				if p.Name() == r {
					sig.Ret = append(sig.Ret, Param{p.Name(), Type})
					continue outer
				}
			}
		}
		params = append(params, Param{p.Name(), Type})
	}
	retType := cleanType(d.Return)
	if !bindgen.IsVoid(d.Return) {
		sig.Ret = append(sig.Ret, Param{"err", retType})
	}

	sig.Name = d.Name
	sig.Params = params
	return sig
}

func cleanType(t cc.Type) string {
	Type := t.String()
	if td := bindgen.TypeDefOf(t); td != "" {
		Type = td
	}

	if bindgen.IsConstType(t) {
		Type = strings.TrimPrefix(Type, "const ")
	}
	if bindgen.IsPointer(t) {
		Type = strings.TrimSuffix(Type, "*")
	}
	return Type
}

var contextualFns = map[string]Param{
	"funcCtx": Param{"ctx", "Ctx"},
}

var retVals = map[string][]string{
	"funcErr": []string{"retVal"},
	"funcCtx": []string{"retVal"},
}

func handleErr(err error) {
	if err != nil {
		panic(err)
	}
}

func main() {
	t, err := bindgen.Parse(bindgen.Model(), "testdata/dummy.h")
	handleErr(err)
	fns, err := functions(t)
	handleErr(err)
	for _, fn := range fns {
		fmt.Println(decl2GoSig(fn.(*bindgen.CSignature)))
	}

}

Output:

func func1i(a int)
func func1f(a foo)
func func1fp(a foo)
func func2i(a int, b int)
func func2f(a foo, b int)
func funcErr(a int) (retVal foo, err error)
func (ctx Ctx) funcCtx(a foo) (retVal foo, err error)

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func Explore 

func Explore(t *cc.TranslationUnit, filters ...FilterFunc) error

Explore is a function used to iterate quickly on a project to translate C functions/types to Go functions/types

Example 
functions := func(decl *cc.Declarator) bool {
	if !strings.HasPrefix(NameOf(decl), "func") {
		return false
	}
	if decl.Type.Kind() == cc.Function {
		return true
	}
	return false
}
enums := func(decl *cc.Declarator) bool {
	if decl.Type.Kind() == cc.Enum {
		return true
	}
	return false
}
others := func(decl *cc.Declarator) bool {
	if decl.Type.Kind() == cc.Ptr || decl.Type.Kind() == cc.Struct {
		return true
	}
	return false
}
tu, err := Parse(Model(), "testdata/dummy.h")
if err != nil {
	panic(err)
}

if err = Explore(tu, functions, enums, others); err != nil {
	panic(err)
}

Output:

func1i
func1f
func1fp
func2i
func2f
funcErr
funcCtx

error
fntype_t

dummy_t
dummy2_t

func GenIgnored 

func GenIgnored(buf io.Writer, t *cc.TranslationUnit, filters ...FilterFunc) error

GenIgnored generates go code for a const data structure that contains all the ignored functions/types

Filename indicates what file needs to be parsed, not the output file.

Example 
functions := func(decl *cc.Declarator) bool {
	if !strings.HasPrefix(NameOf(decl), "func") {
		return false
	}
	if decl.Type.Kind() == cc.Function {
		return true
	}
	return false
}
tu, err := Parse(Model(), "testdata/dummy.h")
if err != nil {
	panic(err)
}

var buf bytes.Buffer
if err := GenIgnored(&buf, tu, functions); err != nil {
	panic(err)
}
fmt.Println(buf.String())

Output:

var ignored = map[string]struct{}{
"func1i":{},
"func1f":{},
"func1fp":{},
"func2i":{},
"func2f":{},
"funcErr":{},
"funcCtx":{},
}

func GenNameMap 

func GenNameMap(buf io.Writer, t *cc.TranslationUnit, varname string, fn func(string) string, filter FilterFunc, init bool) error

GenNameMap generates go code representing a name mapping scheme

filename indicates the file to be parsed, varname indicates the name of the variable.

  • fn is the transformation function.
  • init indicates if the mapping should be generated in a func init(){}
Example 
functions := func(decl *cc.Declarator) bool {
	if !strings.HasPrefix(NameOf(decl), "func") {
		return false
	}
	if decl.Type.Kind() == cc.Function {
		return true
	}
	return false
}

trans := func(a string) string {
	return strings.ToTitle(strings.TrimPrefix(a, "func"))
}
tu, err := Parse(Model(), "testdata/dummy.h")
if err != nil {
	panic(err)
}

var buf bytes.Buffer
if err := GenNameMap(&buf, tu, "m", trans, functions, false); err != nil {
	panic(err)
}
fmt.Println(buf.String())

Output:

var m = map[string]string{
"func1i": "1I",
"func1f": "1F",
"func1fp": "1FP",
"func2i": "2I",
"func2f": "2F",
"funcErr": "ERR",
"funcCtx": "CTX",
}

func IsConstType 

func IsConstType(a cc.Type) bool

IsConstType returns true if the C-type is specified with a `const`

func IsPointer 

func IsPointer(a cc.Type) bool

IsPointer returns true if the C-type is specified as a pointer

func IsVoid 

func IsVoid(a cc.Type) bool

IsVoid returns true if the C type is

func LongestCommonPrefix 

func LongestCommonPrefix(strs ...string) string

LongestCommonPrefix takes a slice of strings, and finds the longest common prefix

This function was taken from github.com/chewxy/lingo/corpus

func Model 

func Model() *cc.Model

func NameOf 

func NameOf(any interface{}) (name string)

NameOf returns the name of a C declarator

func Parse 

func Parse(model *cc.Model, paths ...string) (*cc.TranslationUnit, error)

Parse parses with the given model, as well as having some hard coded predefined definitions that are useful for translating C to Go code

func Snake2Camel 

func Snake2Camel(s string, exported bool) (retVal string)

Snake2Camel converts snake case to camel case. It's not particularly performant. Rather it's a quick and dirty function.

func TypeDefOf 

func TypeDefOf(t cc.Type) (name string)

TypeDefOf returns the type def name of a type. If a type is not a typedef'd type, it returns "".

Types

type CSignature 

type CSignature struct {
	Pos         token.Pos
	Name        string
	Return      cc.Type
	CParameters []cc.Parameter
	Variadic    bool
	Declarator  *cc.Declarator
}

CSignature is a description of a C declaration.

func (*CSignature) Decl 

func (d *CSignature) Decl() *cc.Declarator

func (*CSignature) Parameters 

func (d *CSignature) Parameters() []Parameter

Parameters returns the declaration's CParameters converted to a []Parameter.

func (*CSignature) Position 

func (d *CSignature) Position() token.Position

Position returns the token position of the declaration.

type Declaration 

type Declaration interface {
	Position() token.Position
	Decl() *cc.Declarator
}

Declaration is anything with a position

func Get 

func Get(t *cc.TranslationUnit, filter FilterFunc) ([]Declaration, error)

Get returns a list of declarations given the filter function

type Enum 

type Enum struct {
	Pos        token.Pos
	Name       string
	Type       cc.Type
	Declarator *cc.Declarator
}

Enum is a description of a C enum

func (*Enum) Decl 

func (d *Enum) Decl() *cc.Declarator

func (*Enum) Position 

func (d *Enum) Position() token.Position

type FilterFunc 

type FilterFunc func(*cc.Declarator) bool

FilterFunc is a function to filter types

type Namer 

type Namer interface {
	Name() string
}

type Other 

type Other struct {
	Pos        token.Pos
	Name       string
	Declarator *cc.Declarator
}

Other represents other types that are not part of the "batteries included"ness of this package

func (*Other) Decl 

func (d *Other) Decl() *cc.Declarator

func (*Other) Position 

func (d *Other) Position() token.Position

type ParamKey 

type ParamKey struct {
	Name string
	Type TypeKey
}

ParamKey is a representtive of a param

type Parameter 

type Parameter struct {
	cc.Parameter
	TypeDefName string // can be empty
}

Parameter is a C function parameter.

func (*Parameter) Elem 

func (p *Parameter) Elem() cc.Type

Elem returns the pointer type of a pointer parameter or the element type of an array parameter.

func (*Parameter) IsPointer 

func (p *Parameter) IsPointer() bool

IsPointer returns true if the parameter represents a pointer

func (*Parameter) Kind 

func (p *Parameter) Kind() cc.Kind

Kind returns the C kind of the parameter.

func (*Parameter) Name 

func (p *Parameter) Name() string

Name returns the name of the parameter.

func (*Parameter) Type 

func (p *Parameter) Type() cc.Type

Type returns the C type of the parameter.

type Template 

type Template struct {
	*template.Template
	InContext func() bool
}

Template represents a template of conversion. An optional InContext() function may be provided to check if the template needs to be executed

func Pure 

func Pure(any interface{}) Template

Pure "lifts" a string or *template.Template into a template

type TypeKey 

type TypeKey struct {
	IsPointer bool
	Kind      cc.Kind
	Name      string
}

TypeKey is typically used as a representation of a C type that can be used as a key in a map

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.