tl

package module
v0.0.0-...-98e7a0c Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 11, 2024 License: MIT Imports: 16 Imported by: 2

README

TypeLang


Full native implementation of TL serializing language. Simple and fast.

english русский

🤔 What is it?

TL is a serialization format, created by Telegram and TON devs. This format slightly looks like Protobuf, but has a lot of exotic differences, which creates some issues with working about this format. This package solve a lot of stuff and handle 90% of your routine, related to serialization (and trust me, this routine could be a pain, that's why we created xelaj/tl).

✨ Features

  • Probably the best golang package for parsing TL schemas
  • Auto code generation of implementation your spec
  • Supports TL-B for TON Blockchain
  • And many many more 😲

👨‍💻 How to use

Here are some usage cases, how you can do some stuff:

Basic usage
package main

import "github.com/xelaj/tl"

type RepoUser struct {
    _       struct{} `tl:"someBitflag,bitflag"`
    Nick    string
    Creator bool `tl:",omitempty:someBitflag:0,implicit"`
    Editor  bool `tl:",omitempty:someBitflag:7,implicit"`
}

func (c *RepoUser) CRC() uint32 { return 0x12345678 }

func main() {
    tl.RegisterObjects((*RepoUser)(nil))

    response := &RepoUser{
        Nick:    "Hello user!",
        Creator: true,
        Editor:  true,
    }

    println(tl.Marshal(response))
    // And boom! 💥 You have decoded object!
}

Run in playground

Run codegen
🔘 $ brew install tlgen

     Processing... Done!

🟢 $ cat schema.tl | tlgen generate \
        --package-name="github.com/username/tonlib" \
        --output="./path/to/generate" \
        --file-prefix="tl" \      # ➡ tl_stuff_gen.go or tl_gen.go
        --split-files             # ➡ split by 4 files depend on object type

🟢 $ ls ./path/to/generate

     drwxr-xr-x+   4 username  staff   3.3K Oct 11 2049 .
     drwxr-xr-x  104 username  staff   224B Oct 11 2049 ..
     -rw-rw-r--    4 username  staff   4.1K Oct 11 2049 tl_enums_gen.go
     -rw-rw-r--    4 username  staff   900B Oct 11 2049 tl_init_gen.go
     -rw-rw-r--    4 username  staff   1.8K Oct 11 2049 tl_interfaces_gen.go
     -rw-rw-r--    4 username  staff   3.2K Oct 11 2049 tl_methods_gen.go

🟢 $ # Tadah! You have multiple files, but you can combine them into single one!

View demo

Parse messages without codegen

TODO

Something else

Code examples are here

Full docs are here

⌚️ Getting started

TODO

Why TL serializing is SO ugly?

Well... Read this issue about TON source code, when it was released first time. Remember: current TON developers are absolutely another guys. Use google translate, this issue will answer to all your questions.

Telegram devs are really strange

🤔 Differences between TL and TL-B

Вы наверняка заметили, что логотип этого проекта это на самом деле два логотипа — telegram и ton. Однако, формат в ton обратно несовместим с тем же форматом в telegram. При этом, между ними очень много общего, поэтому вцелом довольно реально объеденить оба формата в одну библиотеку. При этом, в отличии от разработчиков telegram, разработчики ton более благосклонны к кооперации, так что вполне возможно, что поддержка ton будет даже более качественной

🦊 Who use it

💎 Contributing

Please read contributing guide if you want to help. And the help is very necessary!

Don't want code? Read this page! We love nocoders!

🐛 Security bugs?

Please, don't create issue which describes security bug, this can be too offensive! Instead, please read this notification and follow that steps to notify us about problem.

🏋️ TODO

  • Full implementation of Tl serializing
  • Tool to generate golang code
  • Write amazing docs
  • Create full toolset to support TL
  • make support of TL-B (incompatible version, used by TON)
  • Write RFC specification?
  • write good security policy
  • setup taskfile for markdown linter

📒 Running project scripts

This project uses go-task, it's not important to understand, what's going on, (since you can just see into Taskfile.yaml and see all commands). For better experience, you can download go-task and run tasks e.g. via $ task <taskname>. All tasks can be shown via $ task --list-all

👨‍👩‍👧‍👦 Authors

📝 License

This project is licensed under the MIT License - see the LICENSE file for details

Если вы находитесь в россии, или как-либо связаны с российским правительством, (например, являетесь российским налогоплательщиком) на вас распостраняется отдельная лицензия.

One important thing

Even that maintainers of this project are generally from russia, we still stand up with Ukraine, and from beginning of war, decided to stop paying any taxes, or cooperate in any case with government, and companies, connected with government. This is absolutely nothing compared to how much pain putin brought to the fraternal country. And we are responsible for our inaction, and the only thing we can do is to take at least any actions that harm putin’s regime, and help the victims of regime using all resources available for us.

Created with love 💜 and magic 🦄
Xelaj Software, 2022-2024

Documentation

Index

Constants

View Source 
const (
	ErrUnexpectedNil = errorConst("unexpected nil value")
	ErrImplicitInt   = errorConst("value must be converted to int32, int64 or uint32 explicitly")

	ErrBitflagTooHigh   = errorConst("trigger bit is more than 32")
	ErrImplicitNoTarget = errorConst(implicitFlag + " defined without target field to trigger")
	ErrImplicitBitflag  = errorConst("'" + implicitFlag + "' and '" + isBitflagFlag + "' can't be combined")
	ErrTagNameEmpty     = errorConst("tag name is empty")
	ErrInvalidTagOption = errorConst("invalid option")
	ErrInvalidTagFormat = errorConst("invalid tag format")
	ErrTypeIsEmpty      = errorConst("tl field type is empty")
)
View Source 
const (
	MapCrcKey  = "_crc"
	MapTypeKey = "_type"

	WordLen   = 32 / bitsInByte  // length of word in tl is 32 bits
	LongLen   = 64 / bitsInByte  // int64 size in bytes
	DoubleLen = 64 / bitsInByte  // float64 size in bytes
	Int128Len = 128 / bitsInByte // int128 size in bytes
	Int256Len = 256 / bitsInByte // int256 size in bytes

)

Variables

View Source 
var (
	BitflagType      = fieldBitflags{}
	ImplicitBoolType = fieldImplicitBool{}
)

Functions

func GetFieldTypeFromGoType 

func GetFieldTypeFromGoType(t reflect.Type) fieldType

func Marshal 

func Marshal(v any) ([]byte, error)

func RegisterCustom 

func RegisterCustom[T Object](r *ObjectRegistry, constructor func(uint32) T, crcs ...uint32)

func RegisterCustomDefault 

func RegisterCustomDefault[T Object](constructor func(uint32) T, crcs ...uint32)

func RegisterEnum 

func RegisterEnum[T Object](r *ObjectRegistry, enums ...T)

func RegisterEnumDefault 

func RegisterEnumDefault[T Object](enums ...T)

func RegisterObject 

func RegisterObject[T Object](r *ObjectRegistry)

func RegisterObjectDefault 

func RegisterObjectDefault[T Object]()

func Unmarshal 

func Unmarshal(b []byte, res any) error

Types

type Bitflag 

type Bitflag struct {
	TargetField string
	BitPosition uint8
}

func (*Bitflag) String 

func (t *Bitflag) String() string

type BitflagBit 

type BitflagBit struct {
	FieldIndex int // в какое поле пихать бит что поле существует
	BitIndex   int // собственно в какой бит пихать флаг что все ок
}

type Decoder 

type Decoder interface {
	SetRegistry(registry Registry) Decoder

	Decode(res any) error
}

func NewDecoder 

func NewDecoder(r io.Reader) Decoder

NewDecoder returns a new decoder that reads from r.

func NewDecoderWithSize 

func NewDecoderWithSize(r io.Reader, bufSize int) Decoder

NewDecoderWithSize works absolutely like NewDecoder, but it sets buffer with size that you want. It could be useful, when you want to debug serialized data. You can combine this constructor with DumpWithoutRead method, which returns all data, until cache of bufio will be full, or until reader reach end of file.

To make correct cache, in most of cases, you will get length of all serialized message. pass this length and few more bytes (50-60 bytes will be enough). Note that DumpWithoutRead can't guarantee to be stable, so use it only for debugging.

type ErrInvalidBoolCRC 

type ErrInvalidBoolCRC crc32

func (ErrInvalidBoolCRC) Error 

func (e ErrInvalidBoolCRC) Error() string

type ErrInvalidCRC 

type ErrInvalidCRC struct {
	Got  crc32
	Want crc32
}

func (ErrInvalidCRC) Error 

func (e ErrInvalidCRC) Error() string

type ErrObjectNotRegistered 

type ErrObjectNotRegistered crc32

func (ErrObjectNotRegistered) Error 

func (e ErrObjectNotRegistered) Error() string

type ErrPartialWrite 

type ErrPartialWrite struct {
	Has  int
	Want int
}

func (ErrPartialWrite) Error 

func (e ErrPartialWrite) Error() string

type ErrPath 

type ErrPath struct {
	Err  error
	Path string
}

func (ErrPath) Error 

func (e ErrPath) Error() string

func (ErrPath) Unwrap 

func (e ErrPath) Unwrap() error

type ErrRegisteredObjectNotFound 

type ErrRegisteredObjectNotFound struct {
	Data []byte
	Crc  uint32
}

func (ErrRegisteredObjectNotFound) Error 

func (e ErrRegisteredObjectNotFound) Error() string

type ErrUnsupportedType 

type ErrUnsupportedType struct {
	Type reflect.Type
}

func (ErrUnsupportedType) Error 

func (e ErrUnsupportedType) Error() string

type Int128 

type Int128 struct{ *big.Int }

Int128 is alias-like type for fixed size of big int (1024 bit value). It using only for tl objects encoding cause native big.Int isn't supported for en(de)coding.

func NewInt128 

func NewInt128(value int) *Int128

NewInt128 creates int128 with zero value.

func RandomInt128 

func RandomInt128() *Int128

RandomInt128 creates int128 with random value.

func (*Int128) MarshalTL 

func (i *Int128) MarshalTL(s MarshalState) error

MarshalTL implements tl marshaler from this package. Just don't use it by your hands, tl.Encoder does all what you need.

func (*Int128) UnmarshalTL 

func (i *Int128) UnmarshalTL(s UnmarshalState) error

UnmarshalTL implements tl unmarshaler from this package. Just don't use it by your hands, tl.Decoder does all what you need.

type Int256 

type Int256 struct{ *big.Int }

Int256 is alias-like type for fixed size of big int (2048 bit value). It using only for tl objects encoding cause native big.Int isn't supported for en(de)coding.

func NewInt256 

func NewInt256(value int) *Int256

NewInt256 creates int256 with zero value.

func RandomInt256 

func RandomInt256() *Int256

RandomInt256 creates int256 with random value.

func (*Int256) MarshalTL 

func (i *Int256) MarshalTL(s MarshalState) error

MarshalTL implements tl marshaler from this package. Just don't use it by your hands, tl.Encoder does all what you need.

func (*Int256) UnmarshalTL 

func (i *Int256) UnmarshalTL(s UnmarshalState) error

UnmarshalTL implements tl unmarshaler from this package. Just don't use it by your hands, tl.Decoder does all what you need.

type MarshalState 

type MarshalState interface {
	// if object will write bytes the size of which is not divided by the length
	// of the word, it will throw specific error
	io.Writer

	PutBool(bool) error
	PutInt(int32) error
	PutLong(int64) error
	PutCRC(crc32) error
	PutMessage([]byte) error
}

MarshalState provides set of different methods to marshal binary message into specific struct. It's working absolutely like fmt.State, you just need to write data to this abstraction.

type Marshaler 

type Marshaler interface {
	MarshalTL(MarshalState) error
}

type Object 

type Object interface {
	CRC() crc32
}

Object is default interface, which ANY struct must implement to decode it in tl format.

type ObjectRegistry 

type ObjectRegistry struct {
	// contains filtered or unexported fields
}

ObjectRegistry is a type, which handles code generated schema, and could be useful for spawning TL objects. Unlike RawSchemaRegistry, it can work only with predefined go types.

If you are not able to use codegen, use RawSchemaRegistry, it could be slower, but more flexible.

func DefaultRegistry 

func DefaultRegistry() *ObjectRegistry

func NewRegistry 

func NewRegistry() *ObjectRegistry

func (*ObjectRegistry) ConstructObject 

func (r *ObjectRegistry) ConstructObject(crc crc32) (Object, bool)

ConstructObject spawns new object by crc code from registry.

type RealEncoder 

type RealEncoder interface {
	Encode(v any) error
}

func NewEncoder 

func NewEncoder(w io.Writer) RealEncoder

type Registry 

type Registry interface {
	ConstructObject(code crc32) (Object, bool)
}

type StructTag 

type StructTag struct {
	BitFlags *Bitflag
	Name     string
	Type     fieldType
}

func ParseStructTags 

func ParseStructTags(s Object) ([]StructTag, map[int]BitflagBit, error)

func ParseTag 

func ParseTag(tag, defaultName string) (StructTag, error)

ParseTag is a function which parses struct field tag for structes, defined by you.

  • `tag` parameter is a text representation of struct field tag.
  • `defaultName` is a name of struct field, if it's not defined in tag.

how tag could look: 

// field named somefield, it could be empty, value is boolean, so it can
// be encoded implicitly, value stored in some_flag value
`tl:"somefield,omitempty:some_flag:3,implicit"`

// value is abcd, it's required to exist in serialized
`tl:"abcd"`

func (StructTag) Ignore 

func (t StructTag) Ignore() bool

func (StructTag) String 

func (t StructTag) String() string

type UnmarshalState 

type UnmarshalState interface {
	// ReadWords reads words from unmarshaler with fixed size of word.
	io.Reader
	ReadAll() ([]byte, error)
	Peek(seek, size int) ([]byte, error)
	SkipBytes(int)

	PopBool() (bool, error)
	PopInt() (int32, error)
	PopLong() (int64, error)
	PopCRC() (crc32, error)
	PopMessage() ([]byte, error)
}

type Unmarshaler 

type Unmarshaler interface {
	UnmarshalTL(UnmarshalState) error
}

Source Files

View all Source files

Directories

Path Synopsis
cmd
tlgen Module
schema module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.