tzf

package module
v0.13.1-rc2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 28, 2023 License: MIT Imports: 15 Imported by: 12

README

TZF: a fast timezone finder for Go. Go Reference codecov

Note: This package is also available in:

Quick Start

Go
// Use about 150MB memory for init, and 60MB after GC.
package main

import (
	"fmt"

	"github.com/ringsaturn/tzf"
)

func main() {
	finder, err := tzf.NewDefaultFinder()
	if err != nil {
		panic(err)
	}
	fmt.Println(finder.GetTimezoneName(116.6386, 40.0786))
}

If you need 100% accurate query result, use below to got a finder:

// Use about 900MB memory for init, and 660MB after GC.
package main

import (
	"fmt"

	"github.com/ringsaturn/tzf"
	tzfrel "github.com/ringsaturn/tzf-rel"
	"github.com/ringsaturn/tzf/pb"
	"google.golang.org/protobuf/proto"
)

func main() {
	input := &pb.Timezones{}

	// Full data, about 83.5MB
	dataFile := tzfrel.FullData

	if err := proto.Unmarshal(dataFile, input); err != nil {
		panic(err)
	}
	finder, _ := tzf.NewFinderFromPB(input)
	fmt.Println(finder.GetTimezoneName(116.6386, 40.0786))
}
CLI Tool
go install github.com/ringsaturn/tzf/cmd/tzf@latest
tzf -lng 116.3883 -lat 39.9289

Data

Original data download from https://github.com/evansiroky/timezone-boundary-builder .

Preprocessed probuf data can get from https://github.com/ringsaturn/tzf-rel which has Go's embed support. Those files are Protocol Buffers messages for more efficiency binary distribution like Python wheels, you can view the pb/tzinfo.proto or it's HTML format docs for the internal format info.

tzf's data pipeline can be drew as:

graph TD
    Raw[GeoJSON from evansiroky/timezone-boundary-builder]
    Full[Full: Probuf based data]
    Lite[Lite: smaller of Full data]
    Compressed[Compressed: Lite compressed via Polyline]
    Preindex[Tile based data]

    Finder[Finder: Polygon Based Finder]
    FuzzyFinder[FuzzyFinder: Tile based Finder]
    DefaultFinder[DefaultFinder: combine FuzzyFinder and Compressed Finder]

    Raw --> |cmd/geojson2tzpb|Full
    Full --> |cmd/reducetzpb|Lite
    Lite --> |cmd/compresstzpb|Compressed
    Lite --> |cmd/preindextzpb|Preindex

    Full --> |tzf.NewFinderFromPB|Finder
    Lite --> |tzf.NewFinderFromPB|Finder
    Compressed --> |tzf.NewFinderFromCompressed|Finder --> |tzf.NewDefaultFinder|DefaultFinder
    Preindex --> |tzf.NewFuzzyFinderFromPB|FuzzyFinder --> |tzf.NewDefaultFinder|DefaultFinder

The full data(~80MB) could work anywhere but requires more memory usage.

The lite data(~10MB) doesn't work well in some edge places.

You can see points that results diff in this page.

If a little longer init time is acceptable, the compressed data(~5MB) which come from lite data will be more friendly for binary distribution.

The preindex data(~1.78MB) are many tiles. It's used inside the DefaultFinder, which built on FuzzyFinder, to reduce raycasting algorithm execution times.

I wrote an article about the history of tzf, it's Rust port, and it's Rust port' Python binding, view it here.

Performance

Package tzf is designed for high performance geo queries related services like weather forecast API. And most queries could return in very limited time, averagely like 2000 nanoseconds.

Here is what have been done for performance improvements:

  1. Use preindexes to handle most queries, basically about 1000 nanoseconds
  2. Use RTree to filter candidate polygons instead of iter all polygons to reduce Ray Casting algorithm execution times
  3. Use a fine tuned Ray Casting algorithm package https://github.com/tidwall/geojson to check if polygon contain point

That's all. There is no black magics inside package tzf.

Benchmark run version https://github.com/ringsaturn/tzf/releases/tag/v0.10.0

goos: darwin
goarch: amd64
pkg: github.com/ringsaturn/tzf
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
BenchmarkDefaultFinder_GetTimezoneName_Random_WorldCities-16              441309              2778 ns/op              1000 ns/p50            10000 ns/p90            19000 ns/p99
BenchmarkFuzzyFinder_GetTimezoneName_Random_WorldCities-16               1000000              1077 ns/op              1000 ns/p50             2000 ns/p90             2000 ns/p99
BenchmarkGetTimezoneName-16                                               226834              5190 ns/op              5000 ns/p50             5000 ns/p90            22000 ns/p99
BenchmarkGetTimezoneNameAtEdge-16                                         211555              5606 ns/op              5000 ns/p50             6000 ns/p90            23000 ns/p99
BenchmarkGetTimezoneName_Random_WorldCities-16                            163000              7279 ns/op              7000 ns/p50            10000 ns/p90            29000 ns/p99
BenchmarkFullFinder_GetTimezoneName-16                                    212896              5556 ns/op              5000 ns/p50             6000 ns/p90            22000 ns/p99
BenchmarkFullFinder_GetTimezoneNameAtEdge-16                              195381              6262 ns/op              6000 ns/p50             7000 ns/p90            23000 ns/p99
BenchmarkFullFinder_GetTimezoneName_Random_WorldCities-16                 116652              9354 ns/op              8000 ns/p50            15000 ns/p90            31000 ns/p99
PASS
ok      github.com/ringsaturn/tzf       18.321s

Thanks

Documentation

Overview

Package tzf is a package convert (lng,lat) to timezone.

Inspired by timezonefinder https://github.com/jannikmi/timezonefinder, fast python package for finding the timezone of any point on earth (coordinates) offline.

Index

Examples

Constants

This section is empty.

Variables

View Source 
var ErrNoTimezoneFound = errors.New("tzf: no timezone found")

Functions

func SetDropPBTZ added in v0.9.2 

func SetDropPBTZ(opt *Option)

SetDropPBTZ will make Finder not save github.com/ringsaturn/tzf/pb.Timezone in memory

Types

type DefaultFinder added in v0.9.0 

type DefaultFinder struct {
	// contains filtered or unexported fields
}

DefaultFinder is a finder impl combine both FuzzyFinder and Finder.

It's designed for performance first and allow some not so correct return at some area.

func (*DefaultFinder) DataVersion added in v0.13.0 

func (f *DefaultFinder) DataVersion() string

func (*DefaultFinder) GetTimezoneName added in v0.9.0 

func (f *DefaultFinder) GetTimezoneName(lng float64, lat float64) string
Example 
package main

import (
	"fmt"

	"github.com/ringsaturn/tzf"
)

func main() {
	finder, err := tzf.NewDefaultFinder()
	if err != nil {
		panic(err)
	}
	fmt.Println(finder.GetTimezoneName(116.6386, 40.0786))
}

Output:

Asia/Shanghai

func (*DefaultFinder) GetTimezoneNames added in v0.10.0 

func (f *DefaultFinder) GetTimezoneNames(lng float64, lat float64) ([]string, error)
Example 
package main

import (
	"fmt"

	"github.com/ringsaturn/tzf"
)

func main() {
	finder, err := tzf.NewDefaultFinder()
	if err != nil {
		panic(err)
	}
	fmt.Println(finder.GetTimezoneNames(87.6168, 43.8254))
}

Output:

[Asia/Shanghai Asia/Urumqi] <nil>

func (*DefaultFinder) TimezoneNames added in v0.9.0 

func (f *DefaultFinder) TimezoneNames() []string
Example 
package main

import (
	"fmt"

	"github.com/ringsaturn/tzf"
)

func main() {
	finder, err := tzf.NewDefaultFinder()
	if err != nil {
		panic(err)
	}
	fmt.Println(finder.TimezoneNames())
}

Output:

type F added in v0.12.0 

type F interface {
	GetTimezoneName(lng float64, lat float64) string
	GetTimezoneNames(lng float64, lat float64) ([]string, error)
	TimezoneNames() []string
	DataVersion() string
}

func NewDefaultFinder added in v0.9.0 

func NewDefaultFinder() (F, error)

func NewFinderFromCompressed added in v0.8.0 

func NewFinderFromCompressed(input *pb.CompressedTimezones, opts ...OptionFunc) (F, error)

func NewFinderFromPB 

func NewFinderFromPB(input *pb.Timezones, opts ...OptionFunc) (F, error)

func NewFinderFromRawJSON added in v0.2.0 

func NewFinderFromRawJSON(input *convert.BoundaryFile, opts ...OptionFunc) (F, error)

func NewFuzzyFinderFromPB added in v0.9.0 

func NewFuzzyFinderFromPB(input *pb.PreindexTimezones) (F, error)

type Finder 

type Finder struct {
	// contains filtered or unexported fields
}

Finder is based on point-in-polygon search algo.

Memeory will use about 100MB if lite data and 1G if full data. Performance is very stable and very accuate.

func (*Finder) DataVersion added in v0.13.0 

func (f *Finder) DataVersion() string

func (*Finder) GetTimezoneName 

func (f *Finder) GetTimezoneName(lng float64, lat float64) string

GetTimezoneName will use alphabet order and return first matched result.

Example 
package main

import (
	"fmt"

	"github.com/ringsaturn/tzf"

	tzfrel "github.com/ringsaturn/tzf-rel"
	"github.com/ringsaturn/tzf/pb"
	"google.golang.org/protobuf/proto"
)

func main() {
	input := &pb.Timezones{}
	if err := proto.Unmarshal(tzfrel.LiteData, input); err != nil {
		panic(err)
	}
	finder, _ := tzf.NewFinderFromPB(input)
	fmt.Println(finder.GetTimezoneName(116.6386, 40.0786))
}

Output:

Asia/Shanghai

func (*Finder) GetTimezoneNames added in v0.10.0 

func (f *Finder) GetTimezoneNames(lng float64, lat float64) ([]string, error)
Example 
package main

import (
	"fmt"

	"github.com/ringsaturn/tzf"

	tzfrel "github.com/ringsaturn/tzf-rel"
	"github.com/ringsaturn/tzf/pb"
	"google.golang.org/protobuf/proto"
)

func main() {
	input := &pb.Timezones{}
	if err := proto.Unmarshal(tzfrel.LiteData, input); err != nil {
		panic(err)
	}
	finder, _ := tzf.NewFinderFromPB(input)
	fmt.Println(finder.GetTimezoneNames(87.6168, 43.8254))
}

Output:

[Asia/Shanghai Asia/Urumqi] <nil>

func (*Finder) TimezoneNames added in v0.6.2 

func (f *Finder) TimezoneNames() []string
Example 
package main

import (
	"fmt"

	"github.com/ringsaturn/tzf"

	tzfrel "github.com/ringsaturn/tzf-rel"
	"github.com/ringsaturn/tzf/pb"
	"google.golang.org/protobuf/proto"
)

func main() {
	input := &pb.Timezones{}
	if err := proto.Unmarshal(tzfrel.LiteData, input); err != nil {
		panic(err)
	}
	finder, _ := tzf.NewFinderFromPB(input)
	fmt.Println(finder.TimezoneNames())
}

Output:

type FuzzyFinder added in v0.9.0 

type FuzzyFinder struct {
	// contains filtered or unexported fields
}

FuzzyFinder use a tile map to store timezone name. Data are made by github.com/ringsaturn/tzf/cmd/preindextzpb which powerd by github.com/ringsaturn/tzf/preindex.PreIndexTimezones.

func (*FuzzyFinder) DataVersion added in v0.13.0 

func (f *FuzzyFinder) DataVersion() string

func (*FuzzyFinder) GetTimezoneName added in v0.9.0 

func (f *FuzzyFinder) GetTimezoneName(lng float64, lat float64) string
Example 
package main

import (
	"fmt"

	"github.com/ringsaturn/tzf"

	tzfrel "github.com/ringsaturn/tzf-rel"
	"github.com/ringsaturn/tzf/pb"
	"google.golang.org/protobuf/proto"
)

func main() {
	input := &pb.PreindexTimezones{}
	if err := proto.Unmarshal(tzfrel.PreindexData, input); err != nil {
		panic(err)
	}
	finder, _ := tzf.NewFuzzyFinderFromPB(input)
	fmt.Println(finder.GetTimezoneName(116.6386, 40.0786))
}

Output:

Asia/Shanghai

func (*FuzzyFinder) GetTimezoneNames added in v0.10.0 

func (f *FuzzyFinder) GetTimezoneNames(lng float64, lat float64) ([]string, error)
Example 
package main

import (
	"fmt"

	"github.com/ringsaturn/tzf"

	tzfrel "github.com/ringsaturn/tzf-rel"
	"github.com/ringsaturn/tzf/pb"
	"google.golang.org/protobuf/proto"
)

func main() {
	input := &pb.PreindexTimezones{}
	if err := proto.Unmarshal(tzfrel.PreindexData, input); err != nil {
		panic(err)
	}
	finder, _ := tzf.NewFuzzyFinderFromPB(input)
	fmt.Println(finder.GetTimezoneNames(87.6168, 43.8254))
}

Output:

[Asia/Shanghai Asia/Urumqi] <nil>

func (*FuzzyFinder) TimezoneNames added in v0.11.2 

func (f *FuzzyFinder) TimezoneNames() []string
Example 
package main

import (
	"fmt"

	"github.com/ringsaturn/tzf"

	tzfrel "github.com/ringsaturn/tzf-rel"
	"github.com/ringsaturn/tzf/pb"
	"google.golang.org/protobuf/proto"
)

func main() {
	input := &pb.PreindexTimezones{}
	if err := proto.Unmarshal(tzfrel.PreindexData, input); err != nil {
		panic(err)
	}
	finder, _ := tzf.NewFuzzyFinderFromPB(input)
	fmt.Println(finder.TimezoneNames())
}

Output:

type Option added in v0.9.2 

type Option struct {
	DropPBTZ bool
}

type OptionFunc added in v0.9.2 

type OptionFunc = func(opt *Option)

Source Files

View all Source files

Directories

Path Synopsis
cmd
checkboundaryrelease
comparetzpb
compresstzpb
CLI tool to reduce polygon filesize
 CLI tool to reduce polygon filesize
geojson2tzpb
CLI tool to convert GeoJSON based Timezone boundary to tzf's Probuf format.
 CLI tool to convert GeoJSON based Timezone boundary to tzf's Probuf format.
preindextzpb
CLI tool to preindex timezone shape.
 CLI tool to preindex timezone shape.
reducetzpb
CLI tool to reduce polygon filesize
 CLI tool to reduce polygon filesize
tzf
tzf-cli tool for local query.
 tzf-cli tool for local query.
Package preindex
 Package preindex
Package reduce could reduce Polygon size both polygon lines and float precise.
 Package reduce could reduce Polygon size both polygon lines and float precise.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.