sqlite

README

go-whosonfirst-iterate-sql

Go package implementing go-whosonfirst-iterate/emitter functionality for (Who's On First specific) database/sql databases.

Documentation

Example

import (
       "context"
       "fmt"
       "io"

       _ "github.com/mattn/go-sqlite3"
       _ "github.com/whosonfirst/go-whosonfirst-iterate-sql/v2"
       
       "github.com/whosonfirst/go-whosonfirst-iterate/v2/iterator"
)

func main() {

	ctx := context.Background()
     
	iter_cb := func(ctx context.Context, path string, r io.ReadSeeker, args ...interface{}) error {
		fmt.Println(path)
		return nil
	}

	iter, _ := iterator.NewIterator(ctx, "sql://sqlite3", iter_cb)

	iter.IterateURIs(ctx, "whosonfirst.db")
}	

Tools

$> make cli
go build -mod vendor -o bin/count cmd/count/main.go
go build -mod vendor -o bin/emit cmd/emit/main.go

count

$> ./bin/count /usr/local/data/sfomuseum-data-flights-2020-latest.db
2021/02/18 10:57:42 time to index paths (1) 1m46.893753865s
2021/02/18 10:57:42 Counted 752289 records (saw 752289 records)

emit

$> ./bin/emit \
	-geojson \
	-emitter-uri 'sql://sqlite3?include=properties.icao:airline=ANZ' \
	/usr/local/data/sfomuseum-data-flights-2020-latest.db

| jq '.features[]["properties"]["wof:name"]'

"NZ9198 (SFO-IAH)"
"NZ9147 (BFL-SFO)"
"NZ9130 (EWR-SFO)"
"NZ9225 (SFO-LAS)"
"NZ9525 (SFO-MFR)"
"NZ9340 (SFO-PDX)"
"NZ9716 (SNA-SFO)"
"NZ9124 (EWR-SFO)"
"NZ9131 (MIA-SFO)"
"NZ9282 (LAX-SFO)"
"NZ9353 (PDX-SFO)"
"NZ9238 (LAS-SFO)
... and so on

See also

• https://github.com/whosonfirst/go-whosonfirst-iterate
• https://pkg.go.dev/database/sql

Documentation

Overview

Package sqlite implements the `whosonfirst/go-whosonfirst-iterate/v2` interfaces for iterating (Who's On First) documents stored in a SQLite database.

Index

• func NewSQLiteEmitter(ctx context.Context, uri string) (emitter.Emitter, error)
• type SQLiteEmitter
  • func (d *SQLiteEmitter) WalkURI(ctx context.Context, emitter_cb emitter.EmitterCallbackFunc, uri string) error

Functions

func NewSQLiteEmitter

func NewSQLiteEmitter(ctx context.Context, uri string) (emitter.Emitter, error)

NewGitEmitter() returns a new `GitEmitter` instance configured by 'uri' in the form of:

sqlite://?{PARAMETERS}

{PARAMETERS} may be: * `?include=` Zero or more `aaronland/go-json-query` query strings containing rules that must match for a document to be considered for further processing. * `?exclude=` Zero or more `aaronland/go-json-query` query strings containing rules that if matched will prevent a document from being considered for further processing. * `?include_mode=` A valid `aaronland/go-json-query` query mode string for testing inclusion rules. * `?exclude_mode=` A valid `aaronland/go-json-query` query mode string for testing exclusion rules. * `?processes=` An optional number assigning the maximum number of database rows that will be processed simultaneously. (Default is defined by `runtime.NumCPU()`.)

Types

type SQLiteEmitter

type SQLiteEmitter struct {
	emitter.Emitter
	// contains filtered or unexported fields
}

SQLiteEmitter implements the `Emitter` interface for crawling records in a SQLite database (specifically a SQLite database with a 'geojson' table produced by `whosonfirst/go-whosonfirst-sqlite-features` and `whosonfirst/go-whosonfirst-sqlite-features-index`).

func (*SQLiteEmitter) WalkURI

func (d *SQLiteEmitter) WalkURI(ctx context.Context, emitter_cb emitter.EmitterCallbackFunc, uri string) error

WalkURI() walks (crawls) the SQLite database identified by 'uri' and for each file (not excluded by any filters specified when `idx` was created) invokes 'index_cb'.