lines

Documentation

Overview

Package lines provides a virtual table to read data line-by-line.

It is particularly useful for line-oriented datasets, like ndjson or JSON Lines, when paired with SQLite's JSON support.

https://github.com/asg017/sqlite-lines

Example

db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
	lines.Register(c)
	return nil
})
if err != nil {
	log.Fatal(err)
}
defer db.Close()

res, err := http.Get("https://storage.googleapis.com/quickdraw_dataset/full/simplified/calendar.ndjson")
if err != nil {
	log.Fatal(err)
}
defer res.Body.Close()

rows, err := db.Query(`
		SELECT
			line ->> '$.countrycode' as countrycode,
			COUNT(*)
  		FROM lines_read(?)
  		GROUP BY 1
  		ORDER BY 2 DESC
  		LIMIT 5`,
	sqlite3.Pointer(res.Body))
if err != nil {
	log.Fatal(err)
}
defer rows.Close()

var countrycode sql.RawBytes
var count int
for rows.Next() {
	err := rows.Scan(&countrycode, &count)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("%s: %d\n", countrycode, count)
}
if err := rows.Err(); err != nil {
	log.Fatal(err)
}
// Expected output:
// US: 141001
// GB: 22560
// CA: 11759
// RU: 9250
// DE: 8748

Index

• func Register(db *sqlite3.Conn)
• func RegisterFS(db *sqlite3.Conn, fsys fs.FS)

Examples

• Package

Functions

func Register

func Register(db *sqlite3.Conn)

Register registers the lines and lines_read table-valued functions. The lines function reads from a database blob or text. The lines_read function reads from a file or an io.Reader. If a filename is specified, os.Open is used to open the file.

func RegisterFS

func RegisterFS(db *sqlite3.Conn, fsys fs.FS)

RegisterFS registers the lines and lines_read table-valued functions. The lines function reads from a database blob or text. The lines_read function reads from a file or an io.Reader. If a filename is specified, fsys is used to open the file.