geocatalogo

README

geocatalogo

Overview

Geospatial Catalogue in Go

Installation

Requirements

geocatalogo's default backend is Elasticsearch and requires an Elasticsearch endpoint as defined in configuration.

# create directory for local env
mkdir /path/to/golang-env
export GOPATH=/path/to/golang-env
# install dependencies
go get golang.org/x/text/encoding
go get github.com/sirupsen/logrus
go get gopkg.in/yaml.v2
go get gopkg.in/olivere/elastic.v6/..
go get golang.org/x/lint/...
# install geocatalogo
go get github.com/go-spatial/geocatalogo/...
cd $GOPATH/src/github.com/go-spatial/geocatalogo
# set configuration
# (sample in $GOPATH/src/github.com/go-spatial/geocatalogo/geocatalogo-config.env)
cp geocatalogo-config.env local.env
vi local.env  # update accordingly
# GEOCATALOGO_SERVER_OPENAPI: path to OpenAPI Document
# GEOCATALOGO_SERVER_URL: URL of geocatalogo instance for serving via HTTP
# GEOCATALOGO_REPOSITORY_URL: URL to Elasticsearch
. local.env

Running

Using the geocatalogo command line utility

# list commands
geocatalogo

# index a metadata record
geocatalogo index --file=/path/to/record.xml

# index a directory of metadata records
geocatalogo index --dir=/path/to/dir

# dedicated importers

# Landsat on AWS (https://aws.amazon.com/public-datasets/landsat/)
curl http://landsat-pds.s3.amazonaws.com/c1/L8/scene_list.gz | gunzip > /tmp/scene_list
landsat-aws-importer --file /tmp/scene_list

# OpenAerialMap Catalog (https://docs.openaerialmap.org/catalog/)
curl "https://api.openaerialmap.org/meta?limit=5000" > /tmp/oam.json
oam-catalog-importer --file /tmp/scene_list

# search index
geocatalogo search --term=landsat

# search by bbox
geocatalogo search --bbox -152,42,-52,84

# search by time instant
geocatalogo search --time 2018-01-19T18:28:02Z

# search by time range
geocatalogo search --time 2007-11-11T12:43:29Z/2018-01-19T18:28:02Z

# search by collections
geocatalogo search --collections landsat8

# search by any combination exclusively (term, bbox, time)
geocatalogo search --time 2007-11-11T12:43:29Z/2018-01-19T18:28:02Z --bbox -152,42,-52,84 --term landsat

# get a metadata record by id
geocatalogo get --id=12345

# get a metadata record by list of ids
geocatalogo get --id=12345,67890

# run as an HTTP server (default port 8000)
geocatalogo serve
# run as an HTTP server on a custom port
geocatalogo serve --port 8001
# run as an HTTP server honouring the STAC API
geocatalogo serve --api stac

# get version
geocatalogo version

Using the API

// init a Geocatalogue from environment
import (
	"encoding/json"
	"fmt"
	"github.com/go-spatial/geocatalogo"
	"github.com/go-spatial/geocatalogo/metadata/parsers"
)

cat, err := geocatalogo.NewFromEnv()
if err != nil {
	fmt.Println(err)
}

// index a Dublin Core metadata record
source, err := ioutil.ReadFile(file)
if err != nil {
	fmt.Printf("Could not read file: %s\n", err)
}
metadataRecord, err := parsers.ParseCSWRecord(source)
if err != nil {
	fmt.Printf("Could not parse metadata: %s\n", err)
	continue
}
result := cat.Index(metadataRecord)
if !result {
	fmt.Println("Error Indexing")
}

// search records and present records 0 - 10
results := cat.Search("birds", 0, 10)

// get record by id
results := cat.Get("record-id-123")

// process results
for _, result := range results.Records {
	b, _ := json.MarshalIndent(result, "", "    ")
	fmt.Printf("%s\n", b)
}

Development

Running Tests

Releasing

Bugs and Issues

All bugs, enhancements and issues are managed on GitHub.

Contact

• Tom Kralidis

Documentation

Overview

Package geocatalogo provides the main interactions with the geospatial catalogue

Index

• Constants
• Variables
• func EmitResponse(c *GeoCatalogue, w http.ResponseWriter, code int, response []byte)
• func GetURL(url string) string
• func InitLog(cfg *config.Config, log *logrus.Logger) error
• func RenderTemplate(templateString string, data map[string]interface{}) ([]byte, error)
• func Struct2JSON(iface interface{}, prettyPrint bool) []byte
• type GeoCatalogue
  • func New(cfg *config.Config) (*GeoCatalogue, error)
  • func NewFromEnv() (*GeoCatalogue, error)
  • func (c *GeoCatalogue) Get(identifiers []string) search.Results
  • func (c *GeoCatalogue) Index(record metadata.Record) bool
  • func (c *GeoCatalogue) Search(collections []string, term string, bbox []float64, timeVal []time.Time, ...) search.Results
  • func (c *GeoCatalogue) UnIndex() bool

Constants

const VERSION string = "0.2-dev"

VERSION provides the geocatalogo version installed.

Variables

var LogLevels = map[string]logrus.Level{
	"DEBUG": logrus.DebugLevel,
	"INFO":  logrus.InfoLevel,
	"WARN":  logrus.WarnLevel,
	"ERROR": logrus.ErrorLevel,
	"FATAL": logrus.FatalLevel,
	"NONE":  logrus.PanicLevel,
}

LogLevels provides a mapping between configuration and logging level types

Functions

func EmitResponse

func EmitResponse(c *GeoCatalogue, w http.ResponseWriter, code int, response []byte)

EmitResponse provides HTTP response for successful requests

func GetURL

func GetURL(url string) string

GetURL downloads a URL

func InitLog

func InitLog(cfg *config.Config, log *logrus.Logger) error

InitLog initializes the logging facility

func RenderTemplate

func RenderTemplate(templateString string, data map[string]interface{}) ([]byte, error)

RenderTemplate generates template text

func Struct2JSON

func Struct2JSON(iface interface{}, prettyPrint bool) []byte

Struct2JSON generates a byte representation from a struct

Types

type GeoCatalogue

type GeoCatalogue struct {
	Config     config.Config
	Repository *repository.Elasticsearch
}

GeoCatalogue provides the core structure

func New

func New(cfg *config.Config) (*GeoCatalogue, error)

New provides the initializing functionality

func NewFromEnv

func NewFromEnv() (*GeoCatalogue, error)

NewFromEnv provides the initializing functionality using configuration from the environment

func (*GeoCatalogue) Get

func (c *GeoCatalogue) Get(identifiers []string) search.Results

Get retrieves a single metadata record from the Index

func (*GeoCatalogue) Index

func (c *GeoCatalogue) Index(record metadata.Record) bool

Index adds a metadata record to the Index

func (*GeoCatalogue) Search

func (c *GeoCatalogue) Search(collections []string, term string, bbox []float64, timeVal []time.Time, from int, size int) search.Results

Search performs a search/query against the Index

func (*GeoCatalogue) UnIndex

func (c *GeoCatalogue) UnIndex() bool

UnIndex removes a metadata record to the Index