Documentation ¶
Overview ¶
package index provides methods for bulk indexing collections of Who's On First documents in Elasticsearch.
Index ¶
- Constants
- func BulkIndexerFromFlagSet(ctx context.Context, fs *flag.FlagSet) (esutil.BulkIndexer, error)
- func NewBulkIndexerFlagSet(ctx context.Context) (*flag.FlagSet, error)
- func PrepareFuncsFromFlagSet(ctx context.Context, fs *flag.FlagSet) ([]document.PrepareDocumentFunc, error)
- func RunBulkIndexer(ctx context.Context, opts *RunBulkIndexerOptions) (*esutil.BulkIndexerStats, error)
- func RunBulkIndexerWithFlagSet(ctx context.Context, fs *flag.FlagSet) (*esutil.BulkIndexerStats, error)
- func RunES2BulkIndexerWithFlagSet(ctx context.Context, fs *flag.FlagSet) (*es.BulkProcessorStats, error)
- type RunBulkIndexerOptions
Constants ¶
const FLAG_APPEND_SPELUNKER_V1 string = "append-spelunker-v1-properties"
const FLAG_ES_ENDPOINT string = "elasticsearch-endpoint"
const FLAG_ES_INDEX string = "elasticsearch-index"
const FLAG_INDEX_ALT string = "index-alt-files"
const FLAG_INDEX_PROPS string = "index-only-properties"
const FLAG_INDEX_SPELUNKER_V1 string = "index-spelunker-v1"
const FLAG_ITERATOR_URI string = "iterator-uri"
const FLAG_WORKERS string = "workers"
Variables ¶
This section is empty.
Functions ¶
func BulkIndexerFromFlagSet ¶ added in v0.5.0
BulkIndexerFromFlagSet returns a esutil.BulkIndexer instance derived from the values in 'fs'.
func NewBulkIndexerFlagSet ¶
NewBulkIndexerFlagSet creates a new `flag.FlagSet` instance with command-line flags required by the `es-whosonfirst-index` tool.
func PrepareFuncsFromFlagSet ¶ added in v0.5.0
func PrepareFuncsFromFlagSet(ctx context.Context, fs *flag.FlagSet) ([]document.PrepareDocumentFunc, error)
PrepareFuncsFromFlagSet returns a list of zero or more known `document.PrepareDocumentFunc` functions based on the values in 'fs'.
func RunBulkIndexer ¶ added in v0.5.0
func RunBulkIndexer(ctx context.Context, opts *RunBulkIndexerOptions) (*esutil.BulkIndexerStats, error)
RunBulkIndexer will "bulk" index a set of Who's On First documents with configuration details defined in 'opts'.
func RunBulkIndexerWithFlagSet ¶
func RunBulkIndexerWithFlagSet(ctx context.Context, fs *flag.FlagSet) (*esutil.BulkIndexerStats, error)
RunBulkIndexerWithFlagSet will "bulk" index a set of Who's On First documents with configuration details defined in 'fs'.
func RunES2BulkIndexerWithFlagSet ¶ added in v0.4.0
func RunES2BulkIndexerWithFlagSet(ctx context.Context, fs *flag.FlagSet) (*es.BulkProcessorStats, error)
RunBulkIndexerWithFlagSet will "bulk" index a set of Who's On First documents with configuration details defined by `fs`.
Types ¶
type RunBulkIndexerOptions ¶ added in v0.5.0
type RunBulkIndexerOptions struct { // BulkIndexer is a `esutil.BulkIndexer` instance BulkIndexer esutil.BulkIndexer // PrepareFuncs are one or more `document.PrepareDocumentFunc` used to transform a document before indexing PrepareFuncs []document.PrepareDocumentFunc // IteratorURI is a valid `whosonfirst/go-whosonfirst-iterate/v2` URI string. IteratorURI string // IteratorPaths are one or more valid `whosonfirst/go-whosonfirst-iterate/v2` paths to iterate over IteratorPaths []string // IndexAltFiles is a boolean value indicating whether or not to index "alternate geometry" files IndexAltFiles bool }
type RunBulkIndexerOptions contains runtime configurations for bulk indexing
func RunBulkIndexerOptionsFromFlagSet ¶ added in v0.5.0
func RunBulkIndexerOptionsFromFlagSet(ctx context.Context, fs *flag.FlagSet) (*RunBulkIndexerOptions, error)
RunBulkIndexerOptionsFromFlagSet returns a `RunBulkIndexerOptions` instance derived from the values in 'fs'.