Documentation ¶
Index ¶
- Variables
- func SetLog(l *log.Logger)
- type Classifier
- type DocumentMapping
- func (dm *DocumentMapping) AddFieldMapping(fm *FieldMapping)
- func (dm *DocumentMapping) AddFieldMappingsAt(property string, fms ...*FieldMapping)
- func (dm *DocumentMapping) AddSubDocumentMapping(property string, sdm *DocumentMapping)
- func (dm *DocumentMapping) UnmarshalJSON(data []byte) error
- func (dm *DocumentMapping) Validate(cache *registry.Cache, parentName string, ...) error
- type FieldMapping
- func NewBooleanFieldMapping() *FieldMapping
- func NewDateTimeFieldMapping() *FieldMapping
- func NewGeoPointFieldMapping() *FieldMapping
- func NewGeoShapeFieldMapping() *FieldMapping
- func NewIPFieldMapping() *FieldMapping
- func NewKeywordFieldMapping() *FieldMapping
- func NewNumericFieldMapping() *FieldMapping
- func NewTextFieldMapping() *FieldMapping
- func NewVectorFieldMapping() *FieldMapping
- type IndexMapping
- type IndexMappingImpl
- func (im *IndexMappingImpl) AddCustomAnalyzer(name string, config map[string]interface{}) error
- func (im *IndexMappingImpl) AddCustomCharFilter(name string, config map[string]interface{}) error
- func (im *IndexMappingImpl) AddCustomDateTimeParser(name string, config map[string]interface{}) error
- func (im *IndexMappingImpl) AddCustomTokenFilter(name string, config map[string]interface{}) error
- func (im *IndexMappingImpl) AddCustomTokenMap(name string, config map[string]interface{}) error
- func (im *IndexMappingImpl) AddCustomTokenizer(name string, config map[string]interface{}) error
- func (im *IndexMappingImpl) AddDocumentMapping(doctype string, dm *DocumentMapping)
- func (im *IndexMappingImpl) AnalyzeText(analyzerName string, text []byte) (analysis.TokenStream, error)
- func (im *IndexMappingImpl) AnalyzerNameForPath(path string) string
- func (im *IndexMappingImpl) AnalyzerNamed(name string) analysis.Analyzer
- func (im *IndexMappingImpl) DateTimeParserNamed(name string) analysis.DateTimeParser
- func (im *IndexMappingImpl) DefaultSearchField() string
- func (im *IndexMappingImpl) FieldAnalyzer(field string) string
- func (im *IndexMappingImpl) FieldMappingForPath(path string) FieldMapping
- func (im *IndexMappingImpl) MapDocument(doc *document.Document, data interface{}) error
- func (im *IndexMappingImpl) UnmarshalJSON(data []byte) error
- func (im *IndexMappingImpl) Validate() error
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( IndexDynamic = true StoreDynamic = true DocValuesDynamic = true // TODO revisit default? )
control the default behavior for dynamic fields (those not explicitly mapped)
var MappingJSONStrict = false
Functions ¶
Types ¶
type Classifier ¶
type Classifier interface {
Type() string
}
A Classifier is an interface describing any object which knows how to identify its own type. Alternatively, if a struct already has a Type field or method in conflict, one can use BleveType instead.
type DocumentMapping ¶
type DocumentMapping struct { Enabled bool `json:"enabled"` Dynamic bool `json:"dynamic"` Properties map[string]*DocumentMapping `json:"properties,omitempty"` Fields []*FieldMapping `json:"fields,omitempty"` DefaultAnalyzer string `json:"default_analyzer,omitempty"` // StructTagKey overrides "json" when looking for field names in struct tags StructTagKey string `json:"struct_tag_key,omitempty"` }
A DocumentMapping describes how a type of document should be indexed. As documents can be hierarchical, named sub-sections of documents are mapped using the same structure in the Properties field. Each value inside a document can be indexed 0 or more ways. These index entries are called fields and are stored in the Fields field. Entire sections of a document can be ignored or excluded by setting Enabled to false. If not explicitly mapped, default mapping operations are used. To disable this automatic handling, set Dynamic to false.
func NewDocumentDisabledMapping ¶
func NewDocumentDisabledMapping() *DocumentMapping
NewDocumentDisabledMapping returns a new document mapping that will not perform any indexing.
func NewDocumentMapping ¶
func NewDocumentMapping() *DocumentMapping
NewDocumentMapping returns a new document mapping with all the default values.
func NewDocumentStaticMapping ¶
func NewDocumentStaticMapping() *DocumentMapping
NewDocumentStaticMapping returns a new document mapping that will not automatically index parts of a document without an explicit mapping.
func (*DocumentMapping) AddFieldMapping ¶
func (dm *DocumentMapping) AddFieldMapping(fm *FieldMapping)
AddFieldMapping adds the provided FieldMapping for this section of the document.
Example ¶
// you can only add field mapping to those properties which already have a document mapping documentMapping := NewDocumentMapping() subDocumentMapping := NewDocumentMapping() documentMapping.AddSubDocumentMapping("Property", subDocumentMapping) fieldMapping := NewTextFieldMapping() fieldMapping.Analyzer = "en" subDocumentMapping.AddFieldMapping(fieldMapping) fmt.Println(len(documentMapping.Properties["Property"].Fields))
Output: 1
func (*DocumentMapping) AddFieldMappingsAt ¶
func (dm *DocumentMapping) AddFieldMappingsAt(property string, fms ...*FieldMapping)
AddFieldMappingsAt adds one or more FieldMappings at the named sub-document. If the named sub-document doesn't yet exist it is created for you. This is a convenience function to make most common mappings more concise. Otherwise, you would:
subMapping := NewDocumentMapping() subMapping.AddFieldMapping(fieldMapping) parentMapping.AddSubDocumentMapping(property, subMapping)
Example ¶
// you can only add field mapping to those properties which already have a document mapping documentMapping := NewDocumentMapping() subDocumentMapping := NewDocumentMapping() documentMapping.AddSubDocumentMapping("NestedProperty", subDocumentMapping) fieldMapping := NewTextFieldMapping() fieldMapping.Analyzer = "en" documentMapping.AddFieldMappingsAt("NestedProperty", fieldMapping) fmt.Println(len(documentMapping.Properties["NestedProperty"].Fields))
Output: 1
func (*DocumentMapping) AddSubDocumentMapping ¶
func (dm *DocumentMapping) AddSubDocumentMapping(property string, sdm *DocumentMapping)
AddSubDocumentMapping adds the provided DocumentMapping as a sub-mapping for the specified named subsection.
Example ¶
// adds a document mapping for a property in a document // useful for mapping nested documents documentMapping := NewDocumentMapping() subDocumentMapping := NewDocumentMapping() documentMapping.AddSubDocumentMapping("Property", subDocumentMapping) fmt.Println(len(documentMapping.Properties))
Output: 1
func (*DocumentMapping) UnmarshalJSON ¶
func (dm *DocumentMapping) UnmarshalJSON(data []byte) error
UnmarshalJSON offers custom unmarshaling with optional strict validation
func (*DocumentMapping) Validate ¶
func (dm *DocumentMapping) Validate( cache *registry.Cache, parentName string, fieldAliasCtx map[string]*FieldMapping, ) error
type FieldMapping ¶
type FieldMapping struct { Name string `json:"name,omitempty"` Type string `json:"type,omitempty"` // Analyzer specifies the name of the analyzer to use for this field. If // Analyzer is empty, traverse the DocumentMapping tree toward the root and // pick the first non-empty DefaultAnalyzer found. If there is none, use // the IndexMapping.DefaultAnalyzer. Analyzer string `json:"analyzer,omitempty"` // Store indicates whether to store field values in the index. Stored // values can be retrieved from search results using SearchRequest.Fields. Store bool `json:"store,omitempty"` Index bool `json:"index,omitempty"` // IncludeTermVectors, if true, makes terms occurrences to be recorded for // this field. It includes the term position within the terms sequence and // the term offsets in the source document field. Term vectors are required // to perform phrase queries or terms highlighting in source documents. IncludeTermVectors bool `json:"include_term_vectors,omitempty"` IncludeInAll bool `json:"include_in_all,omitempty"` DateFormat string `json:"date_format,omitempty"` // DocValues, if true makes the index uninverting possible for this field // It is useful for faceting and sorting queries. DocValues bool `json:"docvalues,omitempty"` // SkipFreqNorm, if true, avoids the indexing of frequency and norm values // of the tokens for this field. This option would be useful for saving // the processing of freq/norm details when the default score based relevancy // isn't needed. SkipFreqNorm bool `json:"skip_freq_norm,omitempty"` // Dimensionality of the vector Dims int `json:"dims,omitempty"` // Similarity is the similarity algorithm used for scoring // vector fields. // See: index.DefaultSimilarityMetric & index.SupportedSimilarityMetrics Similarity string `json:"similarity,omitempty"` // Applicable to vector fields only - optimization string VectorIndexOptimizedFor string `json:"vector_index_optimized_for,omitempty"` }
A FieldMapping describes how a specific item should be put into the index.
func NewBooleanFieldMapping ¶
func NewBooleanFieldMapping() *FieldMapping
NewBooleanFieldMapping returns a default field mapping for booleans
func NewDateTimeFieldMapping ¶
func NewDateTimeFieldMapping() *FieldMapping
NewDateTimeFieldMapping returns a default field mapping for dates
func NewGeoPointFieldMapping ¶
func NewGeoPointFieldMapping() *FieldMapping
NewGeoPointFieldMapping returns a default field mapping for geo points
func NewGeoShapeFieldMapping ¶
func NewGeoShapeFieldMapping() *FieldMapping
NewGeoShapeFieldMapping returns a default field mapping for geoshapes
func NewIPFieldMapping ¶
func NewIPFieldMapping() *FieldMapping
NewIPFieldMapping returns a default field mapping for IP points
func NewKeywordFieldMapping ¶
func NewKeywordFieldMapping() *FieldMapping
NewKeyworFieldMapping returns a default field mapping for text with analyzer "keyword".
func NewNumericFieldMapping ¶
func NewNumericFieldMapping() *FieldMapping
NewNumericFieldMapping returns a default field mapping for numbers
func NewTextFieldMapping ¶
func NewTextFieldMapping() *FieldMapping
NewTextFieldMapping returns a default field mapping for text
func NewVectorFieldMapping ¶
func NewVectorFieldMapping() *FieldMapping
func (*FieldMapping) Options ¶
func (fm *FieldMapping) Options() index.FieldIndexingOptions
Options returns the indexing options for this field.
func (*FieldMapping) UnmarshalJSON ¶
func (fm *FieldMapping) UnmarshalJSON(data []byte) error
UnmarshalJSON offers custom unmarshaling with optional strict validation
type IndexMapping ¶
type IndexMapping interface { MapDocument(doc *document.Document, data interface{}) error Validate() error DateTimeParserNamed(name string) analysis.DateTimeParser DefaultSearchField() string AnalyzerNameForPath(path string) string AnalyzerNamed(name string) analysis.Analyzer FieldMappingForPath(path string) FieldMapping }
type IndexMappingImpl ¶
type IndexMappingImpl struct { TypeMapping map[string]*DocumentMapping `json:"types,omitempty"` DefaultMapping *DocumentMapping `json:"default_mapping"` TypeField string `json:"type_field"` DefaultType string `json:"default_type"` DefaultAnalyzer string `json:"default_analyzer"` DefaultDateTimeParser string `json:"default_datetime_parser"` DefaultField string `json:"default_field"` StoreDynamic bool `json:"store_dynamic"` IndexDynamic bool `json:"index_dynamic"` DocValuesDynamic bool `json:"docvalues_dynamic"` CustomAnalysis *customAnalysis `json:"analysis,omitempty"` // contains filtered or unexported fields }
An IndexMappingImpl controls how objects are placed into an index. First the type of the object is determined. Once the type is know, the appropriate DocumentMapping is selected by the type. If no mapping was determined for that type, a DefaultMapping will be used.
func NewIndexMapping ¶
func NewIndexMapping() *IndexMappingImpl
NewIndexMapping creates a new IndexMapping that will use all the default indexing rules
func (*IndexMappingImpl) AddCustomAnalyzer ¶
func (im *IndexMappingImpl) AddCustomAnalyzer(name string, config map[string]interface{}) error
AddCustomAnalyzer defines a custom analyzer for use in this mapping. The config map must have a "type" string entry to resolve the analyzer constructor. The constructor is invoked with the remaining entries and returned analyzer is registered in the IndexMapping.
bleve comes with predefined analyzers, like github.com/blevesearch/bleve/analysis/analyzer/custom. They are available only if their package is imported by client code. To achieve this, use their metadata to fill configuration entries:
import ( "github.com/MuratYMT2/bleve/v2/analysis/analyzer/custom" "github.com/MuratYMT2/bleve/v2/analysis/char/html" "github.com/MuratYMT2/bleve/v2/analysis/token/lowercase" "github.com/MuratYMT2/bleve/v2/analysis/tokenizer/unicode" ) m := bleve.NewIndexMapping() err := m.AddCustomAnalyzer("html", map[string]interface{}{ "type": custom.Name, "char_filters": []string{ html.Name, }, "tokenizer": unicode.Name, "token_filters": []string{ lowercase.Name, ... }, })
func (*IndexMappingImpl) AddCustomCharFilter ¶
func (im *IndexMappingImpl) AddCustomCharFilter(name string, config map[string]interface{}) error
AddCustomCharFilter defines a custom char filter for use in this mapping
func (*IndexMappingImpl) AddCustomDateTimeParser ¶
func (im *IndexMappingImpl) AddCustomDateTimeParser(name string, config map[string]interface{}) error
AddCustomDateTimeParser defines a custom date time parser for use in this mapping
func (*IndexMappingImpl) AddCustomTokenFilter ¶
func (im *IndexMappingImpl) AddCustomTokenFilter(name string, config map[string]interface{}) error
AddCustomTokenFilter defines a custom token filter for use in this mapping
func (*IndexMappingImpl) AddCustomTokenMap ¶
func (im *IndexMappingImpl) AddCustomTokenMap(name string, config map[string]interface{}) error
AddCustomTokenMap defines a custom token map for use in this mapping
func (*IndexMappingImpl) AddCustomTokenizer ¶
func (im *IndexMappingImpl) AddCustomTokenizer(name string, config map[string]interface{}) error
AddCustomTokenizer defines a custom tokenizer for use in this mapping
func (*IndexMappingImpl) AddDocumentMapping ¶
func (im *IndexMappingImpl) AddDocumentMapping(doctype string, dm *DocumentMapping)
AddDocumentMapping sets a custom document mapping for the specified type
func (*IndexMappingImpl) AnalyzeText ¶
func (im *IndexMappingImpl) AnalyzeText(analyzerName string, text []byte) (analysis.TokenStream, error)
func (*IndexMappingImpl) AnalyzerNameForPath ¶
func (im *IndexMappingImpl) AnalyzerNameForPath(path string) string
AnalyzerNameForPath attempts to find the best analyzer to use with only a field name will walk all the document types, look for field mappings at the provided path, if one exists and it has an explicit analyzer that is returned.
func (*IndexMappingImpl) AnalyzerNamed ¶
func (im *IndexMappingImpl) AnalyzerNamed(name string) analysis.Analyzer
func (*IndexMappingImpl) DateTimeParserNamed ¶
func (im *IndexMappingImpl) DateTimeParserNamed(name string) analysis.DateTimeParser
func (*IndexMappingImpl) DefaultSearchField ¶
func (im *IndexMappingImpl) DefaultSearchField() string
func (*IndexMappingImpl) FieldAnalyzer ¶
func (im *IndexMappingImpl) FieldAnalyzer(field string) string
FieldAnalyzer returns the name of the analyzer used on a field.
func (*IndexMappingImpl) FieldMappingForPath ¶
func (im *IndexMappingImpl) FieldMappingForPath(path string) FieldMapping
FieldMappingForPath returns the mapping for a specific field 'path'.
func (*IndexMappingImpl) MapDocument ¶
func (im *IndexMappingImpl) MapDocument(doc *document.Document, data interface{}) error
func (*IndexMappingImpl) UnmarshalJSON ¶
func (im *IndexMappingImpl) UnmarshalJSON(data []byte) error
UnmarshalJSON offers custom unmarshaling with optional strict validation
func (*IndexMappingImpl) Validate ¶
func (im *IndexMappingImpl) Validate() error
Validate will walk the entire structure ensuring the following explicitly named and default analyzers can be built