search

package
v0.0.0-...-c5d5a31 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 21, 2020 License: Apache-2.0 Imports: 8 Imported by: 2

Documentation

Overview

Package search provides the core search results.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type ElasticSearch

type ElasticSearch struct {
	*document.ElasticSearch
}

ElasticSearch embeds our main Elasticsearch instance

func (*ElasticSearch) Fetch

func (e *ElasticSearch) Fetch(q string, filter Filter, lang language.Tag, region language.Region, number int, offset int) (*Results, error)

Fetch returns search results for a search query https://www.elastic.co/guide/en/elasticsearch/guide/current/one-lang-docs.html https://www.elastic.co/guide/en/elasticsearch/guide/current/_single_query_string.html#know-your-data The idea here is to first filter out docs that do not want to be indexed. We then search multiple fields for the search query, giving more weight to certain fields. We also are searching the standard analyzer and the language-specific analyzer. We weight the domain > path, path > title, title > description. We also give extra weight for bigram matches (need trigram????): https://www.elastic.co/guide/en/elasticsearch/guide/current/shingles.html Note: "It is not useful to mix not_analyzed fields with analyzed fields in multi_match queries." TODO: A better domain name method...we could use regex ('.*hendrix'), prefix query, etc.

type Fetcher

type Fetcher interface {
	Fetch(q string, s Filter, lang language.Tag, region language.Region, number int, offset int) (*Results, error)
}

Fetcher outlines the methods used to retrieve the core search results

type Filter

type Filter string

Filter is the safe search settings

var Moderate Filter = "moderate"

Moderate indicates a moderate safe search setting

var Off Filter = "off"

Off indicates the weakest safe search settings

var Strict Filter = "strict"

Strict indicates the strongest safe search settings

type Provider

type Provider string

Provider is a search provider

type Results

type Results struct {
	Provider   Provider             `json:"-"`
	Count      int64                `json:"-"`
	Page       string               `json:"-"`
	Previous   string               `json:"-"`
	Next       string               `json:"next"`
	Last       string               `json:"-"`
	Pagination []string             `json:"-"`
	Documents  []*document.Document `json:"documents"`
	Err        error
}

Results are the core search results from a query

func (*Results) AddPagination

func (r *Results) AddPagination(number, page int) *Results

AddPagination adds pagination to the search results

Directories

Path Synopsis
Package crawler is a distributed web crawler.
Package crawler is a distributed web crawler.
cmd
Command crawler demonstrates how to run the crawler
Command crawler demonstrates how to run the crawler
queue
Package queue manages the queue for a distributed crawler
Package queue manages the queue for a distributed crawler
robots
Package robots handles caching robots.txt files
Package robots handles caching robots.txt files
Package document parses URLs and the HTML of a webpage
Package document parses URLs and the HTML of a webpage
cmd

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL