bleve-server

README

bleve-server

RPC server over the text Golang text indexing library bleve.

Purpose

Currently bleve does not support read/write access between more then programs according to issue #1571; so that means, if you have one program accessing bleve then another program cannot access it.

As a result, this stand-alone server was created to allow multiple programs to access without problem. For your Golang program to access bleve, simply make remote procedure calls to this server.

Installation (Golang)

1. Clone the library to your computer.

git clone https://github.com/bartmika/bleve-server.git
cd bleve-server

1. Install the library dependencies.

go mod tidy

1. Before you start the server, make sure you setup the following environment variable.

export BLEVE_SERVER_ADDRESS=127.0.0.1:8001
export BLEVE_SERVER_HOME_DIRECTORY_PATH=""  # Leave blank if you want to save app files in the same folder as your application, else if you want to save somewhere else (ex: "/tmp") then set this value.

1. Verify the server starts up.

go run main.go serve

Installation (Docker)

Alternatively, to start the server using docker then follow these steps.

1. Clone the library to your computer.

git clone https://github.com/bartmika/bleve-server.git
cd bleve-server

1. Build the docker image.

docker build -t bartmika/bleve-server:latest .

1. Run a docker container.

docker run -d -p 8001:8001 --name=bleve-server -e BLEVE_SERVER_ADDRESS="0.0.0.0:8001" -e BLEVE_SERVER_HOME_DIRECTORY_PATH="/db" bartmika/bleve-server:latest

1. Alternatively you can use docker-compose as well via:

docker-compose up

Usage

RPC server over a single running bleve instance

Usage:
  bleve-server [flags]
  bleve-server [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  index       Submit data to index
  query       Perform full-text search of indexed data
  register    Register a bleve index
  serve       Run the rpc server.
  version     Print the version number

Flags:
      --appAddress string   The applications address. (default "127.0.0.1:8001")
  -h, --help                help for bleve-server

Use "bleve-server [command] --help" for more information about a command.

Usage Examples

Command Line Interface

Start Server Command

1. Open up a terminal and register the index file before you begin. If index do not make sense then please read the bleve developer documentation.

go run main.go register --filename=dune.bleve

1. You have successfully created an index. Please note bleve-server supports multiple-index files open concurrently. Every time you want a new index you'll need to rerun the regsiter command.

2. Start bleve-server in your terminal:

go run main.go serve

1. Now you are ready to make rpc calls to the server.

Index Data Command

While your bleve-server is running in terminal, open up another terminal run the following commands:

go run main.go index --filename=dune.bleve --identifier=123456789 --data="The spice extends life"
go run main.go index --filename=dune.bleve --identifier=987654321 --data="The spice is vital for space travel"

Query Command

Start by getting an individual result.

go run main.go query --filename=dune.bleve --search="life"

# OUTPUT:
# UUIDs: &[123456789]

Try another query to get an individual result.

go run main.go query --filename=dune.bleve --search="space travel"

# OUTPUT:
# UUIDs: &[987654321]

Try a query to get multiple results.

go run main.go query --filename=dune.bleve --search="spice"

# OUTPUT:
# UUIDs: &[123456789 987654321]

Application

Here is a sample file of accessing bleve-server over rpc in your code:

package main

import (
    "fmt"
    "log"
    "os"
    "time"

    remote "github.com/bartmika/bleve-server/pkg/rpc_client"
)

// Assume you installed the dependency:
//    go get github.com/bartmika/bleve-server

// Assume you ran the following commands in your `bleve-server` project:
//    go run main.go register --filename=dune.bleve
//    go run main.go serve

// Assume you setup the following environment variable:
//    export BLEVE_SERVER_ADDRESS=127.0.0.1:8001

func main() {
    // Load up our `environment variables` from our operating system.
    addr := os.Getenv("BLEVE_SERVER_ADDRESS") // Example Value: 127.0.0.1:8001

    // Initialize the RPC client.
    rpc := remote.New(addr, 3, 15*time.Second)

    // Index the following data...

    err := rpc.Index("dune.bleve", "123456789", []byte("The spice extends life"))
    if err != nil {
        log.Fatal("doIndex err:", err)
    }
    err = rpc.Index("dune.bleve", "987654321", []byte("The spice is vital for space travel"))
    if err != nil {
        log.Fatal("doIndex err:", err)
    }

    // Try querying...

    uuids, err := rpc.Query("dune.bleve", "life")
    if err != nil {
        log.Fatal("doQuery err:", err)
    }
    fmt.Println("UUIDs:", uuids) // OUTPUT: [123456789]

    uuids, err = rpc.Query("dune.bleve", "space travel")
    if err != nil {
        log.Fatal("doQuery err:", err)
    }
    fmt.Println("UUIDs:", uuids) // OUTPUT: [987654321]

    uuids, err = rpc.Query("dune.bleve", "spice")
    if err != nil {
        log.Fatal("doQuery err:", err)
    }
    fmt.Println("UUIDs:", uuids) // OUTPUT: [123456789, 987654321]
}

Contributing

Found a bug? Want a feature to improve the package? Please create an issue.

License

Made with ❤️ by Bartlomiej Mika.
The project is licensed under the ISC License.

Resource used:

• levesearch/bleve is modern text indexing library for go.
• spf13/cobra is a commander for modern Go CLI interactions.
• spf13/viper is a Go configuration with fangs.