arnedb

package module
v1.2.2-alpha Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 27, 2022 License: MIT Imports: 13 Imported by: 0

README

ArneDB

ArneDB is an embedded document (NoSql) database. There is no server implementation. Just import the package and GO.

ArneDB is not a high-performance database. There are a lot of database products which can achieve high performance. Arnedb provides a lightweight database implementation which is embeddable in any GO app.

The design goals of Arnedb are:

  • Low memory usage: Can be run in resource constrained environments
  • Simplicity: There are only 10 functions.
  • Text file storage: All the data is stored in text based JSON files
Minimum Go Version

Go 1.19+

Overview

Installation

This module can be installed with the go get command:

go get github.com/mgulsoy/arnedb

This module is pure GO implementation. No external libraries required. Only standart libraries used.

Usage

After installing the library, import it into your app file:

import (
    "github.com/mgulsoy/arnedb"
)
Db Management

ArneDB uses a folder to store data. To create or open a database use Open function:

func main() {
    ptrDbInstance, err := arnedb.Open("baseDir","databaseName")
    if err != nil {
        panic(err)
    }
}

The Open function checks whether baseDir exists and then creates databaseName database. A baseDir can contain multiple databases. The database requires no closing operation.

To store documents at first we need to create a collection. To create a collection we use CreateColl function:

func main() {
    // Open or create a collection 
    ptrDbInstance, err := arnedb.Open("baseDir","databaseName")
    if err != nil {
        panic(err)
    }

    //Create a collection
    ptrACollection, err := ptrDbInstance.CreateColl("aCollection")
    if err != nil {
        panic(err)
    }
}

The CreateColl function returns a pointer to a Coll struct. This will enable to interact with the collections created. The Open function loads existing collections. If we want to delete a collection we can use the DeleteColl function.

func main() {
    // ...
    err := ptrDbInstance.DeleteColl("anotherCollection")
    if err != nil {
        panic(err) // Not found or file system error	
    }
}

To get a slice (array) of the names of the collections we can use the GetCollNames function:

func main() {
    // ...
    collNames := ptrDbInstance.GetCollNames()
    if collNames == nil {
        // there is no collection
        fmt.Println("There is no collection")
    }
}

To get a collection we use the GetColl function:

func main() {
    // ...
    collNames := ptrDbInstance.GetCollNames()
    if collNames == nil {
        // there is no collection
        fmt.Println("There is no collection")
    }
}
Collection Operations And Query
Adding Documents

Once we get a collection from db we can add data to it. To add a document to a collection we use the Add function:

func main() {
    // ... 
    
    err := ptrToAColl.Add(someData)
    if err != nil {
        panic(err)
    }
}

Add function returns error if something goes wrong. If no error is returned then adding is successful.

If we want to add multiple documents at once, we use AddAll function.

func main() {
    // ... 
    dataArray := []RecordInstance{data1, data2, data3, data4}
    // This function is a variadic function! Don't forget the ...
    numberOfAdded, err := ptrToAColl.AddAll(dataArray...)
    if err != nil {
        panic(err)
    }
}

The function returns the number of added records. This function writes/commits data to disk at once.

Querying

After adding data, we need to query and get the data from the store. There is no special query language. Query mechanism works with the predicate functions. The operation is similar to the LINQ. A Coll provides these methods:

Also there are function from the package using generics:

GetFirst

To get a single data we use the GetFirst function. This function runs the predicate and returns the first match in a collection. The predicate function signature must match the QueryPredicate type.

func main() {
    // ... 
    // This predicate checks the records for the id is greater then 34
    queryPredicate := func(instance RecordInstance) bool {
        return instance["id"].(float64) > 34
    }
    
    data, err := ptrToAColl.GetFirst(queryPredicate)
    if err != nil {
        panic(err)
    }
    
    if data == nil {
        // This means the predicate matches no data.
        fmt.Println("No data matched!")
        return
    }
    
    fmt.Printf("Data: %+v",data) // show data on the console
}

The function returns nil if there is no match.

GetAll

If we want to get all the records that the predicate match, we use the GetAll function.

func main() {
    // ... 
    // This predicate checks the records for the id is greater then 34
    queryPredicate := func(instance RecordInstance) bool {
        return instance["id"].(float64) > 34
    }
    
    dataSlice, err := ptrToAColl.GetAll(queryPredicate)
    if err != nil {
        panic(err)
    }
    
    if len(dataSlice) == 0 {
        // This means the predicate matches no data.
        fmt.Println("No data matched!")
        return
    }
    
    fmt.Printf("Data: %+v",dataSlice) // show data on the console
}

If the predicate does not match any records, the function returns an empty slice.

GetFirstAsInterface

There is also GetFirstAsInterface function. This function tries to return data as a struct used in the application. This function works a little different with the GetFirst function. Check the example:

type SomeDataType stuct {
	Id              int
	SomeValue       string
	SomeOtherValue  float64
}

func main() {
	// ...
	
    var dataHolder SomeDataType
    var queryPredicate = func(instance interface{}) bool {
        i := instance.(*SomeDataType) // this typecast is required
        return i.Id == 13
    }

    // The holder (3rd) parameter must be an address of a variable
    found, err := ptrToAColl.GetFirstAsInterface(queryPredicate, &dataHolder)
    if err != nil {
        //handle error
        // ...
    }

    if found {
        // data found. You can reach the data with dataHolder
        fmt.Println("Data: ", dataHolder)
        // ...
    } else {
        // Not found, no match
        // if so dataHolder will be nil
        // handle this state ...
    }
}
GetAllAsInterface

There is also GetAllAsInterface function. This function hands the found document to an argument named harvestCallback. This is a callback function. Inside this function you can harvest the data as you wish. Check the example:

type SomeDataType stuct {
	Id              int
	SomeValue       string
	SomeOtherValue  float64
}

func main() {
	
    // ...
	
    var dataHolder SomeDataType
    var queryPredicate = func(instance interface{}) bool {
        i := instance.(*SomeDataType) // this typecast is required
        return i.Id > 0
    }

    var resultCollection = make([]SomeDataType,0) // create an empty slice
    var harvestCB = func(instance interface{}) bool {
        // this is a double indirection. Please pay attention to the * operators!
        i := *instance.(*SomeDataType) // this typecast is required
        resultCollection = append(resultCollection, i) // harvest as you need
        return true // always return true
    }
	
	// The holder (3rd) parameter must be an address of a variable!
    count, err := ptrToAColl.GetAllAsInterface(queryPredicate, harvestCB, &dataHolder)
    if err != nil {
       //handle error
       // ...
    }
    if count > 0 {
        // query result will be in resultCollection
        fmt.Println("Data: ", resultCollection)
        // ...
    } else {
        // Not found, no match
        // if so resultCollection will be empty
        // handle this state ...
    }
}
Count

If you want to get the count of the documents stored, there is the Count function. Here is an example of how to use it:

func main() {
	
    queryPredicate := func(q RecordInstance) bool {
       return true // we want to count all the records. You can also give conditions here.	
    }
	
    n, err := ptrToAColl.Count(queryPredicate)
    if err != nil {
        // handle error...
    } else {
       // no error
       fmt.Println("Record count:",n)
    }
}
GetFirstAs

This is a function from the package. This function works like GetFirstAsInterface method. But this function uses generics and easier to query. The predicate function must have one argument which is a pointer to generic type and must return bool. If a record is found then the record pointer is returned. If nothing found then nil returned.

type SomeDataType stuct {
    Id              int
    SomeValue       string
    SomeOtherValue  float64
}

func main() {
    // ...
	
    record, err := arnedb.GetFirstAs[SomeDataType](ptrToAColl, func(i *SomeDataType) bool{
        return i.Id == 455
    })
    if err != nil {
        //handle error
        // ...
    } else {
        //if there is no error
        if record == nil {
            // no records found.	
        } else {
            // found record
        }
    }

    
}
GetAllAs

This is a function from the package. This function works like GetAllAsInterface method. But this function uses generics and easier to query. The predicate function must have one argument which is a pointer to generic type and must return bool. If a record is found then the record pointer is returned. If nothing found then nil returned.

type SomeDataType stuct {
    Id              int
    SomeValue       string
    SomeOtherValue  float64
}

func main() {
    // ...
	
    records, err := arnedb.GetAllAs[SomeDataType](ptrToAColl, func(i *SomeDataType) bool{
        return i.SomeOtherValue > 12
    })
    if err != nil {
        //handle error
        // ...
    } else {
        //if there is no error
        if len(records)==0 {
            // no records found.	
        } else {
            // found record
        }
    }

    
}
Manipulation

We can delete records by using DeleteFirst and DeleteAll functions. The functions accept a QueryPredicate function as an argument and returns the count of deleted records. If the count is 0 this means no deletion occurred.

func main() {
    // ... 
    // This predicate checks the records for the id is greater then 34
    queryPredicate := func(instance RecordInstance) bool {
        return instance["id"].(float64) > 34
    }
    
    delCount, err := ptrToAColl.DeleteFirst(queryPredicate)
    if err != nil {
        panic(err)
    }
    
    if delCount == 0 {
        // This means the predicate matches no data.
        fmt.Println("No data matched!")
        return
    }
    
    delCount, err = ptrToAColl.DeleteAll(queryPredicate)
    if err != nil {
        panic(err)
    }
    
    if delCount == 0 {
        // This means the predicate matches no data.
        fmt.Println("No data matched!")
        return
    } 
}

We can replace or update records by using these functions:

  • ReplaceFirst : Replaces the first record matched by the query predicate with the given one in place.
  • ReplaceAll : Replaces all the records matched by the query predicate with the given one in place.
  • UpdateFirst : Updates the first record matched by the query predicate by using the update function in place.
  • UpdateAll : Updates all the records matched by the query predicate by using the update function in place.

All these functions return the count of altered records and error. If an error is returned this means there is a problem with the operation and records are not updated. If the count returned is 0 then the query predicate matched no record.

func main() {
    // ... 
    // This predicate checks the records for the id is greater then 34
    queryPredicate := func(instance RecordInstance) bool {
        return instance["id"].(float64) > 34
    }
    
    nCount, err := ptrToAColl.ReplaceFirst(queryPredicate, someNewData)
    if err != nil {
        panic(err)
    }
    
    if nCount == 0 {
        // This means the predicate matches no data.
        fmt.Println("No data matched!")
        return
    }
    
    nCount, err = ptrToAColl.ReplaceAll(queryPredicate, someNewData)
    if err != nil {
        panic(err)
    }
    
    if nCount == 0 {
        // This means the predicate matches no data.
        fmt.Println("No data matched!")
        return
    } 
}

The Update operation accepts an updater function. The function signature must match with the UpdateFunc type.

func main() {
    // ... 
    // This predicate checks the records for the id is greater then 34
    queryPredicate := QueryPredicate(func(instance RecordInstance) bool {
        return instance["id"].(float64) > 34
    })
    
    fUpdt := UpdateFunc(func(ptrRecord *RecordInstance) *RecordInstance {
        (*ptrRecord)["user"] = "Updated First" // change whatever needed to change
        return ptrRecord // and return the result
    })
    
    nCount, err := ptrToAColl.UpdateFirst(queryPredicate, fUpdt)
    if err != nil {
        panic(err)
    }
    
    if nCount == 0 {
        // This means the predicate matches no data.
        fmt.Println("No data matched!")
        return
    }
    
    nCount, err = ptrToAColl.UpdateAll(queryPredicate, fUpdt)
    if err != nil {
        panic(err)
    }
    
    if nCount == 0 {
        // This means the predicate matches no data.
        fmt.Println("No data matched!")
        return
    } 
}

Documentation

Overview

Package arnedb implements database access layer. Using this package one can embed a simple database functionality into his/her application. Arnedb is not a high performance database. There are a lot of database products which can achieve high performance. Arnedb provides a lightweight database implementation which is embeddable in any GO app. Design goals of Arnedb are: * Low memory usage: Can be run in resource constrained environments * Simplicity: Hence the title implies * Text file storage: All the data is stored in text based JSON files

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func GetAllAs 

func GetAllAs[T any](coll *Coll, predicate func(i *T) bool) (result []*T, err error)

GetAllAs function queries given coll and returns all for the predicate match. This function uses generics. Returns a slice of data pointers. If nothing is found then empty slice is returned

func GetFirstAs 

func GetFirstAs[T any](coll *Coll, predicate func(i *T) bool) (result *T, err error)

GetFirstAs function queries given coll and gets the first match of the query. This function uses generics. Returns nil if no data found.

Types

type ArneDB 

type ArneDB struct {
	// Name is the database name
	Name string
	// contains filtered or unexported fields
}

ArneDB represents a single database. There is no limit for databases. (Unless you have enough disk space)

func Open 

func Open(baseDir, dbName string) (*ArneDB, error)

Open function opens an existing or creates a new database.

func (*ArneDB) CreateColl 

func (db *ArneDB) CreateColl(collName string) (*Coll, error)

CreateColl function creates a collection and returns it.

func (*ArneDB) DeleteColl 

func (db *ArneDB) DeleteColl(collName string) error

DeleteColl function deletes a given collection.

func (*ArneDB) GelCollNames 

func (db *ArneDB) GelCollNames() (result []string)

GelCollNames returns all present collection names as []string

func (*ArneDB) GetColl 

func (db *ArneDB) GetColl(collName string) *Coll

GetColl gets the collection by the given name. It returns the pointer if it finds a collection with the given name. If not it returns nil

type Coll 

type Coll struct {

	// Name is the collection name.
	Name string
	// contains filtered or unexported fields
}

Coll represents a single collection of documents. There is no limit for collections

func (*Coll) Add 

func (coll *Coll) Add(data interface{}) error

Add function appends data into a collection

func (*Coll) AddAll 

func (coll *Coll) AddAll(data ...RecordInstance) (int, error)

AddAll function appends multiple data into a collection. If one fails, no data will be committed to storage. Thus, this function acts like a transaction. This function is a variadic function which accepts a SLICE as an argument: 

d := []RecordInstance{ a, b, c}
AddAll(d...)

Or can be called like: 

AddAll(d1,d2,d3)

func (*Coll) Count 

func (coll *Coll) Count(predicate QueryPredicate) (n int, err error)

Count function returns the count of matched records with the predicate function

func (*Coll) DeleteAll 

func (coll *Coll) DeleteAll(predicate QueryPredicate) (n int, err error)

DeleteAll function deletes all the matches of the predicate and returns the count of deletions. n = 0 if no deletions occurred.

func (*Coll) DeleteFirst 

func (coll *Coll) DeleteFirst(predicate QueryPredicate) (n int, err error)

DeleteFirst function deletes the first match of the predicate and returns the count of deleted records. n = 1 if a deletion occurred, n = 0 if none.

func (*Coll) GetAll 

func (coll *Coll) GetAll(predicate QueryPredicate) (result []RecordInstance, err error)

GetAll function queries and gets all the matches of the query predicate.

func (*Coll) GetAllAsInterface 

func (coll *Coll) GetAllAsInterface(predicate QueryPredicateAsInterface, harvestCallback QueryPredicateAsInterface, holder interface{}) (n int, err error)

GetAllAsInterface function queries and gets all the matches of the query predicate. Returns the number of record found or 0 if not. Data is sent into harvestCallback function. So you can harvest the data. There is no generics in GO. So user must handle the type conversion.

func (*Coll) GetFirst 

func (coll *Coll) GetFirst(predicate QueryPredicate) (result RecordInstance, err error)

GetFirst function queries and gets the first match of the query. The function returns nil if no data found.

func (*Coll) GetFirstAsInterface 

func (coll *Coll) GetFirstAsInterface(predicate QueryPredicateAsInterface, holder interface{}) (found bool, err error)

GetFirstAsInterface function queries and gets the first match of the query. The query result can be found in the holder argument. The function returns a boolean value indicating data is found or not.

func (*Coll) ReplaceAll 

func (coll *Coll) ReplaceAll(predicate QueryPredicate, newData interface{}) (n int, err error)

ReplaceAll replaces all the matches of the predicate with the newData and returns the count of updates. Replace is the most costly operation. The library does not provide a method to update parts of a document since document is not known to the system.

func (*Coll) ReplaceFirst 

func (coll *Coll) ReplaceFirst(predicate QueryPredicate, newData interface{}) (n int, err error)

ReplaceFirst replaces the first match of the predicate with the newData and returns the count of updates. Obviously the return value is 1 if update is successful and 0 if not. Update is the most costly operation. The library does not provide a method to update parts of a document since document is not known to the system.

func (*Coll) UpdateAll 

func (coll *Coll) UpdateAll(predicate QueryPredicate, updateFunction UpdateFunc) (n int, err error)

UpdateAll updates all the matches of the predicate in place with the data provided by the updateFunction

func (*Coll) UpdateFirst 

func (coll *Coll) UpdateFirst(predicate QueryPredicate, updateFunction UpdateFunc) (n int, err error)

UpdateFirst updates the first match of predicate in place with the data provided by the updateFunction

type QueryPredicate 

type QueryPredicate func(instance RecordInstance) bool

QueryPredicate is a function type receiving row instance and returning bool

type QueryPredicateAsInterface 

type QueryPredicateAsInterface func(instance interface{}) bool

QueryPredicateAsInterface is a function type receiving record instance as given type and returning bool

type RecordInstance 

type RecordInstance map[string]interface{}

RecordInstance represents a record instance read from data file. It is actually a map.

type UpdateFunc 

type UpdateFunc func(ptrRecord *RecordInstance) *RecordInstance

UpdateFunc alters the data matched by predicate

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.