neptune

package
v2.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 7, 2020 License: MIT Imports: 18 Imported by: 0

Documentation

Overview

This module, when combined with codelist.go, provides code that satisfies the graph.driver.CodeList interface using Gremlin queries into a Neptune database.

It is dedicated to code to satisfy the GetCodeDatasets() method - which is sufficiently complex to merit a module (and tests) of its own.

Index

Constants

This section is empty.

Variables

View Source 
var ErrInvalidFilter = errors.New("nil filter cannot be processed")

ErrInvalidFilter is returned if the provided filter is nil.

Functions

This section is empty.

Types

type ErrAttemptsExceededLimit 

type ErrAttemptsExceededLimit struct {
	WrappedErr error
}

ErrAttemptsExceededLimit is returned when the number of attempts has reached the maximum permitted

func (ErrAttemptsExceededLimit) Error 

func (e ErrAttemptsExceededLimit) Error() string

type NeptuneDB 

type NeptuneDB struct {
	neptune.NeptuneDriver
	// contains filtered or unexported fields
}

func New 

func New(dbAddr string, size, timeout, retries int, errs chan error) (n *NeptuneDB, err error)

func (*NeptuneDB) AddDimensions 

func (n *NeptuneDB) AddDimensions(ctx context.Context, instanceID string, dimensions []interface{}) error

AddDimensions list to the specified instance node

func (*NeptuneDB) AddVersionDetailsToInstance 

func (n *NeptuneDB) AddVersionDetailsToInstance(ctx context.Context, instanceID string, datasetID string, edition string, version int) error

AddVersionDetailsToInstance updates an instance node to contain details of which dataset, edition and version the instance will also be known by

func (*NeptuneDB) CloneNodes 

func (n *NeptuneDB) CloneNodes(ctx context.Context, attempt int, instanceID, codeListID, dimensionName string) (err error)

func (*NeptuneDB) CloneRelationships 

func (n *NeptuneDB) CloneRelationships(ctx context.Context, attempt int, instanceID, codeListID, dimensionName string) (err error)

func (*NeptuneDB) CountInsertedObservations 

func (n *NeptuneDB) CountInsertedObservations(ctx context.Context, instanceID string) (count int64, err error)

CountInsertedObservations returns the current number of observations relating to a given instance

func (*NeptuneDB) CountNodes 

func (n *NeptuneDB) CountNodes(ctx context.Context, instanceID, dimensionName string) (count int64, err error)

func (*NeptuneDB) CreateCodeRelationship 

func (n *NeptuneDB) CreateCodeRelationship(ctx context.Context, instanceID, codeListID, code string) error

CreateCodeRelationship links an instance to a code for the given dimension option

func (*NeptuneDB) CreateInstance 

func (n *NeptuneDB) CreateInstance(ctx context.Context, instanceID string, csvHeaders []string) error

CreateInstance will check if an instance node already exists and create one from the provided details if one does not exist

func (*NeptuneDB) CreateInstanceConstraint 

func (n *NeptuneDB) CreateInstanceConstraint(ctx context.Context, instanceID string) error

CreateInstanceConstraint is not needed for the neptune implementation, as constraints are not a neptune construct

func (*NeptuneDB) CreateInstanceHierarchyConstraints 

func (n *NeptuneDB) CreateInstanceHierarchyConstraints(ctx context.Context, attempt int, instanceID, dimensionName string) error

func (*NeptuneDB) GetCode 

func (n *NeptuneDB) GetCode(ctx context.Context, codeListID, edition string, code string) (*models.Code, error)

GetCode provides a Code struct to represent the requested code list, edition and code string. E.g. ashe-earnings|one-off|hourly-pay-gross. It doesn't need to access the database to form the response, but does so to validate the query. Specifically it can return errors as follows: - The Gremlin query failed to execute. - The query parameter values do not successfully navigate to a Code node. (error is `ErrNotFound`) - Duplicate Code(s) exist that satisfy the search criteria (error is `ErrMultipleFound`)

func (*NeptuneDB) GetCodeDatasets 

func (n *NeptuneDB) GetCodeDatasets(ctx context.Context, codeListID, edition string, code string) (*models.Datasets, error)

GetCodeDatasets searches the database for datasets that are associated with the given code list, code, and code list edition. Specifically those that satisfy all of:

  1. code lists that match the requested code list ID.
  2. code lists of the requested edition.
  3. codes that match the requested code value.
  4. datasets that are related to qualifying codes by *inDataset* edges.
  5. datasets that have the *isPublished* state true.

Each such result from the database (potentially) has the properties:

  • dimensionName (what the dataset calls this dimension)
  • datasetEdition
  • version

The results however include all permuations of dimensionName and datasetEdition - BUT ONLY CITES the most recent dataset *version* of those found for that permuation.

func (*NeptuneDB) GetCodeList 

func (n *NeptuneDB) GetCodeList(ctx context.Context, codeListID string) (
	*models.CodeList, error)

GetCodeList provides a CodeList for a given ID (e.g. "ashe-earnings"), having checked it exists in the database. Nb. The caller is expected to fully qualify the embedded Links field afterwards. It returns an error if: - The Gremlin query failed to execute. - The requested CodeList does not exist. (error is `ErrNotFound`) - Duplicate CodeLists exist with the given ID (error is `ErrMultipleFound`)

func (*NeptuneDB) GetCodeLists 

func (n *NeptuneDB) GetCodeLists(ctx context.Context, filterBy string) (*models.CodeListResults, error)

GetCodeLists provides a list of either all Code Lists, or a list of only those having a boolean property with the name <filterBy> which is set to true. E.g. "geography": true. The caller is expected to fully qualify the embedded Links field afterwards. It returns an error if: - The Gremlin query failed to execute. - A CodeList is encountered that does not have *listID* property.

func (*NeptuneDB) GetCodes 

func (n *NeptuneDB) GetCodes(ctx context.Context, codeListID, edition string) (*models.CodeResults, error)

GetCodes provides a list of Code(s) packaged into a models.CodeResults structure that has been populated by a database query that finds the Code List nodes of the required codeListID (e.g. "ashe-earnings"), and the required edition (e.g. "one-off"), and then harvests the Code nodes that are known to be "usedBy" that Code List. It raises a wrapped error if the database raises a non-transient error, (e.g. malformed query). It raises driver.ErrNotFound if the graph traversal above produces an empty list of codes - including the case of a short-circuit early termination of the query, because no such qualifying code list exists. It returns a wrapped error if a Code is found that does not have a "value" property.

func (*NeptuneDB) GetEdition 

func (n *NeptuneDB) GetEdition(ctx context.Context, codeListID, edition string) (*models.Edition, error)

GetEdition provides an Edition structure for the code list in the database that has both the given codeListID (e.g. "ashed-earnings"), and the given edition string (e.g. "one-off"). Nb. The caller is expected to fully qualify the embedded Links field afterwards. It returns an error if: - The Gremlin query failed to execute. (wrapped error) - No CodeLists exist with the requested codeListID (error is `ErrNotFound`) - A CodeList is found that does not have the "edition" property (error is 'ErrNoSuchProperty') - More than one CodeList exists with the requested ID AND edition (error is `ErrMultipleFound`)

func (*NeptuneDB) GetEditions 

func (n *NeptuneDB) GetEditions(ctx context.Context, codeListID string) (*models.Editions, error)

GetEditions provides a models.Editions structure populated based on the the values in the Code List vertices in the database, that have the provided codeListId. It returns an error if: - The Gremlin query failed to execute. (wrapped error) - No CodeLists are found of the requested codeListID (error is ErrNotFound') - A CodeList is found that does not have the "edition" property (error is 'ErrNoSuchProperty')

func (*NeptuneDB) GetHierarchyCodelist 

func (n *NeptuneDB) GetHierarchyCodelist(ctx context.Context, instanceID, dimension string) (codelistID string, err error)

func (*NeptuneDB) GetHierarchyElement 

func (n *NeptuneDB) GetHierarchyElement(ctx context.Context, instanceID, dimension, code string) (node *models.HierarchyResponse, err error)

func (*NeptuneDB) GetHierarchyRoot 

func (n *NeptuneDB) GetHierarchyRoot(ctx context.Context, instanceID, dimension string) (node *models.HierarchyResponse, err error)

func (*NeptuneDB) InsertDimension 

func (n *NeptuneDB) InsertDimension(ctx context.Context, uniqueDimensions map[string]string, instanceID string, d *models.Dimension) (*models.Dimension, error)

InsertDimension node to neptune and create relationships to the instance node. Where nodes and relationships already exist, ensure they are upserted.

func (*NeptuneDB) InsertObservationBatch 

func (n *NeptuneDB) InsertObservationBatch(ctx context.Context, attempt int, instanceID string, observations []*models.Observation, dimensionNodeIDs map[string]string) error

InsertObservationBatch creates a batch query based on a provided list of observations and attempts to insert them by bulk to the database

func (*NeptuneDB) InstanceExists 

func (n *NeptuneDB) InstanceExists(ctx context.Context, instanceID string) (bool, error)

InstanceExists returns true if an instance already exists with the provided id

func (*NeptuneDB) MarkNodesToRemain 

func (n *NeptuneDB) MarkNodesToRemain(ctx context.Context, attempt int, instanceID, dimensionName string) (err error)

func (*NeptuneDB) RemoveCloneEdges 

func (n *NeptuneDB) RemoveCloneEdges(ctx context.Context, attempt int, instanceID, dimensionName string) (err error)

func (*NeptuneDB) RemoveNodesNotMarkedToRemain 

func (n *NeptuneDB) RemoveNodesNotMarkedToRemain(ctx context.Context, attempt int, instanceID, dimensionName string) (err error)

func (*NeptuneDB) RemoveRemainMarker 

func (n *NeptuneDB) RemoveRemainMarker(ctx context.Context, attempt int, instanceID, dimensionName string) (err error)

func (*NeptuneDB) SetHasData 

func (n *NeptuneDB) SetHasData(ctx context.Context, attempt int, instanceID, dimensionName string) (err error)

func (*NeptuneDB) SetInstanceIsPublished 

func (n *NeptuneDB) SetInstanceIsPublished(ctx context.Context, instanceID string) error

SetInstanceIsPublished sets a flag on an instance node to indicate the published state

func (*NeptuneDB) SetNumberOfChildren 

func (n *NeptuneDB) SetNumberOfChildren(ctx context.Context, attempt int, instanceID, dimensionName string) (err error)

func (*NeptuneDB) StreamCSVRows 

func (n *NeptuneDB) StreamCSVRows(ctx context.Context, instanceID, filterID string, filter *observation.DimensionFilters, limit *int) (observation.StreamRowReader, error)

StreamCSVRows returns a reader allowing individual CSV rows to be read. Rows returned can be limited, to stop this pass in nil. If filter.DimensionFilters is nil, empty or contains only empty values then a StreamRowReader for the entire dataset will be returned.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.