README
¶
Use Cases | Setup | Status | Documentation | Contact
Welcome
Noms is a decentralized database philosophically descendant from the Git version control system.
Like Git, Noms is:
- Versioned: By default, all previous versions of the database are retained. You can trivially track how the database evolved to its current state, easily and efficiently compare any two versions, or even rewind and branch from any previous version.
- Synchronizable: Instances of a single Noms database can be disconnected from each other for any amount of time, then later reconcile their changes efficiently and correctly.
Unlike Git, Noms is a database, so it also:
- Primarily stores structured data, not files and directories (see: the Noms type system)
- Scales well to large amounts of data and concurrent clients
- Supports atomic transactions (a single instance of Noms is CP, but Noms is typically run in production backed by S3, in which case it is "effectively CA")
- Supports efficient indexes (see: Noms prolly-trees)
- Features a flexible query model (see: GraphQL)
A Noms database can reside within a file system or in the cloud:
- The (built-in) NBS
ChunkStoreimplementation provides two back-ends which provide persistence for Noms databases: one for storage in a file system and one for storage in an S3 bucket.
Finally, because Noms is content-addressed, it yields a very pleasant programming model.
Working with Noms is declarative. You don't INSERT new data, UPDATE existing data, or DELETE old data. You simply declare what the data ought to be right now. If you commit the same data twice, it will be deduplicated because of content-addressing. If you commit almost the same data, only the part that is different will be written.
Use Cases
Decentralization
Because Noms is very good at sync, it makes a decent basis for rich, collaborative, fully-decentralized applications.
ClientDB (coming someday)
Embed Noms into mobile applications, making it easier to build offline-first, fully synchronizing mobile applications.
Setup
# You probably want to add this to your environment
export NOMS_VERSION_NEXT=1
go get github.com/attic-labs/noms/cmd/noms
go install github.com/attic-labs/noms/cmd/noms
Run
Import some data:
go install github.com/attic-labs/noms/samples/go/csv/csv-import
curl 'https://data.cityofnewyork.us/api/views/kku6-nxdu/rows.csv?accessType=DOWNLOAD' > /tmp/data.csv
csv-import /tmp/data.csv /tmp/noms::nycdemo
Explore:
noms show /tmp/noms::nycdemo
Should show:
struct Commit {
meta: struct Meta {
date: "2017-09-19T19:33:01Z",
inputFile: "/tmp/data.csv",
},
parents: set {},
value: [ // 236 items
struct Row {
countAmericanIndian: "0",
countAsianNonHispanic: "3",
countBlackNonHispanic: "21",
countCitizenStatusTotal: "44",
countCitizenStatusUnknown: "0",
countEthnicityTotal: "44",
...
Status
Data Format
We are fairly confident in the core data format, and plan to support Noms database version 7 and forward. If you create a database with Noms today, future versions will have migration tools to pull your databases forward.
Roadmap
We plan to implement the following for Noms version 8:
- Horizontal scalability (Done! See: nbs)
- Automatic merge (Done! See: CommitOptions.Policy and the
noms mergesubcommand). - Query language (Done! See ngql)
- Garbage Collection (https://github.com/attic-labs/noms/issues/3374)
- Optional fields (https://github.com/attic-labs/noms/issues/2327)
- Implement migration (https://github.com/attic-labs/noms/issues/3363)
- Fix sync performance with long commit chains (https://github.com/attic-labs/noms/issues/2233)
- Various other smaller bugs and improvements
Learn More About Noms
For the decentralized web: The Decentralized Database
Learn the basics: Technical Overview
Tour the CLI: Command-Line Interface Tour
Tour the Go API: Go SDK Tour
Contact Us
Interested in using Noms? Awesome! We would be happy to work with you to help understand whether Noms is a fit for your problem. Reach out at:
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
util
This is the Command struct used by the noms utility.
|
This is the Command struct used by the noms utility. |
|
go
|
|
|
chunks
Package chunks provides facilities for representing, storing, and fetching content-addressed chunks of Noms data.
|
Package chunks provides facilities for representing, storing, and fetching content-addressed chunks of Noms data. |
|
constants
Package constants collects common constants used in Noms, such as the Noms data format version.
|
Package constants collects common constants used in Noms, such as the Noms data format version. |
|
d
Package d implements several debug, error and assertion functions used throughout Noms.
|
Package d implements several debug, error and assertion functions used throughout Noms. |
|
datas
Package datas defines and implements the database layer used in Noms.
|
Package datas defines and implements the database layer used in Noms. |
|
hash
Package hash implements the hash function used throughout Noms.
|
Package hash implements the hash function used throughout Noms. |
|
marshal
Package marshal implements encoding and decoding of Noms values.
|
Package marshal implements encoding and decoding of Noms values. |
|
perf/suite
Package suite implements a performance test suite for Noms, intended for measuring and reporting long running tests.
|
Package suite implements a performance test suite for Noms, intended for measuring and reporting long running tests. |
|
spec
Package spec provides builders and parsers for spelling Noms databases, datasets and values.
|
Package spec provides builders and parsers for spelling Noms databases, datasets and values. |
|
types
Package types contains most of the data structures available to/from Noms.
|
Package types contains most of the data structures available to/from Noms. |
|
util/datetime
Package datetime implements marshalling of Go DateTime values into Noms structs with type DateTimeType.
|
Package datetime implements marshalling of Go DateTime values into Noms structs with type DateTimeType. |
|
util/exit
Package exit provides a mockable implementation of os.Exit.
|
Package exit provides a mockable implementation of os.Exit. |
|
util/progressreader
Package progressreader provides an io.Reader that reports progress to a callback
|
Package progressreader provides an io.Reader that reports progress to a callback |
|
util/status
Package status prints status messages to a console, overwriting previous values.
|
Package status prints status messages to a console, overwriting previous values. |
|
samples
|
|