Shock

module
v0.9.17 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 9, 2016 License: BSD-2-Clause

README

About

Shock is a platform to support computation, storage, and distribution. Designed from the ground up to be fast, scalable, fault tolerant, federated.

Shock is RESTful. Accessible from desktops, HPC systems, exotic hardware, the cloud and your smartphone.

Shock is for scientific data. One of the challenges of large volume scientific data is that without often complex metadata it is of little to no value. Shock allows storage and querying of complex metadata.

Shock is a data management system. The long term goals of Shock include the ability to annotate, anonymize, convert, filter, perform quality control, and statically subsample at line speed bioinformatics sequence data. Extensible plug-in architecture is in development.

Most importantly Shock is still very much in development. Be patient and contribute.

Shock is actively being developed at github.com/MG-RAST/Shock.

Building

Shock (requires mongodb=>2.0.3, go=>1.1.0 golang.org, git, mercurial and bazaar). You must also set the $GOPATH and $GOROOT environment variables before installing Shock. There are two options for installing Shock.

OPTION 1: The recommended method for installing Shock is to download the Makefile located here to your $GOPATH directory and then run:

make install

OPTION 2: You could alternatively install Shock by running:

go get github.com/MG-RAST/Shock/...

The upside to using OPTION 1 is that this will insert the Shock version number into your Shock server to be displayed when the server is started and this will also generate the Shock documentation locally to be hosted by the server. The built binaries will be located in the env configured $GOPATH/bin/ directory.

Dockerfile

Alternativly your can use Docker to compile Shock. The Dockerfile in directory "docker" in this repository compiles Shock statically.

export TAG=`date +"%Y%m%d.%H%M"`
docker build --force-rm --no-cache --rm -t shock:${TAG} .

If you only want the binary you can create an container from the image an copy the binary to your host. This will copy the shock-server binary to your current working directory

docker run --name shock shock:${TAG} ls
docker cp shock:/go/bin/shock-server .
docker rm shock

MongoDB

In ubuntu you can simply install mongo with:

sudo apt-get install -y mongodb-server

If you do not want to use a package manager to install mongodb, use:

curl -s http://downloads.mongodb.org/linux/mongodb-linux-x86_64-2.4.14.tgz | tar -v -C /mongodb/ -xz

If you do not use a service manager such as systemd, you can start mongodb like this, in foreground:

/mongodb/bin/mongod --dbpath /data/

or in background:

nohup /mongodb/bin/mongod --dbpath /mnt/db/ &

Configuration

The Shock configuration file is in INI file format. There is a template of the config file located at the root level of the repository.

Running

To run:

shock-server -conf <path_to_config_file>

Documentation

For further information about Shock's functionality, please refer to our github wiki.

Directories

Path Synopsis
lib
auth
Package auth implements http request authentication
Package auth implements http request authentication
auth/basic
Package basic implements basic auth decoding and self contained user authentication
Package basic implements basic auth decoding and self contained user authentication
auth/globus
Package globus implements Globus Online Nexus authentication
Package globus implements Globus Online Nexus authentication
auth/mgrast
Package globus implements MG-RAST OAuth authentication
Package globus implements MG-RAST OAuth authentication
conf
Package conf parses start up args and config file
Package conf parses start up args and config file
controller/node
Package node implements /node resource
Package node implements /node resource
controller/node/acl
Package acl implements /node/:id/acl resource
Package acl implements /node/:id/acl resource
controller/preauth
Package preauth implements /preauth resource
Package preauth implements /preauth resource
db
Package db to connect to mongodb
Package db to connect to mongodb
errors
Package contains error strings and patterns for common errors
Package contains error strings and patterns for common errors
logger
Package logger implements async log web api messages
Package logger implements async log web api messages
node/file
Package contains Node File struct and MultiReaderAt implementation
Package contains Node File struct and MultiReaderAt implementation
node/file/format/fasta
Package to read and write FASTA format files
Package to read and write FASTA format files
node/file/format/fastq
Package to read and write FASTQ format files
Package to read and write FASTQ format files
node/file/format/line
Package to read and index lines of a file
Package to read and index lines of a file
node/file/format/multi
Package to read and auto-detect format of fasta & fastq files
Package to read and auto-detect format of fasta & fastq files
node/file/format/sam
Package to read and write SAM format files
Package to read and write SAM format files
node/file/format/seq
Package contains interfaces for fasta & fastq & and multi packages
Package contains interfaces for fasta & fastq & and multi packages
preauth
Package preauth implements persistent storage and retrieval of preauth requests
Package preauth implements persistent storage and retrieval of preauth requests

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL