pipedream

command module

v0.30.0 Latest Latest Go to latest Published: Feb 6, 2021 License: GPL-2.0, GPL-3.0 Imports: 1 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

README ¶

Maintainable automation.

🚧 Status: alpha 🚧

Early development

Please expect breaking changes. Feel free to file an issue if you

find a bug

need a feature added or tweaked for your use case

have any other suggestions

PipeDream is an open-source, general-purpose automation tool.

It is an alternative to shell scripts - just as powerful, but more portable, composable, testable and reliable.

How it works

Define a pipeline in a yaml file. Using a simple, but extensible syntax, you can call, chain, merge, and split pipelines, invoke shell commands, handle errors, write tests, create mocks, and much more.

Getting started

Have a look at the Quick Start Guide, the Documentation or just run the program. It will offer to create a sample pipeline file in your current folder.

Installation

From `npm`

npm i -g @l9/pipedream

Using `homebrew` (Mac OS/linux)

brew install layer9berlin/tap/pipedream

From source (requires Go installed)

git clone https://github.com/Layer9Berlin/PipeDream
cd PipeDream
go build
pipedream

then select Install when promted.

This will compile PipeDream and run it locally, allowing you to execute the install.pipe file (very meta), which in turn installs everything you need to run PipeDream anywhere on your machine. If you make changes to the code, just run PipeDream and execute the installation.pipe again to update your installed version. You can skip the installation step entirely if you only want to run PipeDream in the project folder and don't need any of the utility tools.

If you use PipeDream a lot, it might be convenient to set up an alias like p that is faster to type.

Uninstall

Installed via `npm`

npm uninstall -g @l9/pipedream

Installed via `homebrew`

brew uninstall pipedream

Compiled from source

Simply run

pipedream

in the repo source directory and select Uninstall.

Not sure

If you don't recall how you installed PipeDream, run

pipedream version

and check the via field.

Why wouldn't I just use a shell script?

Shell scripting is an art that takes years to master. Tools like shellcheck and bats can help you improve the code quality of your scripts, but they don't solve the core problems:

There are many traps for beginners to fall into
Each command has its own syntax (not to mention subtle differences between shells)
Error handling is so difficult it's rarely done well
Even simple tasks often require contorted solutions that break for edge cases
A lack of tests and documentation means you generally don't want to adapt an existing script

If you have used a modern CI/CD, orchestration or containerization tool, you are probably already familiar with a different way to write a script: defining tasks in yaml format. PipeDream is the natural extension of these tools to your localhost. Automate anything, anywhere, using a simple syntax that follows the principle of least astonishment.

"It's like Ansible for localhost!"

Anonymous

We have created PipeDream to simplify maintenance and dependency management tasks that used to involve a large number of steps, combining many different tools running both locally and on remote servers.

To illustrate its potential, we have used PipeDream to define a universal dependency manager pipe. It offers a single, consistent interface pulling together results from different package managers.

Documentation ¶

Overview ¶

PipeDream package for maintainable automation

Source Files ¶

View all Source files

pipedream.go

Directories ¶

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL

Path	Synopsis
cmd Implements the `pipedream` command line tool	Implements the `pipedream` command line tool
src Implementation of the `pipedream` functionalities	Implementation of the `pipedream` functionalities
custom Extensions and customizations of native/third party libraries	Extensions and customizations of native/third party libraries
custom/evaluate Custom functions extending the `govaluate` package	Custom functions extending the `govaluate` package
custom/filepath Custom functions extending the native `filepath` package	Custom functions extending the native `filepath` package
custom/io Custom functions extending the native `io` package	Custom functions extending the native `io` package
custom/math Custom functions extending the native `math` package	Custom functions extending the native `math` package
custom/stringmap Convenience functions for maps with string keys	Convenience functions for maps with string keys
custom/strings Custom functions extending the native `strings` package	Custom functions extending the native `strings` package
datastream Provides a composable data stream, which pipes multiple inputs to multiple outputs according to the defined compositions	Provides a composable data stream, which pipes multiple inputs to multiple outputs according to the defined compositions
logging Provides custom logging and activity indication	Provides custom logging and activity indication
logging/fields Provides custom log entry fields	Provides custom log entry fields
middleware Each middleware provides an implementation slice, performing side effects and adapting the run based on provided arguments	Each middleware provides an implementation slice, performing side effects and adapting the run based on provided arguments
middleware/catch The `catch` middleware enables graceful handling of stderr output	The `catch` middleware enables graceful handling of stderr output
middleware/catcheach The `catchEach` middleware enables graceful handling of stderr output based on regex patterns	The `catchEach` middleware enables graceful handling of stderr output based on regex patterns
middleware/dir The `dir` middleware allows for changing the current working directory	The `dir` middleware allows for changing the current working directory
middleware/docker The `docker` middleware enables execution within a Docker (Compose) container	The `docker` middleware enables execution within a Docker (Compose) container
middleware/each The `each` middleware copies some input into several child pipes running simultaneously	The `each` middleware copies some input into several child pipes running simultaneously
middleware/env The `env` middleware manipulates environment variables	The `env` middleware manipulates environment variables
middleware/inherit The `inherit` middleware passes arguments from a parent to its children	The `inherit` middleware passes arguments from a parent to its children
middleware/input The `input` middleware overwrites a pipe's input directly	The `input` middleware overwrites a pipe's input directly
middleware/interpolate The `interpolate` middleware substitutes arguments or inputs into other arguments	The `interpolate` middleware substitutes arguments or inputs into other arguments
middleware/output The `output` middleware overwrites a pipe's output directly	The `output` middleware overwrites a pipe's output directly
middleware/pipe The `pipe` middleware executes several commands in sequence	The `pipe` middleware executes several commands in sequence
middleware/select The `select` middleware shows selection prompts to the user	The `select` middleware shows selection prompts to the user
middleware/shell The `shell` middleware executes commands in a shell	The `shell` middleware executes commands in a shell
middleware/ssh The `ssh` middleware enables remote command execution via SSH	The `ssh` middleware enables remote command execution via SSH
middleware/stack Defines a list of middleware items to be executed when a pipeline is run	Defines a list of middleware items to be executed when a pipeline is run
middleware/sync The `sync` middleware defers execution until a condition is fulfilled	The `sync` middleware defers execution until a condition is fulfilled
middleware/timer The `timer` middleware records execution time	The `timer` middleware records execution time
middleware/when The `when` middleware enables conditional execution	The `when` middleware enables conditional execution
middleware/with The `with` middleware extracts and processes input patterns	The `with` middleware extracts and processes input patterns
parser Provides a parser for pipeline yaml files	Provides a parser for pipeline yaml files
pipeline Provides a model representing a pipeline definition that defines default arguments for a pipeline identifier Provides a representation of a yaml pipeline file as a Go struct Provides models to represent pipeline files and runs Provides a model representing a pipeline run	Provides a model representing a pipeline definition that defines default arguments for a pipeline identifier Provides a representation of a yaml pipeline file as a Go struct Provides models to represent pipeline files and runs Provides a model representing a pipeline run
run Implementation of the default shell command, selecting, parsing and executing pipelines	Implementation of the default shell command, selecting, parsing and executing pipelines
version Implementation of the version command, providing information about the current pipedream command line tool installation	Implementation of the version command, providing information about the current pipedream command line tool installation
util Additional command line tools to handle specific, specialized tasks not covered by pipes or middleware	Additional command line tools to handle specific, specialized tasks not covered by pipes or middleware
semver-compare Parse and compare two or more semantic versions	Parse and compare two or more semantic versions
semver-compare/cmd
traffic-light Augment output with a green, amber or red status indicator	Augment output with a green, amber or red status indicator
traffic-light/cmd