blacksmith

README

Blacksmith

Blacksmith is a low-code platform offering a complete and consistent approach for self-hosted data engineering solutions. Blacksmith allows software engineers to write low-code ETL using the Go language. It also allows data engineers to write templated SQL for TLT and database migrations on top of one or multiple databases.

Any team that is building — or think about building — such a platform knows the tremendous amount of work needed to properly accomplish this mission. Think of Blacksmith as the central piece of your data engineering workflow, leading you to save months of customized and professional work.

By leveraging Blacksmith, organizations benefit a single source of truth for all their data with a unique developer experience.

Production-ready modules

Product offerings

Blacksmith is not an open-source software. This repository only holds the public Go APIs, allowing organizations to build reliable data engineering solutions on top of Blacksmith using Go and SQL. Blacksmith itself is built and distributed as a CLI and as a Docker image.

Blacksmith is available in two Editions:

• Blacksmith Standard Edition addresses the technical complexity of data engineering. It is and will always be free. This Edition focuses on building ETLT platforms with Go and SQL for better data quality and stronger data reliability across systems.
• Blacksmith Enterprise Edition addresses the complexity of collaboration and governance across multi-team and multi-scope data solutions. This Edition adds advanced features for the enterprises, such as migrations management, distributed semaphore, and a built-in dashboard.

Professional services

Along consulting and training, we provide different product offerings as well as different levels of support.

• Request a demo
• Discover our services

License

Repository licensed under the Apache License, Version 2.0.

By downloading, installing, and using Blacksmith, you agree to the Blacksmith Terms and Conditions.

Documentation

Overview

Package blacksmith is the package to create applications on top of Blacksmith.

Blacksmith is a low-code platform offering a complete and consistent approach for self-managed data engineering. Blacksmith allows software engineers to write low-code ETL using the Go language. It also allows data engineers to write templated SQL for TLT and database migrations on top of one or multiple databases.

Any team that is building — or think about building — a data engineering platform knows the tremendous amount of work needed to properly accomplish this mission. Think of Blacksmith as the central piece of your data engineering workflow, leading you to save months of customized and professional work.

A new application can be generated using the Blacksmith CLI:

$ blacksmith generate application --name <name> [--path <path>]

Index

• type License
• type Options

Types

type License

type License struct {

	// Key is the subscription license key. This information can also be set using
	// the environment variable `BLACKSMITH_LICENSE_KEY`
	Key string `json:"-"`

	// Token is the subscription license token. This information can also be set
	// using the environment variable `BLACKSMITH_LICENSE_TOKEN`.
	Token string `json:"-"`
}

License holds the license details for using Blacksmith Enterprise Edition.

type Options

type Options struct {

	// Logger allows you to use a logrus Logger across all Blacksmith adapters and
	// the application built on top of it.
	Logger *logrus.Logger `json:"-"`

	// Supervisor is the options passed to use the supervisor adapter.
	// The supervisor is optional.
	Supervisor *supervisor.Options `json:"supervisor"`

	// Wanderer is the options passed to use the wanderer adapter.
	// The wanderer is optional.
	Wanderer *wanderer.Options `json:"wanderer"`

	// Store is the options passed to use the store adapter.
	Store *store.Options `json:"store"`

	// PubSub is the options passed to use the pubsub adapter.
	// The pusub is optional.
	PubSub *pubsub.Options `json:"pubsub"`

	// Gateway is the options passed to use the gateway service.
	Gateway *service.Options `json:"gateway"`

	// Scheduler is the options passed to use the scheduler service.
	Scheduler *service.Options `json:"scheduler"`

	// Sources is a slice of Blacksmith sources.
	Sources []source.Source `json:"-"`

	// Destinations is a slice of Blacksmith destinations.
	Destinations []destination.Destination `json:"-"`

	// License holds the license details for using Blacksmith Enterprise Edition.
	// This is not necessary for using the Standard Edition.
	License *License `json:"-"`
}

Options is the options a user can pass to create a new application.