privacy-on-beam

README

Privacy on Beam

Privacy on Beam is an end-to-end differential privacy solution built on Apache Beam. It is intended to be usable by all developers, regardless of their differential privacy expertise.

Internally, Privacy on Beam relies on the lower-level building blocks from the differential privacy library and combines them into an "out-of-the-box" solution that takes care of all the steps that are essential to differential privacy, including noise addition, partition selection, and contribution bounding. Thus, rather than using the lower-level differential privacy library, it is recommended to use Privacy on Beam, as it can reduce implementation mistakes.

Privacy on Beam is only available in Go at the moment.

Note that this work is still experimental, as well as the Go SDK for Beam, and is subject to change.

How to Use

Our codelab about computing private statistics with Privacy on Beam demonstrates how to use the library. Source code for the codelab is available in the codelab/ directory.

Full documentation of the API is available as godoc.

Using with the "go" Command

For building Privacy on Beam with the "go" command, you can run the following:

go build -mod=mod ./...

This will build all the packages. -mod=mod is necessary for installing all the dependencies automatically. Otherwise, you'll be asked to install each dependency manually.

Similarly, you can run all the tests with:

go test -mod=mod ./...

If you wish to run the codelab, you can do so by:

cd codelab/main
go run -mod=mod . -example=count -input_file=day_data.csv -output_stats_file=stats.csv -output_chart_file=chart.png

Change example to run other examples. See the codelab documentation for more information.

Both for go run and go test, if you already built the code with go build, you can omit -mod=mod.

Using with Bazel

In order to include Privacy on Beam in your Bazel project, you need to add the following to your WORKSPACE file (change dp_lib_version to the version you want to depend on, or alternatively you can depend on a specific commit; but keep in mind that you have to update dp_lib_tar_sha256 as well):

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "io_bazel_rules_go",
    sha256 = "7c10271940c6bce577d51a075ae77728964db285dac0a46614a7934dc34303e6",
    urls = [
        "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.26.0/rules_go-v0.26.0.tar.gz",
        "https://github.com/bazelbuild/rules_go/releases/download/v0.26.0/rules_go-v0.26.0.tar.gz",
    ],
)

load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")

go_rules_dependencies()

go_register_toolchains(version = "1.16")

http_archive(
    name = "bazel_gazelle",
    sha256 = "62ca106be173579c0a167deb23358fdfe71ffa1e4cfdddf5582af26520f1c66f",
    urls = [
        "https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.23.0/bazel-gazelle-v0.23.0.tar.gz",
        "https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.23.0/bazel-gazelle-v0.23.0.tar.gz",
    ],
)

load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies", "go_repository")

gazelle_dependencies()

dp_lib_version = "1.0.1" # Change to the version you want to use.
dp_lib_tar_sha256 = "c72422dc29b7307334f12b0ff95866002503e2c1d209d16cae0a6f849ebf07f4" # Change to the sha256 of the .tar.gz of the version you want to use.
dp_lib_url = "https://github.com/google/differential-privacy/archive/refs/tags/v" + dp_lib_version + ".tar.gz"

http_archive(
    name = "com_github_google_differential_privacy",
    sha256 = dp_lib_tar_sha256,
    urls = [
        dp_lib_url,
    ],
    strip_prefix = "differential-privacy-" + dp_lib_version,
)

# Load dependencies for Google DP Library base workspace.
load("@com_github_google_differential_privacy//:differential_privacy_deps.bzl", "differential_privacy_deps")
differential_privacy_deps()

# Protobuf transitive dependencies.
load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
protobuf_deps()

http_archive(
    name = "com_google_go_differential_privacy",
    sha256 = dp_lib_tar_sha256,
    urls = [
        dp_lib_url,
    ],
    strip_prefix = "differential-privacy-" + dp_lib_version + "/go",
)

load("@com_google_go_differential_privacy//:go_differential_privacy_deps.bzl", "go_differential_privacy_deps")
go_differential_privacy_deps()

http_archive(
    name = "com_google_privacy_on_beam",
    sha256 = dp_lib_tar_sha256,
    urls = [
        dp_lib_url,
    ],
    strip_prefix = "differential-privacy-" + dp_lib_version + "/privacy-on-beam",
)

load("@com_google_privacy_on_beam//:privacy_on_beam_deps.bzl", "privacy_on_beam_deps")
privacy_on_beam_deps()

Then, you can depend on @com_google_privacy_on_beam in your BUILD files.