autogazelle

README

Autogazelle: fast, automatic build file generation
==================================================

.. _cmd/autogazelle/autogazelle.bash: autogazelle.bash

Autogazelle is a wrapper program that runs Gazelle as part of each
Bazel command. You can add, delete, or rename source files, add imports, then
just ``bazel build``. Your build files are updated automatically during the
build.

*Autogazelle is highly experimental and may change significantly in the future.
Use with caution. See* `Limitations`_ *below.*

Setting up autogazelle
----------------------

Before you begin
~~~~~~~~~~~~~~~~

Make sure the ``bazel_gazelle`` repository is declared in your ``WORKSPACE``.
It should look the the snippet below, but the versions will likely be different.
It may be possible to used Autogazelle from a vendored ``bazel_gazelle``
repository, but this is not directly supported yet.

.. code:: bzl

    load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

    http_archive(
        name = "io_bazel_rules_go",
        sha256 = "91585017debb61982f7054c9688857a2ad1fd823fc3f9cb05048b0025c47d023",
        urls = [
            "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.42.0/rules_go-v0.42.0.zip",
            "https://github.com/bazelbuild/rules_go/releases/download/v0.42.0/rules_go-v0.42.0.zip",
        ],
    )

    http_archive(
        name = "bazel_gazelle",
        sha256 = "b7387f72efb59f876e4daae42f1d3912d0d45563eac7cb23d1de0b094ab588cf",
        urls = [
            "https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.34.0/bazel-gazelle-v0.34.0.tar.gz",
            "https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.34.0/bazel-gazelle-v0.34.0.tar.gz",
        ],
    )

    load("@io_bazel_rules_go//go:def.bzl", "go_rules_dependencies", "go_register_toolchains")
    go_rules_dependencies()
    go_register_toolchains()
    load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies")
    gazelle_dependencies()

Make sure you have a ``gazelle`` rule declared in your root build file.
It should look this this, probably with different options.

.. code:: bzl

  load("@bazel_gazelle//:def.bzl", "gazelle")

  # gazelle:prefix github.com/example/project
  gazelle(name = "gazelle")

Installing the wrapper
~~~~~~~~~~~~~~~~~~~~~~

In order to run autogazelle as part of each Bazel command, you'll need to
copy `cmd/autogazelle/autogazelle.bash`_ to ``tools/bazel`` within your
repository. Make sure the script is executable. The ``bazel`` script in
your ``PATH`` will execute this script instead of the real ``bazel`` binary
if it's present and executable.

Deleting build files
~~~~~~~~~~~~~~~~~~~~

If your build files are completely generated by Gazelle, you may want to
delete them from source control, since they can be easily regenerated.
When Gazelle generates new files, it names them ``BUILD.bazel`` by default,
so you can add ``BUILD.bazel`` to your ``.gitignore`` file after deleting
them.

It's likely that you'll have some build files with manual modifications
you want to keep around. At minimum, you'll need to keep the root build
file since it contains the ``gazelle`` rule. The simplest way to keep these
files is to name the ``BUILD`` instead of ``BUILD.bazel``. Bazel and Gazelle
will both recognize files named ``BUILD``, and Git will not ignore them.

Another option is to name your files ``BUILD.bazel.in`` or ``BUILD.in``.
Autogazelle will copy these files to ``BUILD.bazel`` or ``BUILD`` when it
starts running before invoking Gazelle.

How autogazelle works
---------------------

Autogazelle has three components: a wrapper script, a client, and a server.

The *wrapper script* is a bash script installed in ``tools/bazel`` in workspaces
that use autogazelle. The ``bazel`` command installed in your ``PATH`` will look
for an executable file at this location and will execute it instead of the real
bazel. The script builds and runs the *autogazelle client* using ``bazel run``
before invoking the real bazel binary with the original command-line arguments.

The *client* is a Go program that attempts to connect to the *server*
over a UNIX domain socket. If the server isn't running, the client will
start it and connect. Once connected, the client will wait for the server
to disconnect before exiting. The client does no other work.

The *server* is a Go program (actually the same binary as the client, started
with different options) that listens for connections on a UNIX domain socket.
When it accepts a connection, it runs Gazelle using ``bazel run``, then closes
the connection. The socket is just used to make the client wait until Gazelle
completes; no information is exchanged, other than log messages.  While the
server is waiting for a connection, it watches the file system for changes that
could affect build files. When the server runs Gazelle, it runs only in
directories that have changed. This makes Gazelle run much faster. The server
exits after being idle for an hour.

Limitations
-----------

Dependency resolution
~~~~~~~~~~~~~~~~~~~~~

Autogazelle tries to run Gazelle quickly by only updating certain
directories. To support this, it runs Gazelle with the flags ``-r=false``
(don't recurse through selected directories) and ``-index=false`` (don't
build an index of library targets for dependency resolution). This means
that you'll need to set ``external = "vendored"`` explicitly on your ``gazelle``
rule if you have a vendor directory, and you'll need to add
``# gazelle:resolve`` directives in your root build files for any imports
that should be resolved to custom names.

Platform support
~~~~~~~~~~~~~~~~

Autogazelle uses UNIX-domain sockets to synchronize the client and server. This
only works on UNIX-like platforms; these sockets are not supported on Windows.

Autogazelle uses ``github.com/fsnotify/fsnotify`` to watch the file system. This
library works on multiple platforms, but it won't work on file systems that
don't support watches (e.g., NFS and most other network file systems).

Autogazelle has only been tested on Linux. It is intended to work on macOS, but
this has not been tested yet.

Credits
-------

The original idea for running Gazelle automatically was proposed by Matthew
Moore (@mattmoor). The initial plan was to run Gazelle in a repository rule
which regenerate a repository full of build files on each run.

Erick Fejta (@fejta) prototyped this idea for Kubernetes. You can find the
prototype at https://github.com/kubernetes/test-infra/tree/master/autogo.

Documentation

Overview

autogazelle is a program that tracks changes in a workspace and runs gazelle to incorporate those changes into Bazel build files.

autogazelle has two components: a client and a server. The server watches for file system changes within the workspace and builds a set of build files that need to be updated. The server listens on a UNIX socket. When it accepts a connection, it runs gazelle in modified directories and closes the connection without transmitting anything. The client simply connects to the server and waits for the connection to be closed.

autogazelle is intended to be invoked by autogazelle.bash as a bazel wrapper script. It requires the BUILD_WORKSPACE_DIRECTORY environment variable to be set to the workspace root directory and BAZEL_REAL to be set to the local of the real bazel binary.