siso

command
v0.0.0-...-7f005ae Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 24, 2025 License: BSD-3-Clause Imports: 31 Imported by: 0

README

Siso

Siso is a build tool that aims to significantly speed up Chromium's build.

  • It is a drop-in replacement for Ninja, which means it can be easily used instead of Ninja without requiring a migration or change in developer's workflows.
  • It runs build actions on RBE natively, and falls back to local.
  • It avoids stat, disk and network I/O as much as possible.
  • It reduces CPU usage and memory consumption by sharing in one process memory space.
  • It collects performance metrics for each action during a build and allows to analyze them using cloud trace/cloud profiler.

Key Differences from Ninja

  1. Remote execution capabilities:

    • Ninja: Primarily focuses on local build execution. Ninja supports mainly c/cxx remote compiles by integrating with Reclient.
    • Siso: Offers extensive support for various remote execution. You can find Chromium's remote configurations here. The configurations are currently maintained by the Browser Build team.

  2. Handling of missing output files during restat:

    • Ninja: May silently ignore missing output files when performing restat.
    • Siso: Enforces stricter checks. If an output file is missing during restat, Siso will fail the build.

  3. Handling of missing input files during scheduling:

    • Ninja: May silently ignore missing input files when scheduling build steps.
    • Siso: Enforces strict input dependency checks. If an input file is missing when scheduling a build step, Siso will fail the build.

  4. Targets from special syntax target^

    • Ninja: Specifies only the first target that contains the source file with target^. For example, foo.c^ will specify foo.o.
    • Siso: Expands target^ to all the targets that contains the source file. For example, foo.java^ will get expanded to all Java steps that use files such as javac, errorprone, turbine etc.
    • Siso: For a header file, tries to find one of the source files include the header directly. For example, foo.h^ will be treated as foo.cc^.
    • Requested in crbug.com/396522989

  5. Supports phony_output rule variable

    • Ninja: Doesn't have phony_output. But, Android's forked Ninja has a patch for the rule variable. See also here
    • Siso: Supports the variable for Android builds.

  6. Concurrent builds for the same build directory

    • Ninja: Allows multiple build invocations to run for the same build directory. There can be a race problem.
    • Siso: Locks the build directory so that other build invocations would wait until the current invocation to finish.

  7. Handling of depfile parse error

    • Ninja: Ignores a depfile parse error.
    • Siso: Fails for a depfile parse error.

  8. Unsupported features

    Siso may not support Ninja features if they are not used for Chromium builds. e.g. dynamic dependencies, ninja -t browse etc

FAQ

Please check go/siso-faq.

Status

Siso is the primary build system for builder builds, and is being rolled out to Chrome developers. Chromium and Chrome are only supported projects. The projects that import //build/config from Chromium might be able to use Siso. However, they are not tested or supported, yet.

As of Nov 2024, Siso is used by default for Chromium build on gLinux machine.

As of July 2024, Siso is used in all Chromium and Chrome builders, including official builds released to users.

As of end of 2024 Q1, Siso is used in all CQ builders in Chromium.

As of April 2023, we are dogfooding Siso with invited Chrome developers. Please check go/chrome-build-dogfood for more information.

Development

Please check go/siso-development.

References

Documentation

Overview

Siso is a Ninja-compatible build system optimized for remote execution.

Source Files

View all Source files

Directories

Path Synopsis
auth
cred
Package cred provides gRPC / API credentials to authenticate to network services.
 Package cred provides gRPC / API credentials to authenticate to network services.
Package build is the core package of the build tool.
 Package build is the core package of the build tool.
buildconfig
Package buildconfig provides build config for `siso ninja`.
 Package buildconfig provides build config for `siso ninja`.
cachestore
Package cachestore provides the interface CacheStore.
 Package cachestore provides the interface CacheStore.
metadata
Package metadata provides a data structure to hold build metadata.
 Package metadata provides a data structure to hold build metadata.
ninjabuild
Package ninjabuild provides build steps by ninja.
 Package ninjabuild provides build steps by ninja.
Package execute runs commands.
 Package execute runs commands.
localexec
Package localexec implements local command execution.
 Package localexec implements local command execution.
proto
Package proto provides protocol buffer message for execute.
 Package proto provides protocol buffer message for execute.
remoteexec
Package remoteexec executes cmd with remote exec API.
 Package remoteexec executes cmd with remote exec API.
reproxyexec
Package reproxyexec executes cmd with reproxy.
 Package reproxyexec executes cmd with reproxy.
reproxyexec/reproxytest
Package reproxytest provides fake implementation of reproxy for test.
 Package reproxytest provides fake implementation of reproxy for test.
Package hashfs provides a filesystem with digest hash.
 Package hashfs provides a filesystem with digest hash.
osfs
Package osfs provides OS Filesystem access.
 Package osfs provides OS Filesystem access.
proto
Package proto provides protocol buffer message for hashfs.
 Package proto provides protocol buffer message for hashfs.
o11y
clog
Package clog provides context aware logging.
 Package clog provides context aware logging.
iometrics
Package iometrics manages I/O metrics.
 Package iometrics manages I/O metrics.
monitoring
Package monitoring provides Cloud Monitoring (aka Stackdriver) support.
 Package monitoring provides Cloud Monitoring (aka Stackdriver) support.
pprof
Package pprof provides pprof supports.
 Package pprof provides pprof supports.
pprof/proto
Package proto provides protocol buffer message for profiler.
 Package proto provides protocol buffer message for profiler.
resultstore
Package resultstore uploads to resultstore.
 Package resultstore uploads to resultstore.
trace
Package trace manages execution traces.
 Package trace manages execution traces.
Package reapi provides remote execution API.
 Package reapi provides remote execution API.
bytestreamio
Package bytestreamio provides io interfaces on bytestream service.
 Package bytestreamio provides io interfaces on bytestream service.
digest
Package digest handles content digests of remote executon API.
 Package digest handles content digests of remote executon API.
merkletree
Package merkletree operates on a merkle tree for remote execution API.
 Package merkletree operates on a merkle tree for remote execution API.
merkletree/exporter
Package exporter is an exporter of directory tree from RBE-CAS.
 Package exporter is an exporter of directory tree from RBE-CAS.
merkletree/importer
Package importer is an importer of directory tree into RBE-CAS.
 Package importer is an importer of directory tree into RBE-CAS.
proto
Package proto provides protocol buffer message for reapi.
 Package proto provides protocol buffer message for reapi.
reapitest
Package reapitest provides fake implementation of reapi for test.
 Package reapitest provides fake implementation of reapi for test.
retry
Package retry provides retrying functionalities.
 Package retry provides retrying functionalities.
Package runtimex fixes the following API in standard runtime package.
 Package runtimex fixes the following API in standard runtime package.
Package scandeps provides forged C/C++ dependency scanner.
 Package scandeps provides forged C/C++ dependency scanner.
subcmd
authcheck
Package authcheck provides auth_check subcommand.
 Package authcheck provides auth_check subcommand.
fetch
Package fetch is fetch subcommand to fetch data from CAS.
 Package fetch is fetch subcommand to fetch data from CAS.
fscmd
Package fscmd provides fs subcommand.
 Package fscmd provides fs subcommand.
help
Package help provides version subcommand.
 Package help provides version subcommand.
isolate
Package isolate uploads and computes tree digest for each targets.
 Package isolate uploads and computes tree digest for each targets.
metricscmd
Package metricscmd provides metrics subcommand.
 Package metricscmd provides metrics subcommand.
ninja
Package ninja implements the subcommand `ninja` which parses a `build.ninja` file and builds the requested targets.
 Package ninja implements the subcommand `ninja` which parses a `build.ninja` file and builds the requested targets.
ps
Package ps is ps subcommand to list up active steps of ninja build.
 Package ps is ps subcommand to list up active steps of ninja build.
query
Package query is ninja_query subcommand to query ninja build graph.
 Package query is ninja_query subcommand to query ninja build graph.
recall
Package recall is recall subcommand to recall action by the digest and execute cmd with remote exec API.
 Package recall is recall subcommand to recall action by the digest and execute cmd with remote exec API.
report
Package report is report subcommand to report siso logs.
 Package report is report subcommand to report siso logs.
scandeps
Package scandeps is scandeps subcommand for debugging scandeps.
 Package scandeps is scandeps subcommand for debugging scandeps.
version
Package version provides version subcommand.
 Package version provides version subcommand.
webui
Package webui provides webui subcommand.
 Package webui provides webui subcommand.
sync
semaphore
Package semaphore provives semaphore.
 Package semaphore provives semaphore.
toolsupport
artfsutil
Package artfsutil provides access to artfs.
 Package artfsutil provides access to artfs.
artfsutil/proto
Package proto provides protocol buffer message for artfs.
 Package proto provides protocol buffer message for artfs.
artfsutil/proto/artfs
Package artfs provides protocol buffer message for artfs.
 Package artfs provides protocol buffer message for artfs.
artfsutil/proto/manifest
Package manifest provides protocol buffer message for artfs manifest.
 Package manifest provides protocol buffer message for artfs manifest.
ciderutil/proto
Package proto provides protocol buffer message for ide analysis.
 Package proto provides protocol buffer message for ide analysis.
cmdutil
Package cmdutil provides utilities for cmd.exe.
 Package cmdutil provides utilities for cmd.exe.
cogutil
Package cogutil provides access to cog system.
 Package cogutil provides access to cog system.
cogutil/proto
Package proto provides protocol buffer message for cog system.
 Package proto provides protocol buffer message for cog system.
gccutil
Package gccutil provides utilities of gcc.
 Package gccutil provides utilities of gcc.
makeutil
Package makeutil provides utilities for make.
 Package makeutil provides utilities for make.
msvcutil
Package msvcutil provides utilities of msvc.
 Package msvcutil provides utilities of msvc.
ninjautil
Package ninjautil provides utilitites for ninja.
 Package ninjautil provides utilitites for ninja.
shutil
Package shutil provides utilities for unix shell.
 Package shutil provides utilities for unix shell.
straceutil
Package straceutil provides utilities for strace.
 Package straceutil provides utilities for strace.
watchmanutil
Package watchmanutil provides utilities of watchman.
 Package watchmanutil provides utilities of watchman.
Package ui provides user interface functionalities.
 Package ui provides user interface functionalities.
Package webui implements siso webui.
 Package webui implements siso webui.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.