octocov

README

octocov

octocov is a tool for collecting code metrics (code coverage, code to test ratio and test execution time).

Key features of octocov are:

• Support multiple coverage report formats.
• Support multiple code metrics.
• Support for even generating coverage report badge.
• Have a mechanism to aggregate reports from multiple repositories.

:octocat: GitHub Actions for octocov is here !!

Usage

First, run test with coverage report output.

For example, in case of Go language, add -coverprofile=coverage.out option as follows

$ go test ./... -coverprofile=coverage.out

Add .octocov.yml ( or octocov.yml ) file to your repository, and run octocov

$ octocov

Comment report to pull request

By setting comment:, comment the reports to pull request.

# .octocov.yml
comment:
  enable: true
  hideFooterLink: false # hide octocov link

octocov checks for "Code Coverage" by default. If it is running on GitHub Actions, it will also measure "Test Execution Time".

If you want to measure "Code to Test Ratio", set codeToTestRatio:.

comment:
  enable: true
codeToTestRatio:
  code:
    - '**/*.go'
    - '!**/*_test.go'
  test:
    - '**/*_test.go'

Check for acceptable coverage

By setting coverage.acceptable:, the minimum acceptable coverage is specified.

If it is less than that value, the command will exit with exit status 1.

# .octocov.yml
coverage:
  acceptable: 60%

$ octocov
Error: code coverage is 54.9%, which is below the accepted 60.0%

Check for acceptable code to test ratio

By setting codeToTestRatio.acceptable:, the minimum acceptable "Code to Test Ratio" is specified.

If it is less than that value, the command will exit with exit status 1.

# .octocov.yml
codeToTestRatio:
  code:
    - '**/*.go'
    - '!**/*_test.go'
  test:
    - '**/*_test.go'
  acceptable: 1:1.2

$ octocov
Error: code to test ratio is 1:1.1, which is below the accepted 1:1.2

Check for acceptable test execution time

(on GitHub Actions only)

By setting testExecutionTime.acceptable:, the maximum acceptable "Test Execution Time" is specified.

If it is greater than that value, the command will exit with exit status 1.

# .octocov.yml
testExecutionTime:
  acceptable: 1 min

$ octocov
Error: test execution time is 1m15s, which is below the accepted 1m

Generate report badges self.

By setting coverage.badge.path:, generate the coverage report badge self.

# .octocov.yml
coverage:
  badge:
    path: docs/coverage.svg

By setting codeToTestRatio.badge.path:, generate the code-to-test-ratio report badge self.

# .octocov.yml
codeToTestRatio:
  badge:
    path: docs/ratio.svg

By setting testExecutionTime.badge.path:, generate the test-execution-time report badge self (on GitHub Actions only).

# .octocov.yml
testExecutionTime:
  badge:
    path: docs/time.svg

You can display the coverage badge without external communication by setting a link to this badge image in README.md, etc.

# mytool

![coverage](docs/coverage.svg)

Push report badges self.

By setting push.enable:, git push report badges self.

# .octocov.yml
coverage:
  badge:
    path: docs/coverage.svg
push:
  enable: true

Store report to central datastore

By setting datastore:, store the reports to central datastore.

GitHub

# .octocov.yml
datastore:
  github:
    repository: owner/coverages # central datastore repository
    branch: main                # default: main
    path:                       # default: reports/${GITHUB_REPOSITORY}/report.json

Required environment variables:

• GITHUB_TOKEN
• GITHUB_REPOSITORY
• GITHUB_API_URL (optional)

S3

# .octocov.yml
datastore:
  s3:
    bucket: my-coverage # datastore bucket
    path:               # default: reports/${GITHUB_REPOSITORY}/report.json

Required permission:

• s3:PutObject

Required environment variables:

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_SESSION_TOKEN (optional)

GCS

# .octocov.yml
datastore:
  gcs:
    bucket: my-coverage # datastore bucket
    path:               # default: reports/${GITHUB_REPOSITORY}/report.json

Required permission:

• storage.objects.create
• storage.objects.delete

Required environment variables:

• GOOGLE_APPLICATION_CREDENTIALS

BigQuery

# .octocov.yml
datastore:
  bq:
    project: my-octocov-project # project ID
    dataset: octocov            # dataset ID
    table:                      # table. default: reports

Required permission:

• bigquery.datasets.get
• bigquery.tables.updateData

Required environment variables:

• GOOGLE_APPLICATION_CREDENTIALS

Datastore schema:

Datastore schema

If you want to create a table, execute the following command ( require bigquery.datasets.create ).

$ octocov --create-bq-table

If section

# .octocov.yml
datastore:
  if: env.GITHUB_REF == 'refs/heads/main'
  github:
    repository: owner/coverages

The variables available in the if section are as follows

Variable name	Type	Description
year	int	Year of current time (UTC)
month	int	Month of current time (UTC)
day	int	Day of current time (UTC)
hour	int	Hour of current time (UTC)
weekday	int	Weekday of current time (UTC) (Sunday = 0, ...)
github.event_name	string	Event name of GitHub Actions ( ex. issues, pull_request )
github.event	object	Detailed data for each event of GitHub Actions (ex. github.event.action, github.event.label.name )
env.<env_name>	string	The value of a specific environment variable

Central mode

By enabling central:, octocov acts as a central repository for collecting reports ( example ).

# .octocov.yml for central mode
central:
  enable: true
  root: .          # root directory or index file path of collected coverage reports pages. default: .
  reports: reports # datastore path (url) where reports are stored. default: reports
  badges: badges   # directory where badges are generated. default: badges
  push:
    enable: true   # enable self git push

Use GitHub repository as datastore

When using the central repository as a datastore, perform badge generation via on.push.

# .octocov.yml
datastore:
  github:
    repository: owner/central-repo

# .octocov.yml for central repo
central:
  enable: true
  reports: reports
  push:
    enable: true

Use S3 bucket as datastore

When using the S3 bucket as a datastore, perform badge generation via on.schedule.

# .octocov.yml
datastore:
  s3:
    bucket: my-s3-bucket

# .octocov.yml for central repo
central:
  enable: true
  reports: s3://my-s3-bucket/reports
  push:
    enable: true

Required permission (Central Repo):

• s3:GetObject
• s3:ListObject

Required environment variables (Central Repo):

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_SESSION_TOKEN (optional)

Use GCS bucket as datastore

When using the GCS bucket as a datastore, perform badge generation via on.schedule.

# .octocov.yml
datastore:
  gcs:
    bucket: my-gcs-bucket

# .octocov.yml for central repo
central:
  enable: true
  reports: gs://my-gcs-bucket/reports
  push:
    enable: true

Required permission (Central Repo):

• storage.objects.get
• storage.objects.list
• storage.buckets.get

Required environment variables (Central Repo):

• GOOGLE_APPLICATION_CREDENTIALS

Use BigQuery table as datastore

When using the BigQuery table as a datastore, perform badge generation via on.schedule.

# .octocov.yml
datastore:
  bq:
    project: my-project
    dataset: my-dataset

# .octocov.yml for central repo
central:
  enable: true
  reports: bq://my-project/my-dataset/reports
  push:
    enable: true

Required permission (Central Repo):

• bigquery.jobs.create
• bigquery.tables.getData

Required environment variables (Central Repo):

• GOOGLE_APPLICATION_CREDENTIALS

:NOTICE: When central mode is enabled, other functions are automatically turned off.

Supported coverage report formats

octocov supports multiple coverage report formats.

And octocov searches for the default path for each format.

If you want to specify the path of the report file, set coverage.path

coverage:
  path: /path/to/coverage.txt

Go coverage

Default path: coverage.out

LCOV

Default path: coverage/lcov.info

Support SF DA only

SimpleCov

Default path: coverage/.resultset.json

Clover

Default path: coverage.xml

Cobertura

Default path: coverage.xml

Supported code metrics

• Code Coverage
• Code to Test Ratio
• Test Execution Time (on GitHub Actions only)

Install

deb:

Use dpkg-i-from-url

$ export OCTOCOV_VERSION=X.X.X
$ curl -L https://git.io/dpkg-i-from-url | bash -s -- https://github.com/k1LoW/octocov/releases/download/v$OCTOCOV_VERSION/octocov_$OCTOCOV_VERSION-1_amd64.deb

RPM:

$ export OCTOCOV_VERSION=X.X.X
$ yum install https://github.com/k1LoW/octocov/releases/download/v$OCTOCOV_VERSION/octocov_$OCTOCOV_VERSION-1_amd64.rpm

apk:

Use apk-add-from-url

$ export OCTOCOV_VERSION=X.X.X
$ curl -L https://git.io/apk-add-from-url | sh -s -- https://github.com/k1LoW/octocov/releases/download/v$OCTOCOV_VERSION/octocov_$OCTOCOV_VERSION-1_amd64.apk

homebrew tap:

$ brew install k1LoW/tap/octocov

manually:

Download binary from releases page

go get:

$ go get github.com/k1LoW/octocov

docker:

$ docker pull ghcr.io/k1low/octocov:latest

Documentation

Overview

Copyright © 2021 Ken'ichiro Oyama <k1lowxb@gmail.com>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.