function

package module
v0.0.0-...-75add70 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 8, 2019 License: Apache-2.0 Imports: 8 Imported by: 0

README

CodeDiaper

You can search for a specific string from all the source code on GitHub and check if it has been posted illegally.

Usage

This package uses below services.

  • GitHub API
  • Slack API(Optional)
  • Google Cloud Functions(Optional)

Motivation

I want to detect when a developer accidentally submits a confidential code to GitHub or misconfigures the Public setting. COPYRIGHT is described as a comment of the code in many confidential codes. This tool aims to detect illegal posts by specifying such strings. It seems that this can be achieved using the standard GitHub API, but it only tells you what is contained somewhere in the file. With this tool, you can more accurately detect suspicious code.

QuickStart(Command Line)

Requirements
  • Go more than 1.11
Steps
  1. Get GitHub API Token
  2. Install go get -u github.com/pj-cancan/code-diaper/cmd/codediaper
  3. Run
codediaper -githubToken <Your GitHub Token> \
  -searchWord="Copyright+{2019,2018,2017}+Future+Corporation" \
  -skipOwners=future-architect \
  -skipRepos=vuls,ap4r,uroborosql \
  -skipLibs=lib/ap4r \
  -slackEnabled=false
  1. Result You can see search result. "Copyright 2019 Future Corporation", "Copyright 2018 Future Corporation", etc.

QuickStart(Google Cloud Functions)

Requirements
Steps
  1. Get GitHub API Token
  2. Get Slack API Token
  3. Set Cloud Scheduler
# Mac/Linux
gcloud beta scheduler jobs create pubsub code-diaper --project <YOUR GCP PROJECT> \
  --schedule "55 23 * * *" \
  --topic topic-code-diaper \
  --message-body='{"search":[{"word_list":"<YOUR SEARCH WORD>", "skip_owners":<YOUR SKIP OWNER LIST>", skip_repos":"<YOUR SKIP LIST>"}]}' \
  --time-zone "Asia/Tokyo" \
  --description "This job invokes CloudFunction of code-diaper"

# Windows
gcloud beta scheduler jobs create pubsub code-diaper --project <YOUR GCP PROJECT> ^
  --schedule "55 23 * * *" ^
  --topic topic-code-diaper ^
  --message-body="{\"search_list\":[{\"queries\":[\"<YOUR SEARCH WORD>\"], "skip_owners":<YOUR SKIP OWNER LIST>", \"skip_repos\":\"<YOUR SKIP LIST>\"}]}" ^
  --time-zone "Asia/Tokyo" ^
  --description "This job invokes CloudFunction of code-diaper"

  1. Deploy to Cloud Functions
gcloud functions deploy codeDiaper --project <YOUR GCP PROJECT> \
  --entry-point Subscribe \
  --trigger-resource topic-code-diaper \
  --trigger-event google.pubsub.topic.publish \
  --timeout=540s \
  --runtime go111 \
  --set-env-vars GITHUB_API_TOKEN=<github-api-token> \
  --set-env-vars SLACK_API_TOKEN=<slack-api-token> \
  --set-env-vars SLACK_CHANNEL=<slack-channel-name>
  1. Go to the Cloud Scheduler page and click the run now button of code-diaper

Example

// TODO

Options

CLI Arg Env Notes Type Example
githubToken GITHUB_API_TOKEN GitHub Access Token Required
searchWord SEARCH_WORDS GitHub Search word. Comma separated. Required apple+orange
skipOwnerList SKIP_OWNER_LIST Skip Owner name list. Comma separated. Optional future-architect
skipRepoList SKIP_REPO_LIST Skip repository name list. Comma separated. Optional repo1,repo2
skipLibList SKIP_LIB_LIST Skip library name list. Comma separated. Optional lib/emoji
slackEnabled --- Skip library name list Optional true / false
slackToken SLACK_API_TOKEN Slack Access Token Optional
slackChannel SLACK_CHANNEL Slack Channel ID Optional

Tips:

The GitHub API has a limit on the maximum number of searches for a term. Therefore, it is necessary to set keywords that will reduce the number of searches as much as possible.

This is a trade-off. If too many keywords are set, there is a risk of missing leaked codes.

If there are many false positives, you can exclude them by adding a skip list.

Developer Guide

Install git pre-commit hook script before developing.

# Windows
git clone https://github.com/pj-cancan/code-diaper
copy /Y .\githooks\*.* .\.git\hooks

# Mac/Linux
git clone https://github.com/pj-cancan/code-diaper
cp githooks/* .git/hooks
chmod +x .git/hooks/pre-commit

License

This project is licensed under the Apache License 2.0 License - see the LICENSE file for details

Documentation

Overview

*

  • Copyright (c) 2019-present Future Corporation *
  • Licensed under the Apache License, Version 2.0 (the "License");
  • you may not use this file except in compliance with the License.
  • You may obtain a copy of the License at *
  • http://www.apache.org/licenses/LICENSE-2.0 *
  • Unless required by applicable law or agreed to in writing, software
  • distributed under the License is distributed on an "AS IS" BASIS,
  • WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  • See the License for the specific language governing permissions and
  • limitations under the License.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Subscribe

func Subscribe(ctx context.Context, msg *pubsub.Message) error

CloudFunction entry point

Types

This section is empty.

Directories

Path Synopsis
cmd
*
*
*
*
*
*
*
*

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL