README
¶
CodeDiaper
You can search for a specific string from all the source code on GitHub and check if it has been posted illegally.
Usage
This package uses below services.
- GitHub API
- Slack API(Optional)
- Google Cloud Functions(Optional)
Motivation
I want to detect when a developer accidentally submits a confidential code to GitHub or misconfigures the Public setting. COPYRIGHT is described as a comment of the code in many confidential codes. This tool aims to detect illegal posts by specifying such strings. It seems that this can be achieved using the standard GitHub API, but it only tells you what is contained somewhere in the file. With this tool, you can more accurately detect suspicious code.
QuickStart(Command Line)
Requirements
- Go more than 1.11
Steps
- Get GitHub API Token
- Install
go get -u github.com/pj-cancan/code-diaper/cmd/codediaper
- Run
codediaper -githubToken <Your GitHub Token> \
-searchWord="Copyright+{2019,2018,2017}+Future+Corporation" \
-skipOwners=future-architect \
-skipRepos=vuls,ap4r,uroborosql \
-skipLibs=lib/ap4r \
-slackEnabled=false
- Result You can see search result. "Copyright 2019 Future Corporation", "Copyright 2018 Future Corporation", etc.
QuickStart(Google Cloud Functions)
Requirements
Steps
- Get GitHub API Token
- Get Slack API Token
- Set Cloud Scheduler
# Mac/Linux
gcloud beta scheduler jobs create pubsub code-diaper --project <YOUR GCP PROJECT> \
--schedule "55 23 * * *" \
--topic topic-code-diaper \
--message-body='{"search":[{"word_list":"<YOUR SEARCH WORD>", "skip_owners":<YOUR SKIP OWNER LIST>", skip_repos":"<YOUR SKIP LIST>"}]}' \
--time-zone "Asia/Tokyo" \
--description "This job invokes CloudFunction of code-diaper"
# Windows
gcloud beta scheduler jobs create pubsub code-diaper --project <YOUR GCP PROJECT> ^
--schedule "55 23 * * *" ^
--topic topic-code-diaper ^
--message-body="{\"search_list\":[{\"queries\":[\"<YOUR SEARCH WORD>\"], "skip_owners":<YOUR SKIP OWNER LIST>", \"skip_repos\":\"<YOUR SKIP LIST>\"}]}" ^
--time-zone "Asia/Tokyo" ^
--description "This job invokes CloudFunction of code-diaper"
- Deploy to Cloud Functions
gcloud functions deploy codeDiaper --project <YOUR GCP PROJECT> \
--entry-point Subscribe \
--trigger-resource topic-code-diaper \
--trigger-event google.pubsub.topic.publish \
--timeout=540s \
--runtime go111 \
--set-env-vars GITHUB_API_TOKEN=<github-api-token> \
--set-env-vars SLACK_API_TOKEN=<slack-api-token> \
--set-env-vars SLACK_CHANNEL=<slack-channel-name>
- Go to the Cloud Scheduler page and click the run now button of code-diaper
Example
// TODO
Options
CLI Arg | Env | Notes | Type | Example |
---|---|---|---|---|
githubToken | GITHUB_API_TOKEN | GitHub Access Token | Required | |
searchWord | SEARCH_WORDS | GitHub Search word. Comma separated. | Required | apple+orange |
skipOwnerList | SKIP_OWNER_LIST | Skip Owner name list. Comma separated. | Optional | future-architect |
skipRepoList | SKIP_REPO_LIST | Skip repository name list. Comma separated. | Optional | repo1,repo2 |
skipLibList | SKIP_LIB_LIST | Skip library name list. Comma separated. | Optional | lib/emoji |
slackEnabled | --- | Skip library name list | Optional | true / false |
slackToken | SLACK_API_TOKEN | Slack Access Token | Optional | |
slackChannel | SLACK_CHANNEL | Slack Channel ID | Optional |
Tips:
The GitHub API has a limit on the maximum number of searches for a term. Therefore, it is necessary to set keywords that will reduce the number of searches as much as possible.
This is a trade-off. If too many keywords are set, there is a risk of missing leaked codes.
If there are many false positives, you can exclude them by adding a skip list.
Developer Guide
Install git pre-commit hook script before developing.
# Windows
git clone https://github.com/pj-cancan/code-diaper
copy /Y .\githooks\*.* .\.git\hooks
# Mac/Linux
git clone https://github.com/pj-cancan/code-diaper
cp githooks/* .git/hooks
chmod +x .git/hooks/pre-commit
License
This project is licensed under the Apache License 2.0 License - see the LICENSE file for details
Documentation
¶
Overview ¶
*
- Copyright (c) 2019-present Future Corporation *
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at *
- http://www.apache.org/licenses/LICENSE-2.0 *
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
This section is empty.