README
¶
gh-sql
GitHub SQL is a tool to fetch data from the GitHub API and store it in a SQL database. The ultimate goal is to have the data locally to perform queries on the dataset.
The tool has the following focuses:
- Being targeted towards storing information about specific repositories.
- Discovery of users or repositories is a non-goal.
- Being able to incrementally update the database, fetching what has changed in a repository using the events API (either through polling, even in long intervals (up to 1 month) or through WebSockets).
- Being able to easily "saved" in a single SQL file for quick recovery and replication.
- Replicating the GitHub REST API in read-only operations, without rate limiting (for integration with existing tools that use the GitHub API, by only changing the API endpoint.)
Documentation
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
schema
Package schema specifies the ent schema types for gh-sql's database.
|
Package schema specifies the ent schema types for gh-sql's database. |
|
pkg
|
|
|
model
Package model defines some types used within the ent schemas.
|
Package model defines some types used within the ent schemas. |
|
model/internal/jsoncatcher
Package jsoncatcher compares re-emitted, parsed JSON data with its raw counterpart to find fields which have not been correctly parsed.
|
Package jsoncatcher compares re-emitted, parsed JSON data with its raw counterpart to find fields which have not been correctly parsed. |
|
rest
Package rest implements a REST API for gh-sql, mimicking the GitHub REST API.
|
Package rest implements a REST API for gh-sql, mimicking the GitHub REST API. |
|
sync
Package sync implements a system to fetch GitHub repositories concurrently.
|
Package sync implements a system to fetch GitHub repositories concurrently. |
Click to show internal directories.
Click to hide internal directories.