gddoexp

README

gddoexp

gddoexp (Go Doc Dot Org Expired) was created to indicate if a package from GoDoc should be suppressed from search results. Idea was born from issue 320 of gddo project by @garyburd. The following rules are current applied to verify if a package should be suppressed:

• No other packages reference the analyzed package
• Package wasn't modified in the last 2 years
• Package is a fork with a few commits (fast fork)

A fast fork package is a fork created to made some small changes for a pull request. Currently we tolerate up to 2 commits in a period of 1 week after the fork date.

Install

go get -u github.com/rafaeljusto/gddoexp/...

Remember to add your $GOPATH/bin to your $PATH.

Tools

Please check the specific documentation of each tool in the subdirectories.

Documentation

Overview

Package gddoexp indicate if a package from GoDoc should be suppressed

Index

• Variables
• func AreFastForkPackages(packages []database.Package, auth *GithubAuth) <-chan FastForkResponse
• func IsFastForkPackage(p database.Package, auth *GithubAuth) (fastFork, cache bool, err error)
• func ShouldSuppressPackage(p database.Package, db gddoDB, auth *GithubAuth) (suppress, cache bool, err error)
• func ShouldSuppressPackages(packages []database.Package, db gddoDB, auth *GithubAuth) <-chan SuppressResponse
• type Error
  • func NewError(path string, code ErrorCode, details error) Error
  • func (e Error) Error() string
• type ErrorCode
• type FastForkResponse
• type GithubAuth
  • func (g GithubAuth) String() string
• type SuppressResponse

Variables

var HTTPClient httpClient

HTTPClient is going to be used to perform the Github HTTP requests. We use a global variable, as it is safe for concurrent use by multiple goroutines

var IsCacheResponse func(*http.Response) bool

IsCacheResponse detects if a HTTP response was retrieved from cache or not.

Functions

func AreFastForkPackages

func AreFastForkPackages(packages []database.Package, auth *GithubAuth) <-chan FastForkResponse

AreFastForkPackages determinate if a package is a fast fork or not, but unlike IsFastForkPackage, it can process a list of packages concurrently. An optional argument with the Github authentication can be informed to allow more checks per minute in Github API (we will use token bucket strategy to don't exceed the rate limit).

func IsFastForkPackage

func IsFastForkPackage(p database.Package, auth *GithubAuth) (fastFork, cache bool, err error)

IsFastForkPackage identifies if a package is a fork created only to make small changes for a pull request. An optional argument with the Github authentication can be informed to allow more checks per minute in Github API.

func ShouldSuppressPackage

func ShouldSuppressPackage(p database.Package, db gddoDB, auth *GithubAuth) (suppress, cache bool, err error)

ShouldSuppressPackage determinate if a package should be suppressed or not. It's necessary to inform the GoDoc database to retrieve current stored package information. An optional argument with the Github authentication can be informed to allow more checks per minute in Github API.

func ShouldSuppressPackages

func ShouldSuppressPackages(packages []database.Package, db gddoDB, auth *GithubAuth) <-chan SuppressResponse

ShouldSuppressPackages determinate if a package should be suppressed or not, but unlike ShouldSuppressPackage, it can process a list of packages concurrently. It's necessary to inform the GoDoc database to retrieve current stored package information. An optional argument with the Github authentication can be informed to allow more checks per minute in Github API.

Types

type Error

type Error struct {
	Path    string
	Code    ErrorCode
	Details error
}

Error stores extra information from a low level error indicating the context and path the originated the problem.

func NewError

func NewError(path string, code ErrorCode, details error) Error

NewError will build a godocexp error.

func (Error) Error

func (e Error) Error() string

Error will show the error in a human readable message.

type ErrorCode

type ErrorCode int

ErrorCode stores the type of the error. Useful when we want to perform different actions depending on the error type.

const (
	// ErrorCodeRetrieveImportCounts is used whenever a error occurs while
	// retrieving the import counter from GoDoc database.
	ErrorCodeRetrieveImportCounts ErrorCode = iota

	// ErrorCodeNonGithub is used when the path isn't from Github.
	ErrorCodeNonGithub

	// ErrorCodeGithubFetch is used when there's a problem while retrieving
	// information from Guthub API.
	ErrorCodeGithubFetch

	// ErrorCodeGithubForbidden is used when the rate limit is reached in Github
	// (status 403 Forbidden).
	ErrorCodeGithubForbidden

	// ErrorCodeGithubNotFound is used when the repository wasn't found in Github
	// (status 404 Not Found).
	ErrorCodeGithubNotFound

	// ErrorCodeGithubStatusCode is used when the response status code from
	// Github isn't one of the following: 200 OK, 403 Forbidden or 404 Not Found.
	ErrorCodeGithubStatusCode

	// ErrorCodeGithubParse is used when there's a problem while parsing the
	// JSON response.
	ErrorCodeGithubParse
)

List of possible error codes that can be returned while using this library.

type FastForkResponse

type FastForkResponse struct {
	Path     string
	FastFork bool
	Cache    bool
	Error    error
}

FastForkResponse stores the information of a path verification on an asynchronous check.

type GithubAuth

type GithubAuth struct {
	ID     string
	Secret string
}

GithubAuth store the authentication information to allow a less restrictive rate limit in Github API. Authenticated requests can make up to 5000 requests per hour, otherwise you will be limited in 60 requests per hour (https://developer.github.com/v3/#rate-limiting).

func (GithubAuth) String

func (g GithubAuth) String() string

String build the Github authentication in the request query string format.

type SuppressResponse

type SuppressResponse struct {
	Package  database.Package
	Suppress bool
	Cache    bool
	Error    error
}

SuppressResponse stores the information of a path verification on an asynchronous check.