httpclient

README

httpclient-lib-go

This is a Go comnmon lib for crawler, publiccode-parser-go and publiccode-validator

publiccode.yml is an international standard for describing public software. It is expected to be published in the root of open source repositories. This parser performs syntactic and semantic validation according to the official spec.

Contributing

Contributing is always appreciated. Feel free to open issues, fork or submit a Pull Request. If you want to know more about how to add new fields, check out CONTRIBUTING.md. In order to support other country-specific extensions in addition to Italy some refactoring might be needed.

See also

• Developers Italia backend & crawler - a Go crawler for PC.
• publiccode-parser-go - a Go parser and validator for publiccode.yml files.
• publiccode-validator - a REST API server for publiccode validation.

Maintainers

This software is maintained by the Developers Italia team.

License

© 2020 Team per la Trasformazione Digitale - Presidenza del Consiglio dei Minstri

Licensed under the AGPL-3.0. The version control system provides attribution for specific lines of code.

Documentation

Index

• func HeaderLink(linkHeader, command string) string
• type HTTPResponse
  • func GetURL(URL string, headers map[string]string) (HTTPResponse, error)
  • func PostURL(URL string, headers map[string]string, body io.Reader) (HTTPResponse, error)
  • func Request(URL string, verb string, headers map[string]string, body io.Reader) (HTTPResponse, error)
• type ResponseStatus

Functions

func HeaderLink

func HeaderLink(linkHeader, command string) string

HeaderLink parse the Github Header Link to "next"/"last"/"first"/"prev" link of repositories. Example: HeaderLink(link,"next") or HeaderLink(link, "prev") or HeaderLink(link,"last").

Types

type HTTPResponse

type HTTPResponse struct {
	Body    []byte
	Status  ResponseStatus
	Headers http.Header
}

HTTPResponse wraps body, Status and Headers from the http.Response.

func GetURL

func GetURL(URL string, headers map[string]string) (HTTPResponse, error)

GetURL retrieves data, status and response headers from an URL. It uses some technique to slow down the requests if it get a 429 (Too Many Requests) response.

func PostURL

func PostURL(URL string, headers map[string]string, body io.Reader) (HTTPResponse, error)

PostURL retrieves data, status and response headers from an URL. It uses some technique to slow down the requests if it get a 429 (Too Many Requests) response.

func Request

func Request(URL string, verb string, headers map[string]string, body io.Reader) (HTTPResponse, error)

Request retrieves data, status and response headers from an URL. It uses some technique to slow down the requests if it get a 429 (Too Many Requests) response.

type ResponseStatus

type ResponseStatus struct {
	Text string // e.g. "200 OK"
	Code int    // e.g. 200
}

ResponseStatus contains the status and statusCode of a response.