proxyscrape

Documentation

Overview

package designed to contact proxyscrape's API and pull down a list of proxies based on the options the user selects.

API documentation:

https://docs.proxyscrape.com/#1ec9e5ed-0dce-4511-91e1-ebe99f7bd88d

Index

• Constants
• func ValidateTimeout(timeout int) (err error)
• type OptsFunc
  • func UsingAnonymity(anontype string) OptsFunc
  • func UsingCountry(country string) OptsFunc
  • func UsingProtocol(protocol string) OptsFunc
  • func UsingSSL(ssltype string) OptsFunc
  • func UsingTimeout(timeout int) OptsFunc
• type ProxyList
• type ProxyScraper
  • func NewProxyScraper(opts ...OptsFunc) (scraper *ProxyScraper, err error)
  • func (p *ProxyScraper) GetProxies() (err error)
• type ScraperOptions

Constants

const ANONYMITY_ALL int = 0

set anonymity level to "all".

const ANONYMITY_ANONYMOUS int = 2

set anonymity level to "anonymous".

const ANONYMITY_ELITE int = 3

set anonymity level to "elite".

const ANONYMITY_TRANSPARENT int = 1

set anonymity level to "transparent".

const BASE_URL string = "https://api.proxyscrape.com/v2"

this is the API url for proxyscrape v2.

const HTTPS_ALL int = 0

set the ssl option to "all"

const HTTPS_NO int = 2

set the ssl option to "no"

const HTTPS_YES int = 1

set the ssl option to "yes"

const MAX_TIMEOUT int = 10000

maximum allowed timeout value that ProxyScraper can get.

const PROTO_ALL int = 0

set protocol to all

const PROTO_HTTP int = 1

set protocol to http

const PROTO_SOCKS4 int = 2

set protocol to socks4

const PROTO_SOCKS5 int = 3

set protocol to socks5

Functions

func ValidateTimeout

func ValidateTimeout(timeout int) (err error)

function designed to validate the timeout value for the ProxyScraper.

Types

type OptsFunc

type OptsFunc func(*ScraperOptions) error

function alias used to set the values of a ScraperOptions struct. this is used in the NewProxyScraper function.

func UsingAnonymity

func UsingAnonymity(anontype string) OptsFunc

function designed to set the anonymity option for the current ScraperOptions struct.

func UsingCountry

func UsingCountry(country string) OptsFunc

function designed to set the country option for the current ScraperOptions struct.

func UsingProtocol

func UsingProtocol(protocol string) OptsFunc

function designed to set the protocol option for the current ScraperOptions struct.

func UsingSSL

func UsingSSL(ssltype string) OptsFunc

function designed to set the ssl option for the current ScraperOptions struct.

func UsingTimeout

func UsingTimeout(timeout int) OptsFunc

function designed to set the timeout option for the current ScraperOptions struct.

type ProxyList

type ProxyList struct {
	Proxies []string
}

this represents the data that will be returned by proxyscrape when the API is hit.

type ProxyScraper

type ProxyScraper struct {

	// list of proxies returned by proxyscrape.
	Proxies ProxyList
	// contains filtered or unexported fields
}

this is the object that will be used to contact the API and pull down and store the proxies list.

func NewProxyScraper

func NewProxyScraper(opts ...OptsFunc) (scraper *ProxyScraper, err error)

function designed to create, initialize and return a new ProxyScraper instance that can be used to contact the API.

func (*ProxyScraper) GetProxies

func (p *ProxyScraper) GetProxies() (err error)

function designed to contact the ProxyScraper API and pull down a list of proxies.

type ScraperOptions

type ScraperOptions struct {

	// this represents the anonymity level desired
	// for the proxy. this only applies to HTTP(S) proxies.
	// the values for this are defined as constants.
	Anonymity int

	// this represents the country that the proxy is
	// located in. it is the 2 letter country code.
	// to ignore country, this should be set to all.
	Country string

	// the proxy protocol to filter on. if set to all
	// this will grab all protocols.
	Protocol int

	// specifies whether the proxy can use HTTPS. the
	// only options are "yes", "no", and "all". these
	// are defined as constants.
	SSL int

	// specified the max timeout for the proxy (limit 10K).
	Timeout int
}