session

Documentation

Index

• type BasicAuth
• type CtxArg
• type Session
  • func NewSession(query string, proxy string, multiRateLimiter *ratelimit.MultiLimiter, ...) (*Session, error)
  • func (s *Session) Close()
  • func (s *Session) DiscardHTTPResponse(response *http.Response)
  • func (s *Session) Get(ctx context.Context, getURL, cookies string, headers map[string]string) (*http.Response, error)
  • func (s *Session) HTTPRequest(ctx context.Context, method, requestURL, cookies string, ...) (*http.Response, error)
  • func (s *Session) Post(ctx context.Context, postURL, cookies string, headers map[string]string, ...) (*http.Response, error)
  • func (s *Session) SimpleGet(ctx context.Context, getURL string) (*http.Response, error)
  • func (s *Session) SimplePost(ctx context.Context, postURL, contentType string, body io.Reader) (*http.Response, error)

Types

type BasicAuth

type BasicAuth struct {
	Username string
	Password string
}

BasicAuth request's Authorization header

type CtxArg

type CtxArg string

const (
	CtxSourceArg CtxArg = "source"
)

type Session

type Session struct {
	//DomainExtractor
	Extractor extractor.DomainExtractor
	// Client is the current http client
	Client           *http.Client
	MultiRateLimiter *ratelimit.MultiLimiter
}

Session is the option passed to the source, an option is created uniquely for each source.

func NewSession

func NewSession(query string, proxy string, multiRateLimiter *ratelimit.MultiLimiter, timeout int) (*Session, error)

NewSession creates a new session object for a domain

func (*Session) Close

func (s *Session) Close()

Close the session

func (*Session) DiscardHTTPResponse

func (s *Session) DiscardHTTPResponse(response *http.Response)

DiscardHTTPResponse discards the response content by demand

func (*Session) Get

func (s *Session) Get(ctx context.Context, getURL, cookies string, headers map[string]string) (*http.Response, error)

Get makes a GET request to a URL with extended parameters

func (*Session) HTTPRequest

func (s *Session) HTTPRequest(ctx context.Context, method, requestURL, cookies string, headers map[string]string, body io.Reader, basicAuth BasicAuth) (*http.Response, error)

HTTPRequest makes any HTTP request to a URL with extended parameters

func (*Session) Post

func (s *Session) Post(ctx context.Context, postURL, cookies string, headers map[string]string, body io.Reader) (*http.Response, error)

Post makes a POST request to a URL with extended parameters

func (*Session) SimpleGet

func (s *Session) SimpleGet(ctx context.Context, getURL string) (*http.Response, error)

SimpleGet makes a simple GET request to a URL

func (*Session) SimplePost

func (s *Session) SimplePost(ctx context.Context, postURL, contentType string, body io.Reader) (*http.Response, error)

SimplePost makes a simple POST request to a URL