chromedpundetected

package module
v0.0.0-...-63673d3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 25, 2023 License: MIT Imports: 24 Imported by: 0

README

Undetected chromedp

Go.Dev reference Go Report Card Unit Tests GitHub

A small library that provides a chromedp context with a browser configured to mimick a regular browser to prevent triggering anti-bot measures. This is not a fool proof method, and how you use it will still dictate whether you will run into anti-bot detection, but at least it won't trigger on all the basic detection tests.

The headless option only works on linux, and requires Xvfb to be installed. Could theoretically work on Mac OS with xquartz but I don't have a Mac to test with, so feel free to PR.

A Docker container example is provided in Dockerfile. The most important things to note is to not use the headless chrome image as base, but to normally install chrome or chromium, and to install xvfb. Note that this image is neither secure nor optimized, and merely serves as an example.

package main

import (
	"time"

	"github.com/chromedp/chromedp"

	cu "github.com/Davincible/chromedp-undetected"
)

func main() {
	// New creates a new context for use with chromedp. With this context
	// you can use chromedp as you normally would.
	ctx, cancel, err := cu.New(cu.NewConfig(
		// Remove this if you want to see a browser window.
		cu.WithHeadless(),

		// If the webelement is not found within 10 seconds, timeout.
		cu.WithTimeout(10 * time.Second),
	))
	if err != nil {
		panic(err)
	}
	defer cancel()

	if err := chromedp.Run(ctx,
		// Check if we pass anti-bot measures.
		chromedp.Navigate("https://nowsecure.nl"),
		chromedp.WaitVisible(`//div[@class="hystericalbg"]`),
	); err != nil {
		panic(err)
	}

	fmt.Println("Undetected!")
}

Based on undetected-chromedriver

Utilities

Some utility functions are included I was missing in chromedp itself.

// BlockURLs blocks a set of URLs in Chrome.
func BlockURLs(url ...string) chromedp.ActionFunc 

// LoadCookies will load a set of cookies into the browser.
func LoadCookies(cookies []Cookie) chromedp.ActionFunc

// LoadCookiesFromFile takes a file path to a json file containing cookies,
// and loads in the cookies into the browser.
func LoadCookiesFromFile(path string) chromedp.ActionFunc

// SaveCookies extracts the cookies from the current URL and appends them to
// provided array.
func SaveCookies(cookies *[]Cookie) chromedp.ActionFunc

// SaveCookiesTo extracts the cookies from the current page and saves them
// as JSON to the provided path.
func SaveCookiesTo(path string) chromedp.ActionFunc

// RunCommand runs any Chrome Dev Tools command, with any params.
// 
// In contrast to the native method of chromedp, with this method you can
// directly pass in a map with the data passed to the command.
func RunCommand(method string, params any) chromedp.ActionFunc

// RunCommandWithRes runs any Chrome Dev Tools command, with any params and
// sets the result to the res parameter. Make sure it is a pointer.
// 
// In contrast to the native method of chromedp, with this method you can
// directly pass in a map with the data passed to the command.
func RunCommandWithRes(method string, params, res any) chromedp.ActionFunc

// UserAgentOverride overwrites the Chrome user agent.
// 
// It's better to use this method than emulation.UserAgentOverride.
func UserAgentOverride(userAgent string) chromedp.ActionFunc

// SendKeys does the same as chromedp.SendKeys excepts it randomly waits 100-500ms
// between sending key presses.
func SendKeys(sel any, v string, opts ...chromedp.QueryOption) chromedp.ActionFunc

Documentation

Overview

Package chromedpundetected provides a chromedp context with an undetected Chrome browser.

Package chromedpundetected provides a chromedp context with an undetected Chrome browser.

Index

Constants

View Source 
const (
	// DefaultNoSandbox enables the 'no-sandbox' flag by default.
	DefaultNoSandbox = true
)

Variables

View Source 
var (
	DefaultUserDirPrefix = "chromedp-undetected-"
)

Defaults.

View Source 
var (
	ErrXvfbNotFound = errors.New("xvfb not found. Please install (Linux only)")
)

Errors.

Functions

func BlockURLs 

func BlockURLs(url ...string) chromedp.ActionFunc

BlockURLs blocks a set of URLs in Chrome.

func LoadCookies 

func LoadCookies(cookies []Cookie) chromedp.ActionFunc

LoadCookies will load a set of cookies into the browser.

func LoadCookiesFromFile 

func LoadCookiesFromFile(path string) chromedp.ActionFunc

LoadCookiesFromFile takes a file path to a json file containing cookies, and loads in the cookies into the browser.

func New 

func New(config Config) (context.Context, context.CancelFunc, error)

New creates a context with an undetected Chrome executor.

func RunCommand 

func RunCommand(method string, params any) chromedp.ActionFunc

RunCommand runs any Chrome Dev Tools command, with any params.

In contrast to the native method of chromedp, with this method you can directly pass in a map with the data passed to the command.

func RunCommandWithRes 

func RunCommandWithRes(method string, params, res any) chromedp.ActionFunc

RunCommandWithRes runs any Chrome Dev Tools command, with any params and sets the result to the res parameter. Make sure it is a pointer.

In contrast to the native method of chromedp, with this method you can directly pass in a map with the data passed to the command.

func SaveCookies 

func SaveCookies(cookies *[]Cookie) chromedp.ActionFunc

SaveCookies extracts the cookies from the current URL and appends them to provided array.

func SaveCookiesTo 

func SaveCookiesTo(path string) chromedp.ActionFunc

SaveCookiesTo extracts the cookies from the current page and saves them as JSON to the provided path.

func SendKeys 

func SendKeys(sel any, v string, opts ...chromedp.QueryOption) chromedp.ActionFunc

SendKeys does the same as chromedp.SendKeys excepts it randomly waits 100-500ms between sending key presses.

func UserAgentOverride 

func UserAgentOverride(userAgent string) chromedp.ActionFunc

UserAgentOverride overwrites the Chrome user agent.

It's better to use this method than emulation.UserAgentOverride.

Types

type Config 

type Config struct {
	// Ctx is the base context to use. By default a background context will be used.
	Ctx context.Context `json:"-" yaml:"-"`

	// ContextOptions are chromedp context option.
	ContextOptions []chromedp.ContextOption `json:"-" yaml:"-"`

	// ChromeFlags are additional Chrome flags to pass to the browser.
	//
	// NOTE: adding additional flags can make the detection unstable, so test,
	// and be careful of what flags you add. Mostly intended to configure things
	// like a proxy. Also check if the flags you want to set are not already set
	// by this library.
	ChromeFlags []chromedp.ExecAllocatorOption

	// UserDataDir is the path to the directory where Chrome user data is stored.
	//
	// By default a temporary directory will be used.
	UserDataDir string `json:"userDataDir" yaml:"userDataDir"`

	// LogLevel is the Chrome log level, 0 by default.
	LogLevel int `json:"logLevel" yaml:"logLevel"`

	// NoSandbox dictates whether the no-sanbox flag is added. Defaults to true.
	NoSandbox bool `json:"noSandbox" yaml:"noSandbox"`

	// ChromePath is a specific binary path for Chrome.
	//
	// By default the chrome or chromium on your PATH will be used.
	ChromePath string `json:"chromePath" yaml:"chromePath"`

	// Port is the Chrome debugger port. By default a random port will be used.
	Port int `json:"port" yaml:"port"`

	// Timeout is the context timeout.
	Timeout time.Duration `json:"timeout" yaml:"timeout"`

	// Headless dicates whether Chrome will start headless (without a visible window)
	//
	// It will NOT use the '--headless' option, rather it will use a virtual display.
	// Requires Xvfb to be installed, only available on Linux.
	Headless bool `json:"headless" yaml:"headless"`

	// language to be used otherwise system/OS defaults are used
	// https://developer.chrome.com/docs/webstore/i18n/#localeTable
	Language string
}

Config is a undetected Chrome config.

func NewConfig 

func NewConfig(opts ...Option) Config

NewConfig creates a new config object with defaults. 

type Cookie struct {
	Name     string  `json:"name" yaml:"name"`
	Value    string  `json:"value" yaml:"value"`
	Domain   string  `json:"domain" yaml:"domain"`
	Path     string  `json:"path" yaml:"path"`
	Expires  float64 `json:"expires" yaml:"expires"`
	HTTPOnly bool    `json:"httpOnly" yaml:"httpOnly"`
	Secure   bool    `json:"secure" yaml:"secure"`
}

Cookie is used to set browser cookies.

type Option 

type Option func(*Config)

Option is a functional option.

func WithChromeBinary 

func WithChromeBinary(path string) Option

WithChromeBinary sets the chrome binary path.

func WithChromeFlags 

func WithChromeFlags(opts ...chromedp.ExecAllocatorOption) Option

WithChromeFlags add chrome flags.

func WithContext 

func WithContext(ctx context.Context) Option

WithContext adds a base context.

func WithHeadless 

func WithHeadless() Option

WithHeadless creates a headless chrome instance.

func WithLogLevel 

func WithLogLevel(level int) Option

WithLogLevel sets the chrome log level.

func WithNoSandbox 

func WithNoSandbox(b bool) Option

WithNoSandbox enable/disable sandbox. Disabled by default.

func WithPort 

func WithPort(port int) Option

WithPort sets the chrome debugger port.

func WithTimeout 

func WithTimeout(timeout time.Duration) Option

WithTimeout sets the context timeout.

func WithUserDataDir 

func WithUserDataDir(dir string) Option

WithUserDataDir sets the user data directory to a custom path.

Source Files

View all Source files

Directories

Path Synopsis
util
easyjson
Package easyjson provides utilities for use with easyjson.
 Package easyjson provides utilities for use with easyjson.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.