pirsch

package module
v1.4.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 30, 2020 License: MIT Imports: 18 Imported by: 0

README

Pirsch

GoDoc Go Report Card CircleCI Chat on Discord

State of the project: still under heavy development, as we add more features until we're satisified. Our plan is to add session tracking and extracting more data from the User-Agent header, then it should become more stable.

Pirsch is a server side, no-cookie, drop-in and privacy focused tracking solution for Go. Integrated into a Go application it enables you to track HTTP traffic without invading the privacy of your visitors. The visualization of the data (dashboard) is not part of this project.

The name is in German and refers to a special kind of hunt: the hunter carefully and quietly enters the area to be hunted, he stalks against the wind in order to get as close as possible to the prey without being noticed.

How does it work?

Pirsch generates a unique fingerprint for each visitor. The fingerprint is a hash of the visitors IP, User-Agent and a salt. The salt is re-generated at midnight to separate data for each day.

Each time a visitor opens your page, Pirsch will store a hit. The hits are analyzed later to extract meaningful data and reduce storage usage by aggregation.

The tracking works without invading the visitor's privacy as no cookies are used nor required. Pirsch can track visitors using ad blockers that block trackers like Google Analytics.

Features and limitations

Pirsch tracks the following data points:

  • total visitors on each day
  • visitors per day and hour
  • visitors per day and page
  • visitors per day and language

All timestamps are stored as UTC. Each data point belongs to an (optional) tenant, which can be used to split data between multiple domains for example. If you just integrate Pirsch into your application, you don't need to care about that field. But if you do, you need to set a tenant ID for all columns!

It's theoretically possible to track the visitor flow (which page was seen first, which one was visited next, etc.), but this is not implemented at the moment. Here is a list of the limitations of Pirsch:

  • tracking sessions is not possible at the moment as the salt will prevent you from tracking a user across two days
  • bots might not always be filtered out
  • rare cases where two fingerprints collide, if two visitors are behind the same proxy for example and the User-Agent is the same (or empty)
  • the accuracy might not be as high as with client-side solutions, because Pirsch can only collect information that is available to the server

Usage

To store hits and statistics, Pirsch uses a database. Right now only Postgres is supported, but new ones can easily be added by implementing the Store interface. The schema can be found within the schema directory. Changes will be added to migrations scripts, so that you can add them to your projects database migration or run them manually.

Here is a quick demo on how to use the library:

// create a Postgres store using the sql.DB database connection "db"
store := pirsch.NewPostgresStore(db)

// Tracker is the main component of Pirsch
// the salt is used to prevent anyone from generating fingerprints like yours (to prevent man in the middle attacks), pick something random
// an optional configuration can be used to change things like worker count, timeouts and so on
tracker := pirsch.NewTracker(store, "secret_salt", nil)

// the Processor analyzes hits and stores the reduced data points in store
// it's recommended to run the Process method once a day, but you can run it as often as you want
// the config can be used to enable/disable certain features of the processor
processor := pirsch.NewProcessor(store, nil)
pirsch.RunAtMidnight(processor.Process) // helper function to run a function at midnight (UTC)

http.Handle("/", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    // a call to Hit will track the request
    // note that Pirsch stores the path and URL, therefor you should make sure you only call it for the endpoints you're interested in
    // you can also modify the path by passing in the options argument
    if r.URL.Path == "/" {
        tracker.Hit(r, nil)
    }

    w.Write([]byte("<h1>Hello World!</h1>"))
}))

To analyze hits and processed data you can use the analyzer, which provides convenience functions to extract useful information.

The secret salt passed to NewTracker should not be known outside your organization as it can be used to generate fingerprints equal to yours. Note that while you can generate the salt at random, the fingerprints will change too. To get reliable data configure a fixed salt and treat it like a password.

// this also needs access to the store
analyzer := pirsch.NewAnalyzer(store)

// as an example, lets extract the total number of visitors
// the filter is used to specify the time frame you're looking at (days) and is optional
// if you pass nil, the Analyzer returns data for the past week including today
visitors, err := analyzer.Visitors(&pirsch.Filter{
    From: yesterday(),
    To: today()
})

Read the full documentation for more details and check out this article.

Changelog

1.4.2
  • fixed null fields in model
  • fixed counting visitors multiple times (by using a transaction to rollback changes in case the processor fails)
  • added optional log.Logger to Tracker and PostgresStore
  • removed all panics and log errors instead
1.4.1
  • added relative visitor statistics for OS and browser usage
1.4.0

You need to update the schema by running the v1.4.0.sql migration script!

  • added parsing the User-Agent header to extract the OS, OS version, browser, and browser version
  • added OS, browser and platform statistics to Processor and Analyzer
  • Pirsch now uses a single struct for all statistics called Stats
  • fixed error channel size in Processor
  • a few smaller refactorings
1.3.3
  • fixed extracting Referer header
  • added ref, referer and referrer query parameters for referrers, when Referer header is not present
1.3.2

You need to update the schema by running the v1.3.2.sql migration script!

  • fixed helper function RunAtMidnight not using UTC for all times
  • referer -> referrer
1.3.1
  • added statistics for visitor count per page and referrer
1.3.0

You need to update the schema by running the v1.3.0.sql migration script!

  • added cancel function to RunAtMidnight
  • added helper function for tenant IDs
  • hits for an empty User-Agent will be ignored from now on
  • added configuration options to Processor
  • IgnoreHit and HitFromRequest are now exported functions
  • added options to filter for unwanted referrer, like your own domain
  • added referrer statistics to Processor and Analyzer
  • added method to Analyzer to extract active visitor pages
  • Analyzer results are now sorted by visitors in descending order instead of path and referrer length
1.2.0

You need to update the schema by running the v1.2.0.sql migration script!

  • the processor now returns an error
  • the processor now updates existing statistics in case it has been run before, but keep in mind that it drops hits and therefor breaks tracking users that return on the same day. It's recommended to run the processor for days in the past excluding today
  • (optional) multi-tenancy support to track multiple domains using the same database. In case you don't want to use it, use null as the tenant_id
  • improved IP extraction from X-Forwarded-For, Forwarded and X-Real-IP headers
1.1.1
  • fixed error in case values are too long
  • fixed language statistics not including today
1.1.0
  • added a secret salt to prevent generating fingerprints to identify visitors on other websites (man in the middle)
  • extended bot list
1.0.0

Initial release.

Contribution

Contributions are welcome! You can extend the bot list or processor to extract more useful data, for example. Please open a pull requests for your changes and tickets in case you would like to discuss something or have a question.

To run the tests you'll need a Postgres database and a schema called pirsch. The user and password are set to postgres.

License

MIT

Documentation

Index

Constants

View Source 
const (
	// BrowserChrome represents the Chrome and Chromium browser.
	BrowserChrome = "Chrome"

	// BrowserFirefox represents the Firefox browser.
	BrowserFirefox = "Firefox"

	// BrowserSafari  represents the Safari browser.
	BrowserSafari = "Safari"

	// BrowserOpera represents the Opera browser.
	BrowserOpera = "Opera"

	// BrowserEdge represents the Edge browser.
	BrowserEdge = "Edge"

	// BrowserIE represents the Internet Explorer browser.
	BrowserIE = "IE"

	// OSWindows represents the Windows operating system.
	OSWindows = "Windows"

	// OSMac represents the Mac operating system.
	OSMac = "Mac"

	// OSLinux represents a Linux distribution.
	OSLinux = "Linux"

	// OSAndroid represents the Android operating system.
	OSAndroid = "Android"

	// OSiOS represents the iOS operating system.
	OSiOS = "iOS"

	// OSWindowsMobile represents the Windows Mobile operating system.
	OSWindowsMobile = "Windows Mobile"
)

Variables

View Source 
var NullTenant = NewTenantID(0)

NullTenant can be used to pass no (null) tenant to filters and functions. This is an invalid sql.NullInt64 with a value of 0.

Functions

func Fingerprint 

func Fingerprint(r *http.Request, salt string) string

Fingerprint returns a hash for given request and salt. The hash is unique for the visitor.

func IgnoreHit added in v1.3.0 

func IgnoreHit(r *http.Request) bool

IgnoreHit returns true, if a hit should be ignored for given request, or false otherwise. The easiest way to track visitors is to use the Tracker.

func NewTenantID added in v1.3.0 

func NewTenantID(id int64) sql.NullInt64

NewTenantID is a helper function to return a sql.NullInt64. The ID is considered valid if greater than 0.

func RunAtMidnight 

func RunAtMidnight(f func()) context.CancelFunc

RunAtMidnight calls given function on each day of month on midnight (UTC), unless it is cancelled by calling the cancel function.

Types

type Analyzer 

type Analyzer struct {
	// contains filtered or unexported fields
}

Analyzer provides an interface to analyze processed data and hits.

func NewAnalyzer 

func NewAnalyzer(store Store) *Analyzer

NewAnalyzer returns a new Analyzer for given Store.

func (*Analyzer) ActiveVisitors 

func (analyzer *Analyzer) ActiveVisitors(tenantID sql.NullInt64, d time.Duration) (int, error)

ActiveVisitors returns the number of unique visitors active within the given duration.

func (*Analyzer) ActiveVisitorsPages added in v1.3.0 

func (analyzer *Analyzer) ActiveVisitorsPages(tenantID sql.NullInt64, d time.Duration) ([]Stats, error)

ActiveVisitorsPages returns the number of unique visitors active within the given duration and the corresponding pages.

func (*Analyzer) Browser added in v1.4.0 

func (analyzer *Analyzer) Browser(filter *Filter) ([]Stats, error)

Browser returns the absolute visitor count per browser for given time frame.

func (*Analyzer) HourlyVisitors 

func (analyzer *Analyzer) HourlyVisitors(filter *Filter) ([]Stats, error)

HourlyVisitors returns the absolute and relative visitor count per language for given time frame.

func (*Analyzer) Languages 

func (analyzer *Analyzer) Languages(filter *Filter) ([]Stats, int, error)

Languages returns the absolute and relative visitor count per language for given time frame.

func (*Analyzer) OS added in v1.4.0 

func (analyzer *Analyzer) OS(filter *Filter) ([]Stats, error)

OS returns the absolute visitor count per operating system for given time frame.

func (*Analyzer) PageVisits 

func (analyzer *Analyzer) PageVisits(filter *Filter) ([]Stats, error)

PageVisits returns the visitors per page per day for given time frame.

func (*Analyzer) Pages added in v1.3.1 

func (analyzer *Analyzer) Pages(filter *Filter) ([]Stats, error)

Pages returns the absolute visitor count per page for given time frame.

func (*Analyzer) Platform added in v1.4.0 

func (analyzer *Analyzer) Platform(filter *Filter) (*Stats, error)

Platform returns the relative platform usage for given time frame.

func (*Analyzer) Referrer added in v1.3.2 

func (analyzer *Analyzer) Referrer(filter *Filter) ([]Stats, error)

Referrer returns the absolute visitor count per referrer for given time frame.

func (*Analyzer) ReferrerVisits added in v1.3.2 

func (analyzer *Analyzer) ReferrerVisits(filter *Filter) ([]Stats, error)

ReferrerVisits returns the visitors per referrer per day for given time frame.

func (*Analyzer) Visitors 

func (analyzer *Analyzer) Visitors(filter *Filter) ([]VisitorsPerDay, error)

Visitors returns the visitors per day for the given time frame.

type BaseEntity added in v1.4.0 

type BaseEntity struct {
	ID       int64         `db:"id" json:"id"`
	TenantID sql.NullInt64 `db:"tenant_id" json:"tenant_id"`
}

BaseEntity is the base entity for all other entities.

type Filter 

type Filter struct {
	// TenantID is the optional tenant ID used to filter results.
	TenantID sql.NullInt64

	// From is the start of the selection.
	From time.Time

	// To is the end of the selection.
	To time.Time
}

Filter is used to specify the time frame and tenant for the Analyzer.

func NewFilter added in v1.4.0 

func NewFilter(tenantID sql.NullInt64) *Filter

NewFilter returns a new default filter for given tenant and the past week.

func (*Filter) Days 

func (filter *Filter) Days() int

Days returns the number of days covered by the filter.

type Hit 

type Hit struct {
	BaseEntity

	Fingerprint    string         `db:"fingerprint" json:"fingerprint"`
	Path           sql.NullString `db:"path" json:"path,omitempty"`
	URL            sql.NullString `db:"url" json:"url,omitempty"`
	Language       sql.NullString `db:"language" json:"language,omitempty"`
	UserAgent      sql.NullString `db:"user_agent" json:"user_agent,omitempty"`
	Ref            sql.NullString `db:"ref" json:"ref,omitempty"`
	OS             sql.NullString `db:"os" json:"os,omitempty"`
	OSVersion      sql.NullString `db:"os_version" json:"os_version,omitempty"`
	Browser        sql.NullString `db:"browser" json:"browser,omitempty"`
	BrowserVersion sql.NullString `db:"browser_version" json:"browser_version,omitempty"`
	Desktop        bool           `db:"desktop" json:"desktop"`
	Mobile         bool           `db:"mobile" json:"mobile"`
	Time           time.Time      `db:"time" json:"time"`
}

Hit represents a single data point/page visit and is the central entity of pirsch.

func HitFromRequest added in v1.3.0 

func HitFromRequest(r *http.Request, salt string, options *HitOptions) Hit

HitFromRequest returns a new Hit for given request, salt and HitOptions. The salt must stay consistent to track visitors across multiple calls. The easiest way to track visitors is to use the Tracker.

func (Hit) String 

func (hit Hit) String() string

String implements the Stringer interface.

type HitOptions added in v1.2.0 

type HitOptions struct {
	// TenantID is optionally saved with a hit to split the data between multiple tenants.
	TenantID sql.NullInt64

	// Path can be specified to manually overwrite the path stored for the request.
	// This will also affect the URL.
	Path string

	// ReferrerDomainBlacklist is used to filter out unwanted referrer from the Ref header.
	// This can be used to filter out traffic from your own site or subdomains.
	// To filter your own domain and subdomains, add your domain to the list and set ReferrerDomainBlacklistIncludesSubdomains to true.
	// This way the referrer for blog.mypage.com -> mypage.com won't be saved.
	ReferrerDomainBlacklist []string

	// ReferrerDomainBlacklistIncludesSubdomains set to true to include all subdomains in the ReferrerDomainBlacklist,
	// or else subdomains must explicitly be included in the blacklist.
	// If the blacklist contains domain.com, sub.domain.com and domain.com will be treated as equally.
	ReferrerDomainBlacklistIncludesSubdomains bool
}

HitOptions is used to manipulate the data saved on a hit.

type PostgresConfig added in v1.4.2 

type PostgresConfig struct {
	// Logger is the log.Logger used for logging.
	// The default log will be used printing to os.Stdout with "pirsch" in its prefix in case it is not set.
	Logger *log.Logger
}

PostgresConfig is the optional configuration for the PostgresStore.

type PostgresStore 

type PostgresStore struct {
	DB *sqlx.DB
	// contains filtered or unexported fields
}

PostgresStore implements the Store interface.

func NewPostgresStore 

func NewPostgresStore(db *sql.DB, config *PostgresConfig) *PostgresStore

NewPostgresStore creates a new postgres storage for given database connection and logger.

func (*PostgresStore) ActiveVisitors 

func (store *PostgresStore) ActiveVisitors(tenantID sql.NullInt64, from time.Time) (int, error)

ActiveVisitors implements the Store interface.

func (*PostgresStore) ActiveVisitorsPerPage added in v1.3.0 

func (store *PostgresStore) ActiveVisitorsPerPage(tenantID sql.NullInt64, from time.Time) ([]Stats, error)

ActiveVisitorsPerPage implements the Store interface.

func (*PostgresStore) Commit added in v1.4.2 

func (store *PostgresStore) Commit(tx *sqlx.Tx)

Commit implements the Store interface.

func (*PostgresStore) CountHits added in v1.3.0 

func (store *PostgresStore) CountHits(tenantID sql.NullInt64) int

CountHits implements the Store interface.

func (*PostgresStore) CountVisitorPlatforms added in v1.4.0 

func (store *PostgresStore) CountVisitorPlatforms(tx *sqlx.Tx, tenantID sql.NullInt64, day time.Time) (*VisitorPlatform, error)

CountVisitorPlatforms implements the Store interface.

func (*PostgresStore) CountVisitorsPerBrowserAndVersion added in v1.4.0 

func (store *PostgresStore) CountVisitorsPerBrowserAndVersion(tx *sqlx.Tx, tenantID sql.NullInt64, day time.Time) ([]VisitorsPerBrowser, error)

CountVisitorsPerBrowserAndVersion implements the Store interface.

func (*PostgresStore) CountVisitorsPerDay added in v1.3.0 

func (store *PostgresStore) CountVisitorsPerDay(tx *sqlx.Tx, tenantID sql.NullInt64, day time.Time) (int, error)

CountVisitorsPerDay implements the Store interface.

func (*PostgresStore) CountVisitorsPerDayAndHour added in v1.3.0 

func (store *PostgresStore) CountVisitorsPerDayAndHour(tx *sqlx.Tx, tenantID sql.NullInt64, day time.Time) ([]VisitorsPerHour, error)

CountVisitorsPerDayAndHour implements the Store interface.

func (*PostgresStore) CountVisitorsPerLanguage added in v1.3.0 

func (store *PostgresStore) CountVisitorsPerLanguage(tx *sqlx.Tx, tenantID sql.NullInt64, day time.Time) ([]VisitorsPerLanguage, error)

CountVisitorsPerLanguage implements the Store interface.

func (*PostgresStore) CountVisitorsPerOSAndVersion added in v1.4.0 

func (store *PostgresStore) CountVisitorsPerOSAndVersion(tx *sqlx.Tx, tenantID sql.NullInt64, day time.Time) ([]VisitorsPerOS, error)

CountVisitorsPerOSAndVersion implements the Store interface.

func (*PostgresStore) CountVisitorsPerPage added in v1.3.0 

func (store *PostgresStore) CountVisitorsPerPage(tx *sqlx.Tx, tenantID sql.NullInt64, day time.Time) ([]VisitorsPerPage, error)

CountVisitorsPerPage implements the Store interface.

func (*PostgresStore) CountVisitorsPerReferrer added in v1.3.2 

func (store *PostgresStore) CountVisitorsPerReferrer(tx *sqlx.Tx, tenantID sql.NullInt64, day time.Time) ([]VisitorsPerReferrer, error)

CountVisitorsPerReferrer implements the Store interface.

func (*PostgresStore) Days 

func (store *PostgresStore) Days(tenantID sql.NullInt64) ([]time.Time, error)

Days implements the Store interface.

func (*PostgresStore) DeleteHitsByDay 

func (store *PostgresStore) DeleteHitsByDay(tx *sqlx.Tx, tenantID sql.NullInt64, day time.Time) error

DeleteHitsByDay implements the Store interface.

func (*PostgresStore) HourlyVisitors 

func (store *PostgresStore) HourlyVisitors(tenantID sql.NullInt64, from, to time.Time) ([]Stats, error)

HourlyVisitors implements the Store interface.

func (*PostgresStore) NewTx added in v1.4.2 

func (store *PostgresStore) NewTx() *sqlx.Tx

NewTx implements the Store interface.

func (*PostgresStore) PageVisits 

func (store *PostgresStore) PageVisits(tenantID sql.NullInt64, path string, from, to time.Time) ([]VisitorsPerDay, error)

PageVisits implements the Store interface.

func (*PostgresStore) Paths 

func (store *PostgresStore) Paths(tenantID sql.NullInt64, from, to time.Time) ([]string, error)

Paths implements the Store interface.

func (*PostgresStore) Referrer added in v1.3.2 

func (store *PostgresStore) Referrer(tenantID sql.NullInt64, from, to time.Time) ([]string, error)

Referrer implements the Store interface.

func (*PostgresStore) ReferrerVisits added in v1.3.2 

func (store *PostgresStore) ReferrerVisits(tenantID sql.NullInt64, referrer string, from, to time.Time) ([]VisitorsPerReferrer, error)

ReferrerVisits implements the Store interface.

func (*PostgresStore) Rollback added in v1.4.2 

func (store *PostgresStore) Rollback(tx *sqlx.Tx)

Rollback implements the Store interface.

func (*PostgresStore) Save 

func (store *PostgresStore) Save(hits []Hit) error

Save implements the Store interface.

func (*PostgresStore) SaveVisitorPlatform added in v1.4.0 

func (store *PostgresStore) SaveVisitorPlatform(tx *sqlx.Tx, visitors *VisitorPlatform) error

SaveVisitorPlatform implements the Store interface.

func (*PostgresStore) SaveVisitorsPerBrowser added in v1.4.0 

func (store *PostgresStore) SaveVisitorsPerBrowser(tx *sqlx.Tx, visitors *VisitorsPerBrowser) error

SaveVisitorsPerBrowser implements the Store interface.

func (*PostgresStore) SaveVisitorsPerDay 

func (store *PostgresStore) SaveVisitorsPerDay(tx *sqlx.Tx, visitors *VisitorsPerDay) error

SaveVisitorsPerDay implements the Store interface.

func (*PostgresStore) SaveVisitorsPerHour 

func (store *PostgresStore) SaveVisitorsPerHour(tx *sqlx.Tx, visitors *VisitorsPerHour) error

SaveVisitorsPerHour implements the Store interface.

func (*PostgresStore) SaveVisitorsPerLanguage 

func (store *PostgresStore) SaveVisitorsPerLanguage(tx *sqlx.Tx, visitors *VisitorsPerLanguage) error

SaveVisitorsPerLanguage implements the Store interface.

func (*PostgresStore) SaveVisitorsPerOS added in v1.4.0 

func (store *PostgresStore) SaveVisitorsPerOS(tx *sqlx.Tx, visitors *VisitorsPerOS) error

SaveVisitorsPerOS implements the Store interface.

func (*PostgresStore) SaveVisitorsPerPage 

func (store *PostgresStore) SaveVisitorsPerPage(tx *sqlx.Tx, visitors *VisitorsPerPage) error

SaveVisitorsPerPage implements the Store interface.

func (*PostgresStore) SaveVisitorsPerReferrer added in v1.3.2 

func (store *PostgresStore) SaveVisitorsPerReferrer(tx *sqlx.Tx, visitors *VisitorsPerReferrer) error

SaveVisitorsPerReferrer implements the Store interface.

func (*PostgresStore) VisitorBrowser added in v1.4.0 

func (store *PostgresStore) VisitorBrowser(tenantID sql.NullInt64, from time.Time, to time.Time) ([]Stats, error)

VisitorBrowser implements the Store interface.

func (*PostgresStore) VisitorLanguages 

func (store *PostgresStore) VisitorLanguages(tenantID sql.NullInt64, from, to time.Time) ([]Stats, error)

VisitorLanguages implements the Store interface.

func (*PostgresStore) VisitorOS added in v1.4.0 

func (store *PostgresStore) VisitorOS(tenantID sql.NullInt64, from time.Time, to time.Time) ([]Stats, error)

VisitorOS implements the Store interface.

func (*PostgresStore) VisitorPages added in v1.3.1 

func (store *PostgresStore) VisitorPages(tenantID sql.NullInt64, from time.Time, to time.Time) ([]Stats, error)

VisitorPages implements the Store interface.

func (*PostgresStore) VisitorPlatform added in v1.4.0 

func (store *PostgresStore) VisitorPlatform(tenantID sql.NullInt64, from time.Time, to time.Time) (*Stats, error)

VisitorPlatform implements the Store interface.

func (*PostgresStore) VisitorReferrer added in v1.3.2 

func (store *PostgresStore) VisitorReferrer(tenantID sql.NullInt64, from time.Time, to time.Time) ([]Stats, error)

VisitorReferrer implements the Store interface.

func (*PostgresStore) Visitors 

func (store *PostgresStore) Visitors(tenantID sql.NullInt64, from, to time.Time) ([]VisitorsPerDay, error)

Visitors implements the Store interface.

func (*PostgresStore) VisitorsPerBrowser added in v1.4.0 

func (store *PostgresStore) VisitorsPerBrowser(tenantID sql.NullInt64) []VisitorsPerBrowser

VisitorsPerBrowser implements the Store interface.

func (*PostgresStore) VisitorsPerDay 

func (store *PostgresStore) VisitorsPerDay(tenantID sql.NullInt64) []VisitorsPerDay

VisitorsPerDay implements the Store interface.

func (*PostgresStore) VisitorsPerHour added in v1.3.0 

func (store *PostgresStore) VisitorsPerHour(tenantID sql.NullInt64) []VisitorsPerHour

VisitorsPerHour implements the Store interface.

func (*PostgresStore) VisitorsPerLanguage 

func (store *PostgresStore) VisitorsPerLanguage(tenantID sql.NullInt64) []VisitorsPerLanguage

VisitorsPerLanguage implements the Store interface.

func (*PostgresStore) VisitorsPerOS added in v1.4.0 

func (store *PostgresStore) VisitorsPerOS(tenantID sql.NullInt64) []VisitorsPerOS

VisitorsPerOS implements the Store interface.

func (*PostgresStore) VisitorsPerPage 

func (store *PostgresStore) VisitorsPerPage(tenantID sql.NullInt64) []VisitorsPerPage

VisitorsPerPage implements the Store interface.

func (*PostgresStore) VisitorsPerPlatform added in v1.4.0 

func (store *PostgresStore) VisitorsPerPlatform(tenantID sql.NullInt64) []VisitorPlatform

VisitorsPerPlatform implements the Store interface.

func (*PostgresStore) VisitorsPerReferrer added in v1.3.2 

func (store *PostgresStore) VisitorsPerReferrer(tenantID sql.NullInt64) []VisitorsPerReferrer

VisitorsPerReferrer implements the Store interface.

type Processor 

type Processor struct {
	// contains filtered or unexported fields
}

Processor processes hits to reduce them into meaningful statistics.

func NewProcessor 

func NewProcessor(store Store, config *ProcessorConfig) *Processor

NewProcessor creates a new Processor for given Store and config. Pass nil for the config to use the defaults.

func (*Processor) Process 

func (processor *Processor) Process() error

Process processes all hits in database and deletes them afterwards.

func (*Processor) ProcessTenant added in v1.2.0 

func (processor *Processor) ProcessTenant(tenantID sql.NullInt64) error

ProcessTenant processes all hits in database for given tenant and deletes them afterwards. The tenant can be set to nil if you don't split your data (which is usually the case).

type ProcessorConfig added in v1.3.0 

type ProcessorConfig struct {
	// ProcessVisitors enables/disabled processing the visitor count.
	// The default is true (enabled).
	ProcessVisitors bool

	// ProcessVisitorPerHour enables/disabled processing the visitor count per hour.
	// The default is true (enabled).
	ProcessVisitorPerHour bool

	// ProcessLanguages enables/disabled processing the language count.
	// The default is true (enabled).
	ProcessLanguages bool

	// ProcessPageViews enables/disabled processing the page views.
	// The default is true (enabled).
	ProcessPageViews bool

	// ProcessVisitorPerReferrer enables/disabled processing the visitor count per referrer.
	// The default is true (enabled).
	ProcessVisitorPerReferrer bool

	// ProcessVisitorPerOS enables/disabled processing the visitor count per operating system.
	// The default is true (enabled).
	ProcessVisitorPerOS bool

	// ProcessVisitorPerBrowser enables/disabled processing the visitor count per browser.
	// The default is true (enabled).
	ProcessVisitorPerBrowser bool

	// ProcessPlatform enables/disabled processing the visitor platform.
	// The default is true (enabled).
	ProcessPlatform bool
}

ProcessorConfig is the optional configuration for the Processor.

type Stats added in v1.4.0 

type Stats struct {
	Path                    sql.NullString        `db:"path" json:"path,omitempty"`
	Language                sql.NullString        `db:"language" json:"language,omitempty"`
	Referrer                sql.NullString        `db:"ref" json:"referrer,omitempty"`
	OS                      sql.NullString        `db:"os" json:"os,omitempty"`
	Browser                 sql.NullString        `db:"browser" json:"browser,omitempty"`
	Hour                    int                   `db:"hour" json:"hour,omitempty"`
	Visitors                int                   `db:"visitors" json:"visitors,omitempty"`
	RelativeVisitors        float64               `db:"-" json:"relative_visitors,omitempty"`
	PlatformDesktopVisitors int                   `db:"platform_desktop_visitors" json:"platform_desktop_visitors,omitempty"`
	PlatformDesktopRelative float64               `db:"-" json:"platform_desktop_relative,omitempty"`
	PlatformMobileVisitors  int                   `db:"platform_mobile_visitors" json:"platform_mobile_visitors,omitempty"`
	PlatformMobileRelative  float64               `db:"-" json:"platform_mobile_relative,omitempty"`
	PlatformUnknownVisitors int                   `db:"platform_unknown_visitors" json:"platform_unknown_visitors,omitempty"`
	PlatformUnknownRelative float64               `db:"-" json:"platform_unknown_relative,omitempty"`
	VisitorsPerDay          []VisitorsPerDay      `db:"-" json:"visitors_per_day,omitempty"`
	VisitorsPerReferrer     []VisitorsPerReferrer `db:"-" json:"visitors_per_referrer,omitempty"`
}

Stats bundles all statistics into a single object. The meaning of the data depends on the context. It's not persisted in the database.

type Store 

type Store interface {
	// Save persists a list of hits.
	Save([]Hit) error

	// DeleteHitsByDay deletes all hits on given day.
	DeleteHitsByDay(*sqlx.Tx, sql.NullInt64, time.Time) error

	// SaveVisitorsPerDay persists unique visitors per day.
	SaveVisitorsPerDay(*sqlx.Tx, *VisitorsPerDay) error

	// SaveVisitorsPerHour persists unique visitors per day and hour.
	SaveVisitorsPerHour(*sqlx.Tx, *VisitorsPerHour) error

	// SaveVisitorsPerLanguage persists unique visitors per day and language.
	SaveVisitorsPerLanguage(*sqlx.Tx, *VisitorsPerLanguage) error

	// SaveVisitorsPerPage persists unique visitors per day and page.
	SaveVisitorsPerPage(*sqlx.Tx, *VisitorsPerPage) error

	// SaveVisitorsPerReferrer persists unique visitors per day and referrer.
	SaveVisitorsPerReferrer(*sqlx.Tx, *VisitorsPerReferrer) error

	// SaveVisitorsPerOS persists unique visitors per day and operating system.
	SaveVisitorsPerOS(*sqlx.Tx, *VisitorsPerOS) error

	// SaveVisitorsPerBrowser persists unique visitors per day and browser.
	SaveVisitorsPerBrowser(*sqlx.Tx, *VisitorsPerBrowser) error

	// SaveVisitorPlatform persists visitors per platform and day.
	SaveVisitorPlatform(*sqlx.Tx, *VisitorPlatform) error

	// Days returns the days at least one hit exists for.
	Days(sql.NullInt64) ([]time.Time, error)

	// CountVisitorsPerDay returns the unique visitor count for per day.
	CountVisitorsPerDay(*sqlx.Tx, sql.NullInt64, time.Time) (int, error)

	// CountVisitorsPerDayAndHour returns the unique visitor count per day and hour.
	CountVisitorsPerDayAndHour(*sqlx.Tx, sql.NullInt64, time.Time) ([]VisitorsPerHour, error)

	// CountVisitorsPerLanguage returns the unique visitor count per language and day.
	CountVisitorsPerLanguage(*sqlx.Tx, sql.NullInt64, time.Time) ([]VisitorsPerLanguage, error)

	// CountVisitorsPerPage returns the unique visitor count per page and day.
	CountVisitorsPerPage(*sqlx.Tx, sql.NullInt64, time.Time) ([]VisitorsPerPage, error)

	// CountVisitorsPerReferrer returns the unique visitor count per referrer and day.
	CountVisitorsPerReferrer(*sqlx.Tx, sql.NullInt64, time.Time) ([]VisitorsPerReferrer, error)

	// CountVisitorsPerOSAndVersion returns the unique visitor count per operating system, version and day.
	CountVisitorsPerOSAndVersion(*sqlx.Tx, sql.NullInt64, time.Time) ([]VisitorsPerOS, error)

	// CountVisitorsPerBrowserAndVersion returns the unique visitor count per browser, version and day.
	CountVisitorsPerBrowserAndVersion(*sqlx.Tx, sql.NullInt64, time.Time) ([]VisitorsPerBrowser, error)

	// CountVisitorPlatforms returns the unique visitor count per platform and day.
	CountVisitorPlatforms(*sqlx.Tx, sql.NullInt64, time.Time) (*VisitorPlatform, error)

	// Paths returns distinct paths for page visits.
	// This does not include today.
	Paths(sql.NullInt64, time.Time, time.Time) ([]string, error)

	// Referrer returns distinct referrer for page visits.
	// This does not include today.
	Referrer(sql.NullInt64, time.Time, time.Time) ([]string, error)

	// Visitors returns the visitors within given time frame.
	// This does not include today.
	Visitors(sql.NullInt64, time.Time, time.Time) ([]VisitorsPerDay, error)

	// Stats returns the page visits within given time frame for given path.
	// This does not include today.
	PageVisits(sql.NullInt64, string, time.Time, time.Time) ([]VisitorsPerDay, error)

	// ReferrerVisits returns the referrer visits within given time frame for given referrer.
	// This does not include today.
	ReferrerVisits(sql.NullInt64, string, time.Time, time.Time) ([]VisitorsPerReferrer, error)

	// VisitorPages returns the pages and unique visitor count for given time frame.
	// It does include today.
	VisitorPages(sql.NullInt64, time.Time, time.Time) ([]Stats, error)

	// VisitorLanguages returns the language and unique visitor count for given time frame.
	// It does include today.
	VisitorLanguages(sql.NullInt64, time.Time, time.Time) ([]Stats, error)

	// VisitorReferrer returns the referrer and unique visitor count for given time frame.
	// It does include today.
	VisitorReferrer(sql.NullInt64, time.Time, time.Time) ([]Stats, error)

	// VisitorOS returns the OS and unique visitor count for given time frame.
	// It does include today.
	VisitorOS(sql.NullInt64, time.Time, time.Time) ([]Stats, error)

	// VisitorBrowser returns the browser and unique visitor count for given time frame.
	// It does include today.
	VisitorBrowser(sql.NullInt64, time.Time, time.Time) ([]Stats, error)

	// VisitorPlatform returns the platform and unique visitor count for given time frame.
	// It does include today.
	VisitorPlatform(sql.NullInt64, time.Time, time.Time) (*Stats, error)

	// HourlyVisitors returns unique visitors per hour for given time frame.
	// It does include today.
	HourlyVisitors(sql.NullInt64, time.Time, time.Time) ([]Stats, error)

	// ActiveVisitors returns unique visitors starting at given time.
	ActiveVisitors(sql.NullInt64, time.Time) (int, error)

	// ActiveVisitorsPerPage returns unique visitors per page starting at given time.
	ActiveVisitorsPerPage(sql.NullInt64, time.Time) ([]Stats, error)

	// CountHits returns the number of hits for given tenant ID.
	CountHits(sql.NullInt64) int

	// VisitorsPerDay returns all visitors per day for given tenant ID sorted by days.
	VisitorsPerDay(sql.NullInt64) []VisitorsPerDay

	// VisitorsPerHour returns all visitors per hour for given tenant ID sorted by days.
	VisitorsPerHour(sql.NullInt64) []VisitorsPerHour

	// VisitorsPerLanguage returns all visitors per language for given tenant ID in alphabetical order.
	VisitorsPerLanguage(sql.NullInt64) []VisitorsPerLanguage

	// VisitorsPerPage returns all visitors per page for given tenant ID sorted by days.
	VisitorsPerPage(sql.NullInt64) []VisitorsPerPage

	// VisitorsPerReferrer returns all visitors per referrer for given tenant ID sorted by days.
	VisitorsPerReferrer(sql.NullInt64) []VisitorsPerReferrer

	// VisitorsPerOS returns all visitors per operating system for given tenant ID sorted by days.
	VisitorsPerOS(sql.NullInt64) []VisitorsPerOS

	// VisitorsPerBrowser returns all visitors per browsers for given tenant ID sorted by days.
	VisitorsPerBrowser(sql.NullInt64) []VisitorsPerBrowser

	// VisitorsPerPlatform returns all visitor platforms for given tenant ID sorted by days.
	VisitorsPerPlatform(sql.NullInt64) []VisitorPlatform

	// NewTx creates a new transaction and panic on failure.
	NewTx() *sqlx.Tx

	// Commit commits given transaction and logs the error.
	Commit(*sqlx.Tx)

	// Rollback rolls back given transaction and logs the error.
	Rollback(*sqlx.Tx)
}

Store defines an interface to persists hits and other data. The first parameter (if required) is always the tenant ID and can be left out (pirsch.NullTenant), if you don't want to split your data. This is usually the case if you integrate Pirsch into your application.

type Tracker 

type Tracker struct {
	// contains filtered or unexported fields
}

Tracker is the main component of Pirsch. It provides methods to track requests and store them in a data store. Make sure you call Stop to make sure the hits get stored before shutting down the server.

func NewTracker 

func NewTracker(store Store, salt string, config *TrackerConfig) *Tracker

NewTracker creates a new tracker for given store, salt and config. Pass nil for the config to use the defaults. The salt is mandatory.

func (*Tracker) Flush 

func (tracker *Tracker) Flush()

Flush flushes all hits to store.

func (*Tracker) Hit 

func (tracker *Tracker) Hit(r *http.Request, options *HitOptions)

Hit stores the given request. The request might be ignored if it meets certain conditions. The HitOptions, if passed, will overwrite the Tracker configuration. The actions performed within this function run in their own goroutine, so you don't need to create one yourself.

func (*Tracker) Stop 

func (tracker *Tracker) Stop()

Stop flushes and stops all workers.

type TrackerConfig 

type TrackerConfig struct {
	// Worker sets the number of workers that are used to store hits.
	// Must be greater or equal to 1.
	Worker int

	// WorkerBufferSize is the size of the buffer used to store hits.
	// Must be greater than 0. The hits are stored in batch when the buffer is full.
	WorkerBufferSize int

	// WorkerTimeout sets the timeout used to store hits.
	// This is used to allow the workers to store hits even if the buffer is not full yet.
	// It's recommended to set this to a few seconds.
	WorkerTimeout time.Duration

	// ReferrerDomainBlacklist see HitOptions.ReferrerDomainBlacklist.
	ReferrerDomainBlacklist []string

	// ReferrerDomainBlacklistIncludesSubdomains see HitOptions.ReferrerDomainBlacklistIncludesSubdomains.
	ReferrerDomainBlacklistIncludesSubdomains bool

	// Logger is the log.Logger used for logging.
	// The default log will be used printing to os.Stdout with "pirsch" in its prefix in case it is not set.
	Logger *log.Logger
}

TrackerConfig is the optional configuration for the Tracker.

type UserAgent added in v1.4.0 

type UserAgent struct {
	// Browser is the browser name.
	Browser string

	// BrowserVersion is the browser (non technical) version number.
	BrowserVersion string

	// OS is the operating system.
	OS string

	// OSVersion is the operating system version number.
	OSVersion string
}

UserAgent contains information extracted from the User-Agent header.

func ParseUserAgent added in v1.4.0 

func ParseUserAgent(ua string) UserAgent

ParseUserAgent parses given User-Agent header and returns the extracted information. This just supports major browsers and operating systems, we don't care about browsers and OSes that have no market share, unless you prove us wrong.

func (*UserAgent) IsDesktop added in v1.4.0 

func (ua *UserAgent) IsDesktop() bool

IsDesktop returns true if the user agent is a desktop device.

func (*UserAgent) IsMobile added in v1.4.0 

func (ua *UserAgent) IsMobile() bool

IsMobile returns true if the user agent is a mobile device.

type VisitorPlatform added in v1.4.0 

type VisitorPlatform struct {
	BaseEntity

	Day     time.Time `db:"day" json:"day"`
	Desktop int       `db:"desktop" json:"desktop"`
	Mobile  int       `db:"mobile" json:"mobile"`
	Unknown int       `db:"unknown" json:"unknown"`
}

VisitorPlatform is the visitor count per platform and day.

type VisitorsPerBrowser added in v1.4.0 

type VisitorsPerBrowser struct {
	BaseEntity

	Day            time.Time      `db:"day" json:"day"`
	Browser        sql.NullString `db:"browser" json:"browser"`
	BrowserVersion sql.NullString `db:"browser_version" json:"version"`
	Visitors       int            `db:"visitors" json:"visitors"`
}

VisitorsPerBrowser is the unique visitor count per browser and day.

type VisitorsPerDay 

type VisitorsPerDay struct {
	BaseEntity

	Day      time.Time `db:"day" json:"day"`
	Visitors int       `db:"visitors" json:"visitors"`
}

VisitorsPerDay is the unique visitor count per day.

type VisitorsPerHour 

type VisitorsPerHour struct {
	BaseEntity

	DayAndHour time.Time `db:"day_and_hour" json:"day_and_hour"`
	Visitors   int       `db:"visitors" json:"visitors"`
}

VisitorsPerHour is the unique visitor count per hour and day.

type VisitorsPerLanguage 

type VisitorsPerLanguage struct {
	BaseEntity

	Day      time.Time      `db:"day" json:"day"`
	Language sql.NullString `db:"language" json:"language"`
	Visitors int            `db:"visitors" json:"visitors"`
}

VisitorsPerLanguage is the unique visitor count per language and day.

type VisitorsPerOS added in v1.4.0 

type VisitorsPerOS struct {
	BaseEntity

	Day       time.Time      `db:"day" json:"day"`
	OS        sql.NullString `db:"os" json:"os"`
	OSVersion sql.NullString `db:"os_version" json:"version"`
	Visitors  int            `db:"visitors" json:"visitors"`
}

VisitorsPerOS is the unique visitor count per operating system and day.

type VisitorsPerPage 

type VisitorsPerPage struct {
	BaseEntity

	Day      time.Time      `db:"day" json:"day"`
	Path     sql.NullString `db:"path" json:"path"`
	Visitors int            `db:"visitors" json:"visitors"`
}

VisitorsPerPage is the unique visitor count per path and day.

type VisitorsPerReferrer added in v1.3.2 

type VisitorsPerReferrer struct {
	BaseEntity

	Day      time.Time      `db:"day" json:"day"`
	Ref      sql.NullString `db:"ref" json:"ref"`
	Visitors int            `db:"visitors" json:"visitors"`
}

VisitorsPerReferrer is the unique visitor count per referrer and day.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.