timegopher

package module
v0.0.0-...-a38ff78 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 4, 2022 License: MIT Imports: 11 Imported by: 0

README

TimeGopher

Time flows forward, epoch=0, 1.1.1970 never returns

1.1.1970 never returns, time goes forward

This library is meant for applications that have to deal with timestamps, but do not have syncronized wall clock time always available when it is needed. This library provides way to store monotonic clock and convert it to wall clock time later when synchronization to RTC (by NTP for example) is available. This library provides also way to track software starts, stops (wanted and crashes) and system reboot times. Storage is implemented by fixregsto This library tries also to catch typical errors when dealing with timestamping (sanity checks added)

Typical application would be datalogger kind of embedded application.

NOTICE!! THIS LIBRARY IS NOT YET PRODUCTION READY

Naive solution is to use calendar date&time in some standard format and hope that everything goes fine. Maybe fix timestamps later if system wall clock is synchronized or set manually.

Basic solution when logging timestamps is to use epoch timestamp. wikipedia. It solves issues only with timezones and daylight savings. But still there is issue if epoch time is wrong while data and epoch timestamp is logged. As bad as date and time with timezone information.

One solution is to use monotonic clock that never moves back and never pauses while system is running. Always increases while computer runs. If synchronized wall clock time (epoch) is recorded with monotonic clock then. Golang time uses *monotonic clock when doing calculations with time.Time types.

There are good and bad monotonic sources. Linux have /proc/uptime that tells uptime since kernel was started. One way to create always monotonic clock is to combine system uptime and counter that counts how many boots system have experienced.

This timegopher library was created based on that idea.

Timegopher keeps following timestamp logs (bootcounter, uptime and epoch values)

  • RtcSyncLog, every time when there is need to connect bootcounter and uptime to actual "wall clock"
  • UncertainRtcSyncLog, created for storing uncertain sync events like pick best guess from RTC clock, or user enters actual time. secondary option if RtcSyncLog entry not available Optional feature
  • LastLog, frequently updated live log when software was running. Optional feature. Not needed if start and stop logs are not required and software have some other updating datasources with TimeVariable timestamp
  • StartLog, optional feature. Use if tracking software start timestamps is required in your application
  • StopLog. optional feature. Use if tracking software stop timestamps is required in your application

How to use

The first thing is to create one instance of TimeGopher when actual main application starts (a fresh start after boot or software restart) before any data is stored or timestamped.

While sofware is running, call Refresh method on TimeGopher. IF timestamps are needed on your data. Create those with Convert(time.Now()) on TimeGopher

There is example application examples/templogger that logs cpu temperature. (it also demonstrates some sub system features)

Initializing TimeGopher

Creation of TimeGopher is done by calling NewTimeGopher( .

func NewTimeGopher(
	timeNow time.Time,

	inSync bool,
	coldStart bool,

	rtcSyncLog *TimeFileDb,
	uncertainRtcSyncLog *TimeFileDb, //Optional

	startLog *TimeFileDb, //Optional
	stopLog *TimeFileDb, //Optional

	lastLog *TimeFileDb, //Last alive situation

	latestKnowTimeElsewhere TimeVariable, //If knows from latest stored timestamp on timeseries database
	uptimeCheck *UptimeChecker,
) (TimeGopher, error) {

If there is no need for fine grain control of things and using default disk storage implementation is ok and using time.Now() as time source is ok. Then CreateDefaultTimeGopher helps to generate few variables

Initializing time storage

Parameter coldStart can be resolved by having temporary flag file under /tmp. When linux kernel restarts /tmp folder is cleared. There is helper function FirstCallAfterBoot available for that

isFirstRun, errfirstrun := tgopher.FirstCallAfterBoot("/tmp/coldstart")

Parameter uptimeCheck can be created by CreateUptimeChecker method. It have to be done once. Uptime checker uses more decimals than 0.01s available from /proc/uptime

func CreateUptimeChecker() (UptimeChecker, error) {

Initializing TimeGopher, easy way

func CreateDefaultTimeGopher(
    rtcLogDir string,
    latestKnowTimeElsewhere TimeVariable) 
    
    (TimeGopher, error) {

Then you have to provide dedicated and persisted disk directory for time files as rtcLogDir. (depends on your application)

Running TimeGopher

There are no goroutines in TimeGopher

For that reason Refesh function have to be called. It will update lastLog entries (so TimeGopher can say at next start when previous run stopped). Refresh also handles situation if system got realtime clock synced.

Calling Refresh too often can generates disk activity and exessive usage can lead to SSD/sdcard wearout.

func (p *TimeGopher) Refresh(t time.Time, inSync bool) error

Parameter inSync True means that linux wall clock is set to correct time. Parameter inSync can be resolved by using RtcIsSynced_adjtimex or by some other means. It is prefered to have sync value hold by actual software.

Parameter t can be generated by time.Now() function.

Or in that case just call RefreshNow() function.

func (p *TimeGopher) RefreshNow() error {

Relation in between uptime and epoch time can change if time synchronization fixes epoch time. Software can call RtcDeviation helper function with time.Now(). It will tell how much deviation will be. If it is too much, software must call Refresh function with inSync=false and after that inSync=true values. Then new entry is added to RTC sync log

func (p *TimeGopher) RtcDeviation(t time.Time) (int64, error)

If software runs on device that do not have automatic time sync available (like NTP), but time is set manually by ui or physical buttons. Then application must call DoUncertainTimeSync or DoUncertainTimeSyncNow when that happens

func (p *TimeGopher) DoUncertainTimeSync(t time.Time) error
func (p *TimeGopher) DoUncertainTimeSyncNow() error {

Converting timestamps with TimeGopher

When timeorganizer is created, it provides following conversion functions. Convert function is needed when data timestamps are converted to more storeable TimeVariable format. Instead of time.Time. Actual time.Time can be solved by Unconvert or SolveTime when there have been RTC sync.

func (p *TimeGopher) Convert(t time.Time) (TimeVariable, error)
func (p *TimeGopher) Unconvert(tv TimeVariable) (time.Time, error)
func (p *TimeGopher) SolveTime(boot int32, uptime NsUptime) (time.Time, error) //Calls unconvert

Sometimes software can restart while operational system does not boot (like "quiet restart" style in embedded devices). Software might need to do some initialization procedures at cold start. But not at warms start.

Function IsColdStart is getter function that returns coldStart flag set at creation of TimeGopher

func (p *TimeGopher) IsColdStart() bool

Documentation

Overview

Default time organizer

Helper function that does initialization that is good enough for many use cases

Helper functions for checking RTC status.

Checking is linux wall clock in sync depends on installation and hardware configuration. Initial guess is that Adjtimex should work.

Please add functions for checking synchronization status in some other methods

TimeGopher

AJATUS 18.7.2022 Talletetaan RTC kellosync mukaan jos sync

AJATUS 19.7.2022 Voisiko joitain funktioita laittaa erikseen? Jotta helpompi testattavuus

This module provides uptime.

"Algorithm" is based on fact that golang time.Time includes "hidden" monotonic clock that is used when doing time operations like diff. https://pkg.go.dev/time#hdr-Monotonic_Clocks So it is possible to resolve what time.Time uptime. Assuming that it happens after latest kernel boot (uptime 0). For resolving older uptimes from timestamps, other methods and rtc sync history is needed

/proc/uptime have 0.01s granularity meaning that anything happening more frequently than 100Hz will have repeating timestamps

Index

Constants

View Source 
const (
	DEFAULTDBFILE_UNCERTAINRTC = "uncertain.rtc"
	DEFAULTDBFILE_RTC          = "rtcsync.rtc"
	DEFAULTDBFILE_STARTLOG     = "start.time"
	DEFAULTDBFILE_STOPLOG      = "stop.time"
	DEFAULTDBFILE_ALIVELOG     = "alive.time"
)
View Source 
const (
	WARMSTARTFILE    = "/tmp/warmstart"
	WARMSTARTCONTENT = "warm start"
)
View Source 
const (
	RECORDSIZE_TIMEVARIABLE_NORTC = 12
	RECORDSIZE_TIMEVARIABLE_RTC   = 20
)
View Source 
const (
	TIME_OK   = iota //Clock synchronized, no leap second adjustment pending.
	TIME_INS         //Indicates that a leap second will be added at the end of the UTC day
	TIME_DEL         //Indicates that a leap second will be deleted at the end of the UTC day.
	TIME_OOP         //Insertion of a leap second is in progress.
	TIME_WAIT        // A leap-second insertion or deletion has been completed. This value will be returned until the next ADJ_STATUS operation clears the STA_INS and STA_DEL flags.
	TIME_ERROR
)

https://man7.org/linux/man-pages/man2/adjtimex.2.html

View Source 
const (
	STA_PLL       = 0x0001 /* enable PLL updates (rw) */
	STA_PPSFREQ   = 0x0002 /* enable PPS freq discipline (rw) */
	STA_PPSTIME   = 0x0004 /* enable PPS time discipline (rw) */
	STA_FLL       = 0x0008 /* select frequency-lock mode (rw) */
	STA_INS       = 0x0010 /* insert leap (rw) */
	STA_DEL       = 0x0020 /* delete leap (rw) */
	STA_UNSYNC    = 0x0040 /* clock unsynchronized (rw) */
	STA_FREQHOLD  = 0x0080 /* hold frequency (rw) */
	STA_PPSSIGNAL = 0x0100 /* PPS signal present (ro) */
	STA_PPSJITTER = 0x0200 /* PPS signal jitter exceeded (ro) */
	STA_PPSWANDER = 0x0400 /* PPS signal wander exceeded (ro) */
	STA_PPSERROR  = 0x0800 /* PPS signal calibration error (ro) */
	STA_CLOCKERR  = 0x1000 /* clock hardware fault (ro) */
	STA_NANO      = 0x2000 /* resolution (0 = us, 1 = ns) (ro) */
	STA_MODE      = 0x4000 /* mode (0 = PLL, 1 = FLL) (ro) */
	STA_CLK       = 0x8000 /* clock source (0 = A, 1 = B) (ro) */
)
View Source 
const EPOCH70S = 10 * 365 * 24 * 60 * 60 * 1000 * 1000 * 1000

Used for sanity check, was a joke at first

View Source 
const (
	PROCUPTIME = "/proc/uptime"
)

Variables

This section is empty.

Functions

func FirstCallAfterBoot 

func FirstCallAfterBoot(flagfilename string) (bool, error)

FirstStartAfterBoot, helper function. Call and tell is this the first time. Creates file.

func RtcIsSynced_adjtimex 

func RtcIsSynced_adjtimex() (bool, error)

RtcIsSynced_adjtimex uses syscall.Adjtimex for checking is wall clock synchronized

Types

type NsEpoch 

type NsEpoch int64

func (NsEpoch) Diff 

func (p NsEpoch) Diff(ref NsEpoch) NsEpoch

Diff compares NsEpoch, returns always positive

func (NsEpoch) Seconds 

func (p NsEpoch) Seconds() float64

Seconds converts to NsEpoch

type NsUptime 

type NsUptime int64

func GetDirectUptime 

func GetDirectUptime() (NsUptime, error)

For reference 10ms resolution only

func (NsUptime) Diff 

func (p NsUptime) Diff(ref NsUptime) NsUptime

Diff compares uptime, returns always positive

type TimeFileDb 

type TimeFileDb struct {
	// contains filtered or unexported fields
}

func CreateTimeFileDb 

func CreateTimeFileDb(storage fixregsto.FixRegSto, storeRTC bool) (TimeFileDb, error)

CreateTimeFileDb restores content from FixRegSto storage and initializes TimeFileDb struct

func (*TimeFileDb) All 

func (p *TimeFileDb) All() ([]TimeVariable, error)

func (*TimeFileDb) GetFirstN 

func (p *TimeFileDb) GetFirstN(n int) ([]TimeVariable, error)

func (*TimeFileDb) GetLatestN 

func (p *TimeFileDb) GetLatestN(n int) ([]TimeVariable, error)

func (*TimeFileDb) GetOnBoot 

func (p *TimeFileDb) GetOnBoot(boot int32) ([]TimeVariable, error)

func (*TimeFileDb) Insert 

func (p *TimeFileDb) Insert(t TimeVariable) error

func (*TimeFileDb) Len 

func (p *TimeFileDb) Len() (int, error)

func (*TimeFileDb) SearchTimeVariable 

func (p *TimeFileDb) SearchTimeVariable(epoch NsEpoch) (TimeVariable, error)

func (*TimeFileDb) SolveBootNumber 

func (p *TimeFileDb) SolveBootNumber(epoch NsEpoch) (int32, error)

Search vs solve

func (*TimeFileDb) SolveEpoch 

func (p *TimeFileDb) SolveEpoch(boot int32, uptime NsUptime) (NsEpoch, error)

type TimeGopher 

type TimeGopher struct {
	RtcMaxDeviation NsEpoch //deviation in nanosecond from RTC when in sync. Set variable default value if need to change settings

	UncertainRtcSyncLog *TimeFileDb //For manual sync
	RtcSyncLog          *TimeFileDb
	StartLog            *TimeFileDb //boot number and uptime
	StopLog             *TimeFileDb //boot number and uptime needed
	LastLog             *TimeFileDb //Last alive situation

	UptimeCheck *UptimeChecker //Create externally, better for testing
	// contains filtered or unexported fields
}

func CreateDefaultTimeGopher 

func CreateDefaultTimeGopher(rtcLogDir string, latestKnowTimeElsewhere TimeVariable) (TimeGopher, error)

Create default that is good for embedded linux use This function acts also as example use Not possible to unit test well.

func NewTimeGopher 

func NewTimeGopher(
	timeNow time.Time,

	inSync bool,
	coldStart bool,

	rtcSyncLog *TimeFileDb,
	uncertainRtcSyncLog *TimeFileDb,

	startLog *TimeFileDb,
	stopLog *TimeFileDb,

	lastLog *TimeFileDb,

	latestKnowTimeElsewhere TimeVariable,
	uptimeCheck *UptimeChecker,
) (TimeGopher, error)

Parameters: 

timeNow, give time.Now() as parameter
inSync, set true if system time is synchronized when TimeGopher is created. Resolve for example with RtcIsSynced_adjtimex()
coldStart, set true if first start. Resolve this for example with FirstCallAfterBoot(WARMSTARTFILE)
rtcSyncLog, TimeFileDb pointer for storing certain sync events
uncertainRtcSyncLog, TimeFileDb pointer for storing uncertain sync events. Nil if not needed
startLog TimeFileDb, TimeFileDb pointer for storing entries when software starts (colds and warms). Nil if not needed
stopLog TimeFileDb, TimeFileDb pointer for storing entries when software stops (entries added at next TimeGopher init). Nil if not needed
lastLog TimeFileDb, TimeFileDb pointer for keeping up situation status when sofware was running
latestKnowTimeElsewhere TimeVariable, //If some other time stamp information is kept outside TimeGopher, get latest entry here
uptimeCheck, Pointer for uptime checker. There can be many implementations depeding on needs. (or unit test requires dummy version)

func (*TimeGopher) Convert 

func (p *TimeGopher) Convert(t time.Time) (TimeVariable, error)

Convert time at current boot to TimeVariable

func (*TimeGopher) DoUncertainTimeSync 

func (p *TimeGopher) DoUncertainTimeSync(t time.Time) error

UncertainTimeSync called by library user, after realtime clock is set from unreliable source like set manually This function adds time to uncertain RTC sync log. Uncertain sync is used if certain sync is not available

func (*TimeGopher) DoUncertainTimeSyncNow 

func (p *TimeGopher) DoUncertainTimeSyncNow() error

DoUncertainTimeSyncNow is helper function for calling DoUncertainTimeSync

func (*TimeGopher) GetLatestTime 

func (p *TimeGopher) GetLatestTime() (TimeVariable, error)

GetLatestTime picks the last entry of any TimeFileDb entry inside TimeGopher instance. Used internally and for diagnostics

func (*TimeGopher) IsColdStart 

func (p *TimeGopher) IsColdStart() bool

IsColdStart() returns true if TimeOrganized have created at first time after boot One use for this function is for checking, is there need to do something "after boot" on system

func (*TimeGopher) Refresh 

func (p *TimeGopher) Refresh(t time.Time, inSync bool) error

Refresh function is called as often as application requires. Calling frequently creates frequent synclog entries so determining when sofware was running

func (*TimeGopher) RefreshNow 

func (p *TimeGopher) RefreshNow() error

RefreshNow is helper function for Refresh

func (*TimeGopher) RtcDeviation 

func (p *TimeGopher) RtcDeviation(t time.Time) (NsEpoch, error)

RtcDeviation gets RTC deviation now. Deviation can happen if timekeeping jumps Based on this, decide is RTC update needed

func (*TimeGopher) SolveTime 

func (p *TimeGopher) SolveTime(boot int32, uptime NsUptime) (time.Time, error)

SolveTime converts boot number and uptime to golang time.Time

func (*TimeGopher) Unconvert 

func (p *TimeGopher) Unconvert(tv TimeVariable) (time.Time, error)

Unconvert converts TimeVariable to time.Time, vased on what is synchronization is added. Helper function for SolveTime

type TimeVariable 

type TimeVariable struct {
	BootNumber int32
	Uptime     NsUptime
	Epoch      NsEpoch
}

TimeVariable is way to store timestamps. Epoch can solved by know TimeVariables with same bootNumber

func ParseTimeVariable 

func ParseTimeVariable(raw []byte, storeRTC bool) (TimeVariable, error)

ParseTimeVariable parses TimeVariable from binary format

func (*TimeVariable) After 

func (p *TimeVariable) After(u TimeVariable) bool

After reports whether the TimeVariable instant p is after u.

func (*TimeVariable) Before 

func (p *TimeVariable) Before(u TimeVariable) bool

After reports whether the TimeVariable instant p is before u.

func (*TimeVariable) Equal 

func (p *TimeVariable) Equal(u TimeVariable) bool

Equal reports whether p and u are equal

func (*TimeVariable) SolveEpoch 

func (p *TimeVariable) SolveEpoch(bootNumber int32, uptime NsUptime) (NsEpoch, error)

SolveEpoch calculates propotional epoch from uptime

func (*TimeVariable) SolveUptime 

func (p *TimeVariable) SolveUptime(bootNumber int32, epoch NsEpoch) (NsUptime, error)

SolveUptime calculates propotional uptime from epoch and does validity check

func (*TimeVariable) ToBinary 

func (p *TimeVariable) ToBinary(storeRTC bool) ([]byte, error)

ToBinary creates binary presentation of time variable. Some variables do not need epoch

type TimeVariableList 

type TimeVariableList []TimeVariable

func ParseTimeVariableList 

func ParseTimeVariableList(raw []byte, storeRTC bool) (TimeVariableList, error)

ParseTimeVariableList parses from raw byte array. Check length validity

func (*TimeVariableList) GetFirstsInBoot 

func (p *TimeVariableList) GetFirstsInBoot() TimeVariableList

GetFirstsInBoot picks first entries from each boot by index

func (*TimeVariableList) GetVariablesInBoot 

func (p *TimeVariableList) GetVariablesInBoot(boot int32) TimeVariableList

GetVariablesInBoot give all entries with specific boot number

func (TimeVariableList) Len 

func (e TimeVariableList) Len() int

Len number of TimeVariables in list

func (TimeVariableList) Less 

func (e TimeVariableList) Less(i, j int) bool

Less function for sorting

func (*TimeVariableList) SearchTimeVariable 

func (p *TimeVariableList) SearchTimeVariable(epoch NsEpoch) (TimeVariable, error)

SearchTimeVariable from list that is nearest to parameter epoch (at same boot)

func (*TimeVariableList) SolveBootNumber 

func (p *TimeVariableList) SolveBootNumber(epoch NsEpoch) (int32, error)

SolveBootNumber searches from list. Assumption is that array is sorted.. old at low indexes... newest at higher indexes

func (*TimeVariableList) SolveEpoch 

func (p *TimeVariableList) SolveEpoch(bootNumber int32, uptime NsUptime) (NsEpoch, error)

SolveEpoch picks TimeVariable entry at defined boot number just before or at uptime and uses that for solving epoch

func (TimeVariableList) String 

func (p TimeVariableList) String() string

String representation from TimeVariableList with newline at end

func (TimeVariableList) Swap 

func (e TimeVariableList) Swap(i, j int)

Swap function for sorting

type UptimeChecker 

type UptimeChecker struct {
	// contains filtered or unexported fields
}

func CreateUptimeChecker 

func CreateUptimeChecker() (UptimeChecker, error)

Creates uptime checker, reads uptime and sets creation time

func (*UptimeChecker) UptimeNano 

func (p *UptimeChecker) UptimeNano(tNow time.Time) (NsUptime, error)

UptimeNano resolves what is uptime on specific timestamp

Source Files

View all Source files

Directories

Path Synopsis
example
templogger

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.