server

package
v1.23.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 2, 2021 License: Apache-2.0 Imports: 34 Imported by: 0

Documentation

Index

Constants

View Source 
const BaseRank = 100

BaseRank is the base ranking score for sources. If a source is prefered, it should be given a score lower than BaseRank in StatsRanking. If a source is not prefered, it should be given a score higher than BaseRank in StatsRanking

Variables

View Source 
var RelatedLocationsPrefixMap = map[bool]map[bool]string{
	// contains filtered or unexported fields
}

RelatedLocationsPrefixMap is a map from different scenarios to key prefix for RelatedLocations cache.

The three levels of keys are: - Whether related locations have the same ancestor. - Whether related locations have the same place type. - Whether closeness computaion is per capita.

View Source 
var StatsRanking = map[RankKey]int{
	{"CensusPEP", "CensusPEPSurvey"}:                                      0,
	{"CensusACS5YearSurvey", "CensusACS5yrSurvey"}:                        1,
	{"CensusACS5YearSurvey_AggCountry", "dcAggregate/CensusACS5yrSurvey"}: 1,
	{"CensusUSAMedianAgeIncome", "CensusACS5yrSurvey"}:                    1,
	{"EurostatData", "EurostatRegionalPopulationData"}:                    2,
	{"WorldDevelopmentIndicators", ""}:                                    3,
	{"BLS_LAUS", "BLSSeasonallyUnadjusted"}:                               0,
	{"EurostatData", ""}:                                                  1,
	{"NYT_COVID19", "NYT_COVID19_GitHub"}:                                 0,
	{"CDC500", "AgeAdjustedPrevalence"}:                                   0,
	{"WikidataPopulation", "WikidataPopulation"}:                          1001,
}

StatsRanking is used to rank multiple source series for the same StatisticalVariable, where lower value means higher ranking. The ranking score ranges from 0 to 100.

Functions

func BuildParentSvgMap added in v1.21.0 

func BuildParentSvgMap(rawSvg map[string]*pb.StatVarGroupNode) map[string][]string

BuildParentSvgMap gets the mapping of svg/sv id to the parent svg for that svg/sv.

func GetRawSvg added in v1.20.2 

func GetRawSvg(ctx context.Context, baseTable *bigtable.Table) (
	map[string]*pb.StatVarGroupNode, error)

GetRawSvg gets the raw svg mapping.

func NewBtTable 

func NewBtTable(
	ctx context.Context, projectID, instanceID, tableID string) (
	*bigtable.Table, error)

NewBtTable creates a new bigtable.Table instance.

func ReadBranchTableName added in v1.14.0 

func ReadBranchTableName(
	ctx context.Context, bucket, versionFile string) (string, error)

ReadBranchTableName reads branch cache folder from GCS.

func SetupBigtable added in v1.14.0 

func SetupBigtable(
	ctx context.Context, data map[string]string) (*bigtable.Table, error)

SetupBigtable creates a testing bigtable instance used for testing

Types

type Cache added in v1.20.2 

type Cache struct {
	// ParentSvg is a map of sv/svg id to a list of its parent svgs sorted alphabetically.
	ParentSvg map[string][]string
	// SvgInfo is a map of svg id to its information.
	SvgInfo                   map[string]*pb.StatVarGroupNode
	SvgSearchIndex            *SearchIndex
	BlocklistedSvgSearchIndex *SearchIndex
}

Cache holds cached data for the mixer server.

func NewCache added in v1.20.2 

func NewCache(ctx context.Context, baseTable *bigtable.Table) (*Cache, error)

NewCache initializes the cache for stat var hierarchy.

type CohortByRank added in v1.19.1 

type CohortByRank []*pb.SourceSeries

CohortByRank implements sort.Interface for []*SourceSeries based on the rank score. Each source series data is keyed by the place dcid.

Note this has the same data type as SeriesByRank but is used to compare cohort instead of time series.

func (CohortByRank) Len added in v1.19.1 

func (a CohortByRank) Len() int

func (CohortByRank) Less added in v1.19.1 

func (a CohortByRank) Less(i, j int) bool

func (CohortByRank) Swap added in v1.19.1 

func (a CohortByRank) Swap(i, j int)

type Metadata 

type Metadata struct {
	Mappings         []*base.Mapping
	OutArcInfo       map[string]map[string][]translator.OutArcInfo
	InArcInfo        map[string][]translator.InArcInfo
	SubTypeMap       map[string]string
	Bq               string
	BtProject        string
	BranchBtInstance string
}

Metadata represents the metadata used by the server.

func NewMetadata 

func NewMetadata(
	bqDataset, storeProject, branchInstance, schemaPath string) (*Metadata, error)

NewMetadata initialize the metadata for translator.

type Node 

type Node struct {
	Dcid   string   `json:"dcid,omitempty"`
	Name   string   `json:"name,omitempty"`
	ProvID string   `json:"provenanceId,omitempty"`
	Value  string   `json:"value,omitempty"`
	Types  []string `json:"types,omitempty"`
}

Node represents a information about a node.

type ObsProp 

type ObsProp struct {
	Mmethod string
	Operiod string
	Unit    string
	Sfactor string
}

ObsProp represents properties for a StatObservation.

type ObsTimeSeries 

type ObsTimeSeries struct {
	Data          map[string]float64 `json:"data,omitempty"`
	PlaceName     string             `json:"placeName,omitempty"`
	PlaceDcid     string             `json:"placeDcid,omitempty"`
	SourceSeries  []*SourceSeries    `json:"sourceSeries,omitempty"`
	ProvenanceURL string             `json:"provenanceUrl,omitempty"`
}

ObsTimeSeries repesents multiple time series data.

type PlacePopInfo 

type PlacePopInfo struct {
	PlaceID      string `json:"dcid,omitempty"`
	PopulationID string `json:"population,omitempty"`
}

PlacePopInfo contains basic info for a place and a population.

type PlaceStatsVar 

type PlaceStatsVar struct {
	StatVarIds []string `json:"statVarIds,omitempty"`
}

PlaceStatsVar contains the place stats var list.

type PropLabelCache 

type PropLabelCache struct {
	InLabels  []string `json:"inLabels"`
	OutLabels []string `json:"outLabels"`
}

PropLabelCache represents the json structure returned by the BT Prop cache

type PropValueCache 

type PropValueCache struct {
	Nodes []*Node `json:"entities,omitempty"`
}

PropValueCache represents the json structure returned by the BT PropVal cache

type RankKey 

type RankKey struct {
	Prov    string
	Mmethod string
}

RankKey represents keys used for ranking.

type RankingInfo added in v1.20.2 

type RankingInfo struct {
	// ApproxNumPv is an estimate of the number of PVs in the sv/svg.
	ApproxNumPv int
	// RankingName is the name we will be using to rank this sv/svg against other
	// sv/svg.
	RankingName string
}

RankingInfo holds the ranking information for each stat var hierarchy search result.

type RelatedPlacesInfo 

type RelatedPlacesInfo struct {
	RelatedPlaces  []string `json:"relatedPlaces,omitempty"`
	RankFromTop    int32    `json:"rankFromTop,omitempty"`
	RankFromBottom int32    `json:"rankFromBottom,omitempty"`
}

RelatedPlacesInfo represents the json structure returned by the RelatedPlaces cache.

type SearchIndex added in v1.21.0 

type SearchIndex struct {
	// contains filtered or unexported fields
}

SearchIndex holds the index for searching stat var (group).

func BuildStatVarSearchIndex added in v1.21.0 

func BuildStatVarSearchIndex(
	rawSvg map[string]*pb.StatVarGroupNode,
	blocklist bool) *SearchIndex

BuildStatVarSearchIndex builds the search index for the stat var hierarchy.

type SeriesByRank 

type SeriesByRank []*pb.SourceSeries

SeriesByRank implements sort.Interface for []*SourceSeries based on the rank score. Each source series data is keyed by the observation date.

This is the protobuf version of byRank.

func (SeriesByRank) Len 

func (a SeriesByRank) Len() int

func (SeriesByRank) Less 

func (a SeriesByRank) Less(i, j int) bool

func (SeriesByRank) Swap 

func (a SeriesByRank) Swap(i, j int)

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server holds resources for a mixer server

func NewServer 

func NewServer(
	bqClient *bigquery.Client,
	baseTable *bigtable.Table,
	branchTable *bigtable.Table,
	metadata *Metadata,
	cache *Cache) *Server

NewServer creates a new server instance.

func (*Server) GetBioPageData added in v1.22.0 

func (s *Server) GetBioPageData(
	ctx context.Context, in *pb.GetBioPageDataRequest) (
	*pb.GraphNode, error)

GetBioPageData implements API for Mixer.GetBioPageData.

TODO(shifucun): This is only a mini version with partial data. Use pre-computed data from Bigtable when it's ready.

func (*Server) GetLocationsRankings 

func (s *Server) GetLocationsRankings(ctx context.Context,
	in *pb.GetLocationsRankingsRequest) (*pb.GetLocationsRankingsResponse, error)

GetLocationsRankings implements API for Mixer.GetLocationsRankings.

func (*Server) GetPlaceObs 

func (s *Server) GetPlaceObs(ctx context.Context, in *pb.GetPlaceObsRequest) (
	*pb.SVOCollection, error)

GetPlaceObs implements API for Mixer.GetPlaceObs.

func (*Server) GetPlacePageData added in v1.22.0 

func (s *Server) GetPlacePageData(
	ctx context.Context, in *pb.GetPlacePageDataRequest) (
	*pb.GetPlacePageDataResponse, error)

GetPlacePageData implements API for Mixer.GetPlacePageData.

TODO(shifucun):For each related place, it is supposed to have dcid, name and population but it's not complete now as the client in most cases only requires the dcid. Should consider have the full name, even with parent place abbreviations like "CA" filled in here so the client won't bother to fetch those again.

func (*Server) GetPlaceStatDateWithinPlace 

func (s *Server) GetPlaceStatDateWithinPlace(
	ctx context.Context, in *pb.GetPlaceStatDateWithinPlaceRequest) (
	*pb.GetPlaceStatDateWithinPlaceResponse, error)

GetPlaceStatDateWithinPlace implements API for Mixer.GetPlaceStatDateWithinPlace. Endpoint: /place/stat/date/within-place

func (*Server) GetPlaceStatVars 

func (s *Server) GetPlaceStatVars(
	ctx context.Context, in *pb.GetPlaceStatVarsRequest) (
	*pb.GetPlaceStatVarsResponse, error)

GetPlaceStatVars implements API for Mixer.GetPlaceStatVars.

func (*Server) GetPlaceStatVarsUnionV1 added in v1.20.3 

func (s *Server) GetPlaceStatVarsUnionV1(
	ctx context.Context, in *pb.GetPlaceStatVarsUnionRequest) (
	*pb.GetPlaceStatVarsUnionResponse, error)

GetPlaceStatVarsUnionV1 implements API for Mixer.GetPlaceStatVarsUnionV1.

func (*Server) GetPlaceStatsVar 

func (s *Server) GetPlaceStatsVar(
	ctx context.Context, in *pb.GetPlaceStatsVarRequest) (
	*pb.GetPlaceStatsVarResponse, error)

GetPlaceStatsVar implements API for Mixer.GetPlaceStatsVar. TODO(shifucun): Migrate clients to use GetPlaceStatVars and deprecate this.

func (*Server) GetPlacesIn 

func (s *Server) GetPlacesIn(ctx context.Context, in *pb.GetPlacesInRequest) (
	*pb.GetPlacesInResponse, error)

GetPlacesIn implements API for Mixer.GetPlacesIn.

func (*Server) GetPropertyLabels 

func (s *Server) GetPropertyLabels(ctx context.Context,
	in *pb.GetPropertyLabelsRequest) (*pb.GetPropertyLabelsResponse, error)

GetPropertyLabels implements API for Mixer.GetPropertyLabels.

func (*Server) GetPropertyValues 

func (s *Server) GetPropertyValues(ctx context.Context,
	in *pb.GetPropertyValuesRequest) (*pb.GetPropertyValuesResponse, error)

GetPropertyValues implements API for Mixer.GetPropertyValues.

func (*Server) GetRelatedLocations 

func (s *Server) GetRelatedLocations(ctx context.Context,
	in *pb.GetRelatedLocationsRequest) (*pb.GetRelatedLocationsResponse, error)

GetRelatedLocations implements API for Mixer.GetRelatedLocations.

func (*Server) GetStatAll 

func (s *Server) GetStatAll(ctx context.Context, in *pb.GetStatAllRequest) (
	*pb.GetStatAllResponse, error)

GetStatAll implements API for Mixer.GetStatAll. Endpoint: /stat/set/series/all Endpoint: /stat/all

func (*Server) GetStatSeries 

func (s *Server) GetStatSeries(
	ctx context.Context, in *pb.GetStatSeriesRequest) (
	*pb.GetStatSeriesResponse, error)

GetStatSeries implements API for Mixer.GetStatSeries. Endpoint: /stat/series TODO(shifucun): consilidate and dedup the logic among these similar APIs.

func (*Server) GetStatSet 

func (s *Server) GetStatSet(ctx context.Context, in *pb.GetStatSetRequest) (
	*pb.GetStatSetResponse, error)

GetStatSet implements API for Mixer.GetStatSet. Endpoint: /stat/set

func (*Server) GetStatSetSeries added in v1.14.0 

func (s *Server) GetStatSetSeries(ctx context.Context, in *pb.GetStatSetSeriesRequest) (
	*pb.GetStatSetSeriesResponse, error)

GetStatSetSeries implements API for Mixer.GetStatSetSeries. Endpoint: /v1/stat/set/series

func (*Server) GetStatSetWithinPlace added in v1.19.1 

func (s *Server) GetStatSetWithinPlace(
	ctx context.Context, in *pb.GetStatSetWithinPlaceRequest) (
	*pb.GetStatSetResponse, error)

GetStatSetWithinPlace implements API for Mixer.GetStatSetWithinPlace. Endpoint: /stat/set/within-place

func (*Server) GetStatValue 

func (s *Server) GetStatValue(ctx context.Context, in *pb.GetStatValueRequest) (
	*pb.GetStatValueResponse, error)

GetStatValue implements API for Mixer.GetStatValue. Endpoint: /stat (/stat/value)

func (*Server) GetStatVarGroup added in v1.18.0 

func (s *Server) GetStatVarGroup(
	ctx context.Context, in *pb.GetStatVarGroupRequest) (
	*pb.StatVarGroups, error)

GetStatVarGroup implements API for Mixer.GetStatVarGroup.

func (*Server) GetStatVarGroupNode added in v1.21.0 

func (s *Server) GetStatVarGroupNode(
	ctx context.Context, in *pb.GetStatVarGroupNodeRequest) (
	*pb.StatVarGroupNode, error)

GetStatVarGroupNode implements API for Mixer.GetStatVarGroupNode.

func (*Server) GetStatVarPath added in v1.21.0 

func (s *Server) GetStatVarPath(
	ctx context.Context, in *pb.GetStatVarPathRequest) (
	*pb.GetStatVarPathResponse, error)

GetStatVarPath implements API for Mixer.GetStatVarPath.

func (*Server) GetStatVarSummary added in v1.21.0 

func (s *Server) GetStatVarSummary(
	ctx context.Context, in *pb.GetStatVarSummaryRequest) (
	*pb.GetStatVarSummaryResponse, error)

GetStatVarSummary implements API for Mixer.GetStatVarSummary.

func (*Server) GetStats 

func (s *Server) GetStats(ctx context.Context, in *pb.GetStatsRequest) (
	*pb.GetStatsResponse, error)

GetStats implements API for Mixer.GetStats. Endpoint: /stat/set/series Endpoint: /bulk/stats

func (*Server) GetTriples 

func (s *Server) GetTriples(ctx context.Context, in *pb.GetTriplesRequest) (
	*pb.GetTriplesResponse, error)

GetTriples implements API for Mixer.GetTriples.

func (*Server) GetVersion added in v1.18.0 

func (s *Server) GetVersion(ctx context.Context,
	in *pb.GetVersionRequest) (*pb.GetVersionResponse, error)

GetVersion implements API for Mixer.GetVersion.

func (*Server) Query 

func (s *Server) Query(
	ctx context.Context, in *pb.QueryRequest) (*pb.QueryResponse, error)

Query implements API for Mixer.Query.

func (*Server) Search 

func (s *Server) Search(
	ctx context.Context, in *pb.SearchRequest) (*pb.SearchResponse, error)

Search implements API for Mixer.Search.

func (*Server) SearchStatVar added in v1.21.0 

func (s *Server) SearchStatVar(
	ctx context.Context, in *pb.SearchStatVarRequest) (
	*pb.SearchStatVarResponse, error,
)

SearchStatVar implements API for Mixer.SearchStatVar.

func (*Server) SubscribeBranchCacheUpdate 

func (s *Server) SubscribeBranchCacheUpdate(
	ctx context.Context, pubsubProjectID, branchCacheBucket, subscriberPrefix,
	pubsubTopic string) (*pubsub.Subscription, error)

SubscribeBranchCacheUpdate subscribe server for branch cache update.

func (*Server) Translate 

func (s *Server) Translate(ctx context.Context,
	in *pb.TranslateRequest) (*pb.TranslateResponse, error)

Translate implements API for Mixer.Translate.

type SourceSeries 

type SourceSeries struct {
	ImportName        string             `json:"importName,omitempty"`
	ObservationPeriod string             `json:"observationPeriod,omitempty"`
	MeasurementMethod string             `json:"measurementMethod,omitempty"`
	ScalingFactor     string             `json:"scalingFactor,omitempty"`
	Unit              string             `json:"unit,omitempty"`
	ProvenanceURL     string             `json:"provenanceUrl,omitempty"`
	Val               map[string]float64 `json:"val,omitempty"`
}

SourceSeries represents time series data for a particular source.

type StatisticalVariable 

type StatisticalVariable struct {
	PopType                string            `json:"popType,omitempty"`
	PVs                    map[string]string `json:"pvs,omitempty"`
	MeasuredProp           string            `json:"measuredProp,omitempty"`
	MeasurementMethod      string            `json:"measurementMethod,omitempty"`
	MeasurementDenominator string            `json:"measurementDeonominator,omitempty"`
	MeasurementQualifier   string            `json:"measurementQualifier,omitempty"`
	ScalingFactor          string            `json:"scalingFactor,omitempty"`
	Unit                   string            `json:"unit,omitempty"`
	StatType               string            `json:"statType,omitempty"`
}

StatisticalVariable contains key info of population and observation.

type Triple 

type Triple struct {
	SubjectID    string   `json:"subjectId,omitempty"`
	SubjectName  string   `json:"subjectName,omitempty"`
	SubjectTypes []string `json:"subjectTypes,omitempty"`
	Predicate    string   `json:"predicate,omitempty"`
	ObjectID     string   `json:"objectId,omitempty"`
	ObjectName   string   `json:"objectName,omitempty"`
	ObjectValue  string   `json:"objectValue,omitempty"`
	ObjectTypes  []string `json:"objectTypes,omitempty"`
	ProvenanceID string   `json:"provenanceId,omitempty"`
}

Triple represents a triples entry in the BT triples cache.

type TriplesCache 

type TriplesCache struct {
	Triples []*Triple `json:"triples"`
}

TriplesCache represents the json structure returned by the BT triples cache

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.