schema

package
v1.5.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 1, 2022 License: MIT Imports: 17 Imported by: 1

Documentation

Index

Examples

Constants

View Source 
const (
	GeoPointArrayFormat  = "array"
	GeoPointObjectFormat = "object"
)

Formats specific to GeoPoint field type.

View Source 
const (
	AnyDateFormat = "any"
)

Formats.

View Source 
const InvalidPosition = -1

InvalidPosition is returned by GetField call when it refers to a field that does not exist in the schema.

View Source 
const (
	// SampleAllRows can be passed to schema.SampleLimit(int) to sample all rows.
	// schema.SampleLimit(int) is an optional argument to
	// schema.Infer(table.Table, ...InferOpts)
	SampleAllRows = -1
)

Variables

This section is empty.

Functions

This section is empty.

Types

type Constraints added in v0.1.4 

type Constraints struct {
	// Required indicates whether this field is allowed to be null.
	// Schema.MissingValues define how the string representation can
	// represent null values.
	Required bool `json:"required,omitempty"`

	// Unique indicates whether this field is allowed to have duplicates.
	// This constrain is only relevant for Schema.CastTable
	Unique bool `json:"unique,omitempty"`

	Maximum   string `json:"maximum,omitempty"`
	Minimum   string `json:"minimum,omitempty"`
	MinLength int    `json:"minLength,omitempty"`
	MaxLength int    `json:"maxLength,omitempty"`
	Pattern   string `json:"pattern,omitempty"`

	// Enum indicates that the value of the field must exactly match a value in the enum array.
	// The values of the fields could need encoding, depending on the type.
	// It applies to all field types.
	Enum []interface{} `json:"enum,omitempty"`
	// contains filtered or unexported fields
}

Constraints can be used by consumers to list constraints for validating field values.

type ConversionError added in v1.5.2 

type ConversionError struct {
	Errors []RowConversionError
}

ConversionError aggregates all errors that happened during a conversion operation (i.e., CastTable or UncastTable).

func (*ConversionError) Error added in v1.5.2 

func (ce *ConversionError) Error() string

Error returns a very simple string version of all errors found during conversion.

type Field 

type Field struct {
	// Name of the field. It is mandatory and shuold correspond to the name of field/column in the data file (if it has a name).
	Name   string    `json:"name"`
	Type   FieldType `json:"type,omitempty"`
	Format string    `json:"format,omitempty"`
	// A human readable label or title for the field.
	Title string `json:"title,omitempty"`
	// A description for this field e.g. "The recipient of the funds"
	Description string `json:"description,omitempty"`

	// Boolean properties. Define set of the values that represent true and false, respectively.
	// https://specs.frictionlessdata.io/table-schema/#boolean
	TrueValues  []string `json:"trueValues,omitempty"`
	FalseValues []string `json:"falseValues,omitempty"`

	// A string whose value is used to represent a decimal point within the number. The default value is ".".
	DecimalChar string `json:"decimalChar,omitempty"`
	// A string whose value is used to group digits within the number. The default value is null. A common value is "," e.g. "100,000".
	GroupChar string `json:"groupChar,omitempty"`
	// If true the physical contents of this field must follow the formatting constraints already set out.
	// If false the contents of this field may contain leading and/or trailing non-numeric characters which
	// are going to be stripped. Default value is true:
	BareNumber bool `json:"bareNumber,omitempty"`

	// MissingValues is a map which dictates which string values should be treated as null
	// values.
	MissingValues map[string]struct{} `json:"-"`

	// Constraints can be used by consumers to list constraints for validating
	// field values.
	Constraints Constraints `json:"constraints,omitempty"`
}

Field describes a single field in the table schema. More: https://specs.frictionlessdata.io/table-schema/#field-descriptors

func (*Field) Cast added in v1.1.2 

func (f *Field) Cast(value string) (interface{}, error)

Cast casts the passed-in string against field type. Returns an error if the value can not be cast or any field constraint can not be satisfied.

Example 
in := `{
		"name": "id",
		"type": "string",
		"format": "default",
		"constraints": {
			"required": true,
			"minLen": "5",
			"maxLen": "10",
			"pattern": ".*11$",
			"enum":["1234511"]
		}
	}`
var field Field
json.Unmarshal([]byte(in), &field)
v, err := field.Cast("1234511")
if err != nil {
	panic(err)
}
fmt.Println(v)

Output:

1234511

func (*Field) TestString added in v0.1.2 

func (f *Field) TestString(value string) bool

TestString checks whether the value can be unmarshalled to the field type.

func (*Field) Uncast added in v1.1.2 

func (f *Field) Uncast(in interface{}) (string, error)

Uncast uncasts the passed-in value into a string. It returns an error if the the type of the passed-in value can not be converted to field type.

func (*Field) UnmarshalJSON 

func (f *Field) UnmarshalJSON(data []byte) error

UnmarshalJSON sets *f to a copy of data. It will respect the default values described at: https://specs.frictionlessdata.io/table-schema/

type FieldType added in v1.1.2 

type FieldType string

FieldType defines the field types. 

const (
	IntegerType   FieldType = "integer"
	StringType    FieldType = "string"
	BooleanType   FieldType = "boolean"
	NumberType    FieldType = "number"
	DateType      FieldType = "date"
	ObjectType    FieldType = "object"
	ArrayType     FieldType = "array"
	DateTimeType  FieldType = "datetime"
	TimeType      FieldType = "time"
	YearMonthType FieldType = "yearmonth"
	YearType      FieldType = "year"
	DurationType  FieldType = "duration"
	GeoPointType  FieldType = "geopoint"
	AnyType       FieldType = "any"
)

Field types.

type Fields 

type Fields []Field

Fields represents a list of schema fields.

func (Fields) Len 

func (f Fields) Len() int

func (Fields) Less 

func (f Fields) Less(i, j int) bool

func (Fields) Swap 

func (f Fields) Swap(i, j int)

type ForeignKeyReference 

type ForeignKeyReference struct {
	Resource          string      `json:"resource,omitempty"`
	Fields            []string    `json:"-"`
	FieldsPlaceholder interface{} `json:"fields,omitempty"`
}

ForeignKeyReference represents the field reference by a foreign key.

type ForeignKeys 

type ForeignKeys struct {
	Fields            []string            `json:"-"`
	FieldsPlaceholder interface{}         `json:"fields,omitempty"`
	Reference         ForeignKeyReference `json:"reference,omitempty"`
}

ForeignKeys defines a schema foreign key

type GeoPoint 

type GeoPoint struct {
	Lon float64 `json:"lon,omitempty"`
	Lat float64 `json:"lat,omitempty"`
}

GeoPoint represents a "geopoint" cell. More at: https://specs.frictionlessdata.io/table-schema/#geopoint

func (*GeoPoint) UnmarshalJSON 

func (p *GeoPoint) UnmarshalJSON(data []byte) error

UnmarshalJSON sets *f to a copy of data. It will respect the default values

type InferOpts added in v1.1.2 

type InferOpts func(c *inferConfig) error

InferOpts defines functional options for inferring a schema.

func SampleLimit added in v1.1.2 

func SampleLimit(limit int) InferOpts

SampleLimit specifies the maximum number of rows to sample for inference.

func WithPriorityOrder added in v1.1.2 

func WithPriorityOrder(precedendeOrder []FieldType) InferOpts

WithPriorityOrder allows users to specify the priority order of types used to infer fields.

type RowConversionError added in v1.5.2 

type RowConversionError struct {
	LineNumber int
	Err        error
}

RowConversionError stores information about an error converting (cast or uncasting) a single row.

type Schema 

type Schema struct {
	Fields                Fields        `json:"fields,omitempty"`
	PrimaryKeyPlaceholder interface{}   `json:"primaryKey,omitempty"`
	PrimaryKeys           []string      `json:"-"`
	ForeignKeys           []ForeignKeys `json:"foreignKeys,omitempty"`
	MissingValues         []string      `json:"missingValues,omitempty"`
}

Schema describes tabular data.

func Infer 

func Infer(tab table.Table, opts ...InferOpts) (*Schema, error)

Infer infers a schema from a slice of the tabular data. For columns that contain cells that can inferred as different types, the most popular type is set as the field type. For instance, a column with values 10.1, 10, 10 will inferred as being of type "integer".

Example (WithPrecedence) 
tab := table.FromSlices(
	[]string{"Person", "Height"},
	[][]string{
		[]string{"Foo", "0"},
		[]string{"Bar", "0"},
	})

s, _ := Infer(
	tab,
	WithPriorityOrder([]FieldType{NumberType, BooleanType, YearType, IntegerType, GeoPointType, YearMonthType, DateType, DateTimeType, TimeType, DurationType, ArrayType, ObjectType}))
fmt.Println("Fields:")
for _, f := range s.Fields {
	fmt.Printf("{Name:%s Type:%s Format:%s}\n", f.Name, f.Type, f.Format)
}

Output:

Fields:
{Name:Person Type:string Format:default}
{Name:Height Type:number Format:default}

func InferImplicitCasting 

func InferImplicitCasting(tab table.Table, opts ...InferOpts) (*Schema, error)

InferImplicitCasting uses a implicit casting for infering the type of columns that have cells of diference types. For instance, a column with values 10.1, 10, 10 will inferred as being of type "number" ("integer" can be implicitly cast to "number").

For medium to big tables, this method is faster than the Infer.

Example 
tab := table.FromSlices(
	[]string{"Person", "Height"},
	[][]string{
		[]string{"Foo", "5"},
		[]string{"Bar", "4"},
		[]string{"Bez", "5.5"},
	})
s, _ := InferImplicitCasting(tab)
fmt.Println("Fields:")
for _, f := range s.Fields {
	fmt.Printf("{Name:%s Type:%s Format:%s}\n", f.Name, f.Type, f.Format)
}

Output:

Fields:
{Name:Person Type:string Format:default}
{Name:Height Type:number Format:default}

func LoadFromFile added in v0.1.3 

func LoadFromFile(path string) (*Schema, error)

LoadFromFile loads and parses a schema descriptor from a local file.

func LoadRemote added in v0.1.3 

func LoadRemote(url string) (*Schema, error)

LoadRemote downloads and parses a schema descriptor from the specified URL.

func Read 

func Read(r io.Reader) (*Schema, error)

Read reads and parses a descriptor to create a schema.

Example - Reading a schema from a file: 

f, err := os.Open("foo/bar/schema.json")
if err != nil {
  panic(err)
}
s, err := Read(f)
if err != nil {
  panic(err)
}
fmt.Println(s)

func (*Schema) CastColumn added in v1.1.2 

func (s *Schema) CastColumn(col []string, name string, out interface{}) error

CastColumn loads and casts all rows from a single column.

The result argument must necessarily be the address for a slice. The slice may be nil or previously allocated.

Example 
// Lets assume we have a schema ...
s := Schema{Fields: []Field{{Name: "Name", Type: StringType}, {Name: "Age", Type: IntegerType, Constraints: Constraints{Unique: true}}}}
// And a Table.
t := table.FromSlices([]string{"Name", "Age"}, [][]string{
	{"Foo", "42"},
	{"Bar", "43"}})
// And we would like to process the column Age using Go types. First we need to create a
// slice to hold the column contents.
var ages []float64
// Extract the column.
col, _ := t.ReadColumn("Age")
// And profit!
s.CastColumn(col, "Age", &ages)
fmt.Print(ages)

Output:

[42 43]

func (*Schema) CastRow 

func (s *Schema) CastRow(row []string, out interface{}) error

CastRow casts the passed-in row to schema types and stores it in the value pointed by out. The out value must be pointer to a struct. Only exported fields will be unmarshalled. The lowercased field name is used as the key for each exported field.

If a value in the row cannot be marshalled to its respective schema field (Field.Unmarshal), this call will return an error. Furthermore, this call is also going to return an error if the schema field value can not be unmarshalled to the struct field type.

Example 
// Lets assume we have a schema ...
s := Schema{Fields: []Field{{Name: "Name", Type: StringType}, {Name: "Age", Type: IntegerType}}}

// And a Table.
t := table.FromSlices([]string{"Name", "Age"}, [][]string{
	{"Foo", "42"},
	{"Bar", "43"}})

// And we would like to process them using Go types. First we need to create a struct to
// hold the content of each row.
// The tag tableheader maps the field to the schema. If no tag is set the name of the field
// has to be the same like inside the schema.
type person struct {
	MyName string `tableheader:"Name"`
	Age    int
}

// Now it is a matter of iterate over the table and Cast each row.
iter, _ := t.Iter()
for iter.Next() {
	var p person
	s.CastRow(iter.Row(), &p)
	fmt.Printf("%+v\n", p)
}

Output:

{MyName:Foo Age:42}
{MyName:Bar Age:43}

func (*Schema) CastTable added in v1.1.2 

func (s *Schema) CastTable(tab table.Table, out interface{}) error

CastTable loads and casts all table rows in a best effort manner. Line-by-line errors will be reported as *ConversionError type

The result argument must necessarily be the address for a slice. The slice may be nil or previously allocated.

Example 
// Lets assume we have a schema ...
s := Schema{Fields: []Field{{Name: "Name", Type: StringType}, {Name: "Age", Type: IntegerType, Constraints: Constraints{Unique: true}}}}

// And a Table.
t := table.FromSlices([]string{"Name", "Age"}, [][]string{
	{"Foo", "42"},
	{"Bar", "43"}})

// And we would like to process them using Go types. First we need to create a struct to
// hold the content of each row.
// The tag tableheader maps the field to the schema. If no tag is set the name of the field
// has to be the same like inside the schema.
type person struct {
	MyName string `tableheader:"Name"`
	Age    int
}
var people []person
s.CastTable(t, &people)
fmt.Print(people)

Output:

[{Foo 42} {Bar 43}]

func (*Schema) GetField 

func (s *Schema) GetField(name string) (*Field, int)

GetField fetches the index and field referenced by the name argument.

func (*Schema) HasField 

func (s *Schema) HasField(name string) bool

HasField returns checks whether the schema has a field with the passed-in.

func (*Schema) MarshalJSON 

func (s *Schema) MarshalJSON() ([]byte, error)

MarshalJSON returns the JSON encoding of s.

func (*Schema) SaveToFile 

func (s *Schema) SaveToFile(path string) error

SaveToFile writes the schema descriptor in local file.

func (*Schema) String added in v1.1.2 

func (s *Schema) String() string

String returns an human readable version of the schema.

func (*Schema) UncastRow added in v1.1.2 

func (s *Schema) UncastRow(in interface{}) ([]string, error)

UncastRow uncasts struct into a row. This method can only uncast structs (or pointer to structs) and will error out if nil is passed. The order of the cells in the returned row is the schema declaration order.

Example 
// Lets assume we have a schema.
s := Schema{Fields: []Field{{Name: "Name", Type: StringType}, {Name: "Age", Type: IntegerType}}}

// And would like to create a CSV out of this list. The tag tableheader maps
// the field to the schema name. If no tag is set the name of the field
// has to be the same like inside the schema.
people := []struct {
	MyName string `tableheader:"Name"`
	Age    int
}{{"Foo", 42}, {"Bar", 43}}

// First create the writer and write the header.
w := table.NewStringWriter()
w.Write([]string{"Name", "Age"})

// Then write the list
for _, person := range people {
	row, _ := s.UncastRow(person)
	w.Write(row)
}
w.Flush()
fmt.Print(w.String())

Output:

Name,Age
Foo,42
Bar,43

func (*Schema) UncastTable added in v1.1.2 

func (s *Schema) UncastTable(in interface{}) ([][]string, error)

UncastTable uncasts each element (struct) of the passed-in slice and

Example 
// Lets assume we have a schema.
s := Schema{Fields: []Field{{Name: "Name", Type: StringType}, {Name: "Age", Type: IntegerType}}}

// And would like to create a CSV out of this list. The tag tableheader maps
// the field to the schema name. If no tag is set the name of the field
// has to be the same like inside the schema.
people := []struct {
	MyName string `tableheader:"Name"`
	Age    int
}{{"Foo", 42}, {"Bar", 43}}

// Then uncast the people slice into a slice of rows.
rows, _ := s.UncastTable(people)

// Now, simply write it down.
w := table.NewStringWriter()
w.Write([]string{"Name", "Age"})
w.WriteAll(rows)
w.Flush()
fmt.Print(w.String())

Output:

Name,Age
Foo,42
Bar,43

func (*Schema) UnmarshalJSON 

func (s *Schema) UnmarshalJSON(data []byte) error

UnmarshalJSON sets *f to a copy of data. It will respect the default values described at: https://specs.frictionlessdata.io/table-schema/

func (*Schema) Validate 

func (s *Schema) Validate() error

Validate checks whether the schema is valid. If it is not, returns an error describing the problem. More at: https://specs.frictionlessdata.io/table-schema/

func (*Schema) Write 

func (s *Schema) Write(w io.Writer) error

Write writes the schema descriptor.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.