README
¶
CSV Importer
Imports a CSV file as List<T> where T is a struct with fields corresponding to the CSV's column headers. The struct spec can also be set manually with the -header flag.
Usage
$ cd csv-import
$ go build
$ ./csv-import <PATH> http://localhost:8000::foo
Some places for CSV files
- https://data.cityofnewyork.us/api/views/kku6-nxdu/rows.csv?accessType=DOWNLOAD
- http://www.opendatacache.com/
CSV Exporter
Export a dataset in CSV format to stdout with column headers.
Usage
$ cd csv-export
$ go build
$ ./csv-export http://localhost:8000:foo
Documentation
¶
Index ¶
- Variables
- func GetListElemDesc(l types.List, vr types.ValueReader) types.StructDesc
- func GetMapElemDesc(m types.Map, vr types.ValueReader) types.StructDesc
- func KindsToStrings(kinds KindSlice) []string
- func MakeStructTypeFromHeaders(headers []string, structName string, kinds KindSlice) (typ *types.Type, fieldOrder []int, kindMap []types.NomsKind)
- func NewCSVReader(res io.Reader, comma rune) *csv.Reader
- func ReadToList(r *csv.Reader, structName string, headers []string, kinds KindSlice, ...) (l types.List, t *types.Type)
- func ReadToMap(r *csv.Reader, structName string, headersRaw []string, pkIdx int, ...) types.Map
- func StringToRune(delimiter string) (rune, error)
- func StringToValue(s string, k types.NomsKind) (types.Value, error)
- func WriteList(l types.List, sd types.StructDesc, comma rune, output io.Writer)
- func WriteMap(m types.Map, sd types.StructDesc, comma rune, output io.Writer)
- type KindSlice
Constants ¶
This section is empty.
Variables ¶
var StringToKind = func(kindMap map[types.NomsKind]string) map[string]types.NomsKind { m := map[string]types.NomsKind{} for k, v := range kindMap { m[v] = k } return m }(types.KindToString)
StringToKind maps names of valid NomsKinds (e.g. Bool, Number, etc) to their associated types.NomsKind
Functions ¶
func GetListElemDesc ¶
func GetListElemDesc(l types.List, vr types.ValueReader) types.StructDesc
GetListElemDesc ensures that l is a types.List of structs, pulls the types.StructDesc that describes the elements of l out of vr, and returns the StructDesc.
func GetMapElemDesc ¶
func GetMapElemDesc(m types.Map, vr types.ValueReader) types.StructDesc
GetMapElemDesc ensures that m is a types.Map of structs, pulls the types.StructDesc that describes the elements of m out of vr, and returns the StructDesc.
func KindsToStrings ¶
KindsToStrings looks up each element of kinds in the types.KindToString map and returns a slice of answers
func MakeStructTypeFromHeaders ¶
func MakeStructTypeFromHeaders(headers []string, structName string, kinds KindSlice) (typ *types.Type, fieldOrder []int, kindMap []types.NomsKind)
MakeStructTypeFromHeaders creates a struct type from the headers using |kinds| as the type of each field. If |kinds| is empty, default to strings.
func NewCSVReader ¶
NewCSVReader returns a new csv.Reader that splits on comma and asserts that all rows contain the same number of fields as the first.
func ReadToList ¶
func ReadToList(r *csv.Reader, structName string, headers []string, kinds KindSlice, vrw types.ValueReadWriter) (l types.List, t *types.Type)
ReadToList takes a CSV reader and reads data into a typed List of structs. Each row gets read into a struct named structName, described by headers. If the original data contained headers it is expected that the input reader has already read those and are pointing at the first data row. If kinds is non-empty, it will be used to type the fields in the generated structs; otherwise, they will be left as string-fields. In addition to the list, ReadToList returns the typeDef of the structs in the list.
func ReadToMap ¶
func ReadToMap(r *csv.Reader, structName string, headersRaw []string, pkIdx int, kinds KindSlice, vrw types.ValueReadWriter) types.Map
ReadToMap takes a CSV reader and reads data into a typed Map of structs. Each row gets read into a struct named structName, described by headers. If the original data contained headers it is expected that the input reader has already read those and are pointing at the first data row. If kinds is non-empty, it will be used to type the fields in the generated structs; otherwise, they will be left as string-fields.
func StringToRune ¶
StringToRune returns the rune contained in delimiter or an error.
func StringToValue ¶
StringToValue takes a piece of data as a string and attempts to convert it to a types.Value of the appropriate types.NomsKind.
Types ¶
type KindSlice ¶
KindSlice is an alias for []types.NomsKind. It's needed because types.NomsKind are really just 8 bit unsigned ints, which are what Go uses to represent 'byte', and this confuses the Go JSON marshal/unmarshal code -- it treats them as byte arrays and base64 encodes them!
func StringsToKinds ¶
StringsToKinds looks up each element of strs in the StringToKind map and returns a slice of answers
Source Files
Directories