README
¶
CSV Importer
Imports a CSV file as List<T> where T is a struct with fields corresponding to the CSV's column headers. The struct spec can also be set manually with the -header flag.
Usage
$ cd csv-import
$ go build
$ ./csv-import <PATH> http://localhost:8000:foo
Some places for CSV files
- https://data.cityofnewyork.us/api/views/kku6-nxdu/rows.csv?accessType=DOWNLOAD
- http://www.opendatacache.com/
CSV Exporter
Export a dataset in CSV format to stdout with column headers.
Usage
$ cd csv-export
$ go build
$ ./csv-export http://localhost:8000:foo
Documentation
¶
Index ¶
- Variables
- func KindsToStrings(kinds KindSlice) []string
- func MakeStructTypeFromHeaders(headers []string, structName string, kinds KindSlice) (typ *types.Type, fieldOrder []int, kindMap []types.NomsKind)
- func NewCSVReader(res io.Reader, comma rune) *csv.Reader
- func ReadToList(r *csv.Reader, structName string, headers []string, kinds KindSlice, ...) (l types.List, t *types.Type)
- func ReadToMap(r *csv.Reader, headersRaw []string, pkIdx int, kinds KindSlice, ...) types.Map
- func StringToRune(delimiter string) (rune, error)
- func StringToValue(s string, k types.NomsKind) (types.Value, error)
- func ValueToListAndElemDesc(v types.Value, vr types.ValueReader) (types.List, types.StructDesc)
- func Write(l types.List, sd types.StructDesc, comma rune, output io.Writer)
- type KindSlice
Constants ¶
This section is empty.
Variables ¶
var StringToKind = func(kindMap map[types.NomsKind]string) map[string]types.NomsKind { m := map[string]types.NomsKind{} for k, v := range kindMap { m[v] = k } return m }(types.KindToString)
StringToKind maps names of valid NomsKinds (e.g. Bool, Float32, etc) to their associated types.NomsKind
Functions ¶
func KindsToStrings ¶
KindsToStrings looks up each element of kinds in the types.KindToString map and returns a slice of answers
func MakeStructTypeFromHeaders ¶
func MakeStructTypeFromHeaders(headers []string, structName string, kinds KindSlice) (typ *types.Type, fieldOrder []int, kindMap []types.NomsKind)
MakeStructTypeFromHeaders creates a struct type from the headers using |kinds| as the type of each field. If |kinds| is empty, default to strings.
func NewCSVReader ¶
NewCSVReader returns a new csv.Reader that splits on comma and asserts that all rows contain the same number of fields as the first.
func ReadToList ¶
func ReadToList(r *csv.Reader, structName string, headers []string, kinds KindSlice, vrw types.ValueReadWriter) (l types.List, t *types.Type)
Read takes a CSV reader and reads it into a typed List of structs. Each row gets read into a struct named structName, described by headers. If the original data contained headers it is expected that the input reader has already read those and are pointing at the first data row. If kinds is non-empty, it will be used to type the fields in the generated structs; otherwise, they will be left as string-fields. In addition to the list, Read returns the typeRef for the structs in the list, and last the typeDef of the structs.
func StringToRune ¶
StringToRune returns the rune contained in delimiter or an error.
func StringToValue ¶
StringToValue takes a piece of data as a string and attempts to convert it to a types.Value of the appropriate types.NomsKind.
func ValueToListAndElemDesc ¶
func ValueToListAndElemDesc(v types.Value, vr types.ValueReader) (types.List, types.StructDesc)
ValueToListAndElemDesc ensures that v is a types.List of structs, pulls the types.StructDesc that describes the elements of v out of vr, and returns the List and related StructDesc.
Types ¶
type KindSlice ¶
KindSlice is an alias for []types.NomsKind. It's needed because types.NomsKind are really just 8 bit unsigned ints, which are what Go uses to represent 'byte', and this confuses the Go JSON marshal/unmarshal code -- it treats them as byte arrays and base64 encodes them!
func StringsToKinds ¶
StringsToKinds looks up each element of strs in the StringToKind map and returns a slice of answers
Source Files
Directories