Documentation ¶
Overview ¶
Package sculptor is flexible and powerful Go library for transforming data from various formats (CSV, JSON, etc.) into desired Go struct types. Getting Start with
package main import ( "fmt" "github.com/esonhugh/sculptor" ) type TestStruct struct { Name string `select:"name"` Pass string `select:"pass"` } // json like `{"user": "username","pass": "114514","content": "123444", "id":2}` // csv like `user,pass,content,id\nusername,114514,123444,2` Doc := sculptor.NewDataSculptor("your_test.json"). SetDocType(sculptor.JSON_DOCUMENT). SetQuery("name", "user"). SetQuery("pass", "pass"). SetTargetStruct(&TestStruct{}) go Doc.Do() for i := range Doc.ConstructedOutput { fmt.Println(i) }
Index ¶
- Constants
- Variables
- func Merge(sculptors ...*DataSculptor) chan any
- type DataSculptor
- func (d *DataSculptor) CurrentTarget() any
- func (d *DataSculptor) Do()
- func (d *DataSculptor) Error() error
- func (d *DataSculptor) SetBufSize(size int)
- func (d *DataSculptor) SetCSVDelimiter(r rune) *DataSculptor
- func (d *DataSculptor) SetCustomFunc(f ...Func) *DataSculptor
- func (d *DataSculptor) SetDocType(docType DocumentType) *DataSculptor
- func (d *DataSculptor) SetFallbackFunc(f ...Func) *DataSculptor
- func (d *DataSculptor) SetOption(o Options) *DataSculptor
- func (d *DataSculptor) SetQuery(tagName string, Query string) *DataSculptor
- func (d *DataSculptor) SetScanner(dataParser parser.RawDataParser) *DataSculptor
- func (d *DataSculptor) SetTargetStruct(targetStruct any) *DataSculptor
- type DocumentType
- type Func
- type Options
Constants ¶
const ( JSON_DOCUMENT = DocumentType("json") CSV_DOCUMENT = DocumentType("csv") )
DataExtrator types supported
Variables ¶
var DefaultOptions = Options{ Latency: 1 * time.Millisecond, BufSize: 10, TagKey: "select", DebugLevel: log.WarnLevel, }
DefaultOptions is the default options used by DataSculptor
Functions ¶
func Merge ¶ added in v0.1.2
func Merge(sculptors ...*DataSculptor) chan any
Merge merges multiple DataSculptor output in one channel
Types ¶
type DataSculptor ¶
type DataSculptor struct { // DocType is the type of document to be extracted DocType DocumentType // Filename is processed filename or path Filename string // CTX set for goruntime if thread process. CTX context.Context // ConstructedOutput is the channel to send the extracted data out ConstructedOutput chan any // contains filtered or unexported fields }
DocumentType is core struct in package. it contains all the information required to extract data from a document during runtime.
func NewDataSculptor ¶
func NewDataSculptor(file string) *DataSculptor
NewDataSculptor returns a new DataSculptor with initialized values
func NewDataSculptorWithOptionsAndCTX ¶ added in v0.1.2
func NewDataSculptorWithOptionsAndCTX(file string, o Options, ctx context.Context) *DataSculptor
NewDataSculptorWithOptions returns a new DataSculptor with initialized values
func (*DataSculptor) CurrentTarget ¶
func (d *DataSculptor) CurrentTarget() any
CurrentTarget func returns the current target struct during process. Helpful in SetFallbackFunc and SetCustomFunc.
func (*DataSculptor) Do ¶
func (d *DataSculptor) Do()
Do func extract the data from file and put it into the ConstructedOutput chan use go Do() to make it as a new runtime.
func (*DataSculptor) Error ¶
func (d *DataSculptor) Error() error
Error() func returns the last error occurred while processing the record. Helpful in SetFallbackFunc and SetCustomFunc.
func (*DataSculptor) SetBufSize ¶ added in v0.1.2
func (d *DataSculptor) SetBufSize(size int)
SetBufSize will reset the buffer size of the channel
func (*DataSculptor) SetCSVDelimiter ¶
func (d *DataSculptor) SetCSVDelimiter(r rune) *DataSculptor
SetCSVDelimiter sets the delimiter for the CSV document if you set the document type to CSV. Else it will make error.
func (*DataSculptor) SetCustomFunc ¶
func (d *DataSculptor) SetCustomFunc(f ...Func) *DataSculptor
SetCustomFunc sets the customFunc which will be called between constructing targetStruct complete and sending the extracted data to the channel.
func (*DataSculptor) SetDocType ¶
func (d *DataSculptor) SetDocType(docType DocumentType) *DataSculptor
SetDocType sets the document type for the given filename. If Supported, It will automatically set the scanner for the given document type. If not supported, it will panic. If you want Set you own scanners please use SetScanner() and follow the interface.
func (*DataSculptor) SetFallbackFunc ¶
func (d *DataSculptor) SetFallbackFunc(f ...Func) *DataSculptor
SetFallbackFunc sets the fallbackFunc which will be called when framework can't handle the record.
func (*DataSculptor) SetOption ¶
func (d *DataSculptor) SetOption(o Options) *DataSculptor
SetOption set options to replace default options
func (*DataSculptor) SetQuery ¶
func (d *DataSculptor) SetQuery(tagName string, Query string) *DataSculptor
SetQuery sets the query for the given tag name
func (*DataSculptor) SetScanner ¶
func (d *DataSculptor) SetScanner(dataParser parser.RawDataParser) *DataSculptor
SetScanner sets the scanner (parser.RawDataParser) which used to extract data from the document. SetScanner is helpful if you want to use your own scanner to process your file.
func (*DataSculptor) SetTargetStruct ¶
func (d *DataSculptor) SetTargetStruct(targetStruct any) *DataSculptor
SetTargetStruct sets the target struct with the given struct pointer. Helpful in SetFallbackFunc and SetCustomFunc. It will be called when init before the Do() func.
type DocumentType ¶
type DocumentType string
type Func ¶
type Func func(*DataSculptor) error
Func is alias of DataSculptor customFunc and fallbackFunc type
Directories ¶
Path | Synopsis |
---|---|
Package parser is transformers collection of parsing any type of social_engineering_data to our own data struct
|
Package parser is transformers collection of parsing any type of social_engineering_data to our own data struct |
test
|
|