README
¶
FinancialDataCrawlingPlatform
基于Pholcus网路爬虫框架的金融数据爬取平台
Directories
¶
| Path | Synopsis |
|---|---|
|
src
|
|
|
github.com/acroca/go-symbols
The gosymbols command prints type information for package-level symbols.
|
The gosymbols command prints type information for package-level symbols. |
|
github.com/cweill/gotests
Package gotests contains the core logic for generating table-driven tests.
|
Package gotests contains the core logic for generating table-driven tests. |
|
github.com/cweill/gotests/gotests
A commandline tool for generating table-driven Go tests.
|
A commandline tool for generating table-driven Go tests. |
|
github.com/cweill/gotests/gotests/process
Package process is a thin wrapper around the gotests library.
|
Package process is a thin wrapper around the gotests library. |
|
github.com/cweill/gotests/internal/goparser
Package goparse contains logic for parsing Go files.
|
Package goparse contains logic for parsing Go files. |
|
github.com/derekparker/delve/pkg/dwarf/frame
Package frame contains data structures and related functions for parsing and searching through Dwarf .debug_frame data.
|
Package frame contains data structures and related functions for parsing and searching through Dwarf .debug_frame data. |
|
github.com/derekparker/delve/pkg/proc
Package proc is a low-level package that provides methods to manipulate the process we are debugging.
|
Package proc is a low-level package that provides methods to manipulate the process we are debugging. |
|
github.com/derekparker/delve/pkg/terminal
Package terminal implements functions for responding to user input and dispatching to appropriate backend commands.
|
Package terminal implements functions for responding to user input and dispatching to appropriate backend commands. |
|
github.com/golang/lint
Package lint contains a linter for Go source code.
|
Package lint contains a linter for Go source code. |
|
github.com/golang/lint/golint
golint lints the Go source files named on its command line.
|
golint lints the Go source files named on its command line. |
|
github.com/henrylee2cn/pholcus
Pholcus（幽灵蛛）是一款纯Go语言编写的支持分布式的高并发、重量级爬虫软件，定位于互联网数据采集，为具备一定Go或JS编程基础的人提供一个只需关注规则定制的功能强大的爬虫工具。
|
Pholcus（幽灵蛛）是一款纯Go语言编写的支持分布式的高并发、重量级爬虫软件，定位于互联网数据采集，为具备一定Go或JS编程基础的人提供一个只需关注规则定制的功能强大的爬虫工具。 |
|
github.com/henrylee2cn/pholcus/app
app interface for graphical user interface.
|
app interface for graphical user interface. |
|
github.com/henrylee2cn/pholcus/app/downloader/surfer
surfer是一款Go语言编写的高并发web下载器，支持 GET/POST/HEAD 方法及 http/https 协议，同时支持固定UserAgent自动保存cookie与随机大量UserAgent禁用cookie两种模式，高度模拟浏览器行为，可实现模拟登录等功能。
|
surfer是一款Go语言编写的高并发web下载器，支持 GET/POST/HEAD 方法及 http/https 协议，同时支持固定UserAgent自动保存cookie与随机大量UserAgent禁用cookie两种模式，高度模拟浏览器行为，可实现模拟登录等功能。 |
|
github.com/henrylee2cn/pholcus/app/downloader/surfer/agent
Package agent generates user agents strings for well known browsers and for custom browsers.
|
Package agent generates user agents strings for well known browsers and for custom browsers. |
| 数据收集 | |
| 结果收集与输出 | |
|
github.com/henrylee2cn/pholcus/cmd
[spider frame (golang)] Pholcus（幽灵蛛）是一款纯Go语言编写的高并发、分布式、重量级爬虫软件，支持单机、服务端、客户端三种运行模式，拥有Web、GUI、命令行三种操作界面；规则简单灵活、批量任务并发、输出方式丰富（mysql/mongodb/csv/excel等）、有大量Demo共享；同时她还支持横纵向两种抓取模式，支持模拟登录和任务暂停、取消等一系列高级功能； （官方QQ群：Go大数据 42731170，欢迎加入我们的讨论）。
|
[spider frame (golang)] Pholcus（幽灵蛛）是一款纯Go语言编写的高并发、分布式、重量级爬虫软件，支持单机、服务端、客户端三种运行模式，拥有Web、GUI、命令行三种操作界面；规则简单灵活、批量任务并发、输出方式丰富（mysql/mongodb/csv/excel等）、有大量Demo共享；同时她还支持横纵向两种抓取模式，支持模拟登录和任务暂停、取消等一系列高级功能； （官方QQ群：Go大数据 42731170，欢迎加入我们的讨论）。 |
|
github.com/henrylee2cn/pholcus/common/config
Package config is used to parse config Usage: import(
|
Package config is used to parse config Usage: import( |
|
github.com/henrylee2cn/pholcus/common/goquery
Package goquery implements features similar to jQuery, including the chainable syntax, to manipulate and query an HTML document.
|
Package goquery implements features similar to jQuery, including the chainable syntax, to manipulate and query an HTML document. |
|
github.com/henrylee2cn/pholcus/common/mahonia
This package is a character-set conversion library for Go.
|
This package is a character-set conversion library for Go. |
|
github.com/henrylee2cn/pholcus/common/ping
taken from http://golang.org/src/pkg/net/ipraw_test.go
|
taken from http://golang.org/src/pkg/net/ipraw_test.go |
|
github.com/henrylee2cn/pholcus/common/pool
通用资源池，动态增加资源实例，并支持空闲资源定时回收功能。
|
通用资源池，动态增加资源实例，并支持空闲资源定时回收功能。 |
|
github.com/henrylee2cn/pholcus/common/session
Package session provider
|
Package session provider |
|
github.com/henrylee2cn/pholcus/common/websocket
Package websocket implements a client and server for the WebSocket protocol as specified in RFC 6455.
|
Package websocket implements a client and server for the WebSocket protocol as specified in RFC 6455. |
|
github.com/henrylee2cn/pholcus/web
[spider frame (golang)] Pholcus（幽灵蛛）是一款纯Go语言编写的高并发、分布式、重量级爬虫软件，支持单机、服务端、客户端三种运行模式，拥有Web、GUI、命令行三种操作界面；规则简单灵活、批量任务并发、输出方式丰富（mysql/mongodb/csv/excel等）、有大量Demo共享；同时她还支持横纵向两种抓取模式，支持模拟登录和任务暂停、取消等一系列高级功能； （官方QQ群：Go大数据 42731170，欢迎加入我们的讨论）。
|
[spider frame (golang)] Pholcus（幽灵蛛）是一款纯Go语言编写的高并发、分布式、重量级爬虫软件，支持单机、服务端、客户端三种运行模式，拥有Web、GUI、命令行三种操作界面；规则简单灵活、批量任务并发、输出方式丰富（mysql/mongodb/csv/excel等）、有大量Demo共享；同时她还支持横纵向两种抓取模式，支持模拟登录和任务暂停、取消等一系列高级功能； （官方QQ群：Go大数据 42731170，欢迎加入我们的讨论）。 |
|
github.com/josharian/impl
impl generates method stubs for implementing an interface.
|
impl generates method stubs for implementing an interface. |
|
github.com/rogpeppe/godef
Godef prints the source location of definitions in Go programs.
|
Godef prints the source location of definitions in Go programs. |
|
github.com/rogpeppe/godef/go/ast
Package ast declares the types used to represent syntax trees for Go packages.
|
Package ast declares the types used to represent syntax trees for Go packages. |
|
github.com/rogpeppe/godef/go/parser
A parser for Go source files.
|
A parser for Go source files. |
|
github.com/rogpeppe/godef/go/printer
Package printer implements printing of AST nodes.
|
Package printer implements printing of AST nodes. |
|
github.com/rogpeppe/godef/go/scanner
Package scanner implements a scanner for Go source text.
|
Package scanner implements a scanner for Go source text. |
|
github.com/rogpeppe/godef/go/sym
The sym package provides a way to iterate over and change the symbols in Go source files.
|
The sym package provides a way to iterate over and change the symbols in Go source files. |
|
github.com/rogpeppe/godef/go/token
This package defines constants representing the lexical tokens of the Go programming language and basic operations on tokens (printing, predicates).
|
This package defines constants representing the lexical tokens of the Go programming language and basic operations on tokens (printing, predicates). |
|
github.com/rogpeppe/godef/go/types
Types infers source locations and types from Go expressions.
|
Types infers source locations and types from Go expressions. |
|
github.com/sqs/goreturns/returns
Package returns implements a Go pretty-printer (like package "go/format") that also adds zero-value return values as necessary to incomplete return statements.
|
Package returns implements a Go pretty-printer (like package "go/format") that also adds zero-value return values as necessary to incomplete return statements. |
|
github.com/tpng/gopkgs
Command gopkgs list your installed Go packages for import.
|
Command gopkgs list your installed Go packages for import. |
|
golang.org/x/tools/benchmark/parse
Package parse provides support for parsing benchmark results as generated by 'go test -bench'.
|
Package parse provides support for parsing benchmark results as generated by 'go test -bench'. |
|
golang.org/x/tools/blog
Package blog implements a web server for articles written in present format.
|
Package blog implements a web server for articles written in present format. |
|
golang.org/x/tools/blog/atom
Package atom defines XML data structures for an Atom feed.
|
Package atom defines XML data structures for an Atom feed. |
|
golang.org/x/tools/cmd/benchcmp
The benchcmp command displays performance changes between benchmarks.
|
The benchcmp command displays performance changes between benchmarks. |
|
golang.org/x/tools/cmd/bundle
Bundle creates a single-source-file version of a source package suitable for inclusion in a particular target package.
|
Bundle creates a single-source-file version of a source package suitable for inclusion in a particular target package. |
|
golang.org/x/tools/cmd/callgraph
callgraph: a tool for reporting the call graph of a Go program.
|
callgraph: a tool for reporting the call graph of a Go program. |
|
golang.org/x/tools/cmd/compilebench
Compilebench benchmarks the speed of the Go compiler.
|
Compilebench benchmarks the speed of the Go compiler. |
|
golang.org/x/tools/cmd/cover
Cover is a program for analyzing the coverage profiles generated by 'go test -coverprofile=cover.out'.
|
Cover is a program for analyzing the coverage profiles generated by 'go test -coverprofile=cover.out'. |
|
golang.org/x/tools/cmd/digraph
The digraph command performs queries over unlabelled directed graphs represented in text form.
|
The digraph command performs queries over unlabelled directed graphs represented in text form. |
|
golang.org/x/tools/cmd/eg
The eg command performs example-based refactoring.
|
The eg command performs example-based refactoring. |
|
golang.org/x/tools/cmd/fiximports
The fiximports command fixes import declarations to use the canonical import path for packages that have an "import comment" as defined by https://golang.org/s/go14customimport.
|
The fiximports command fixes import declarations to use the canonical import path for packages that have an "import comment" as defined by https://golang.org/s/go14customimport. |
|
golang.org/x/tools/cmd/go-contrib-init
The go-contrib-init command helps new Go contributors get their development environment set up for the Go contribution process.
|
The go-contrib-init command helps new Go contributors get their development environment set up for the Go contribution process. |
|
golang.org/x/tools/cmd/godex
The godex command prints (dumps) exported information of packages or selected package objects.
|
The godex command prints (dumps) exported information of packages or selected package objects. |
|
golang.org/x/tools/cmd/godoc
Godoc extracts and generates documentation for Go programs.
|
Godoc extracts and generates documentation for Go programs. |
|
golang.org/x/tools/cmd/goimports
Command goimports updates your Go import lines, adding missing ones and removing unreferenced ones.
|
Command goimports updates your Go import lines, adding missing ones and removing unreferenced ones. |
|
golang.org/x/tools/cmd/gomvpkg
The gomvpkg command moves go packages, updating import declarations.
|
The gomvpkg command moves go packages, updating import declarations. |
|
golang.org/x/tools/cmd/gorename
The gorename command performs precise type-safe renaming of identifiers in Go source code.
|
The gorename command performs precise type-safe renaming of identifiers in Go source code. |
|
golang.org/x/tools/cmd/gotype
The gotype command, like the front-end of a Go compiler, parses and type-checks a single Go package.
|
The gotype command, like the front-end of a Go compiler, parses and type-checks a single Go package. |
|
golang.org/x/tools/cmd/goyacc
Goyacc is a version of yacc for Go.
|
Goyacc is a version of yacc for Go. |
|
golang.org/x/tools/cmd/guru
guru: a tool for answering questions about Go source code.
|
guru: a tool for answering questions about Go source code. |
|
golang.org/x/tools/cmd/guru/serial
Package serial defines the guru's schema for -json output.
|
Package serial defines the guru's schema for -json output. |
|
golang.org/x/tools/cmd/heapview
heapview is a tool for viewing Go heap dumps.
|
heapview is a tool for viewing Go heap dumps. |
|
golang.org/x/tools/cmd/heapview/internal/core
Package core provides functions for reading core dumps and examining their contained heaps.
|
Package core provides functions for reading core dumps and examining their contained heaps. |
|
golang.org/x/tools/cmd/html2article
This program takes an HTML file and outputs a corresponding article file in present format.
|
This program takes an HTML file and outputs a corresponding article file in present format. |
|
golang.org/x/tools/cmd/present
Present displays slide presentations and articles.
|
Present displays slide presentations and articles. |
|
golang.org/x/tools/cmd/ssadump
ssadump: a tool for displaying and interpreting the SSA form of Go programs.
|
ssadump: a tool for displaying and interpreting the SSA form of Go programs. |
|
golang.org/x/tools/cmd/stress
The stress utility is intended for catching sporadic failures.
|
The stress utility is intended for catching sporadic failures. |
|
golang.org/x/tools/cmd/stringer
Stringer is a tool to automate the creation of methods that satisfy the fmt.Stringer interface.
|
Stringer is a tool to automate the creation of methods that satisfy the fmt.Stringer interface. |
|
golang.org/x/tools/cmd/tip
Command tip is the tip.golang.org server, serving the latest HEAD straight from the Git oven.
|
Command tip is the tip.golang.org server, serving the latest HEAD straight from the Git oven. |
|
golang.org/x/tools/cmd/toolstash
Toolstash provides a way to save, run, and restore a known good copy of the Go toolchain and to compare the object files generated by two toolchains.
|
Toolstash provides a way to save, run, and restore a known good copy of the Go toolchain and to compare the object files generated by two toolchains. |
|
golang.org/x/tools/container/intsets
Package intsets provides Sparse, a compact and fast representation for sparse sets of int values.
|
Package intsets provides Sparse, a compact and fast representation for sparse sets of int values. |
|
golang.org/x/tools/cover
Package cover provides support for parsing coverage profiles generated by "go test -coverprofile=cover.out".
|
Package cover provides support for parsing coverage profiles generated by "go test -coverprofile=cover.out". |
|
golang.org/x/tools/go/ast/astutil
Package astutil contains common utilities for working with the Go AST.
|
Package astutil contains common utilities for working with the Go AST. |
|
golang.org/x/tools/go/buildutil
Package buildutil provides utilities related to the go/build package in the standard library.
|
Package buildutil provides utilities related to the go/build package in the standard library. |
|
golang.org/x/tools/go/callgraph
Package callgraph defines the call graph and various algorithms and utilities to operate on it.
|
Package callgraph defines the call graph and various algorithms and utilities to operate on it. |
|
golang.org/x/tools/go/callgraph/cha
Package cha computes the call graph of a Go program using the Class Hierarchy Analysis (CHA) algorithm.
|
Package cha computes the call graph of a Go program using the Class Hierarchy Analysis (CHA) algorithm. |
|
golang.org/x/tools/go/callgraph/rta
This package provides Rapid Type Analysis (RTA) for Go, a fast algorithm for call graph construction and discovery of reachable code (and hence dead code) and runtime types.
|
This package provides Rapid Type Analysis (RTA) for Go, a fast algorithm for call graph construction and discovery of reachable code (and hence dead code) and runtime types. |
|
golang.org/x/tools/go/callgraph/static
Package static computes the call graph of a Go program containing only static call edges.
|
Package static computes the call graph of a Go program containing only static call edges. |
|
golang.org/x/tools/go/gccgoexportdata
Package gccgoexportdata provides functions for reading export data files containing type information produced by the gccgo compiler.
|
Package gccgoexportdata provides functions for reading export data files containing type information produced by the gccgo compiler. |
|
golang.org/x/tools/go/gcexportdata
Package gcexportdata provides functions for locating, reading, and writing export data files containing type information produced by the gc compiler.
|
Package gcexportdata provides functions for locating, reading, and writing export data files containing type information produced by the gc compiler. |
|
golang.org/x/tools/go/gcimporter15
Package gcimporter15 provides various functions for reading gc-generated object files that can be used to implement the Importer interface defined by the Go 1.5 standard library package.
|
Package gcimporter15 provides various functions for reading gc-generated object files that can be used to implement the Importer interface defined by the Go 1.5 standard library package. |
|
golang.org/x/tools/go/internal/gccgoimporter
Package gccgoimporter implements Import for gccgo-generated object files.
|
Package gccgoimporter implements Import for gccgo-generated object files. |
|
golang.org/x/tools/go/loader
Package loader loads a complete Go program from source code, parsing and type-checking the initial packages plus their transitive closure of dependencies.
|
Package loader loads a complete Go program from source code, parsing and type-checking the initial packages plus their transitive closure of dependencies. |
|
golang.org/x/tools/go/pointer
Package pointer implements Andersen's analysis, an inclusion-based pointer analysis algorithm first described in (Andersen, 1994).
|
Package pointer implements Andersen's analysis, an inclusion-based pointer analysis algorithm first described in (Andersen, 1994). |
|
golang.org/x/tools/go/ssa
Package ssa defines a representation of the elements of Go programs (packages, types, functions, variables and constants) using a static single-assignment (SSA) form intermediate representation (IR) for the bodies of functions.
|
Package ssa defines a representation of the elements of Go programs (packages, types, functions, variables and constants) using a static single-assignment (SSA) form intermediate representation (IR) for the bodies of functions. |
|
golang.org/x/tools/go/ssa/interp
Package ssa/interp defines an interpreter for the SSA representation of Go programs.
|
Package ssa/interp defines an interpreter for the SSA representation of Go programs. |
|
golang.org/x/tools/go/types/typeutil
Package typeutil defines various utilities for types, such as Map, a mapping from types.Type to interface{} values.
|
Package typeutil defines various utilities for types, such as Map, a mapping from types.Type to interface{} values. |
|
golang.org/x/tools/godoc
Package godoc is a work-in-progress (2013-07-17) package to begin splitting up the godoc binary into multiple pieces.
|
Package godoc is a work-in-progress (2013-07-17) package to begin splitting up the godoc binary into multiple pieces. |
|
golang.org/x/tools/godoc/analysis
Package analysis performs type and pointer analysis and generates mark-up for the Go source view.
|
Package analysis performs type and pointer analysis and generates mark-up for the Go source view. |
|
golang.org/x/tools/godoc/redirect
Package redirect provides hooks to register HTTP handlers that redirect old godoc paths to their new equivalents and assist in accessing the issue tracker, wiki, code review system, etc.
|
Package redirect provides hooks to register HTTP handlers that redirect old godoc paths to their new equivalents and assist in accessing the issue tracker, wiki, code review system, etc. |
|
golang.org/x/tools/godoc/static
Package static exports a map of static file content that supports the godoc user interface.
|
Package static exports a map of static file content that supports the godoc user interface. |
|
golang.org/x/tools/godoc/util
Package util contains utility types and functions for godoc.
|
Package util contains utility types and functions for godoc. |
|
golang.org/x/tools/godoc/vfs
Package vfs defines types for abstract file system access and provides an implementation accessing the file system of the underlying OS.
|
Package vfs defines types for abstract file system access and provides an implementation accessing the file system of the underlying OS. |
|
golang.org/x/tools/godoc/vfs/gatefs
Package gatefs provides an implementation of the FileSystem interface that wraps another FileSystem and limits its concurrency.
|
Package gatefs provides an implementation of the FileSystem interface that wraps another FileSystem and limits its concurrency. |
|
golang.org/x/tools/godoc/vfs/httpfs
Package httpfs implements http.FileSystem using a godoc vfs.FileSystem.
|
Package httpfs implements http.FileSystem using a godoc vfs.FileSystem. |
|
golang.org/x/tools/godoc/vfs/mapfs
Package mapfs file provides an implementation of the FileSystem interface based on the contents of a map[string]string.
|
Package mapfs file provides an implementation of the FileSystem interface based on the contents of a map[string]string. |
|
golang.org/x/tools/godoc/vfs/zipfs
Package zipfs file provides an implementation of the FileSystem interface based on the contents of a .zip file.
|
Package zipfs file provides an implementation of the FileSystem interface based on the contents of a .zip file. |
|
golang.org/x/tools/imports
Package imports implements a Go pretty-printer (like package "go/format") that also adds or removes import statements as necessary.
|
Package imports implements a Go pretty-printer (like package "go/format") that also adds or removes import statements as necessary. |
|
golang.org/x/tools/playground
Package playground registers HTTP handlers at "/compile" and "/share" that proxy requests to the golang.org playground service.
|
Package playground registers HTTP handlers at "/compile" and "/share" that proxy requests to the golang.org playground service. |
|
golang.org/x/tools/playground/socket
Package socket implements an WebSocket-based playground backend.
|
Package socket implements an WebSocket-based playground backend. |
|
golang.org/x/tools/present
The present file format
|
The present file format |
|
golang.org/x/tools/refactor/eg
Package eg implements the example-based refactoring tool whose command-line is defined in golang.org/x/tools/cmd/eg.
|
Package eg implements the example-based refactoring tool whose command-line is defined in golang.org/x/tools/cmd/eg. |
|
golang.org/x/tools/refactor/importgraph
Package importgraph computes the forward and reverse import dependency graphs for all packages in a Go workspace.
|
Package importgraph computes the forward and reverse import dependency graphs for all packages in a Go workspace. |
|
golang.org/x/tools/refactor/rename
Package rename contains the implementation of the 'gorename' command whose main function is in golang.org/x/tools/cmd/gorename.
|
Package rename contains the implementation of the 'gorename' command whose main function is in golang.org/x/tools/cmd/gorename. |
|
golang.org/x/tools/refactor/satisfy
Package satisfy inspects the type-checked ASTs of Go packages and reports the set of discovered type constraints of the form (lhs, rhs Type) where lhs is a non-trivial interface, rhs satisfies this interface, and this fact is necessary for the package to be well-typed.
|
Package satisfy inspects the type-checked ASTs of Go packages and reports the set of discovered type constraints of the form (lhs, rhs Type) where lhs is a non-trivial interface, rhs satisfies this interface, and this fact is necessary for the package to be well-typed. |
|
sourcegraph.com/sqs/goreturns/returns
Package returns implements a Go pretty-printer (like package "go/format") that also adds zero-value return values as necessary to incomplete return statements.
|
Package returns implements a Go pretty-printer (like package "go/format") that also adds zero-value return values as necessary to incomplete return statements. |
Click to show internal directories.
Click to hide internal directories.