yiigo

README

yiigo 2.0 🎉🎉🎉

简单易用的 Go Web 微框架

特点

• 采用 Glide 管理依赖包
• 采用 toml 配置文件
• 采用 zap 日志记录
• 采用 sqlx 处理SQL查询
• 支持多 MySQL 连接
• 支持多 PostgreSQL 连接
• 支持多 mongo 连接
• 支持多 redis 连接
• 支持 gomail 邮件发送
• 支持 session 存取
• 支持爬虫模拟登录

获取

# Glide (推荐)
glide init
glide get github.com/iiinsomnia/yiigo

# go get
go get github.com/iiinsomnia/yiigo

使用

1、import yiigo

// 自动根据配置初始化组件
import "github.com/iiinsomnia/yiigo"

2、resolve dependencies

# 获取 yiigo 所需依赖包
glide update

文档

• API Reference
• Example

说明

• 在 main.go 所在目录创建 env.toml 配置文件，具体组件配置可以参考 env.toml.example
• 组件在 env.toml 中配置后会自动初始化加载，没有配置的组件不会初始化加载
• yiigo 组件以外的应用配置均可在 env.toml 中配置，使用 yiigo.ENV 的相关方法获取配置值
• MySQL、postgres、mongo、redis 多连接配置参考 env.toml.example 中的多数据库配置部分 (注释部分)
• golang.org 上 go get 不下来的库，可以在这里获取
• 如爬虫不需要模拟登录，则只需要使用 goquery 即可

Enjoy 😊

Documentation

Overview

Package yiigo 简单易用的 Go Web 微框架

获取：(推荐使用Glide)

glide init
glide get github.com/iiinsomnia/yiigo

go get github.com/iiinsomnia/yiigo

使用：(自动根据配置初始化组件)

import "github.com/iiinsomnia/yiigo"

获取相关依赖包：

glide update

For more details, see the documentation for the types and methods.

Index

• Variables
• func DBConn(conn ...string) (*sqlx.DB, error)
• func Date(timestamp int64, format ...string) string
• func Err(c *gin.Context, code int, msg ...string)
• func HTTPGet(url string, headers map[string]string, timeout ...time.Duration) ([]byte, error)
• func HTTPPost(url string, body []byte, headers map[string]string, timeout ...time.Duration) ([]byte, error)
• func IP2long(ip string) int64
• func InSliceFloat64(x float64, a []float64) bool
• func InSliceInt(x int, a []int) bool
• func InSliceInt64(x int64, a []int64) bool
• func InSliceString(x string, a []string) bool
• func InsertSQL(table string, data interface{}) (string, []interface{})
• func IsXhr(c *gin.Context) bool
• func Long2IP(ip int64) string
• func MD5(s string) string
• func MongoSession(conn ...string) (*mgo.Session, error)
• func MyTimeEncoder(t time.Time, enc zapcore.PrimitiveArrayEncoder)
• func OK(c *gin.Context, data ...interface{})
• func PGConn(conn ...string) (*sqlx.DB, error)
• func PGInsertSQL(table string, data interface{}) (string, []interface{})
• func PGUpdateSQL(query string, data interface{}, args ...interface{}) (string, []interface{})
• func RedisPool(conn ...string) (*redis.Pool, error)
• func SearchInt64s(a []int64, x int64) int
• func SeqID(session *mgo.Session, db string, collection string, seqs ...int64) (int64, error)
• func SortInt64s(a []int64)
• func UniqueFloat64(a []float64) []float64
• func UniqueInt(a []int) []int
• func UniqueInt64(a []int64) []int64
• func UniqueString(a []string) []string
• func UpdateSQL(query string, data interface{}, args ...interface{}) (string, []interface{})
• type HTTPSCert
• type Int64Slice
  • func (p Int64Slice) Len() int
  • func (p Int64Slice) Less(i, j int) bool
  • func (p Int64Slice) Swap(i, j int)
• type Mailer
  • func (m *Mailer) Send() error
• type ReqBody
• type Sequence
• type Spider
  • func NewSpider(cookieFile string, cert *HTTPSCert, timeout ...time.Duration) (*Spider, error)
  • func (s *Spider) HTTPGet(reqBody *ReqBody) ([]byte, error)
  • func (s *Spider) HTTPPost(reqBody *ReqBody) ([]byte, error)
• type X

Variables

var (
	// DB default mysql connection
	DB *sqlx.DB
)

var Env *env

Env enviroment

var ErrEnvNil = errors.New("yiigo: env config not found")

ErrEnvNil returned when config not found.

var Logger *zap.Logger

Logger yiigo logger

var (
	// Mongo default mongo session
	Mongo *mgo.Session
)

var (
	// PG default postgres connection
	PG *sqlx.DB
)

var (
	// Redis default redis connection pool
	Redis *redis.Pool
)

var YiiClient = &http.Client{
	Timeout: 5 * time.Second,
}

YiiClient HTTP request client

Functions

func DBConn

func DBConn(conn ...string) (*sqlx.DB, error)

DBConn returns a mysql connection.

func Date

func Date(timestamp int64, format ...string) string

Date format a local time/date and returns a string formatted according to the given format string using the given timestamp of int64. The default format is: 2006-01-02 15:04:05.

func Err

func Err(c *gin.Context, code int, msg ...string)

Err returns error of an API.

func HTTPGet

func HTTPGet(url string, headers map[string]string, timeout ...time.Duration) ([]byte, error)

HTTPGet http get request

func HTTPPost

func HTTPPost(url string, body []byte, headers map[string]string, timeout ...time.Duration) ([]byte, error)

HTTPPost http post request

func IP2long

func IP2long(ip string) int64

IP2long converts a string containing an (IPv4) Internet Protocol dotted address into a long integer

func InSliceFloat64

func InSliceFloat64(x float64, a []float64) bool

InSliceFloat64 checks if x exists in a slice of float64s and returns TRUE if x is found.

func InSliceInt

func InSliceInt(x int, a []int) bool

InSliceInt checks if x exists in a slice of ints and returns TRUE if x is found.

func InSliceInt64

func InSliceInt64(x int64, a []int64) bool

InSliceInt64 checks if x exists in a slice of int64s and returns TRUE if x is found.

func InSliceString

func InSliceString(x string, a []string) bool

InSliceString checks if x exists in a slice of strings and returns TRUE if x is found.

func InsertSQL

func InsertSQL(table string, data interface{}) (string, []interface{})

InsertSQL returns mysql insert sql and binds. param data expects struct, []struct, yiigo.X, []yiigo.X.

func IsXhr

func IsXhr(c *gin.Context) bool

IsXhr checks if a request is xml-http-request (ajax).

func Long2IP

func Long2IP(ip int64) string

Long2IP converts an long integer address into a string in (IPv4) Internet standard dotted format

func MD5

func MD5(s string) string

MD5 calculate the md5 hash of a string.

func MongoSession

func MongoSession(conn ...string) (*mgo.Session, error)

MongoSession returns a mongo session.

func MyTimeEncoder

func MyTimeEncoder(t time.Time, enc zapcore.PrimitiveArrayEncoder)

MyTimeEncoder zap time encoder.

func OK

func OK(c *gin.Context, data ...interface{})

OK returns success of an API.

func PGConn

func PGConn(conn ...string) (*sqlx.DB, error)

PGConn returns a postgres connection.

func PGInsertSQL

func PGInsertSQL(table string, data interface{}) (string, []interface{})

PGInsertSQL returns postgres insert sql and binds. param data expects struct, []struct, yiigo.X, []yiigo.X.

func PGUpdateSQL

func PGUpdateSQL(query string, data interface{}, args ...interface{}) (string, []interface{})

PGUpdateSQL returns postgres update sql and binds. param query expects eg: "UPDATE table SET $1 WHERE id = $2". param data expects struct, yiigo.X.

func RedisPool

func RedisPool(conn ...string) (*redis.Pool, error)

RedisPool returns a redis connection pool.

func SearchInt64s

func SearchInt64s(a []int64, x int64) int

SearchInt64s searches for x in a sorted slice of ints and returns the index as specified by Search. The return value is the index to insert x if x is not present (it could be len(a)). The slice must be sorted in ascending order.

func SeqID

func SeqID(session *mgo.Session, db string, collection string, seqs ...int64) (int64, error)

SeqID returns _id auto_increment to mongo.

func SortInt64s

func SortInt64s(a []int64)

SortInt64s sorts a slice of int64s in increasing order.

func UniqueFloat64

func UniqueFloat64(a []float64) []float64

UniqueFloat64 takes an input slice of float64s and returns a new slice of float64s without duplicate values.

func UniqueInt

func UniqueInt(a []int) []int

UniqueInt takes an input slice of ints and returns a new slice of ints without duplicate values.

func UniqueInt64

func UniqueInt64(a []int64) []int64

UniqueInt64 takes an input slice of int64s and returns a new slice of int64s without duplicate values.

func UniqueString

func UniqueString(a []string) []string

UniqueString takes an input slice of strings and returns a new slice of strings without duplicate values.

func UpdateSQL

func UpdateSQL(query string, data interface{}, args ...interface{}) (string, []interface{})

UpdateSQL returns mysql update sql and binds. param query expects eg: "UPDATE table SET ? WHERE id = ?". param data expects struct, yiigo.X.

Types

type HTTPSCert

type HTTPSCert struct {
	CertPem           string
	KeyUnencryptedPem string
}

HTTPSCert https cert

type Int64Slice

type Int64Slice []int64

Int64Slice attaches the methods of Interface to []int64, sorting a increasing order.

func (Int64Slice) Len

func (p Int64Slice) Len() int

func (Int64Slice) Less

func (p Int64Slice) Less(i, j int) bool

func (Int64Slice) Swap

func (p Int64Slice) Swap(i, j int)

type Mailer

type Mailer struct {
	Title    string
	Subject  string
	From     string
	To       []string
	Cc       []string
	Content  string
	Attach   []string
	Charset  string
	Encoding gomail.Encoding
}

Mailer email

func (*Mailer) Send

func (m *Mailer) Send() error

Send send an email.

type ReqBody

type ReqBody struct {
	// URL 请求地址
	URL string
	// Host 请求头Host
	Host string
	// PostData post参数
	PostData url.Values
	// SetCookie 请求是否需要加cookie
	SetCookie bool
	// SaveCookie 是否保存返回的cookie
	SaveCookie bool
	// ClearOldCookie 是否需要清空原来的cookie
	CleanOldCookie bool
	// Referer 请求头Referer
	Referer string
}

ReqBody http request body

type Sequence

type Sequence struct {
	ID  string `bson:"_id"`
	Seq int64  `bson:"seq"`
}

Sequence model for _id auto_increment of mongo

type Spider

type Spider struct {
	// contains filtered or unexported fields
}

Spider 爬虫基础类 包括：http、https(CA证书)、cookie处理

做爬虫时需用到另外两个库：

1、gbk 转 utf8：gopkg.in/iconv.v1 [https://github.com/qiniu/iconv]
2、页面 dom 处理：github.com/PuerkitoBio/goquery

HTTPS CA证书需用 `openssl` 转化为 `pem` 格式：cert.pem、key.pem

func NewSpider

func NewSpider(cookieFile string, cert *HTTPSCert, timeout ...time.Duration) (*Spider, error)

NewSpider return a new spider

func (*Spider) HTTPGet

func (s *Spider) HTTPGet(reqBody *ReqBody) ([]byte, error)

HTTPGet http get请求

func (*Spider) HTTPPost

func (s *Spider) HTTPPost(reqBody *ReqBody) ([]byte, error)

HTTPPost http post请求

type X

type X map[string]interface{}

X is a convenient alias for a map[string]interface{}.