util

package
v1.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 17, 2023 License: BSD-3-Clause Imports: 51 Imported by: 0

Documentation

Index

Constants

View Source 
const GB = 1024 * 1024 * 1024
View Source 
const Version = `2.7.9`

Variables

View Source 
var (
	UrlPrecise      = "UrlPrecise"
	CacheName       = "CacheName"
	EnableSubfinder = "EnableSubfinder"
)
View Source 
var (
	HttpProxy   string // 代理
	CeyeApi     string // Ceye api
	CeyeDomain  string // Ceye domain
	Fuzzthreads = 32   // 2,4,8,16,32,采用2的N次方的数字
)
View Source 
var Abs404 = "/goscan404"
View Source 
var BaseReg = regexp.MustCompile("(?i)Basic\\s*realm\\s*=\\s*")

匹配响应中 www-Authenticate 是否有认证要求都信息

View Source 
var Config4scanAll = Config4scanAllModel{}
View Source 
var Ctx_global, StopAll = context.WithCancel(RootContext)

全局关闭所有线程

View Source 
var CustomHeaders []string

自定义http 头

View Source 
var DbName = "config/goscan_db"
View Source 
var DeleteMe = regexp.MustCompile("rememberMe=deleteMe")

多次使用，一次性编译效率更高

View Source 
var EnableHoneyportDetection = true
View Source 
var EsUrl string
View Source 
var G_Options interface{}
View Source 
var InterruptTimeout = 200 * time.Millisecond

InterruptTimeout timeout for interrupt signal when exiting a Cmd

View Source 
var KillTimeout = 1000 * time.Millisecond

KillTimeout timeout for kill signal when exiting a Cmd

View Source 
var MyName = "goscan"
View Source 
var NoColor bool
View Source 
var Output = ""

out filename

View Source 
var PocCheck_pipe = make(chan *PocCheck, 64)

go POC 检测管道，避免循环引用

View Source 
var RootContext = context.Background()

全局控制

View Source 
var SupplyChainReg *regexp.Regexp

提取供应链信息

View Source 
var SzPwd string
View Source 
var TmpFile = map[string][]*os.File{}
View Source 
var UrlMt []*regexp.Regexp = []*regexp.Regexp{
	regexp.MustCompile("^http[s]:\\/\\/[^\\/]+\\/?$"),
	regexp.MustCompile("^http[s]:\\/\\/[^\\/]+\\/[^\\/]+$")}
View Source 
var UserHomeDir string = "./"
View Source 
var Wg *sync.WaitGroup = &sync.WaitGroup{}

全局线程控制

Functions

func AsynCmd 

func AsynCmd(fnCbk func(line string), szCmd string, args ...string) error

异步执行命令

func CheckHeader 

func CheckHeader(header *http.Header, szUrl string)

头信息同一检查，并调用合适到go poc进一步爆破、检测 

1、需要认证
2、shiro

func CheckHoneyportDetection4HeaderServer 

func CheckHoneyportDetection4HeaderServer(server, szUrl string) bool

检查 蜜罐 Server信息，check Honeypor server info

func CheckHvNmap 

func CheckHvNmap() bool

func CheckShiroCookie 

func CheckShiroCookie(header *http.Header) int

检查 cookie Shiro CVE_2016_4437 cookie 其他POC cookie同一检查入口

func Close 

func Close()

关闭数据库连接

func CloseAll 

func CloseAll()

关闭所有资源

func CloseAllHttpClient 

func CloseAllHttpClient()

func CloseCache 

func CloseCache()

关闭cache

func CloseHttpClient 

func CloseHttpClient(szUrl string)

func Convert2Domains 

func Convert2Domains(x string) []string

兼容hacker one 域名表示方式,以下格式支持 *.xxx.com *.xxx.xx1.*

func Create 

func Create[T any](mod *T) int64

通用,insert

func Dnslogchek 

func Dnslogchek(randomstr string) bool

func DoBody 

func DoBody(szUrl, szBody string, head *http.Header)

body中开发商信息提取

func DoCmd 

func DoCmd(args ...string) (string, error)

最佳的方法是将命令写到临时文件，并通过bash进行执行

func DoInit 

func DoInit(config *embed.FS)

func DoLog4j 

func DoLog4j(szUrl string)

1、检测 $HOME/MyWork/log4j-scan 存在就执行 python3 版本log4j检测 2、相同目标只执行一次，基于内存缓存 3、只支持：https://github.com/hktalent/log4j-scan 版本

func DoSyncFunc 

func DoSyncFunc(cbk func())

异步执行方法，只适合无返回值、或使用管道返回值的方法 程序main整体等待

func FileExists 

func FileExists(s string) bool

判断文件是否存在

func GetAny 

func GetAny[T any](key string) (T, error)

func GetAsAny 

func GetAsAny(key string) interface{}

获取interface

func GetClient 

func GetClient(szUrl string) *http.Client

func GetClient4Cc 

func GetClient4Cc(szUrl string) *http.Client

func GetCount 

func GetCount[T any](mod T, args ...interface{}) int64

通用 求T类型count，支持条件 对T表，mod类型表，args 的where求count

func GetCustomHeadersRaw 

func GetCustomHeadersRaw() string

X-Forwarded-Host: 127.0.0.1 X-Forwarded-For: 127.0.0.1 X-Originating-IP: 127.0.0.1 X-Remote-IP: 127.0.0.1 X-Remote-Addr: 127.0.0.1 X-Client-IP: 127.0.0.1 X-Host: 127.0.0.1

获取 自定义头信息等raw模式

func GetHostInfo 

func GetHostInfo() (result *host.InfoStat, err error)

2.主机信息

func GetJson4Query 

func GetJson4Query(source interface{}, path string) interface{}

func GetLocalIP 

func GetLocalIP() (ip string)

1.主机IP

func GetMemInfo 

func GetMemInfo() (float64, []map[string]interface{})

5.内存信息

func GetNetInfo 

func GetNetInfo() (result []net.IOCountersStat, err error)

6.获取网卡信息

func GetNetSpeed 

func GetNetSpeed() (speed map[string]map[string]uint64, err error)

7.计算上下行带宽

func GetNmap 

func GetNmap() string

func GetOne 

func GetOne[T any](rst *T, args ...interface{}) *T

通用 查询返回T类型、表一条数据

func GetPointVal 

func GetPointVal(i interface{}) interface{}

func GetSha1 

func GetSha1(a ...interface{}) string

获取 Sha1

func GetSubQueryList 

func GetSubQueryList[T1, T2, T3 any](mode T1, preLd T3, aRst []T2, nPageSize int, Offset int, conds ...interface{}) *[]T2

通用 查询模型T1类型 mode，并关联T1类型对子类型T3 preLd 设置 nPageSize 和便宜Offset 以及其他查询条件conds

func GetSubQueryLists 

func GetSubQueryLists[T1, T2 any](mode T1, preLd string, aRst []T2, nPageSize int, Offset int, conds ...interface{}) *[]T2

通用 查询模型T1类型 mode，并关联T1类型对子类型T3 preLd 设置 nPageSize 和便宜Offset 以及其他查询条件conds

func GetTableName 

func GetTableName[T any](mod T) string

通用 获取T类型mod表名

func GetTempFile 

func GetTempFile(t string) *os.File

临时结果文件，例如 nmap

func GetVal 

func GetVal(key string) string

优先使用配置文件中的配置，否则从环境变量中读取

func GetVal4Any 

func GetVal4Any[T any](key string) T

其他类型

func GetVal4File 

func GetVal4File(key, szDefault string) string

读区配置中的字典文件

func GetVal4Filedefault 

func GetVal4Filedefault(key, szDefault string) string

读区配置中的字典文件

func GetValAsBool 

func GetValAsBool(key string) bool

获取配置为bool

func GetValAsInt 

func GetValAsInt(key string, nDefault int) int

获取配置为int

func GetValByDefault 

func GetValByDefault(key, dftvl string) string

func HoneyportDetection 

func HoneyportDetection(host string) bool

添加蜜罐检测，并自动跳过目标，默认false跳过蜜罐检测 考虑内存缓存结果

func Init1 

func Init1(config *embed.FS)

初始化到开头

func Init2 

func Init2()

初始化配置文件信息，这个必须先执行

func InitCHcc 

func InitCHcc()

func InitDb 

func InitDb(dst ...interface{}) *gorm.DB

go - 交叉编译go-sqlite3 https://www.modb.pro/db/329524 ./tools/Check_CVE_2020_26134 -config="/Users/51pwn/MyWork/mybugbounty/allDomains.txt" 获取Gorm db连接、操作对象

func InitModle 

func InitModle(x ...interface{})

初始化模型

func IntInSlice 

func IntInSlice(i int, slice []int) bool

判断 i 是否存在slice中

func IsPointed 

func IsPointed(i interface{}) bool

func IsStruct 

func IsStruct(i interface{}) bool

判断对象是否为struct

func Log 

func Log(v ...any)

func MergeParms2Obj 

func MergeParms2Obj(obj interface{}, args ...interface{}) interface{}

若干参数依赖注入到对象 obj中 

util.MergeParms2Obj(&ms, args...)

func Mkdirs 

func Mkdirs(s string)

func ParseOption 

func ParseOption[T any](key string, opt *T) *T

从配置json中读取naabu、httpx、nuclei等的细化配置

func PutAny 

func PutAny[T any](key string, data T)

func RandStringRunes 

func RandStringRunes(n int) string

func RandomStr 

func RandomStr() string

func ReadCsv 

func ReadCsv(filename string, data CSVReader) error

下一步计划：加载osvdb 并驱动执行

func RegInitFunc 

func RegInitFunc(cbk func())

func RemoveDuplication_map 

func RemoveDuplication_map(arr []string) []string

func SendAData 

func SendAData[T any](k string, data []T, szType ESaveType)

k is id

func SendAnyData 

func SendAnyData(data interface{}, szType ESaveType)

一定得有全局得线程等待

func SendLog 

func SendLog(szUrl, szVulType, Msg, Payload string)

1、优化代码，统一结果输出，便于维护

func SendReq 

func SendReq(data1 interface{}, id string, szType ESaveType)

发送数据到ES

func SetHeader 

func SetHeader(m *http.Header)

func SliceInAny 

func SliceInAny[T any](i T, slice []T) bool

判断 i 是否存在slice中

func SliceInString 

func SliceInString(str string, slice []string) bool

func SliceRemoveDuplicates 

func SliceRemoveDuplicates(slice []string) []string

数组去重

func StrContains 

func StrContains(s1, s2 string) bool

字符串包含关系，且大小写不敏感

func StringInSlice 

func StringInSlice(str string, slice []string) bool

判断 str 是否存在slice中

func SupplyChain 

func SupplyChain(szUrl, szBody string, head *http.Header)

提取供应链信息 相同上下文、成功时只提取一次 提取header信息：server、X*，不同上下文提取

func TestIs404Page 

func TestIs404Page(szUrl string) (page *Page, r01 *Response, err error, ok bool)

func TestRepeat 

func TestRepeat(a ...interface{}) bool

func TestRepeat4Save 

func TestRepeat4Save(key string, a ...interface{}) (interface{}, bool)

func UpInsert 

func UpInsert[T any](mod *T, query string, args ...interface{}) int64

更新失败再插入新数据，确保只有一条数据

func Update 

func Update[T any](mod *T, query string, args ...interface{}) int64

通用,update 指定id更新T类型mod数据

func UpdategoscanVersionToLatest 

func UpdategoscanVersionToLatest(verbose bool) error

更新到最新版本

Types

type CSVReader 

type CSVReader interface {
	ReadLine(line []string)
}

type Cmd 

type Cmd struct {
	*exec.Cmd
	// contains filtered or unexported fields
}

func (*Cmd) AsynCmd 

func (r *Cmd) AsynCmd(fnCbk func(line string), szCmd string, args ...string) error

func (*Cmd) Command 

func (cmd *Cmd) Command(name string, arg ...string) *Cmd

func (*Cmd) Exit 

func (cmd *Cmd) Exit() error

func (*Cmd) Interrupt 

func (cmd *Cmd) Interrupt()

Interrupt sends an os.Interrupt to the process if running

func (*Cmd) Start 

func (cmd *Cmd) Start() error

func (*Cmd) WriteInput 

func (r *Cmd) WriteInput(args ...string)

write input, for interactive

func (*Cmd) WriteInput4Cbk 

func (r *Cmd) WriteInput4Cbk(fnCbk func() *string)

基于回调获取输入

type Config4scanAllModel 

type Config4scanAllModel struct {
	EsUlr           string `json:"EsUlr"`
	EnableSubfinder string `json:"EnableSubfinder"`
	UrlPrecise      string `json:"UrlPrecise"`
}

type CpuInfo 

type CpuInfo struct {
	CpuAvg float64     `json:"cpuAvg"`
	CpuAll []CpuSingle `json:"cpuAll"`
}

func GetCpuPercent 

func GetCpuPercent() (result CpuInfo, err error)

4.CPU使用率

type CpuSingle 

type CpuSingle struct {
	Num     string `json:"num"`
	Percent int    `json:"percent"`
}

CPU

type ESaveType 

type ESaveType string
const (
	Naabu     ESaveType = "naabu"
	Httpx     ESaveType = "httpx"
	Hydra     ESaveType = "hydra"
	Nmap      ESaveType = "nmap"
	Goscan    ESaveType = "goscan"
	Subfinder ESaveType = "subfinder"
)

type KvDbOp 

type KvDbOp struct {
	DbConn *badger.DB
}

https://colobu.com/2017/10/11/badger-a-performant-k-v-store/ https://juejin.cn/post/6844903814571491335 

var Cache1 *KvDbOp

func NewKvDbOp 

func NewKvDbOp() *KvDbOp

func (*KvDbOp) Close 

func (r *KvDbOp) Close()

func (*KvDbOp) Delete 

func (r *KvDbOp) Delete(key string) error

func (*KvDbOp) Get 

func (r *KvDbOp) Get(key string) (szRst []byte, err error)

https://www.modb.pro/db/87317

func (*KvDbOp) GetKeyForData 

func (r *KvDbOp) GetKeyForData(key string) (szRst []byte)

https://www.modb.pro/db/87317

func (*KvDbOp) Init 

func (r *KvDbOp) Init(szDb string) error

func (*KvDbOp) Put 

func (r *KvDbOp) Put(key string, data []byte)

func (*KvDbOp) SetExpiresAt 

func (r *KvDbOp) SetExpiresAt(ExpiresAt uint64)

type Page 

type Page struct {
	IsBackUpPath bool         // 备份、敏感泄露文件检测请求url
	IsBackUpPage bool         // 发现备份、敏感泄露文件
	Title        *string      // 标题
	LocationUrl  *string      // 跳转页面
	Is302        bool         // 是302页面
	Is403        bool         // 403页面
	Url          *string      // 作为本地永久缓存key，提高执行效率
	BodyStr      *string      // body = trim() + ToLower
	BodyLen      int          // body 长度
	Header       *http.Header // 基于指针，节约内存空间
	StatusCode   int          // 状态码
	Resqonse     *Response    // 基于指针，节约内存空间
}

fuzz请求返回的结果 尽可能使用指针，节约内存开销

type Part 

type Part struct {
	Path        string  `json:"path"`
	FsType      string  `json:"fstype"`
	Total       float64 `json:"total"`
	Free        float64 `json:"free"`
	Used        float64 `json:"used"`
	UsedPercent int     `json:"usedPercent"`
}

分区

type Parts 

type Parts []Part

分区集合

func GetDiskInfo 

func GetDiskInfo() (result Parts, err error)

3.磁盘信息

type PocCheck 

type PocCheck struct {
	Wappalyzertechnologies *[]string
	URL                    string
	FinalURL               string
	Checklog4j             bool
}

管道通讯使用

type Response 

type Response struct {
	Status        string
	StatusCode    int
	Body          string
	Header        *http.Header // 不用负责对象，引用，节约内存开销
	ContentLength int
	RequestUrl    string
	Location      string
}

fuzz 响应对象封装

func GetResponse 

func GetResponse(username string, password string, urlstring string, method string, postdata string, isredirect bool, headers map[string]string) (resp1 *Response, reqbody, location string, err error)

func HttpRequset 

func HttpRequset(urlstring string, method string, postdata string, isredirect bool, headers map[string]string) (*Response, error)

需要考虑缓存 

1、缓解网络不好的情况
2、缓存有效期为当天
3、缓存命中需和请求的数据完全匹配

func HttpRequsetBasic 

func HttpRequsetBasic(username string, password string, urlstring string, method string, postdata string, isredirect bool, headers map[string]string) (*Response, error)

http密码爆破

func TestIs404 

func TestIs404(szUrl string) (r01 *Response, err error, ok bool)

绝对404检测 相同 url 本实例中只检测一次

type SimpleVulResult 

type SimpleVulResult struct {
	Url     string `json:"url"`
	VulKind string `json:"vulKind"` // 结果分类
	VulType string `json:"vulType"` // 漏洞类型
	Payload string `json:"payload"`
	Msg     string `json:"msg"`
}

简单结果

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.