README
¶
Laika sniffs content for emails, phone numbers and social media urls.
Features
-
Plain text
-
No protection
- Emails
- Phone Numbers
- Social Media URLs
-
HTML Entities
- Emails
- Phone Numbers
- Social Media URLs
-
HTML Comments
- Emails
- Phone Numbers
- Social Media URLs
-
CSS Display none
- Emails
- Phone Numbers
- Social Media URLs
-
JS Concatenation
- ~Emails
- Phone Numbers
- Social Media URLs
-
HTML Symbol substitution
- Emails
- Phone Numbers
- Social Media URLs
-
-
Clickable link
-
No protection
- Emails
- Phone Numbers
- Social Media URLs
-
HTML entities
- ~Emails
- Phone Numbers
- Social Media URLs
-
URL encoding
- Emails
- Phone Numbers
- Social Media URLs
-
Concatenation JS
- Emails
- Phone Numbers
- Social Media URLs
-
CLI
Installation
go install github.com/bastean/laika/cmd/laika@latest
Usage
laika -h
Usage: laika [OPTIONS] sources...
Sniffs the content of the sources
E.g.: laika -jsonStore "laika" -urls -emails http://localhost:8080
-emails
Sniff emails in the content (Required)
-jsonStore string
Store filepath to save the sniffed content (default "In Memory")
-silent
Do not show the sniffed content
-urls
If the sources for sniffing content are urls (Required)
Sniff Test Server
Installation
go install github.com/bastean/laika/cmd/laika-server@latest
Usage
laika-server -h
Usage: laika-server [OPTIONS]
Sniff Test Server
E.g.: laika-server -p 8080
-p int
Port (default 8080)
Package
Installation
go get github.com/bastean/laika
Update
go get -u github.com/bastean/laika
Usage
In Memory Store
package main
import (
"log"
"github.com/bastean/laika"
)
func main() {
// Test Server
sources := []string{"http://localhost:8080"}
// Alias for "nil"
inMemory := laika.NewInMemoryStore()
fromZero := laika.NewEmptyData()
sniff := laika.New(fromZero)
sniff.SetStore(inMemory)
// Sniffs all HTML recursively
sniff.ContentFromUrls(sources)
// Sniffs all emails in the sniffed HTML
sniff.EmailsFromContent()
// Return an array with all emails sniffed from the HTML
emails := sniff.SniffedEmails()
log.Println(emails)
}
Local Json Store
package main
import (
"log"
"github.com/bastean/laika"
)
func main() {
// Test Server
sources := []string{"http://localhost:8080"}
localJson := laika.NewLocalJsonStore(".", "laika")
fromExistingData, err := laika.ReadDataFromStore(localJson)
if err != nil {
fromExistingData = laika.NewEmptyData()
}
sniff := laika.New(fromExistingData)
sniff.SetStore(localJson)
// Sniffs all HTML recursively
sniff.ContentFromUrls(sources)
// Sniffs all emails in the sniffed HTML
sniff.EmailsFromContent()
// Return an array with all emails sniffed from the HTML
emails := sniff.SniffedEmails()
log.Println(emails)
// Saves the sniffed data in the store
sniff.SaveSniffed()
}
-
laika.json
{ "Sniffed": { "localhost:8080": [ { "Source": "/", "Content": "<html>...</html>", "Found": { "Emails": ["email@example.com", "..."] } }, { "Source": "/dashboard", "Content": "<html>...</html>", "Found": { "Emails": ["email@example.com", "..."] } }, { "Source": "/dashboard/admin", "Content": "<html>...</html>", "Found": { "Emails": ["email@example.com", "..."] } } ] } }
Screenshots
Sniff Test Server
Tech Stack
Base
Please see
Contributing
- Contributions and Feedback are always welcome!
License
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewEmptyData ¶
func NewInMemoryStore ¶
func NewInMemoryStore() repository.Repository
func NewLocalJsonStore ¶
func NewLocalJsonStore(path, filename string) repository.Repository
func ReadDataFromStore ¶
func ReadDataFromStore(persistence repository.Repository) (aggregate.Data, error)
Types ¶
type Laika ¶
type Laika struct {
Data aggregate.Data
Store repository.Repository
SaveData *saveData.SaveData
Scraper model.Scraper
SniffFromUrls *sniffFromUrls.SniffFromUrls
}
func (*Laika) SaveSniffed ¶
func (laika *Laika) SaveSniffed()
func (*Laika) SetStore ¶
func (laika *Laika) SetStore(persistence repository.Repository)
func (*Laika) SniffedEmails ¶
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
pkg
|
|
|
cmd/server/components/layout
templ: version: v0.2.598
|
templ: version: v0.2.598 |
|
cmd/server/components/partials
templ: version: v0.2.598
|
templ: version: v0.2.598 |
|
test
|
|
Click to show internal directories.
Click to hide internal directories.