sxkcd
S(earch)xkcd is a xkcd search engine that supports full-text
search and an extensive query syntax.
Try it out here!
usage: sxkcd [server|download] [OPTIONS] [FILE]
Options:
-v, --version Version info
-h, --help Show help
server:
-f, --file Read data from file
-p, --port Server port
-r, --redis Redis connection URI [host:port]
-i, --reindex Reindex existing data with new file
download:
-n, --num Download single comic by number
-f, --file Download all comics to file
Start your own instance of sxkcd
with the provided docker-compose.yml
:
# download all comic data
$ docker compose run \
--rm \
--no-deps \
app download -f /data/comics.json
$ docker-compose -d
Querying
sxkcd
supports union, negation, prefix matching and filtering by custom date ranges
# foo OR bar
> foo|bar
# exclude foo
> -foo
# prefix matching
> foo*
# filter by comic number
> #420
# comic number range
> #420-425
# filter by dates in ISO-8601 format
> @date: 2022-01-01, 2022-02-01
# from date to present
> @date: 2022-08-01
How it Works
sxkcd
is a webserver built with Go and Svelte. It
downloads all data from xkcd and explainxkcd, before
cleaning and combining it into a single JSON file. When sxkcd
is started, it
indexes all JSON data into Redis Stack as JSON
documents. Redis
Stack offers native support for indexing and querying JSON documents, providing
full-text search and an extensive query
syntax for all JSON
data and its sub-elements.
Every day, sxkcd
automatically updates the database with the newest comic, if it
doesn't already exist.
Install
Run with docker-compose:
# download all comic data
$ docker compose run \
--rm \
--no-deps \
app download -f /data/comics.json
$ docker-compose -d
Run locally:
# download all comic data
$ sxkcd download -f data/comics.json
# start a redis instance with data persistence
$ redis-server --appendonly yes
$ sxkcd server -p 6380 -r localhost:6379 -f data/comics.json
If Redis is started with persistence, sxkcd
can be restarted without any data
files. If we wish to reindex all data in the database with a new file, we can
run sxkcd
with the --reindex
flag:
$ sxkcd server -p 6380 -r localhost:6379 -f data/new.json --reindex
This will replace all existing data with that in the new file.
Development
sxkcd
is built with
Build from Source
Clone the repository:
$ git clone https://github.com/kencx/sxkcd.git
The frontend's static files are embedded in the Go binary. They must generated
prior to building the Go binary:
$ cd ui && npm ci --quiet
# Build static files in ui/build
$ cd ui && npm run build
# Build binary
$ go mod download
$ make build
Build Docker Image
You can also choose to build a local docker image:
$ make build
Local Development
# Start frontend
$ cd ui
$ npm install
$ npm run dev
# Start redis docker container
$ docker-compose up -d redis
# Start backend
$ go mod download
$ make build
$ ./sxkcd server --port 6380 --redis localhost:6379 --file data/comics.json
Visit localhost:5173
.
Acknowledgements
sxkcd is entirely inspired by classes.wtf
MIT