xlog

README

XLog

Local-first personal knowledge management application with focus on enriching markdown files and surfacing implicit links between pages. Uses Goldmark for rendering markdown and CodeMirror for editing.

Core Features

• Serve any file from current directory. rendering any Markdown file to HTML format.
• Supports Github flavor markdown (GFM)
• One statically compiled binary
• Has a list of tools defined by extensions. triggered with Ctrl+K
• Special style if there is an image at the start of a page to make it look like a cover photo.
• Minimal third-party dependencies.
• The first Emoji used in the page will be considered the icon of the page and displayed beside the title
• Shows task list (Done/Total tasks) beside page link (auto links)

Installation

go install github.com/emad-elsaid/xlog/cmd/xlog@latest

Usage

Usage of xlog:
  -bind string
        IP and port to bind the web server to (default "127.0.0.1:3000")
  -build string
        Build all pages as static site in this directory
  -index string
        Index file name used as home page (default "index")
  -readonly
        Should xlog hide write operations, read-only means all write operations will be disabled
  -sidebar
        Should render sidebar. (default true)
  -sitemap.domain string
        domain name without protocol or trailing / to use for sitemap loc
  -sitename string
        Site name is the name that appears on the header beside the logo and in the title tag (default "XLOG")
  -source string
        Directory that will act as a storage (default current directory)

Now you can access xlog with localhost:3000

Generating static site

I used Xlog to generate my personal blog. it uses github workflow to do that here is an example.

Overriding Assets

assets is served from public directory if it exists in the source directory. otherwise the default assets are served from xlog binary. any file under public that has the same name as the ones xlog uses will be used instead of the default files.

Extensions

Extensions are defined under /extensions sub package. each extension is a subpackage. All extensions are imported by default in cmd/xlog/xlog.go has the side effect of registering the extension hooks. removing the extension from the list of imports will removing the features it provides.

Extension	Description
Autolink	Shorten a link string so it wouldn't take unnecessary space
Autolink pages	Convert a page name mentions in the middle of text to a link
Emoji	Emoji autocomplete while editing
File operations	Add a tool item to delete and rename current page
Hashtags	Add support for hashtags #hashtag syntax
Link preview	Preview tweets, Facebook posts, youtube videos, Giphy links
Opengraph	Adds Opengraph meta tags for title, type, image
Recent	Adds an item to sidebar to list all pages ordered by last modified page file.
Search	Full text search
Shortcode	adds a way for short codes (one line and block)
Star	Star pages to pin them to sidebar
Upload file	Add support for upload files, screenshots, audio and camera recording
Versions	Keeps list of pages older versions
Manifest	adds manifest.json to head tag and output proper JSON value.
Sitemap	adds support for sitemap.xml for search engine crawling

Writing Your Own Extension

• Extensions are Go modules that imports github.com/emad-elsaid/xlog package
• Extension defines init() function which calls xlog public functions to define new features
• To use an extension it needs to be imported to your own copy of cmd/xlog/xlog.go with . github.com/user/extension
• Extensions can define a new feature in View page or Edit page. checkout the public API for full list of available functions.

Core Extensible Features

• Add any HTTP route with your handler function
• Add autocomplete list for the editor
• Add any Goldmark extension for parsing or rendering
• Add a Preprocessor function to process page content before converting to HTML
• Listen to Page events such as Write or Delete.
• Define a helper function for templates
• Add a directory to be parsed as a template
• Add widgets in selected areas of the view page such as before or after rendered HTML
• Add a Tool to the list of tools triggered with Ctrl+K which can execute arbitrary Javascript.
• Add a route to be exported in case of building static site
• Add arbitrary link to pages or any URL

Checkout Go Doc for the full list of Public API

License

Xlog is released under MIT license

Logo

Cassette tape icons created by Good Ware - Flaticon

Screenshots

Documentation

Index

• Constants
• Variables
• func EachPage(ctx context.Context, f func(Page))
• func FindAllInAST[t ast.Node](n ast.Node, kind ast.NodeKind) (a []t)
• func FindInAST[t ast.Node](n ast.Node, kind ast.NodeKind) (found t, ok bool)
• func IgnoreDirectory(r *regexp.Regexp)
• func Listen(e PageEvent, h PageEventHandler)
• func Partial(path string, data Locals) template.HTML
• func RegisterAutocomplete(a Autocomplete)
• func RegisterBuildPage(p string, encloseInDir bool)
• func RegisterCommand(c func(Page) []Command)
• func RegisterHelper(name string, f interface{})
• func RegisterLink(l func(Page) []Link)
• func RegisterPreprocessor(f Preprocessor)
• func RegisterProperty(a func(Page) []Property)
• func RegisterQuickCommand(c func(Page) []Command)
• func RegisterTemplate(t fs.FS, subDir string)
• func RegisterWidget(s widgetSpace, priority float32, f widgetFunc)
• func RenderWidget(s widgetSpace, p Page, r Request) (o template.HTML)
• func Start()
• func Trigger(e PageEvent, p Page)
• func Vars(r Request) map[string]string
• type Autocomplete
• type Command
• type HandlerFunc
• type Link
• type Locals
• type Output
  • func BadRequest(msg string) Output
  • func InternalServerError(err error) Output
  • func NoContent() Output
  • func NotFound(msg string) Output
  • func PlainText(text string) Output
  • func Redirect(url string) Output
  • func Render(path string, data Locals) Output
  • func Unauthorized(msg string) Output
• type Page
  • func NewPage(name string) Page
• type PageEvent
• type PageEventHandler
• type Preprocessor
• type Property
• type Request
• type Response
• type Route
  • func Delete(path string, handler HandlerFunc, ...) Route
  • func Get(path string, handler HandlerFunc, ...) Route
  • func Match(route http.HandlerFunc, checks ...RouteCheck) Route
  • func Post(path string, handler HandlerFunc, ...) Route
• type RouteCheck
• type Suggestion

Constants

const (
	SIDEBAR_WIDGET    widgetSpace = iota // widgets rengered in the sidebar
	AFTER_VIEW_WIDGET                    // widgets rendered after the content of the view page
	HEAD_WIDGET                          // widgets rendered in page <head> tag
)

List of widgets spaces that extensions can use to register a widgetFunc to inject content into.

Variables

var (
	SOURCE   string // path to markdown files directory
	BUILD    string // path to write built files
	READONLY bool   // is xlog in readonly mode
	SITENAME string // name of knowledgebase
	SIDEBAR  bool   // is sidebar displayed
	INDEX    string // name of the index page markdown file
)

var (

	// a function that renders CSRF hidden input field
	CSRF = csrf.TemplateField
)

var MarkDownRenderer = goldmark.New(
	goldmark.WithExtensions(
		extension.GFM,
		extension.DefinitionList,
		extension.Footnote,
		highlighting.NewHighlighting(
			highlighting.WithCustomStyle(styles.Dracula),
		),
		emoji.Emoji,
	),

	goldmark.WithRendererOptions(
		html.WithHardWraps(),
		html.WithUnsafe(),
	),
)

The instance of markdown renderer. this is what takes the page content and converts it to HTML. it defines what features to use from goldmark and what options to turn on

Functions

func EachPage

func EachPage(ctx context.Context, f func(Page))

EachPage iterates on all available pages. many extensions uses it to get all pages and maybe parse them and extract needed information

func FindAllInAST

func FindAllInAST[t ast.Node](n ast.Node, kind ast.NodeKind) (a []t)

Extract all nodes of a specific type from the AST

func FindInAST

func FindInAST[t ast.Node](n ast.Node, kind ast.NodeKind) (found t, ok bool)

This is a function that takes an AST node and walks the tree depth first recursively calling itself in search for a node of a specific kind can be used to find first image, link, paragraph...etc

func IgnoreDirectory

func IgnoreDirectory(r *regexp.Regexp)

IgnoreDirectory Register a pattern to be ignored when walking directories.

func Listen

func Listen(e PageEvent, h PageEventHandler)

Register an event handler to be executed when PageEvent is triggered. extensions can use this to register hooks under specific page events. extensions that keeps a cached version of the pages list for example needs to register handlers to update its cache

func Partial

func Partial(path string, data Locals) template.HTML

func RegisterAutocomplete

func RegisterAutocomplete(a Autocomplete)

RegisterAutocomplete registers an autocomplete function. it should be used by an extension to register a new autocomplete function. these functions are going to be executed when rendering the edit page.

func RegisterBuildPage

func RegisterBuildPage(p string, encloseInDir bool)

RegisterBuildPage registers a path of a page to export when building static version of the knowledgebase. encloseInDir will write the output to p/index.html instead instead of writing to p directly. that can help have paths with no .html extension to be served with the exact name.

func RegisterCommand

func RegisterCommand(c func(Page) []Command)

RegisterCommand registers a new command

func RegisterHelper

func RegisterHelper(name string, f interface{})

RegisterHelper registers a new helper function. all helpers are used when compiling templates. so registering helpers function must happen before the server starts as compiling templates happened right before starting the http server.

func RegisterLink

func RegisterLink(l func(Page) []Link)

Register a new links function, should return a list of Links

func RegisterPreprocessor

func RegisterPreprocessor(f Preprocessor)

RegisterPreprocessor registers a Preprocessor function. extensions should use this function to register a preprocessor.

func RegisterProperty

func RegisterProperty(a func(Page) []Property)

RegisterProperty registers a function that returns a set of properties for the page

func RegisterQuickCommand

func RegisterQuickCommand(c func(Page) []Command)

func RegisterTemplate

func RegisterTemplate(t fs.FS, subDir string)

RegisterTemplate registers a filesystem that contains templates, specifying subDir as the subdirectory name that contains the templates. templates are registered such that the latest registered directory override older ones. template file extensions are signified by '.html' extension and the file path can be used as template name without this extension

func RegisterWidget

func RegisterWidget(s widgetSpace, priority float32, f widgetFunc)

RegisterWidget Register a function to a widget space. functions registered will be executed in order of priority lower to higher when rendering view or edit page. the return values of these widgetfuncs will pass down to the template and injected in reserved places.

func RenderWidget

func RenderWidget(s widgetSpace, p Page, r Request) (o template.HTML)

This is used by view and edit routes to render all widgetfuncs registered for specific widget space.

func Start

func Start()

Define the catch all HTTP routes, parse CLI flags and take actions like building the static pages and exit, or start the HTTP server

func Trigger

func Trigger(e PageEvent, p Page)

Trigger event handlers for a specific page event. page methods use this function to trigger all registered handlers when the page is edited or deleted for example.

func Vars

func Vars(r Request) map[string]string

Vars returns a map of key = dynamic segment in the path, value = the value of the segment registering a path like `"/edit/{page:.+}"` calling `Vars in the handler will return a map with one key `page` and the value is that part of the path in r

Types

type Autocomplete

type Autocomplete interface {
	StartChar() string
	Suggestions() []*Suggestion
}

Autocomplete defines what character triggeres the autocomplete feature and what is the list to display in this case.

type Command

type Command interface {
	// Icon returns the Fontawesome icon class name for the Command
	Icon() string
	// Name of the command. to be displayed in the list
	Name() string
	// Link returns the link/url/path of the command if any
	Link() string
	// OnClick action. a Javascript code to invoke when the command is executed
	OnClick() template.JS
	// Widget a HTML snippet to embed in the page that include any needed
	// assets, HTML, JS the command needs
	Widget() template.HTML
}

Command define a command that a user can invoke in view or edit page on a Page.

type HandlerFunc

type HandlerFunc func(http.ResponseWriter, *http.Request) Output

HandlerFunc is the type of an HTTP handler function + returns output function. it makes it easier to return the output directly instead of writing the output to w then return.

type Link

type Link interface {
	// Icon returns the fontawesome icon class name or emoji
	Icon() string
	// Name returns the link text
	Name() string
	// Link returns the Href property for the link (URL, Path, ...etc)
	Link() string
}

Link represent link for the user interface, default theme renders it in the sidebar

type Locals

type Locals map[string]interface{} // passed to templates

map of string to any value used for template rendering

type Output

type Output = http.HandlerFunc

alias of http.HandlerFunc as output is expected from defined http handlers

func BadRequest

func BadRequest(msg string) Output

BadRequest returns an output function that writes BadRequest http response

func InternalServerError

func InternalServerError(err error) Output

InternalServerError returns an output function that writes InternalServerError http response

func NoContent

func NoContent() Output

NoContent returns an output function that writes NoContent http status

func NotFound

func NotFound(msg string) Output

NotFound returns an output function that writes 404 NotFound to http response

func PlainText

func PlainText(text string) Output

PlainText returns an output function that writes text to response writer

func Redirect

func Redirect(url string) Output

Redirect returns an output function that writes Found http response to provided URL

func Render

func Render(path string, data Locals) Output

Render returns an output function that renders partial with data and writes it as response

func Unauthorized

func Unauthorized(msg string) Output

Unauthorized returns an output function that writes Unauthorized http response

type Page

type Page interface {
	// Name returns page name without '.md' extension
	Name() string
	// returns the filename, makes sure it converts slashes to backslashes when
	// needed. this is safe to use when trying to access the file that represent the
	// page
	FileName() string
	// checks if the page underlying file exists on disk or not.
	Exists() bool
	// Renders the page content to HTML. it makes sure all preprocessors are called
	Render() template.HTML
	// Reads the underlying file and returns the content
	Content() string
	// Deletes the file and makes sure it triggers the AfterDelete event
	Delete() bool
	// Overwrite page content with new content. making sure to trigger before and
	// after write events.
	Write(string) bool
	// ModTime Return the last modification time of the underlying file
	ModTime() time.Time
	// Parses the page content and returns the Abstract Syntax Tree (AST).
	// extensions can use it to walk the tree and modify it or collect statistics or
	// parts of the page. for example the following "Emoji" function uses it to
	// extract the first emoji.
	AST() ast.Node
	// Returns the first emoji of the page.
	Emoji() string
}

a Type that represent a page.

func NewPage

func NewPage(name string) Page

NewPage Creates an instance of Page with name. if no name is passed it's assumed INDEX

type PageEvent

type PageEvent int

a type used to define events to be used when the page is manipulated for example modified, renamed, deleted...etc.

const (
	BeforeWrite PageEvent = iota
	AfterWrite
	AfterDelete
)

List of page events. extensions can use these events to register a function to be executed when this event is triggered. extensions that require to be notified when the page is created or overwritten or deleted should register an event handler for the interesting events.

type PageEventHandler

type PageEventHandler func(Page) error

a function that handles a page event. this should be implemented by an extension and then registered. it will get executed when the event is triggered

type Preprocessor

type Preprocessor func(string) string

A Preprocessor is a function that takes the whole page content and returns a modified version of the content. extensions should define this type and register is so that when page is rendered it will execute all of them in order like a pipeline each function output is passed as an input to the next. at the end the last preprocessor output is then rendered to HTML

type Property

type Property interface {
	// Icon returns the fontawesome icon class name or emoji
	Icon() string
	// Name returns the link text
	Name() string
}

Property represent a piece of information about the current page such as last update time, number of versions, number of words, reading time...etc

type Request

type Request = *http.Request

alias *http.Request for shorter handler declaration

type Response

type Response = http.ResponseWriter

alias http.ResponseWriter for shorter handler declaration

type Route

type Route struct {
	// contains filtered or unexported fields
}

A route has handler function and set of RouteChecks if all checks are true, the last request will be passed to the handler function

func Delete

func Delete(path string, handler HandlerFunc, middlewares ...func(http.HandlerFunc) http.HandlerFunc) Route

Delete defines a new route that gets executed when the request matches path and method is http Delete. the list of middlewares are executed in order

func Get

func Get(path string, handler HandlerFunc, middlewares ...func(http.HandlerFunc) http.HandlerFunc) Route

Get defines a new route that gets executed when the request matches path and method is http Get. the list of middlewares are executed in order

func Match

func Match(route http.HandlerFunc, checks ...RouteCheck) Route

Match Adds a new HTTP handler function to the list of routes with a list of checks functions. the list of checks are executed when a request comes in if all of them returned true the handler function gets executed.

func Post

func Post(path string, handler HandlerFunc, middlewares ...func(http.HandlerFunc) http.HandlerFunc) Route

Post defines a new route that gets executed when the request matches path and method is http Post. the list of middlewares are executed in order

type RouteCheck

type RouteCheck func(Request) (Request, bool)

Checks a request for conditions, may modify request returning the new request and if conditions are met.

Can be used to check request method, path or other attributes against expected values.

type Suggestion

type Suggestion struct {
	Text        string // The text that gets injected in the editor if this option is chosen
	DisplayText string // The display text for this item in the menu. this can be more cosmetic. it has to start with the StartChar as the editor filter the list using this field
}

Suggestions represent an item in the list of autocomplete menu in the edit page