s3gen

package module
v0.0.14 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 30, 2024 License: Apache-2.0 Imports: 33 Imported by: 0

README

S3Gen

S3Gen is a simple static site generator written in Go. It lets you do some frequent things very easily:

  1. Compile your posts and content written in markdown (.md) into html files to be served statically.
  2. Use Go templating and a rich set of "builtin" functions for you to use.
  3. Use custom templates for different pages in your website/blog.
  4. Generates an RSS feed so your site can be consumed via a feed reader
  5. Generates a sitemap useable by search engines
  6. Library mode to programatically serve static sites inside another binary

Examples

My Personal Blog

Getting Started

With a Go compiler
go install github.com/panyam/s3gen@latest
Precompiled Binary

TBD

Setup and Usage

This library is mainly for developers who are interested in hosting a blog but want control and customization easily. There are very few conventions needed for this.

Create a Site object

Documentation

Index

Constants

View Source 
const (
	// When a resource is first encountered it is in pending state to indicate it needs to be loaded
	ResourceStatePending = iota

	// Marked when a resource has been loaded without any errors.
	ResourceStateLoaded

	// When a previously loaded resource has been deleted.
	ResourceStateDeleted

	// To indicate that a resource is not found (for some reason)
	ResourceStateNotFound

	// Loading of a resource failed (the status will be in the error field)
	ResourceStateFailed
)

Variables

This section is empty.

Functions

This section is empty.

Types

type DefaultPage added in v0.0.12 

type DefaultPage struct {
	// Site this page belongs to - can this be in multiple - then create different
	// page instances
	Site *Site

	// The slug url for this page
	Slug string

	Title string

	Link string

	Summary string

	CreatedAt time.Time
	UpdatedAt time.Time

	IsDraft bool

	CanonicalUrl string

	Tags []string

	// The resource that corresponds to this page
	Res *Resource

	// Loaded, Pending, NotFound, Failed
	State int

	// Any errors with this resource
	Error error
}

The default page type. Each type can have its own page type and can be overridden in the Site.GetPage method.

func (*DefaultPage) LoadFrom added in v0.0.12 

func (page *DefaultPage) LoadFrom(res *Resource) error

type FrontMatter 

type FrontMatter struct {
	// Whether the front matter for the resource has been loaded or not
	Loaded bool

	// Parsed data from front matter
	Data map[string]any

	// Length of the frontmatter in bytes (will be set after it is loaded)
	Length int64
}

Each Resource may have front matter. Front matter is lazily loaded and parsed in a resource. Our Resources specifically keep a reference to the front matter which can be used later on during rendering

type HTMLResourceHandler added in v0.0.12 

type HTMLResourceHandler struct {
	Template *htmpl.Template
	// contains filtered or unexported fields
}

func NewHTMLResourceHandler added in v0.0.12 

func NewHTMLResourceHandler(templatesDir string) *HTMLResourceHandler

func (*HTMLResourceHandler) GenerateTargets added in v0.0.12 

func (m *HTMLResourceHandler) GenerateTargets(r *Resource, deps map[string]*Resource) (err error)

Generates all target resources for a given resources. Note that before this method is called, LoadResource and LoadParamValues must have be called otherwise this wont work well on resources which are parametric

func (*HTMLResourceHandler) LoadParamValues added in v0.0.12 

func (m *HTMLResourceHandler) LoadParamValues(res *Resource) error

func (*HTMLResourceHandler) LoadResource added in v0.0.12 

func (m *HTMLResourceHandler) LoadResource(res *Resource) error

func (*HTMLResourceHandler) RenderContent added in v0.0.12 

func (m *HTMLResourceHandler) RenderContent(res *Resource, w io.Writer) error

func (*HTMLResourceHandler) RenderResource added in v0.0.12 

func (m *HTMLResourceHandler) RenderResource(outres *Resource, content any, writer io.Writer) error

type MDResourceHandler added in v0.0.12 

type MDResourceHandler struct {
	Template *ttmpl.Template
	// contains filtered or unexported fields
}

func NewMDResourceHandler added in v0.0.12 

func NewMDResourceHandler(templatesDir string) *MDResourceHandler

func (*MDResourceHandler) GenerateTargets added in v0.0.12 

func (m *MDResourceHandler) GenerateTargets(r *Resource, deps map[string]*Resource) (err error)

Generates all target resources for a given resources. Note that before this method is called, LoadResource and LoadParamValues must have be called otherwise this wont work well on resources which are parametric

func (*MDResourceHandler) LoadParamValues added in v0.0.12 

func (m *MDResourceHandler) LoadParamValues(res *Resource) error

func (*MDResourceHandler) LoadResource added in v0.0.12 

func (m *MDResourceHandler) LoadResource(res *Resource) error

func (*MDResourceHandler) RenderContent added in v0.0.12 

func (m *MDResourceHandler) RenderContent(res *Resource, w io.Writer) error

Renders just the content section within the resource

func (*MDResourceHandler) RenderResource added in v0.0.12 

func (m *MDResourceHandler) RenderResource(outres *Resource, content any, writer io.Writer) error

type Page 

type Page interface {
	LoadFrom(*Resource) error
}

type PageTemplate added in v0.0.12 

type PageTemplate struct {
	Name   string
	Params map[any]any
}

PageTemplates are data used to render a page. Typically this needs the name of the template being rendered and the (possibly nested) parameters need by that specific template.

type Resource 

type Resource struct {
	// The Site that owns this resources.  Resources belong to a site and cannot be shared across multiple sites
	Site *Site

	// Fullpath of the Resource uniquely identifying it within a Site
	FullPath string

	// Created timestamp on disk
	CreatedAt time.Time

	// Updated time stamp on disk
	UpdatedAt time.Time

	// The resource this is derived/copied/rendered from. This will only be set for output resources
	Source *Resource

	// The ResourceState - Loaded, Pending, NotFound, Failed
	State int

	// Any errors with this resource (eg during load)
	Error error

	// The destination page if this resource is for a target page
	Page any

	// If this is a parametric resources - this returns the space of all parameters
	// possible for this resource based on how it is loaded and its config it takes
	// For example a blog page of the form /a/b/c/[name].md could have 10 distinct values
	// for the "name" parameter.  Those will be populated here by the content processor
	// For now we are only looking at single level of parameters.  In the future we will
	// consider multiple parameters, eg: /[param1]/[param2]...
	ParamValues []string
	// Name of the parameter
	ParamName string

	NeedsIndex bool
	IsIndex    bool

	// True if the resource is parametric and can result in several instances
	IsParametric bool
	// contains filtered or unexported fields
}

All files in our site are represented by the Resource type. Each resource in identified by a unique path. A resource can be processed or transformed to result in more Resources (eg an input post markdown resource would be rendered as an output html resource).

func (*Resource) AddParam 

func (r *Resource) AddParam(param string) *Resource

This methods add a new "parameter" to the resource. Parametric resources are a way to ensure that a given reosurce (eg a page) can take several instances. For example the content page `<content_root>/tags/[tag].md` can resultin multiple files of the form `<output_folder>/tags/tag1/index.html`, `<output_folder>/tags/tag2/index.html` and so on. This is evaluated by rendering the the source file (`<content_root>/tags/[tag].md`) in the "nameless" mode where the template would call the AddParam method once for each new child resources (eg tag1, tag2...)

func (*Resource) AddParams 

func (r *Resource) AddParams(params []string) *Resource

Very similar to the Addparam but allows adding a list of parameters in one call.

func (*Resource) DirName 

func (r *Resource) DirName() string

Returns the resource's directory

func (*Resource) EnsureDir 

func (r *Resource) EnsureDir()

Ensures that a resource's parent directory exists

func (*Resource) Ext 

func (r *Resource) Ext() string

Returns the extension of the resource's path

func (*Resource) FrontMatter 

func (r *Resource) FrontMatter() *FrontMatter

Load's the resource's front matter and parses if it has not been done so before.

func (*Resource) Info 

func (r *Resource) Info() os.FileInfo

Get the resource's os level FileInfo

func (*Resource) IsDir 

func (r *Resource) IsDir() bool

Returns true if the resource is a directory

func (*Resource) Load 

func (r *Resource) Load() *Resource

Load's the resource from disk including any front matter it might have.

func (*Resource) ReadAll 

func (r *Resource) ReadAll() ([]byte, error)

Read all the content bytes after the front-matter in this file.

func (*Resource) Reader 

func (r *Resource) Reader() (io.ReadCloser, error)

Returns a reader for all the content bytes after the front matter for a resource if it exists. This is handy to prevent loading entire large files into memory (unlike ReadAll).

func (*Resource) RelPath 

func (r *Resource) RelPath() string

Returns the path relative to the content root

func (*Resource) Reset 

func (r *Resource) Reset()

Reset's the Resource's state to Pending so it can be reloaded

func (*Resource) WithoutExt 

func (r *Resource) WithoutExt(all bool) string

Returns the resource without the extension.

type ResourceFilterFunc 

type ResourceFilterFunc func(res *Resource) bool

Types of functions that filter resources (usually in a list call)

type ResourceHandler added in v0.0.12 

type ResourceHandler interface {
	// Loads resource data from the appropriate input path
	LoadResource(r *Resource) error

	// Generates all target/child resources for a given resources.
	// Note that before this method is called, LoadResource and LoadParamValues
	// must have be called otherwise this wont work well on resources which are parametric
	GenerateTargets(r *Resource, deps map[string]*Resource) error

	// Loads the parameter values for a resource.  Parametric resources are an important concept.
	// Parametric resources are a way to ensure that a given reosurce (eg a page) can take several instances.
	// For example the content page `<content_root>/tags/[tag].md` can resultin multiple files of the form
	// `<output_folder>/tags/tag1/index.html`, `<output_folder>/tags/tag2/index.html` and so on.  This is
	// evaluated by rendering the the source file (`<content_root>/tags/[tag].md`)  in the "nameless" mode
	// where the template would call the AddParam method once for each new child resources (eg tag1, tag2...)
	// until all param names are resolved.  And then once for each parameter the template is rendered again
	// so that the correspond output page is generated.  This may seem like a lot but since static sites need
	// all combinations generated upfront, this is fine as long as the number of variations are small.
	// This method performs the "nameless" mode rendering to gather all parameter values a parametric page can take.
	LoadParamValues(r *Resource) error

	// Renders just the content section within the resource so that it can be embedded in the "larger" layout page.
	RenderContent(res *Resource, w io.Writer) error

	// Once the content (ie the main body) of the source is rendered,
	// it needs to be wrapped up in a larger view so it finally looks
	// like a rendered page.  This method should be called on the
	// Destination resource to perform the final wrapping.
	RenderResource(res *Resource, content any, w io.Writer) error
}

Loads a resource of diferent types from storage

type ResourceSortFunc 

type ResourceSortFunc func(a *Resource, b *Resource) bool

Types of function used for sorting of resources. returns true if a < b, false otherwise.

type Site 

type Site struct {
	TemplateStore

	// ContentRoot is the root of all your pages.
	// One structure we want to place is use folders to emphasis url structure too
	// so a site with mysite.com/a/b/c/d
	// would have the file structure:
	// <ContentRoot>/a/b/c/d.<content_type>
	ContentRoot string

	// Final output directory where resources are generated/published to
	OutputDir string

	// The http path prefix the site is prefixed in,
	// The site could be served from a subpath in the domain, eg:
	// eg
	//		myblog.com								=> PathPrefix = "/"
	//		myblog.com/blog						=> PathPrefix = "/blog"
	//		myblog.com/blogs/blog1		=> PathPrefix = "/blogs/blog1"
	//
	// There is no restriction on this.  There could be other routes (eg /products page)
	// that could be served by a different router all together in parallel to say /blog.
	// This is only needed so that the generator knows where to "root" the blog in the URL
	PathPrefix string

	// A list of folders where static files could be served from along with their
	// http path prefixes.  This is an array of strings of the form
	// [ path1, folder1, path2, folder2, path3, folder3 ....]
	StaticFolders []string

	// ResourceHandlers tell us how to "process" a content of a given type.
	// types are denoted by extensions for now later on we could do something else
	ResourceHandlers map[string]ResourceHandler

	// When walking the content root for files, this callback specify which directories
	// are to be ignored.
	IgnoreDirFunc func(dirpath string) bool

	// When walking the content root for files, this callback specify which files
	// are to be ignored.
	IgnoreFileFunc func(filepath string) bool

	// Whether to enable live reload/rebuild of changed files or not
	LiveReload bool
	LazyLoad   bool

	DefaultPageTemplate PageTemplate
	GetTemplate         func(res *Resource, out *PageTemplate)
	CreatePage          func(res *Resource)

	BuildFrequency time.Duration
	// contains filtered or unexported fields
}

The site object is one of the most central types in s3gen. It contains all configuration metadata for the site (eg input/output directories, template directories, static routes etc). The Site is the central point for managing the building, live reloading, templating etc needed to build and serve a static site. This Site object also provides a http.HandlerFunc which can be used to server it via a http.Server.

func (*Site) AddEdge added in v0.0.12 

func (s *Site) AddEdge(srcpath string, destpath string) bool

Add a dependency edge between two resources identified by their full paths. Returns true if edge was added without incurring a cycle, returns false if edge would have resulted in a cycle.

func (*Site) DefaultFuncMap added in v0.0.4 

func (s *Site) DefaultFuncMap() htmpl.FuncMap

func (*Site) EdgeExists added in v0.0.12 

func (s *Site) EdgeExists(srcpath string, destpath string) bool

Returns true if an edge exists between a source and a destination resource

func (*Site) GenerateSitemap added in v0.0.14 

func (s *Site) GenerateSitemap() map[string]any

func (*Site) GetResource 

func (s *Site) GetResource(fullpath string) *Resource

Loads a resource and validates it. Note that a resources may not necessarily be in memory just because it is loaded. Just a Resource pointer is kept and it can be streamed etc

func (*Site) GetResourceHandler added in v0.0.12 

func (s *Site) GetResourceHandler(rs *Resource) ResourceHandler

Every resource needs a ResourceHandler. This is a factory method to return one given a resource. TODO - Today there is no way to override this. In the future this will be turned into a method attribute

func (*Site) GetRouter 

func (s *Site) GetRouter() *mux.Router

Returns a Router instance that can serve this as a site under a larger prefix.

func (*Site) HandleStatic added in v0.0.6 

func (s *Site) HandleStatic(path, folder string) *Site

Add a new static http path and the folder from which its contents can be served.

func (*Site) Init 

func (s *Site) Init() *Site

Initializes the Site

func (*Site) Json added in v0.0.4 

func (s *Site) Json(path string, fieldpath string) (any, error)

func (*Site) ListResources 

func (s *Site) ListResources(filterFunc ResourceFilterFunc,
	sortFunc ResourceSortFunc,
	offset int, count int) (foundResources []*Resource)

A method to list all the resource in our site in the content root. This method also allows pagination, filtering and sorting of resources.

func (*Site) PathExists added in v0.0.12 

func (s *Site) PathExists(srcpath string, destpath string) bool

func (*Site) PathRelUrl 

func (s *Site) PathRelUrl(path string) string

Returns the full url for a path relative to the site's prefix path. If the Site's prefix path is /a/b/c, then PathRelUrl("d") would return /a/b/c/d

func (*Site) Rebuild 

func (s *Site) Rebuild(rs []*Resource)

This is the heart of the build process. This method is called with a list of resources that have to be reprocessed (either due to periodic updates or change events etc). Resources in our site form a graph and each resource is processed by a ResourceHandler appropriate for it The content processor can create more resources that may need an update because they are dependant on this resource. By allowing a list of resources to be processed in a batch it is more efficient to perform batch dependencies instead of doing repeated builds for each change.

func (*Site) RemoveEdge added in v0.0.12 

func (s *Site) RemoveEdge(srcpath string, destpath string) bool

Removes a dependency edge between two resources identified by their full paths

func (*Site) RemoveEdgesFrom added in v0.0.12 

func (s *Site) RemoveEdgesFrom(srcpath string)

Remove all resources depended by a given path

func (*Site) RemoveEdgesTo added in v0.0.12 

func (s *Site) RemoveEdgesTo(destpath string)

Removes all resources that a given path depends on

func (*Site) RemoveResource added in v0.0.12 

func (s *Site) RemoveResource(path string) *Resource

Remove a resources from this graph along with all its dependencies

func (*Site) ServeHTTP 

func (s *Site) ServeHTTP(w http.ResponseWriter, r *http.Request)

The base entry point for a serving a site with our customer handler - also implementing the http.Handler interface

func (*Site) StopWatching 

func (s *Site) StopWatching()

Disables/Stops watching for changes to content files.

func (*Site) Watch added in v0.0.2 

func (s *Site) Watch()

Starts watching for changes to content files so that the site can be rebuilt.

type TemplateStore added in v0.0.14 

type TemplateStore struct {
	// A map of template functions available for both text and html templates
	CommonFuncMap htmpl.FuncMap

	// Global templates dirs
	// A list of GLOBs that will point to several html templates our generator will parse and use
	HtmlTemplates []string

	HtmlFuncMap htmpl.FuncMap

	// A list of GLOBs that will point to several text templates our generator will parse and use
	TextTemplates []string

	TextFuncMap ttmpl.FuncMap
	// contains filtered or unexported fields
}

func (*TemplateStore) DefaultFuncMap added in v0.0.14 

func (s *TemplateStore) DefaultFuncMap() htmpl.FuncMap

Returns the default function map to be used in the html templates.

func (*TemplateStore) HtmlTemplate added in v0.0.14 

func (s *TemplateStore) HtmlTemplate(clone bool) *htmpl.Template

Returns the parsed HTML templates used for rendering a given template content If the clone parameter is true then a clone of the template is returned. This is useful if we ever need to reuse the template while it is in the middle of an execution.

func (*TemplateStore) RenderHtml added in v0.0.14 

func (s *TemplateStore) RenderHtml(templateName string, params map[string]any) (template.HTML, error)

Renders a html template

func (*TemplateStore) RenderText added in v0.0.14 

func (s *TemplateStore) RenderText(templateName string, params map[string]any) (template.HTML, error)

Renders a text template

func (*TemplateStore) RenderView added in v0.0.14 

func (s *TemplateStore) RenderView(writer io.Writer, v views.ViewRenderer, templateName string) error

Site extension to render a view

func (*TemplateStore) TextTemplate added in v0.0.14 

func (s *TemplateStore) TextTemplate(clone bool) *ttmpl.Template

Returns the parsed Text templates used for rendering a given template content If the clone parameter is true then a clone of the template is returned. This is useful if we ever need to reuse the template while it is in the middle of an execution.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.