resources

README

resources

Resources is a package for extracting urls of dependant files for displaying a web page. It's primary use is for establishing the list of assets an archive would need to cache in order to properly display a target resource

License & Copyright

Copyright (C) 2017 Data Together This program is free software: you can redistribute it and/or modify it under the terms of the GNU AFFERO General Public License as published by the Free Software Foundation, version 3.0.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

See the LICENSE file for details.

Getting Involved

We would love involvement from more people! If you notice any errors or would like to submit changes, please see our Contributing Guidelines.

We use GitHub issues for tracking bugs and feature requests and Pull Requests (PRs) for submitting changes

Installation

Use in any golang package with:

import "github.com/datatogether/resources"

Documentation

Overview

Resources is a package for extracting urls of dependant files for displaying a web page. It's primary use is for establishing the list of assets an archive would need to cache in order to properly display a target resource

Index

• type Extractor
  • func NewExtractor() *Extractor
  • func (e Extractor) ExtractHtmlResources(rec *warc.Record) (urls []string, err error)
  • func (e Extractor) ExtractResponseUrls(rec *warc.Record) ([]string, error)

Types

type Extractor

type Extractor struct {
	Tags map[string][]string
}

func NewExtractor

func NewExtractor() *Extractor

func (Extractor) ExtractHtmlResources

func (e Extractor) ExtractHtmlResources(rec *warc.Record) (urls []string, err error)

func (Extractor) ExtractResponseUrls

func (e Extractor) ExtractResponseUrls(rec *warc.Record) ([]string, error)