io

README

IO Module

The IO Module contains useful methods for parsing and writing N-Triples files.

TODO

• Implement full N-Triples specification as in https://www.w3.org/TR/n-triples/

• There is also a library called rdf2go that is able to parse Turtle format files, which is a superset of N-Triple files. Maybe use that in the future, but make sure it is faster and safer.

Documentation

Index

• func CreateAndOpenWithGzip(filePath string) io.WriteCloser
• func InterpreteIriRef(token []byte) (text []byte)
• func InterpreteLangLiteral(token []byte) (text []byte, lang []byte)
• func TrimExtensions(fileName string) (fileBase string)
• func UniversalReader(fileName string) (reader io.ReadCloser, err error)
• type GZipWriteCloser
  • func (gzwc GZipWriteCloser) Close() (err error)
  • func (gzwc GZipWriteCloser) Write(p []byte) (n int, err error)
• type Triple
  • func (t *Triple) Output()
• type TripleParser
  • func NewTripleParser(filePath string) (*TripleParser, error)
  • func (tp *TripleParser) Close() error
  • func (tp *TripleParser) NextTriple(argNumTokens ...int) (*Triple, error)

Functions

func CreateAndOpenWithGzip

func CreateAndOpenWithGzip(filePath string) io.WriteCloser

CreateAndOpenWithGzip opens a file with a GZip writer and returns an structure capable to closing all underlying open files.

func InterpreteIriRef

func InterpreteIriRef(token []byte) (text []byte)

InterpreteIriRef will interprete an IRIREF and return the string that is enclosed by the tag delimiters. It is assumed that the token has no leading nor trailing spaces.

func InterpreteLangLiteral

func InterpreteLangLiteral(token []byte) (text []byte, lang []byte)

InterpreteLangLiteral will interprete a literal with language tag and return the text and language code. It is assumed that the token has no leading nor trailing spaces.

func TrimExtensions

func TrimExtensions(fileName string) (fileBase string)

TrimExtensions will remove the extension of a fileName if it resembles an extension used by compression algorithms (.gz, .bz2, .gbz). Then it will remove the Data format extension (.nt) if it follows next.

func UniversalReader

func UniversalReader(fileName string) (reader io.ReadCloser, err error)

UniversalReader opens a file for read mode. It is able to automatically decompress files that use either GZ, BGZ or BZ2. It will also display a loading bar on stdout.

Types

type GZipWriteCloser

type GZipWriteCloser struct {
	// contains filtered or unexported fields
}

GZipWriteCloser implements the necessary methods to write and make a cascading close of all underlying files.

func (GZipWriteCloser) Close

func (gzwc GZipWriteCloser) Close() (err error)

Close will close the gzip writer and underlying file handle. TODO: There might be a more consise way to do this, but I was not sure if defer can actually

return something.

func (GZipWriteCloser) Write

func (gzwc GZipWriteCloser) Write(p []byte) (n int, err error)

Write writes a byte stream to the gzip writer.

type Triple

type Triple struct {
	Subject   []byte
	Predicate []byte
	Object    []byte
	Line      []byte // Holds the entire line including terminating dot (but no newline)
}

Triple represents an RDF entry in the N-Triple file. TODO: Check if having []byte does not make problems with the buffer they are pointing to. What

happens to the triple if the buffer flushes?

func (*Triple) Output

func (t *Triple) Output()

Output the values of a triple to stdout.

type TripleParser

type TripleParser struct {
	// contains filtered or unexported fields
}

TripleParser reads an internal file and produces triples from it.

func NewTripleParser

func NewTripleParser(filePath string) (*TripleParser, error)

NewTripleParser opens a file and returns the relevant triple parser that will produce triples.

func (*TripleParser) Close

func (tp *TripleParser) Close() error

Close the handlers for the scanner and underlying file.

func (*TripleParser) NextTriple

func (tp *TripleParser) NextTriple(argNumTokens ...int) (*Triple, error)

NextTriple returns the next triple that is read from the internal file.

Arguments:

numTokens int : by adding the optional argument you can decide how many tokens
                should be parsed. With 0, no tokens are parsed. With 1, 2 or 3
                all tokens up to including Subject, Predicate and Object are
                parsed. Non-parsed tokens are nil slices.

Example:

triple, err := tripleParser.NextTriple(2)  // consume line and parse subject and predicate.

TODO: The parsing is a subset of the actual N-Triple syntax.