mxftest

package module
v0.0.0-...-32049d6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 13, 2024 License: BSD-3-Clause Imports: 13 Imported by: 0

README

MXF Test

MRX Unit Test is a library for testing MXF (Material Exchange Format) and MRX (Metarex.media) files.

This library provides the api for designing your own tests. To see these tests in action check out the MXF test repo, the examples folder, or the replit demo playground.

All tests generate an easy to understand report detailing what was checked, the specification it was testing to and the results.

Contents

Introduction

This library is designed to allow you to quickly write easy to understand MXF tests.

This library provides the tools for:

  • Generating a test report in a simple format
  • Writing your own tests to specification
  • Writing conditional specifications
  • An API for quickly searching MXF files

The Report

The report details the tests that were run, the specification the tests were run against and if they passed. In the event of test failures an extra message is given, describing the change from the expected behaviour / result.

All Reports follow the same structure, no matter how many tests and specifications were run when generating it.

The test report is as a yaml file and the first field is the testpass field so you can tell if the mxf passed the tests at a glance.

The report contains the following fields and subfields:

  • testpass - Has the report passed?
  • skippedTests - An array of any tests that were not run, it has the sub fields:
    • testkey - The key the test was looking for to run
    • desc - A brief description of the skipped test
  • tests - An array of the batch of tests that were run in the report, it has the following sub fields.
    • header - Header description for that batch of tests
    • tests - The tests that were run. These have the following fields:
      • message - The message of what the test is testing. It always starts with the specification being covered.
      • checks - An array of the assertions the test run and if they passed or failed. If it fails an additional errorMessage field is also given.
      • pass - Did all the tests pass?
      • passcount - The count of passed tests
      • failcount - The count of failed tests

An example report is given below

testpass: true
tests:
    - header: testing mxf file structure
      tests:
        - message: |
            RDD47:2018,5.4,shall,3: Checking that the generic partition positions match the expected positions at the end of the file
          checks:
            - pass: true
            - pass: true
            - pass: true
            - pass: true
        - message: |
            ST377-1:2019,7.1,Table5,7: Checking the previous partition pointer is the correct byte position for the header partition at byte offset 0
          checks:
            - pass: true
      pass: true
      passcount: 5
      failcount: 0
skippedTests:
    - testkey: generickey
      desc: a skipped partition test

Writing Tests to Specification

The MXF test API is designed so that you can read a MXF specification and type it straight to go. Reducing the need to handle the mxf file and just focus on testing the segments relevant to your specification.

The MXF file structure is broken down into a easily navigable tree, so you can write a test and assign in to the sections of the MXF file it applies to. Therefore you can skip the finding and validating parts of the MXF file that may be relevant. e.g. This test only applies to groups with a Universal Label of 060e2b34.02530105.0e090502.00000000

The MXF tree is detailed here and is important for understanding how the tests work. Once you have read how to traverse the MXF file, a complete demo of writing tests is provided [here.](#Building Custom Tests)

Traversing the MXF file

The MXF file is broken down into an abstract syntax tree, so you can navigate and test the areas of the file that matter to you. Not interested in index tables? Then don't worry you don't need to think abut them.

The abstract syntax tree of an MXF file has three types of Nodes:

  1. The MXF Node - this represents the whole MXF file and has children partitions of type Partition Node.
  2. Partition Node - these nodes represent a partition and have 3 types of children, Essence, Metadata and Index table. All of which are of type Node.
  3. Node - these represent any other type of data in an MXF file, and have children of type node. These are recursive and can have several nested layers of node children.

The following diagram gives a visualisation of the AST layout:


flowchart TD
  A[MXF Node] --> B(Partition)
  A --> C(Partition)
  A --> D(Partition)
  B --> BM(Metadata Nodes) 
  C --> CE(Essence Nodes)
  C --> CM(Metadata Nodes)
  D --> DE(Essence Nodes)
  D --> DM(Metadata Nodes)

When designing the MXF tests, each test needs to target one of these specific node types. Where each node has these common uses:

  • The MXF Node is used for testing the overall structure of the file.
  • Partition Nodes are for checking the essence, the partition properties or its metadata.
  • Node is used checking that groups within the header metadata.

This is not the limit of the node uses in tests, just a guide for test design and where to start.

The tests are processed and run in the following order:

flowchart TD
    A[All Specifications] -->|Run tag tests| C{Does the tag pass}
    C -->|Tag pass| RTS[Add spec tests to mxf tests]
    C -->|Tag fail| E(Stop Specification here)
    RTS --> |Run tests |Loop[Run Structure Tests]
    Loop --> |Loop through partitions|Part[Run partition Test]
    Part --> mdp{Is metadata present}
    mdp --> |Yes| md[run metadata tests]
    md --> cont
    mdp --> |no| cont{Are there more partitions} 
    cont -->|Next Partition| Part
    cont --> |No more partitions to test|Finish(All tests run)
Building Custom Tests

This section will walk you through the complete process of writing MXF tests using the mxftest API, starting from reading a specification document right up to integrating it into your test environment.

With all specifications shall and shall not are given as keywords for strict specification requirements. These are the key words we shall be using to define the tests in this demo.

This section has the following contents:

When writing the code from this demo, make sure you have the following imports.

import (
  "io"
  "github.com/metarex-media/mrx-tool/mrxUnitTest"
  mxf2go "github.com/metarex-media/mxf-to-go"
  . "github.com/onsi/gomega"
)

Which can be imported with

go get github.com/metarex-media/mrx-tool/mrxUnitTest
go get github.com/metarex-media/mxf-to-go
go get github.com/onsi/gomega

A full interactive playground of these demos is available here, feel free to play around with it to some more practice writing and understanding tests.

Data Validation Tests

The contents are the most important bit of an MXF file, so this is where we begin our testing journey. The vast majority of MXF tests written will be utilising this test format, as the data has much more variety than the file structure itself. In the examples we are test to the gps.mxf example file, this contains data of the Metarex type MRX.123.456.789.gps.

For simplicity we shall be using made up documentation to describe data for this demo. These docs have the following requirement All data contained in this file shall satisfy the schema of MRX.123.456.789.gps

This means every time data is encountered in the MRX file, it has to be

  1. A json file, because MRX.123.456.789.gps has a content type "application/json"
  2. The json files pass the gps schema. They are not just any old json

To check the data we set up sniff tests, these are used for confirming the data type (in this case are they json), and running any further checks once the data is confirmed (does the data satisfy the schema for this demo). We can use the mxftest/jsonhandle library for running these tests, which can be set up with the following code, when you set up the test specification.


//go:embed testdata/gpsSchema.json
var gpsSchema []byte

// GPSSpecifications returns all the specifications
// associated with the demo MRX.123.456.789.GPS namespace
func GPSSpecifications(sc mxftest.SniffContext) mxftest.Specifications {

  // Initialising the schmea check function
  // with the sniff key GPSSchema
  schemaSniff, _ := jsonhandle.SchemaCheck(sc, gpsSchema, "GPSSchema")

  return *mxftest.NewSpecification(
    // Assign a sniff test to check files are json and valid gps data
    mxftest.WithSniffTest(mxftest.SniffTest{
      // The function to check data is json
      DataID: jsonhandle.DataIdentifier, 
      // The schema function as part of ths sniff test
      Sniffs: []mxftest.Sniffer{schemaSniff}}),
      // Assign the rest of the tests below
      ...


  )

}

Now we have sniffed the data, we can write the tests to check the results of the sniff tests. We start by writing a partition test function (partitions contain the data), they have a structure of:

func testGenericPartition(doc io.ReadSeeker, header *PartitionNode) func(t Test) {
  return func(t Test) {
  }
}

To validate the essence of the partition, we check that there are only gps files present. We do this by utilising the partition search function to look for non gps files in the partition, where we are searching using the sniffing that we instansiated earlier. This is done with the search phrase select * from essence where sniff:GPSSchema <> pass, which is select all nodes, where the data did not pass the GPS json schema. Now we have found the contents of the partition we need to test against them.

The test is constructed with the t.Test() function, which describes; the test being run, a specification field and a series of assertions that the tests run.

The assertions are the comparisons we make when testing and for this library, they are required to be gomega assertions, otherwise the tests will panic and fail :( .

When writing the assertions we use the following phrasing to help build the tests:

  • We expect t.Expect(gpsSearchErr) that there shall be no error Shall(BeNil()) for finding the gps data - t.Expect(gpsSearchErr).Shall(BeNil())
  • We expect t.Expect(len(nongps)) that there shall be no non gps metadata in the essence Shall(Equal(0) - t.Expect(len(nongps)).Shall(Equal(0))
  • We expect t.Expect(len(header.Essence)) that essence shall be in the generic partition ShallNot(Equal(0)) - t.Expect(len(header.Essence)).ShallNot(Equal(0))

Now we have all the ingredients we can put them together into one test function.

func testBodyPartitionForGPS(_ io.ReadSeeker, header *mxftest.PartitionNode) func(t mxftest.Test) {
  return func(t mxftest.Test) {

    // check all the files are ttml
    nongps, gpsSearchErr := header.Search("select * from essence where sniff:GPSSchema <> pass")

    t.Test("checking that the partition only contains gps json files", mxftest.NewSpecificationDetails("DemoSpec", "X.X", "shall", 1),
      t.Expect(gpsSearchErr).Shall(BeNil()),
      t.Expect(len(nongps)).Shall(Equal(0), "Non GPS files found in the partition"),
      t.Expect(len(header.Essence)).ShallNot(Equal(0), "no essence found in the generic partition"),
    )
  }
}

Which we can integrate into the test specification with the following code:

mxftest.WithPartitionTests(
  mxftest.PartitionTest{PartitionType: mxftest.Body, Test: testBodyPartitionForGPS},
),

Where the PartitionType is the partition type of the node being tested, in this case it's mxftest.Body, which is the body partition where data is kept.

Congratulations you have, written your first MXF test, you have:

  • Written a test to specification requirement
  • Set up a sniff function to comb your data
  • Checked those sniff results in a test
  • Utilised the partition search function
  • Added the test to a specification
Technical Tests

These demos are for walking through testing the structure of an MXF file, because some specification require certain properties to be present in the file metadata, or there to be certain structural requirements. These demos will take you through both types of tests.

Structural Tests

This demo will take you through testing the structural elements of an MXF file. For this demo we will be using R133, a European Broadcasting Union specification on using Subtitle files in MXF, it is a free specification and is not hidden behind a paywall.

On page 10 of R133 the following requirements are given "All Generic Stream Partitions that contain subtitles shall be placed at the end of the MXF file"

This means all the generic partition segments in the mxf file, must come after the header and body partitions, and before the footer and Random Index Pack (where present).

The structure test which takes the form of:

func checkStructure(doc io.ReadSeeker, mxf *MXFNode) func(t Test) {
  return func(t Test) {

  }
}

Note how it takes the MXF node as the input, so that the whole structure of the MXF file can be checked.

For this test we need to find every generic partition, then the footer and random index pack. We find these using the mxfNode search function, searching for the partitions based on their types e.g. "select * from partitions where type = body". We can then calculate the expected positions of the generic partitions, where the partition positions are found with (end partition - generic partition count) + generic partition position. The end partition is the count of the partitions in the file minus the footer and random index packs positions, if they are present.

The test is constructed with the t.Test() function, which describes; the test being run, a specification field and a series of assertions that the tests run.

The assertions are the comparisons we make when testing and for this library, they are required to be gomega assertions, otherwise the tests will panic and fail :( .

When writing the assertions we use the following phrasing to help build the tests:

  • We expect t.Expect(gpErr) that there shall be no error Shall(BeNil()) fot the finding the generic partitions - t.Expect(gpErr).Shall(BeNil())
  • We expect t.Expect(footErr) that there shall be no error Shall(BeNil()) fot the finding the footer partition - t.Expect(footErr).Shall(BeNil())
  • We expect t.Expect(ripErr) that there shall be no error Shall(BeNil()) fot the finding the random index pack - t.Expect(ripErr).Shall(BeNil())
  • We expect t.Expect(expectedParts) the expected generic partitions shall match the actual positions Shall(Equal(GenericCountPositions)) - t.Expect(expectedParts).Shall(Equal(GenericCountPositions))

The completed code for this test is given below.

func checkStructure(doc io.ReadSeeker, mxf *MXFNode) func(t Test) {
  return func(t Test) {

    // find the generic paritions
    genericParts, gpErr := mxf.Search("select * from partitions where type = " + GenericStreamPartition)
    // find the generic partitions positions
    GenericCountPositions := make([]int, len(genericParts))
    for i, gcp := range genericParts {
      GenericCountPositions[i] = gcp.PartitionPos
    }

    // is there a footer partition?
    endPos := len(mxf.Partitions)
    footerParts, footErr := mxf.Search("select * from partitions where type = " + FooterPartition)
    if len(footerParts) != 0 {
      endPos--
    }

    // is there a Random Index Partition
    ripParts, ripErr := mxf.Search("select * from partitions where type = " + RIPPartition)
    if len(ripParts) != 0 {
      endPos--
    }

    // calculate the expected partitions
    expectedParts := make([]int, len(GenericCountPositions))
    for j := range expectedParts {
      expectedParts[j] = endPos - len(expectedParts) + j
    }

    // run the test comparing the positions
    t.Test("Checking that the generic partition positions match the expected positions at the end of the file", NewSpec("EBUR133:2012", "5.1", "shall", 3),
      t.Expect(gpErr).Shall(BeNil()),
      t.Expect(footErr).Shall(BeNil()),
      t.Expect(ripErr).Shall(BeNil()),
      t.Expect(expectedParts).Shall(Equal(GenericCountPositions)),
    )
  }
}

This is then integrated into the test specification with the following code:

mxftest.WithStructureTests(checkStructure)

Congratulations you have, written your first MXF structure test, you have:

  • Written a test to specification requirement
  • Utilised the MXF Node search function
  • Added the test to a specification
File Metadata tests

This sections will take you through writing tests for checking the file metadata. We will be using the RDD 47 as this describes new file metadata that is required. This has two statements that we will be testing to in Section 9.2 and 9.3 of RDD 47.

  1. The File Descriptor sets are those structural metadata sets in the Header Metadata that describe the essence and metadata elements defined in this document. The ISXD Data Essence Descriptor shall be a sub-class of the Generic Data Essence Descriptor defined in SMPTE ST 377-1. File Descriptor sets shall be present in the Header Metadata for each Essence Element
  2. The DataEssenceCoding item shall be present in the ISXD Data Essence Descriptor

These statements boil down to:

  1. The ISXD Data Essence Descriptor is in the file metadata
  2. The ISXD Data Essence Descriptor has the field DataEssenceCoding, with the set value of 060E2B34.04010105.0E090606.00000000.

To test for these conditions we need a node test, which takes the form of:

func testISXDDescriptor(doc io.ReadSeeker, isxdDesc *mxftest.Node, primer map[string]string) func(t mxftest.Test) {
  return func(t Test) {
  }
}

First we check if the ISXD Data Essence Descriptor exists, if the ISXD node passed to the function is nil then it doesn't exist and the test fails. Then if it does exist we can decode it with themxftest.DecodeGroupNode function and check the fields it contains, we can look for theDataEssenceCoding field and check if the value matches 060E2B34.04010105.0E090606.00000000.

The test is constructed with the t.Test() function, which describes; the test being run, a specification field and a series of assertions that the tests run.

The assertions are the comparisons we make when testing and for this library, they are required to be gomega assertions, otherwise the tests will panic and fail :( .

When writing the assertions we use the following phrasing to help build the tests. For the first test we use:

  • We expect t.Expect(isxdDesc) that the ISXD descriptor shall exist ShallNot(BeNil()) - t.Expect(isxdDesc).ShallNot(BeNil())

Then once we have validated the ISXD descriptor exists we use:

  • We expect t.Expect(err) that there shall be no error decoding the descriptor Shall(BeNil()) - t.Expect(err).Shall(BeNil())
  • We expect t.Expect(isxdDecode["DataEssenceCoding"]) that the descriptor field of DataEssenceCoding shall equal 060E2B34.04010105.0E090606.00000000 Shall(Equal(mxf2go.TAUID{...} - t.Expect(isxdDecode["DataEssenceCoding"]).Shall(Equal(mxf2go.TAUID{...})))

Now we have all the ingredients we can put them together into one test function.

func testISXDDescriptor(_ io.ReadSeeker, header *mxftest.PartitionNode) func(t mxftest.Test) {
  return func(t mxftest.Test) {

    // rdd-47:2009/11.5.3/shall/4
    t.Test("Checking that the ISXD descriptor is present in the header metadata", mxftest.NewSpecificationDetails(ISXDDoc, "9.2", "shall", 1),
      t.Expect(isxdDesc).ShallNot(BeNil()),
    )

    if isxdDesc != nil {
      // decode the group
      isxdDecode, err := mxftest.DecodeGroupNode(doc, isxdDesc, primer)

      t.Test("Checking that the data essence coding field is present in the ISXD descriptor", mxftest.NewSpecificationDetails(ISXDDoc, "9.3", "shall", 1),
        t.Expect(err).Shall(BeNil()),
        t.Expect(isxdDecode["DataEssenceCoding"]).Shall(Equal(mxf2go.TAUID{
          Data1: 101591860,
          Data2: 1025,
          Data3: 261,
          Data4: mxf2go.TUInt8Array8{14, 9, 6, 6, 0, 0, 0, 0},
        })))
    }
  }
}

The mxf2go.TAUID{...} value translates to 060E2B34.04010105.0E090606.00000000and is used because the DataEssenceCoding is declared as a AUID in the SMPTE registers.

The test is integrated into the test specification with the following code:

mxftest.WithNodeTests(
  mxftest.NodeTest{UL: mxf2go.GISXDUL[13:], Test: testISXDDescriptor},
)

Where the UL is the ul of the node being tested, in this case it's 060E2B34.02530105.0E090502.00000000, which we found the using the mxf2go library.

Congratulations you have, written your first MXF node test, you have:

  • Decoded a node
  • Constructed a node test
  • Added it to a specification
Specification Tags

Tags are test functions, but for identifying if the MXF file to be tested is an intended target of the specification tests or not. Tags are run before the tests, if the tags fail then those specification tests do not run on the mxf. E.g. The ISXD tags do not pass when run on an MXF file that only contains audio, because this file is not a valid ISXD file.

These tests have the same format as the tests described earlier in the demos, so all these demos are applicable for writing tags.

These are declared in the specification with the following commands.

An example tag used for ISXD is

func ISXDNodeTag(doc io.ReadSeeker, isxdDesc *mxftest.Node, primer map[string]string) func(t mxftest.Test) {

  return func(t mxftest.Test) {
    // all thats needed for isxd is a descriptor
    t.Test("Checking that the ISXD descriptor is present in the header metadata", mxftest.NewSpecificationDetails(ISXDDoc, "9.2", "shall", 1),
      t.Expect(isxdDesc).ShallNot(BeNil()),
    )
  }
}

This code checks if the ISXD descriptor is present, which signals the existence of ISXD data in a file.

It is added to the specification with the following code.

mxftest.WithNodeTags(mxftest.NodeTest{UL: mxf2go.GISXDUL[13:], Test: ISXDNodeTag})
Data Sniffing

As part of the test suite sniff tests are included. These sniff tests take a quick look at the data and glean what they can. With data type (e.g. xml, json etc) being checked first, then any further checks related to that data type being carried out.

These results are then stored in the with the node, and can be searched as part of the tests on that node.

The only field that is reserved for the sniff test is "ContentType", which stores the content type of the data that has been sniffed.

Sniffer functions are broken down into two types:

  • Data identifiers - These are for finding out the content type of data, there's one of these per data type
  • Data sniffers - These find out information about the known data type, several sniffers can be run per data identifier.
Data Identifiers

Firstly data identifiers have a function that passes if the data is valid or not. e.g. a Json identifier would look like this, where all it does is check that the byte stream is valid JSON

func ValidJSON(dataStream []byte) bool {
    var js json.RawMessage

    return json.Unmarshal(dataStream, &js) == nil
}

Any file that does not contain json will be caught by the json unmarshaller, and the json.RawMessage object takes the least amount of computational effort to write to. So we can effectively take a glance at the data, without any further strain on the rest of the tests.

Which would then be wrapped up as a dataIdentifier in the following code.

const (
 // MIME is the xml mimetype
 Content mxftest.CType = "application/json"
)

// DataIdentifier is the json identifier function
var DataIdentifier = mxftest.DataIdentifier{DataFunc: ValidJson, ContentType: Content}
Data Sniffers

A Sniffer function the does something to the data, to try and glean some extra information about it.

For example a simple version of the mxftest/xmlhandle path sniffer function looks like.


// PathSniffer searches an XML document for that path
// and stores the key value of the Node
func PathSniffer(sc mxftest.SniffContext, path string) mxftest.Sniffer {

  pathKey := contKey{path: path, functionName: "the path sniffer function using xpath"}
  sniffFunc := sc.GetData(pathKey)

  if sniffFunc != nil {
    return sniffFunc.(mxftest.Sniffer)
  }

  var xmlSniff mxftest.Sniffer



  mid := func(data []byte) mxftest.SniffResult {
    doc, _ := xmlquery.Parse(bytes.NewBuffer(data))
    // this means find the root of ttml is tt
    out := xmlquery.FindOne(doc, path)

    if out == nil {

      return mxftest.SniffResult{}
    }

    return mxftest.SniffResult{Key: path, Field: out.Data, Certainty: 100}
  }

  xmlSniff = &mid
  sc.CacheData(pathKey, xmlSniff)

  return xmlSniff
}

The function first checks if the sniff test has been built, returning the cached value if it already has. This is to stop sniff tests being repeated, otherwise the same test could be run on the data to no ones benefit. Next if the function is new, it builds a functions that parses the data to a xpath library, which searches runs the search term. This function is then returned to be run on the data.

Things to add

This repo is a work in progress, so there are still some features to add, including the following:

  • Improved sql search style functionality

Think we've missed something to add? Please create an issue or pull request with the proposed update.

Documentation

Overview

package mxftest contains the test functions and interfaces for testing mxf/mrx files against their specifications. And for developing new tests for testing the mxf files.

Index

Constants

View Source 
const (
	// keys for identifying the type of partition.
	HeaderPartition        = "header"
	BodyPartition          = "body"
	GenericStreamPartition = "genericstreampartition"
	FooterPartition        = "footer"
	RIPPartition           = "rip"
)
View Source 
const (
	// ContentTypeKey is the key used for storing the ContentType of data from a sniff test.
	// It can be used with the node search functions
	ContentTypeKey = "ContentType"
)

Variables

This section is empty.

Functions

func DecodeGroup 

func DecodeGroup(group *klv.KLV, primer map[string]string) (map[string]any, error)

DecodeGroup decodes a group KLV into a map[string]any, where the key of the map is the name of the field and the any is the decoded value. Any unknown fields will not be decoded and are skipped from the returned values.

The primer is a map of map[shorthandKey]fullUL

func DecodeGroupNode 

func DecodeGroupNode(doc io.ReadSeeker, node *Node, primer map[string]string) (map[string]any, error)

DecodeGroupNode decodes a Node into a map[string]any, where the key of the map is the name of the field and the any is the decoded value. Any unknown fields will not be decoded and are skipped from the returned values.

The primer is a map of map[shorthandKey]fullUL

func FullNameMask 

func FullNameMask(key []byte, maskedBytes ...int) string

FullNameMask converts an array of 16 bytes into the universal label format of "%02x%02x%02x%02x.%02x%02x%02x%02x.%02x%02x%02x%02x.%02x%02x%02x%02x"

Any masked bytes are masked as 7f, and the maskbyte value is the integer position in the byte array (starting at 0).

func MRXTest 

func MRXTest(doc io.ReadSeeker, w io.Writer, testspecs ...Specifications) error

MRXTest tests an MRX file against the specifications given to it, if no specifications are passed then no tests are run. These test results are then logged as an yaml file to the io.Writer.

func NodeToKLV 

func NodeToKLV(stream io.ReadSeeker, node *Node) (*klv.KLV, error)

NodeToKLV converts a node to a KLV object

func ReferenceExtract 

func ReferenceExtract(field any, reftype Ref) [][]byte

ReferenceExtract extracts all references of a given type from a field. It returns each reference as the bytes. It looks at the type of the field and checks its name for the <reftype>Reference fields, if the field is not a reference object, then no references are returned.

This is for use with the github.com/metarex-media/mxf-to-go repository as it contains a large number of variables that are references or contain nested references. A switch statement would be unfeasible for every type and this is currently the next best thing.

func Sniff 

func Sniff(data []byte, sniffers map[*DataIdentifier][]Sniffer) map[string]*SniffResult

Sniff checks a stream of bytes to find the data type. it then performs any further sniff tests based on the type of data it finds, if it finds any data types that match. It stops searching after finding a data type that matches.

func WithNodeTags 

func WithNodeTags(nodeTags ...NodeTest) func(s *Specifications)

WithNodeTags adds the Node tests to the specifications object

func WithNodeTests 

func WithNodeTests(nodeTests ...NodeTest) func(s *Specifications)

WithNodeTests adds the Node tests to the specifications object

func WithPartitionTags 

func WithPartitionTags(partitionTags ...PartitionTest) func(s *Specifications)

WithPartitionTags adds the partition tests to the specification Tag

func WithPartitionTests 

func WithPartitionTests(partitionTests ...PartitionTest) func(s *Specifications)

WithPartitionTests adds the partition tests to the specification

func WithSniffTest 

func WithSniffTest(sniffTest SniffTest) func(s *Specifications)

WithSniffTest adds the SniffTest for the specification. That is the test that is run on any data found within the file.

func WithStructureTag 

func WithStructureTag(structureTags ...func(doc io.ReadSeeker, mxf *MXFNode) func(t Test)) func(s *Specifications)

WithStructureTag adds the tests to the specification tag.

func WithStructureTests 

func WithStructureTests(structureTests ...func(doc io.ReadSeeker, mxf *MXFNode) func(t Test)) func(s *Specifications)

WithStructureTests adds the structure tests to the specification. It does not check for repeats so make sure you do not repeat tests.

Types

type Assertions 

type Assertions interface {
	Shall(matcher types.GomegaMatcher, optionalDescription ...interface{}) bool
	ShallNot(matcher types.GomegaMatcher, optionalDescription ...interface{}) bool
	types.Assertion
}

Assertions wraps the gomega types assertions with the additional Shall and ShallNot assertion

type CType 

type CType string

CType is used for declaring content types of a data type. These are a separate type to make them easy to identify with autocomplete etc.

type DataIdentifier 

type DataIdentifier struct {
	DataFunc    func([]byte) bool
	ContentType CType
}

DataIdentifier contains the data type and function for checking if the data stream is that type.

type EssenceProperties 

type EssenceProperties struct {
	EssUL string
}

EssenceProperties contains the properties of an essence object

func (EssenceProperties) ID 

func (e EssenceProperties) ID() string

ID returns the of the essence, it always returns ""

func (EssenceProperties) Label 

func (e EssenceProperties) Label() []string

Label returns the labels associated with the essence. it always returns []string{"essence"}

func (EssenceProperties) UL 

func (e EssenceProperties) UL() string

UL returns the Universal Label of the essence

type Expecter 

type Expecter interface {
	Expect(actual interface{}, extra ...interface{}) Assertions
}

Expecter is a workaround to wrap the gomega/internal expect object

type GroupProperties 

type GroupProperties struct {
	UUID           mxf2go.TUUID
	UniversalLabel string
	GroupLabel     []string
}

GroupProperties contains the properties of an group object

func (GroupProperties) ID 

func (gp GroupProperties) ID() string

ID returns the of the group, formatted as "00000000.00000000.00000000.00000000"

func (GroupProperties) Label 

func (gp GroupProperties) Label() []string

Label returns an labels associated with a group

func (GroupProperties) UL 

func (gp GroupProperties) UL() string

UL returns the Universal Label of the group

type MXFAssertions 

type MXFAssertions struct {
	// contains filtered or unexported fields
}

MXFAssertions wraps the basic types.assertions with some extra names to allow the MXf specification to be written as tests.

It satisfies the Assertions methods.

func (MXFAssertions) Error 

func (e MXFAssertions) Error() types.Assertion

func (MXFAssertions) NotTo 

func (e MXFAssertions) NotTo(matcher types.GomegaMatcher, optionalDescription ...interface{}) bool

func (MXFAssertions) Shall 

func (e MXFAssertions) Shall(matcher types.GomegaMatcher, optionalDescription ...interface{}) bool

Shall wraps the To assertion and behaves in the same way

func (MXFAssertions) ShallNot 

func (e MXFAssertions) ShallNot(matcher types.GomegaMatcher, optionalDescription ...interface{}) bool

ShallNot wraps the ToNot assertion and behaves in the same way

func (MXFAssertions) Should 

func (e MXFAssertions) Should(matcher types.GomegaMatcher, optionalDescription ...interface{}) bool

func (MXFAssertions) ShouldNot 

func (e MXFAssertions) ShouldNot(matcher types.GomegaMatcher, optionalDescription ...interface{}) bool

func (MXFAssertions) To 

func (e MXFAssertions) To(matcher types.GomegaMatcher, optionalDescription ...interface{}) bool

func (MXFAssertions) ToNot 

func (e MXFAssertions) ToNot(matcher types.GomegaMatcher, optionalDescription ...interface{}) bool

func (MXFAssertions) WithOffset 

func (e MXFAssertions) WithOffset(offset int) types.Assertion

type MXFNode 

type MXFNode struct {
	Partitions []*PartitionNode
	Tests      tests[MXFNode]
	// contains filtered or unexported fields
}

MXFNode is the parent node of the MXF file. it contains its partitions as children and the list of tests to run on the node.

func MakeAST 

func MakeAST(stream io.Reader, buffer chan *klv.KLV, size int, specs Specifications) (*MXFNode, error)

Make AST generates an Abstract Syntax Tree (AST) of an MXF file.

As part of the AST tests are assigned to the nodes in the tree, these tests are declared as specifications.

func (*MXFNode) FlagFail 

func (m *MXFNode) FlagFail()

Flag fail sets the test pass to fail

func (MXFNode) Search 

func (m MXFNode) Search(searchfield string) ([]*PartitionNode, error)

Search follows SQL for finding things within a partition e.g. select * from essence where partition <> header

Available tables are:

  • partition

Available fields are:

  • essence - the count of essence
  • type - the partition types
  • metadata - the count of metadata

Available operators are:

  • AND - all conditions have to be true

The search command is not case sensitive

type MXFProperty 

type MXFProperty interface {
	// symbol returns the MXF UL associated with the node.
	// if there is one
	UL() string
	// ID returns the ID associated with the property
	ID() string
	// Returns the type of that node
	// e.g. essence, partition or the group type like Descriptivemetadata
	Label() []string
}

MXFProperty contains the properties of and MXF object

type Node 

type Node struct {
	Key, Length, Value Position
	Properties         MXFProperty
	// talk through the children role with Bruce
	// but keep as this
	Tests tests[Node]

	Children []*Node
	Sniffs   map[string]*SniffResult `yaml:"-"`
	// contains filtered or unexported fields
}

Node is a object in the abstact syntax tree it can be a child, a parent or both.

func (*Node) FlagFail 

func (n *Node) FlagFail()

Flag fail sets the test pass to fail then calls the same on its parent.

func (Node) Search 

func (n Node) Search(searchfield string) ([]*Node, error)

Search follows SQL syntax for nodes within a nodes children

e.g. select * where UL = 060e2b34.027f0101.0d010101.01010f00

"from" is not used as there are no tables within a node, the node is a table unto itself.

Available fields are:

  • ul
  • sniff:{field name} - e.g. sniff:/root searches the sniff value of root

The search command is not case sensitive

type NodeTest 

type NodeTest struct {
	// UL is the Universal Label of the Node being targeted.
	// e.g. 060e2b34.02530105.0e090502.00000000
	// All Universal labels take this form
	UL string
	// The test function
	Test func(doc io.ReadSeeker, node *Node, primer map[string]string) func(t Test)
}

NodeTest contains the key and a test for a partition test

type Nodes 

type Nodes interface {
	Node | PartitionNode | MXFNode
}

Nodes are the different nodes in the Abstract syntax tree

type Parent 

type Parent interface {
	FlagFail() // a function that recursively calls the parents when a test is failed
}

Parent is for declaring the parent of a node without out giving full control of that Node.

type Partition 

type Partition struct {
	Signature         string // Must be, hex: 06 0E 2B 34
	PartitionLength   int    // All but first block size
	MajorVersion      uint16 // Must be, hex: 01 00
	MinorVersion      uint16
	SizeKAG           uint32
	ThisPartition     uint64
	PreviousPartition uint64
	FooterPartition   uint64 // First block size
	HeaderByteCount   uint64
	IndexByteCount    uint64
	IndexSID          uint32
	BodyOffset        uint64
	BodySID           uint32

	// useful information from the partition
	PartitionType     string
	IndexTable        bool
	TotalHeaderLength int
	MetadataStart     int
}

Partition is the layout of an mxf partition with type accurate fields (or as close as possible)

func PartitionExtract 

func PartitionExtract(partitionKLV *klv.KLV) Partition

PartitionExtract extracts the partition from a KLV packet

type PartitionNode 

type PartitionNode struct {
	Parent             *MXFNode `yaml:"-"`
	Key, Length, Value Position
	HeaderMetadata     []*Node
	Essence            []*Node
	IndexTable         *Node
	Props              PartitionProperties
	Tests              tests[PartitionNode]

	PartitionPos int
	// contains filtered or unexported fields
}

PartitionNode is the node for every MXF partition/ It contains the different types of content as different arrays of nodes.

func (*PartitionNode) FlagFail 

func (p *PartitionNode) FlagFail()

Flag fail sets the test pass to fail then calls the same on its parent.

func (PartitionNode) Search 

func (p PartitionNode) Search(searchfield string) ([]*Node, error)

Search follows SQL syntax for nodes within a partition

e.g. select * from essence where UL <> 060e2b34.01020105.0e090502.017f017f

Available tables are:

  • essence
  • metadata

Available fields are:

  • ul
  • sniff:{field name} - e.g. sniff:/root searches the sniff value of root

The search command is not case sensitive

type PartitionProperties 

type PartitionProperties struct {
	PartitionCount int // the count of the partition along the MXF
	PartitionType  string
	Primer         map[string]string
	EssenceOrder   []string
}

PartitionProperties contains the properties of a partition object

func (PartitionProperties) ID 

func (p PartitionProperties) ID() string

ID returns the ID associated with a partition, it always returns ""

func (PartitionProperties) Label 

func (p PartitionProperties) Label() []string

Label returns the labels associated with the partition. it always returns []string{"partition"}

func (PartitionProperties) Symbol 

func (p PartitionProperties) Symbol() string

Symbol returns the type of the partition

type PartitionTest 

type PartitionTest struct {
	//
	PartitionType PartitionType
	// The test function
	Test func(doc io.ReadSeeker, partition *PartitionNode) func(t Test)
}

PartitionTest contains the partition type and the test for a partition test. The partition types identifies the type of partition this test is targeting.

type PartitionType 

type PartitionType string

PartitionType is a string value of the partition type 

const (
	// The key used for identifying the header partition
	// it also flags footer partitions.
	Header PartitionType = "header"
	// the key for using identifying essence streams
	Body PartitionType = "essence"
	// The key used for identifying the Generic Body partition
	GenericBody PartitionType = "generickey"
)

type Position 

type Position struct {
	Start, End int
}

Position contains the start and end position of the Node in the byte stream.

type RIP 

type RIP struct {
	Sid        uint32
	ByteOffset uint64
}

RIP is the random index position struct

type Ref 

type Ref string

Ref is the type for identifying reference types 

const (
	// StrongRef is the reference type for strong references
	StrongRef Ref = "StrongReference"
	// WeakRef is the reference type for weak references
	WeakRef Ref = "WeakReference"
)

type Report 

type Report struct {
	// Did the overall test pass
	TestPass bool
	// the tests and their results
	Tests        []TestSection
	SkippedTests []skippedTest `yaml:"skippedTests,omitempty"`
}

Report is the report structure of the MXF test report

type SniffContext 

type SniffContext struct {
	// contains filtered or unexported fields
}

SniffContext is an array of bytes for preventing multiple sniff functions

func NewSniffContext 

func NewSniffContext() SniffContext

func (*SniffContext) CacheData 

func (s *SniffContext) CacheData(key, data any)

CacheData, caches a item in the sniff context, it can be retrieved with GetData()

func (*SniffContext) GetData 

func (s *SniffContext) GetData(key any) any

GetData returns the the data for a sniff context if no data is present a nil object is returned.

type SniffResult 

type SniffResult struct {
	// The sniff test key
	Key string
	// The sniff test result field
	Field string
	// The data of the sniff test
	Data CType
	// what certainty did the sniff test past as a %
	Certainty float64
}

SniffResult is the result of the sniff test.

type SniffTest 

type SniffTest struct {
	DataID DataIdentifier
	Sniffs []Sniffer
}

SniffTest contains an identifier and the tests to run on any data that is identified.

type Sniffer 

type Sniffer *func(data []byte) SniffResult

Sniffer takes a stream of bytes, sniffs it (a quick look at the data) then returns a result of the sniff.

type SnifferUpdate 

type SnifferUpdate struct {
	Sniff           func(data []byte) SniffResult
	SniffProperties props
}

Sniffer takes a stream of bytes, sniffs it (a quick look at the data) then returns a result of the sniff.

type SpecificationDetails 

type SpecificationDetails struct {
	DocName, Section, Command string
	CommandCount              int
}

SpecificationDetails contains the information about the specification that made the test. It can be written with %s formatting

func NewSpecificationDetails 

func NewSpecificationDetails(docName, section, command string, commandCount int) SpecificationDetails

NewSpecificationDetails generates a new specificationDetails struct

func (SpecificationDetails) String 

func (s SpecificationDetails) String() string

String allows spec details to be written as a shorthand string

type Specifications 

type Specifications struct {
	// node specifications for groups, map is UL node test
	Node map[string][]markedTestWithPrimer[Node]

	// test aprtitions the partition tyoe is the map key
	Part map[string][]markedTest[PartitionNode]

	// array of mxf structural tests
	MXF []markedTest[MXFNode]

	// Sniff Tests to check the data
	SniffTests SniffTest
	// contains filtered or unexported fields
}

Specifications contains all the information for a test specification.

It contains:

  • The tests for each MXF Node
  • Tags to ensure the specification runs on the correct files
  • Data Sniff tests

These are all optional fields, a specification utilises as many or as few fields as required to validate the file.

func NewSpecification 

func NewSpecification(options ...func(*Specifications)) *Specifications

NewSpecification generates a new Specifications object. It is tailored to the options provided to generate custom specifications and the order in which the options are specified are the order in which the executed. If no options are provided and empty specifications object is returned. An empty specification is still a valid specification

type Test 

type Test interface {
	Tester
	Expecter
	// contains filtered or unexported methods
}

Test interface is the MXF test parameters

type TestContext 

type TestContext struct {
	// contains filtered or unexported fields
}

TestContext is the global context for all the MRX tests.

func NewTestContext 

func NewTestContext(dest io.Writer) *TestContext

NewTestContext generates a new testContext that writes to des.

ensure EndTest() is called to flush the results to the writer.

func (*TestContext) EndTest 

func (tc *TestContext) EndTest() error

EndTest flushes the tests as a complete yaml. End Test must be called to write the results to the io.Writer

func (*TestContext) Header 

func (s *TestContext) Header(message string, tests func(t Test))

Header is a wrapper for the tests, adding more context to the results, and then running the tests.

func (*TestContext) RegisterSkippedTest 

func (tc *TestContext) RegisterSkippedTest(key, desc string)

RegisterSkippedTest adds a skipped test to the test report.

type TestResult 

type TestResult struct {
	Message string
	Checks  []check
}

TestResult is the result of a test run

type TestSection 

type TestSection struct {
	// the header message for that batch of tests
	Header string
	// the tests themselves
	Tests []TestResult
	// the results
	Pass                 bool
	PassCount, FailCount int
}

TestSection contains the information for a batch of tests

type Tester 

type Tester interface {
	Test(message string, specDetail SpecificationDetails, asserts ...bool)
}

Tester is a workaround to wrap the gomega/internal test object

Source Files

View all Source files

Directories

Path Synopsis
package example contains example implementations of the mxftest repo
 package example contains example implementations of the mxftest repo
xmlhandle contains the xml data interactions, for using in sniff tests of MXF data.
 xmlhandle contains the xml data interactions, for using in sniff tests of MXF data.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.