mxj

package module
v2.8.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 20, 2023 License: MIT Imports: 14 Imported by: 0

README

mxj - to/from maps, XML and JSON

Decode/encode XML to/from map[string]interface{} (or JSON) values, and extract/modify values from maps by key or key-path, including wildcards.

mxj supplants the legacy x2j and j2x packages. If you want the old syntax, use mxj/x2j and mxj/j2x packages.

Installation

Using go.mod:
go get github.com/clbanning/mxj/v2@v2.7	
import "github.com/clbanning/mxj/v2"

... or just vendor the package.

Related Packages

https://github.com/clbanning/checkxml provides functions for validating XML data.

Refactor Encoder - 2020.05.01

Issue #70 highlighted that encoding large maps does not scale well, since the original logic used string appends operations. Using bytes.Buffer results in linear scaling for very large XML docs. (Metrics based on MacBook Pro i7 w/ 16 GB.)
Nodes      m.XML() time
54809       12.53708ms
109780      32.403183ms
164678      59.826412ms
482598     109.358007ms

Refactor Decoder - 2015.11.15

For over a year I've wanted to refactor the XML-to-map[string]interface{} decoder to make it more performant. I recently took the time to do that, since we were using github.com/clbanning/mxj in a production system that could be deployed on a Raspberry Pi. Now the decoder is comparable to the stdlib JSON-to-map[string]interface{} decoder in terms of its additional processing overhead relative to decoding to a structure value. As shown by:
BenchmarkNewMapXml-4         	  100000	     18043 ns/op
BenchmarkNewStructXml-4      	  100000	     14892 ns/op
BenchmarkNewMapJson-4        	  300000	      4633 ns/op
BenchmarkNewStructJson-4     	  300000	      3427 ns/op
BenchmarkNewMapXmlBooks-4    	   20000	     82850 ns/op
BenchmarkNewStructXmlBooks-4 	   20000	     67822 ns/op
BenchmarkNewMapJsonBooks-4   	  100000	     17222 ns/op
BenchmarkNewStructJsonBooks-4	  100000	     15309 ns/op

Notices

2022.11.28: v2.7 - add SetGlobalKeyMapPrefix to change default prefix, '#', for default keys
2022.11.20: v2.6 - add NewMapForattedXmlSeq for XML docs formatted with whitespace character
2021.02.02: v2.5 - add XmlCheckIsValid toggle to force checking that the encoded XML is valid
2020.12.14: v2.4 - add XMLEscapeCharsDecoder to preserve XML escaped characters in Map values
2020.10.28: v2.3 - add TrimWhiteSpace option
2020.05.01: v2.2 - optimize map to XML encoding for large XML docs.
2019.07.04: v2.0 - remove unnecessary methods - mv.XmlWriterRaw, mv.XmlIndentWriterRaw - for Map and MapSeq.
2019.07.04: Add MapSeq type and move associated functions and methods from Map to MapSeq.
2019.01.21: DecodeSimpleValuesAsMap - decode to map[<tag>:map["#text":<value>]] rather than map[<tag>:<value>]
2018.04.18: mv.Xml/mv.XmlIndent encodes non-map[string]interface{} map values - map[string]string, map[int]uint, etc.
2018.03.29: mv.Gob/NewMapGob support gob encoding/decoding of Maps.
2018.03.26: Added mxj/x2j-wrapper sub-package for migrating from legacy x2j package.
2017.02.22: LeafNode paths can use ".N" syntax rather than "[N]" for list member indexing.
2017.02.10: SetFieldSeparator changes field separator for args in UpdateValuesForPath, ValuesFor... methods.
2017.02.06: Support XMPP stream processing - HandleXMPPStreamTag().
2016.11.07: Preserve name space prefix syntax in XmlSeq parser - NewMapXmlSeq(), etc.
2016.06.25: Support overriding default XML attribute prefix, "-", in Map keys - SetAttrPrefix().
2016.05.26: Support customization of xml.Decoder by exposing CustomDecoder variable.
2016.03.19: Escape invalid chars when encoding XML attribute and element values - XMLEscapeChars().
2016.03.02: By default decoding XML with float64 and bool value casting will not cast "NaN", "Inf", and "-Inf".
            To cast them to float64, first set flag with CastNanInf(true).
2016.02.22: New mv.Root(), mv.Elements(), mv.Attributes methods let you examine XML document structure.
2016.02.16: Add CoerceKeysToLower() option to handle tags with mixed capitalization.
2016.02.12: Seek for first xml.StartElement token; only return error if io.EOF is reached first (handles BOM).
2015.12.02: XML decoding/encoding that preserves original structure of document. See NewMapXmlSeq()
            and mv.XmlSeq() / mv.XmlSeqIndent().
2015-05-20: New: mv.StringIndentNoTypeInfo().
            Also, alphabetically sort map[string]interface{} values by key to prettify output for mv.Xml(),
            mv.XmlIndent(), mv.StringIndent(), mv.StringIndentNoTypeInfo().
2014-11-09: IncludeTagSeqNum() adds "_seq" key with XML doc positional information.
            (NOTE: PreserveXmlList() is similar and will be here soon.)
2014-09-18: inspired by NYTimes fork, added PrependAttrWithHyphen() to allow stripping hyphen from attribute tag.
2014-08-02: AnyXml() and AnyXmlIndent() will try to marshal arbitrary values to XML.
2014-04-28: ValuesForPath() and NewMap() now accept path with indexed array references.

Basic Unmarshal XML to map[string]interface{}

type Map map[string]interface{}

Create a Map value, 'mv', from any map[string]interface{} value, 'v':

mv := Map(v)

Unmarshal / marshal XML as a Map value, 'mv':

mv, err := NewMapXml(xmlValue) // unmarshal
xmlValue, err := mv.Xml()      // marshal

Unmarshal XML from an io.Reader as a Map value, 'mv':

mv, err := NewMapXmlReader(xmlReader)         // repeated calls, as with an os.File Reader, will process stream
mv, raw, err := NewMapXmlReaderRaw(xmlReader) // 'raw' is the raw XML that was decoded

Marshal Map value, 'mv', to an XML Writer (io.Writer):

err := mv.XmlWriter(xmlWriter)
raw, err := mv.XmlWriterRaw(xmlWriter) // 'raw' is the raw XML that was written on xmlWriter

Also, for prettified output:

xmlValue, err := mv.XmlIndent(prefix, indent, ...)
err := mv.XmlIndentWriter(xmlWriter, prefix, indent, ...)
raw, err := mv.XmlIndentWriterRaw(xmlWriter, prefix, indent, ...)

Bulk process XML with error handling (note: handlers must return a boolean value):

err := HandleXmlReader(xmlReader, mapHandler(Map), errHandler(error))
err := HandleXmlReaderRaw(xmlReader, mapHandler(Map, []byte), errHandler(error, []byte))

Converting XML to JSON: see Examples for NewMapXml and HandleXmlReader.

There are comparable functions and methods for JSON processing.

Arbitrary structure values can be decoded to / encoded from Map values:

mv, err := NewMapStruct(structVal)
err := mv.Struct(structPointer)

Extract / modify Map values

To work with XML tag values, JSON or Map key values or structure field values, decode the XML, JSON or structure to a `Map` value, 'mv', or cast a `map[string]interface{}` value to a `Map` value, 'mv', then:
paths := mv.PathsForKey(key)
path := mv.PathForKeyShortest(key)
values, err := mv.ValuesForKey(key, subkeys)
values, err := mv.ValuesForPath(path, subkeys)
count, err := mv.UpdateValuesForPath(newVal, path, subkeys)

Get everything at once, irrespective of path depth:

leafnodes := mv.LeafNodes()
leafvalues := mv.LeafValues()

A new Map with whatever keys are desired can be created from the current Map and then encoded in XML or JSON. (Note: keys can use dot-notation.)

newMap, err := mv.NewMap("oldKey_1:newKey_1", "oldKey_2:newKey_2", ..., "oldKey_N:newKey_N")
newMap, err := mv.NewMap("oldKey1", "oldKey3", "oldKey5") // a subset of 'mv'; see "examples/partial.go"
newXml, err := newMap.Xml()   // for example
newJson, err := newMap.Json() // ditto

Usage

The package is fairly well self-documented with examples.

Also, the subdirectory "examples" contains a wide range of examples, several taken from golang-nuts discussions.

XML parsing conventions

Using NewMapXml()

  • Attributes are parsed to map[string]interface{} values by prefixing a hyphen, -, to the attribute label. (Unless overridden by PrependAttrWithHyphen(false) or SetAttrPrefix().)
  • If the element is a simple element and has attributes, the element value is given the key #text for its map[string]interface{} representation. (See the 'atomFeedString.xml' test data, below.)
  • XML comments, directives, and process instructions are ignored.
  • If CoerceKeysToLower() has been called, then the resultant keys will be lower case.

Using NewMapXmlSeq()

  • Attributes are parsed to map["#attr"]map[<attr_label>]map[string]interface{}values where the <attr_label> value has "#text" and "#seq" keys - the "#text" key holds the value for <attr_label>.
  • All elements, except for the root, have a "#seq" key.
  • Comments, directives, and process instructions are unmarshalled into the Map using the keys "#comment", "#directive", and "#procinst", respectively. (See documentation for more specifics.)
  • Name space syntax is preserved:
    • <ns:key>something</ns.key> parses to map["ns:key"]interface{}{"something"}
    • xmlns:ns="http://myns.com/ns" parses to map["xmlns:ns"]interface{}{"http://myns.com/ns"}

Both

  • By default, "Nan", "Inf", and "-Inf" values are not cast to float64. If you want them to be cast, set a flag to cast them using CastNanInf(true).

XML encoding conventions

  • 'nil' Map values, which may represent 'null' JSON values, are encoded as <tag/>. NOTE: the operation is not symmetric as <tag/> elements are decoded as tag:"" Map values, which, then, encode in JSON as "tag":"" values.
  • ALSO: there is no guarantee that the encoded XML doc will be the same as the decoded one. (Go randomizes the walk through map[string]interface{} values.) If you plan to re-encode the Map value to XML and want the same sequencing of elements look at NewMapXmlSeq() and mv.XmlSeq() - these try to preserve the element sequencing but with added complexity when working with the Map representation.

Running "go test"

Because there are no guarantees on the sequence map elements are retrieved, the tests have been written for visual verification in most cases. One advantage is that you can easily use the output from running "go test" as examples of calling the various functions and methods.

Motivation

I make extensive use of JSON for messaging and typically unmarshal the messages into map[string]interface{} values. This is easily done using json.Unmarshal from the standard Go libraries. Unfortunately, many legacy solutions use structured XML messages; in those environments the applications would have to be refactored to interoperate with my components.

The better solution is to just provide an alternative HTTP handler that receives XML messages and parses it into a map[string]interface{} value and then reuse all the JSON-based code. The Go xml.Unmarshal() function does not provide the same option of unmarshaling XML messages into map[string]interface{} values. So I wrote a couple of small functions to fill this gap and released them as the x2j package.

Over the next year and a half additional features were added, and the companion j2x package was released to address XML encoding of arbitrary JSON and map[string]interface{} values. As part of a refactoring of our production system and looking at how we had been using the x2j and j2x packages we found that we rarely performed direct XML-to-JSON or JSON-to_XML conversion and that working with the XML or JSON as map[string]interface{} values was the primary value. Thus, everything was refactored into the mxj package.

Documentation

Overview

Marshal/Unmarshal XML to/from map[string]interface{} values (and JSON); extract/modify values from maps by key or key-path, including wildcards.

mxj supplants the legacy x2j and j2x packages. The subpackage x2j-wrapper is provided to facilitate migrating from the x2j package. The x2j and j2x subpackages provide similar functionality of the old packages but are not function-name compatible with them.

Note: this library was designed for processing ad hoc anonymous messages. Bulk processing large data sets may be much more efficiently performed using the encoding/xml or encoding/json packages from Go's standard library directly.

Related Packages:

checkxml: github.com/clbanning/checkxml provides functions for validating XML data.

Notes:

2022.11.28: v2.7 - add SetGlobalKeyMapPrefix to change default prefix, '#', for default keys
2022.11.20: v2.6 - add NewMapForattedXmlSeq for XML docs formatted with whitespace character
2021.02.02: v2.5 - add XmlCheckIsValid toggle to force checking that the encoded XML is valid
2020.12.14: v2.4 - add XMLEscapeCharsDecoder to preserve XML escaped characters in Map values
2020.10.28: v2.3 - add TrimWhiteSpace option
2020.05.01: v2.2 - optimize map to XML encoding for large XML docs.
2019.07.04: v2.0 - remove unnecessary methods - mv.XmlWriterRaw, mv.XmlIndentWriterRaw - for Map and MapSeq.
2019.07.04: Add MapSeq type and move associated functions and methods from Map to MapSeq.
2019.01.21: DecodeSimpleValuesAsMap - decode to map[<tag>:map["#text":<value>]] rather than map[<tag>:<value>].
2018.04.18: mv.Xml/mv.XmlIndent encodes non-map[string]interface{} map values - map[string]string, map[int]uint, etc.
2018.03.29: mv.Gob/NewMapGob support gob encoding/decoding of Maps.
2018.03.26: Added mxj/x2j-wrapper sub-package for migrating from legacy x2j package.
2017.02.22: LeafNode paths can use ".N" syntax rather than "[N]" for list member indexing.
2017.02.21: github.com/clbanning/checkxml provides functions for validating XML data.
2017.02.10: SetFieldSeparator changes field separator for args in UpdateValuesForPath, ValuesFor... methods.
2017.02.06: Support XMPP stream processing - HandleXMPPStreamTag().
2016.11.07: Preserve name space prefix syntax in XmlSeq parser - NewMapXmlSeq(), etc.
2016.06.25: Support overriding default XML attribute prefix, "-", in Map keys - SetAttrPrefix().
2016.05.26: Support customization of xml.Decoder by exposing CustomDecoder variable.
2016.03.19: Escape invalid chars when encoding XML attribute and element values - XMLEscapeChars().
2016.03.02: By default decoding XML with float64 and bool value casting will not cast "NaN", "Inf", and "-Inf".
            To cast them to float64, first set flag with CastNanInf(true).
2016.02.22: New mv.Root(), mv.Elements(), mv.Attributes methods let you examine XML document structure.
2016.02.16: Add CoerceKeysToLower() option to handle tags with mixed capitalization.
2016.02.12: Seek for first xml.StartElement token; only return error if io.EOF is reached first (handles BOM).
2015-12-02: NewMapXmlSeq() with mv.XmlSeq() & co. will try to preserve structure of XML doc when re-encoding.
2014-08-02: AnyXml() and AnyXmlIndent() will try to marshal arbitrary values to XML.

SUMMARY

type Map map[string]interface{}

Create a Map value, 'mv', from any map[string]interface{} value, 'v':
   mv := Map(v)

Unmarshal / marshal XML as a Map value, 'mv':
   mv, err := NewMapXml(xmlValue) // unmarshal
   xmlValue, err := mv.Xml()      // marshal

Unmarshal XML from an io.Reader as a Map value, 'mv':
   mv, err := NewMapXmlReader(xmlReader)         // repeated calls, as with an os.File Reader, will process stream
   mv, raw, err := NewMapXmlReaderRaw(xmlReader) // 'raw' is the raw XML that was decoded

Marshal Map value, 'mv', to an XML Writer (io.Writer):
   err := mv.XmlWriter(xmlWriter)
   raw, err := mv.XmlWriterRaw(xmlWriter) // 'raw' is the raw XML that was written on xmlWriter

Also, for prettified output:
   xmlValue, err := mv.XmlIndent(prefix, indent, ...)
   err := mv.XmlIndentWriter(xmlWriter, prefix, indent, ...)
   raw, err := mv.XmlIndentWriterRaw(xmlWriter, prefix, indent, ...)

Bulk process XML with error handling (note: handlers must return a boolean value):
   err := HandleXmlReader(xmlReader, mapHandler(Map), errHandler(error))
   err := HandleXmlReaderRaw(xmlReader, mapHandler(Map, []byte), errHandler(error, []byte))

Converting XML to JSON: see Examples for NewMapXml and HandleXmlReader.

There are comparable functions and methods for JSON processing.

Arbitrary structure values can be decoded to / encoded from Map values:
   mv, err := NewMapStruct(structVal)
   err := mv.Struct(structPointer)

To work with XML tag values, JSON or Map key values or structure field values, decode the XML, JSON
or structure to a Map value, 'mv', or cast a map[string]interface{} value to a Map value, 'mv', then:
   paths := mv.PathsForKey(key)
   path := mv.PathForKeyShortest(key)
   values, err := mv.ValuesForKey(key, subkeys)
   values, err := mv.ValuesForPath(path, subkeys) // 'path' can be dot-notation with wildcards and indexed arrays.
   count, err := mv.UpdateValuesForPath(newVal, path, subkeys)

Get everything at once, irrespective of path depth:
   leafnodes := mv.LeafNodes()
   leafvalues := mv.LeafValues()

A new Map with whatever keys are desired can be created from the current Map and then encoded in XML
or JSON. (Note: keys can use dot-notation. 'oldKey' can also use wildcards and indexed arrays.)
   newMap, err := mv.NewMap("oldKey_1:newKey_1", "oldKey_2:newKey_2", ..., "oldKey_N:newKey_N")
   newMap, err := mv.NewMap("oldKey1", "oldKey3", "oldKey5") // a subset of 'mv'; see "examples/partial.go"
   newXml, err := newMap.Xml()   // for example
   newJson, err := newMap.Json() // ditto

XML PARSING CONVENTIONS

Using NewMapXml()

- Attributes are parsed to `map[string]interface{}` values by prefixing a hyphen, `-`,
  to the attribute label. (Unless overridden by `PrependAttrWithHyphen(false)` or
  `SetAttrPrefix()`.)
- If the element is a simple element and has attributes, the element value
  is given the key `#text` for its `map[string]interface{}` representation.  (See
  the 'atomFeedString.xml' test data, below.)
- XML comments, directives, and process instructions are ignored.
- If CoerceKeysToLower() has been called, then the resultant keys will be lower case.

Using NewMapXmlSeq()

- Attributes are parsed to `map["#attr"]map[<attr_label>]map[string]interface{}`values
  where the `<attr_label>` value has "#text" and "#seq" keys - the "#text" key holds the
  value for `<attr_label>`.
- All elements, except for the root, have a "#seq" key.
- Comments, directives, and process instructions are unmarshalled into the Map using the
  keys "#comment", "#directive", and "#procinst", respectively. (See documentation for more
  specifics.)
- Name space syntax is preserved:
   - <ns:key>something</ns.key> parses to map["ns:key"]interface{}{"something"}
   - xmlns:ns="http://myns.com/ns" parses to map["xmlns:ns"]interface{}{"http://myns.com/ns"}

Both

- By default, "Nan", "Inf", and "-Inf" values are not cast to float64.  If you want them
  to be cast, set a flag to cast them  using CastNanInf(true).

XML ENCODING CONVENTIONS

  • 'nil' Map values, which may represent 'null' JSON values, are encoded as "<tag/>". NOTE: the operation is not symmetric as "<tag/>" elements are decoded as 'tag:""' Map values, which, then, encode in JSON as '"tag":""' values..
  • ALSO: there is no guarantee that the encoded XML doc will be the same as the decoded one. (Go randomizes the walk through map[string]interface{} values.) If you plan to re-encode the Map value to XML and want the same sequencing of elements look at NewMapXmlSeq() and mv.XmlSeq() - these try to preserve the element sequencing but with added complexity when working with the Map representation.

Index

Examples

Constants

View Source
const (
	Cast         = true // for clarity - e.g., mxj.NewMapXml(doc, mxj.Cast)
	SafeEncoding = true // ditto - e.g., mv.Json(mxj.SafeEncoding)
)
View Source
const (
	DefaultElementTag = "element"
)
View Source
const (
	DefaultRootTag = "doc"
)
View Source
const (
	NoAttributes = true // suppress LeafNode values that are attributes
)

Variables

View Source
var CustomDecoder *xml.Decoder

CustomDecoder can be used to specify xml.Decoder attribute values, e.g., Strict:false, to be used. By default CustomDecoder is nil. If CustomeDecoder != nil, then mxj.XmlCharsetReader variable is ignored and must be set as part of the CustomDecoder value, if needed.

Usage:
	mxj.CustomDecoder = &xml.Decoder{Strict:false}
View Source
var JsonUseNumber bool

Decode numericvalues as json.Number type Map values - see encoding/json#Number. NOTE: this is for decoding JSON into a Map with NewMapJson(), NewMapJsonReader(), etc.; it does not affect NewMapXml(), etc. The XML encoders mv.Xml() and mv.XmlIndent() do recognize json.Number types; a JSON object can be decoded to a Map with json.Number value types and the resulting Map can be correctly encoded into a XML object.

View Source
var KeyNotExistError = errors.New("Key does not exist")
View Source
var NO_ROOT = NoRoot // maintain backwards compatibility
View Source
var NoRoot = errors.New("no root key")

NoRoot is returned by NewXmlSeq, etc., when a comment, directive or procinstr element is parsed in the XML data stream and the element is not contained in an XML object with a root element.

View Source
var PathNotExistError = errors.New("Path does not exist")
View Source
var XmlCharsetReader func(charset string, input io.Reader) (io.Reader, error)

If XmlCharsetReader != nil, it will be used to decode the XML, if required. Note: if CustomDecoder != nil, then XmlCharsetReader is ignored; set the CustomDecoder attribute instead.

  import (
	     charset "code.google.com/p/go-charset/charset"
	     github.com/clbanning/mxj
	 )
  ...
  mxj.XmlCharsetReader = charset.NewReader
  m, merr := mxj.NewMapXml(xmlValue)

Functions

func AnyXml

func AnyXml(v interface{}, tags ...string) ([]byte, error)

Encode arbitrary value as XML.

Note: unmarshaling the resultant XML may not return the original value, since tag labels may have been injected to create the XML representation of the value.

 Encode an arbitrary JSON object.
	package main

	import (
		"encoding/json"
		"fmt"
		"github.com/aaronschweig/mxj"
	)

	func main() {
		jsondata := []byte(`[
			{ "somekey":"somevalue" },
			"string",
			3.14159265,
			true
		]`)
		var i interface{}
		err := json.Unmarshal(jsondata, &i)
		if err != nil {
			// do something
		}
		x, err := mxj.AnyXmlIndent(i, "", "  ", "mydoc")
		if err != nil {
			// do something else
		}
		fmt.Println(string(x))
	}

	output:
		<mydoc>
		  <somekey>somevalue</somekey>
		  <element>string</element>
		  <element>3.14159265</element>
		  <element>true</element>
		</mydoc>

An extreme example is available in examples/goofy_map.go.

Alternative values for DefaultRootTag and DefaultElementTag can be set as: AnyXml( v, myRootTag, myElementTag).

func AnyXmlIndent

func AnyXmlIndent(v interface{}, prefix, indent string, tags ...string) ([]byte, error)

Encode an arbitrary value as a pretty XML string. Alternative values for DefaultRootTag and DefaultElementTag can be set as: AnyXmlIndent( v, "", " ", myRootTag, myElementTag).

func BeautifyXml

func BeautifyXml(b []byte, prefix, indent string) ([]byte, error)

BeautifyXml (re)formats an XML doc similar to Map.XmlIndent(). It preserves comments, directives and process instructions,

func CastNanInf

func CastNanInf(b ...bool)

Cast "Nan", "Inf", "-Inf" XML values to 'float64'. By default, these values will be decoded as 'string'.

func CastValuesToBool

func CastValuesToBool(b ...bool)

CastValuesToBool can be used to skip casting to bool when "cast" argument is 'true' in NewMapXml, etc. Default is true.

func CastValuesToFloat

func CastValuesToFloat(b ...bool)

CastValuesToFloat can be used to skip casting to float64 when "cast" argument is 'true' in NewMapXml, etc. Default is true.

func CastValuesToInt

func CastValuesToInt(b ...bool)

CastValuesToInt tries to coerce numeric valus to int64 or uint64 instead of the default float64. Repeated calls with no argument will toggle this on/off, or this handling will be set with the value of 'b'.

func CoerceKeysToLower

func CoerceKeysToLower(b ...bool)

Coerce all tag values to keys in lower case. This is useful if you've got sources with variable tag capitalization, and you want to use m.ValuesForKeys(), etc., with the key or path spec in lower case.

CoerceKeysToLower() will toggle the coercion flag true|false - on|off
CoerceKeysToLower(true|false) will set the coercion flag on|off

NOTE: only recognized by NewMapXml, NewMapXmlReader, and NewMapXmlReaderRaw functions as well as
      the associated HandleXmlReader and HandleXmlReaderRaw.

func CoerceKeysToSnakeCase

func CoerceKeysToSnakeCase(b ...bool)

CoerceKeysToSnakeCase changes the default, false, to the specified value, b. Note: the attribute prefix will be a hyphen, '-', or what ever string value has been specified using SetAttrPrefix.

func DecodeSimpleValuesAsMap

func DecodeSimpleValuesAsMap(b ...bool)

DecodeSimpleValuesAsMap forces all values to be decoded as map["#text":<value>]. If called with no argument, the decoding is toggled on/off.

By default the NewMapXml functions decode simple values without attributes as map[<tag>:<value>]. This function causes simple values without attributes to be decoded the same as simple values with attributes - map[<tag>:map["#text":<value>]].

func DisableTrimWhiteSpace

func DisableTrimWhiteSpace(b ...bool)

DisableTrimWhiteSpace set if the white space should be trimmed or not. By default white space is always trimmed. If no argument is provided, trim white space will be disabled.

func HandleJsonReader

func HandleJsonReader(jsonReader io.Reader, mapHandler func(Map) bool, errHandler func(error) bool) error

Bulk process JSON using handlers that process a Map value.

'rdr' is an io.Reader for the JSON (stream).
'mapHandler' is the Map processing handler. Return of 'false' stops io.Reader processing.
'errHandler' is the error processor. Return of 'false' stops io.Reader  processing and returns the error.
Note: mapHandler() and errHandler() calls are blocking, so reading and processing of messages is serialized.
      This means that you can stop reading the file on error or after processing a particular message.
      To have reading and handling run concurrently, pass argument to a go routine in handler and return 'true'.
Example
package main

import ()

func main() {
	/*
		See: bulk_test.go for working example.
		Run "go test" in package directory then scroll back to find output.

		Basic logic for bulk JSON to XML processing is similar to that for
		bulk XML to JSON processing as outlined in the HandleXmlReader example.
		The test case is also a good example.
	*/
}
Output:

func HandleJsonReaderRaw

func HandleJsonReaderRaw(jsonReader io.Reader, mapHandler func(Map, []byte) bool, errHandler func(error, []byte) bool) error

Bulk process JSON using handlers that process a Map value and the raw JSON.

'rdr' is an io.Reader for the JSON (stream).
'mapHandler' is the Map and raw JSON - []byte - processor. Return of 'false' stops io.Reader processing.
'errHandler' is the error and raw JSON processor. Return of 'false' stops io.Reader processing and returns the error.
Note: mapHandler() and errHandler() calls are blocking, so reading and processing of messages is serialized.
      This means that you can stop reading the file on error or after processing a particular message.
      To have reading and handling run concurrently, pass argument(s) to a go routine in handler and return 'true'.
Example
package main

import ()

func main() {
	/*
		See: bulkraw_test.go for working example.
		Run "go test" in package directory then scroll back to find output.

		Basic logic for bulk JSON to XML processing is similar to that for
		bulk XML to JSON processing as outlined in the HandleXmlReader example.
		The test case is also a good example.
	*/
}
Output:

func HandleXMPPStreamTag

func HandleXMPPStreamTag(b ...bool)

HandleXMPPStreamTag causes decoder to parse XMPP <stream:stream> elements. If called with no argument, XMPP stream element handling is toggled on/off. (See xmppStream_test.go for example.)

If called with NewMapXml, NewMapXmlReader, New MapXmlReaderRaw the "stream"
element will be  returned as:
	map["stream"]interface{}{map[-<attrs>]interface{}}.
If called with NewMapSeq, NewMapSeqReader, NewMapSeqReaderRaw the "stream"
element will be returned as:
	map["stream:stream"]interface{}{map["#attr"]interface{}{map[string]interface{}}}
	where the "#attr" values have "#text" and "#seq" keys. (See NewMapXmlSeq.)

func HandleXmlReader

func HandleXmlReader(xmlReader io.Reader, mapHandler func(Map) bool, errHandler func(error) bool) error

Bulk process XML using handlers that process a Map value.

'rdr' is an io.Reader for XML (stream)
'mapHandler' is the Map processor. Return of 'false' stops io.Reader processing.
'errHandler' is the error processor. Return of 'false' stops io.Reader processing and returns the error.
Note: mapHandler() and errHandler() calls are blocking, so reading and processing of messages is serialized.
      This means that you can stop reading the file on error or after processing a particular message.
      To have reading and handling run concurrently, pass argument to a go routine in handler and return 'true'.
Example
package main

import ()

func main() {
	/*
		Bulk processing XML to JSON seems to be a common requirement.
		See: bulk_test.go for working example.
		     Run "go test" in package directory then scroll back to find output.

		The logic is as follows.

			// need somewhere to write the JSON.
			var jsonWriter io.Writer

			// probably want to log any errors in reading the XML stream
			var xmlErrLogger io.Writer

			// func to handle Map value from XML Reader
			func maphandler(m mxj.Map) bool {
				// marshal Map as JSON
				jsonVal, err := m.Json()
				if err != nil {
					// log error
					return false // stops further processing of XML Reader
				}

				// write JSON somewhere
				_, err = jsonWriter.Write(jsonVal)
				if err != nil {
					// log error
					return false // stops further processing of XML Reader
				}

				// continue - get next XML from Reader
				return true
			}

			// func to handle error from unmarshaling XML Reader
			func errhandler(errVal error) bool {
				// log error somewhere
				_, err := xmlErrLogger.Write([]byte(errVal.Error()))
				if err != nil {
					// log error
					return false // stops further processing of XML Reader
				}

				// continue processing
				return true
			}

			// func that starts bulk processing of the XML
				...
				// set up io.Reader for XML data - perhaps an os.File
				...
				err := mxj.HandleXmlReader(xmlReader, maphandler, errhandler)
				if err != nil {
					// handle error
				}
				...
	*/
}
Output:

func HandleXmlReaderRaw

func HandleXmlReaderRaw(xmlReader io.Reader, mapHandler func(Map, []byte) bool, errHandler func(error, []byte) bool) error

Bulk process XML using handlers that process a Map value and the raw XML.

'rdr' is an io.Reader for XML (stream)
'mapHandler' is the Map and raw XML - []byte - processor. Return of 'false' stops io.Reader processing.
'errHandler' is the error and raw XML processor. Return of 'false' stops io.Reader processing and returns the error.
Note: mapHandler() and errHandler() calls are blocking, so reading and processing of messages is serialized.
      This means that you can stop reading the file on error or after processing a particular message.
      To have reading and handling run concurrently, pass argument(s) to a go routine in handler and return 'true'.
See NewMapXmlReaderRaw for comment on performance associated with retrieving raw XML from a Reader.
Example
package main

import ()

func main() {
	/*
		See: bulkraw_test.go for working example.
		Run "go test" in package directory then scroll back to find output.

		Basic logic for bulk XML to JSON processing is in HandleXmlReader example;
		the only major difference is in handler function signatures so they are passed
		the raw XML.  (Read documentation on NewXmlReader regarding performance.)
	*/
}
Output:

func IncludeTagSeqNum

func IncludeTagSeqNum(b ...bool)

IncludeTagSeqNum - include a "_seq":N key:value pair with each inner tag, denoting its position when parsed. This is of limited usefulness, since list values cannot be tagged with "_seq" without changing their depth in the Map. So THIS SHOULD BE USED WITH CAUTION - see the test cases. Here's a sample of what you get.

	<Obj c="la" x="dee" h="da">
		<IntObj id="3"/>
		<IntObj1 id="1"/>
		<IntObj id="2"/>
		<StrObj>hello</StrObj>
	</Obj>

parses as:

	{
	Obj:{
		"-c":"la",
		"-h":"da",
		"-x":"dee",
		"intObj":[
			{
				"-id"="3",
				"_seq":"0" // if mxj.Cast is passed, then: "_seq":0
			},
			{
				"-id"="2",
				"_seq":"2"
			}],
		"intObj1":{
			"-id":"1",
			"_seq":"1"
			},
		"StrObj":{
			"#text":"hello", // simple element value gets "#text" tag
			"_seq":"3"
			}
		}
	}

func LeafUseDotNotation

func LeafUseDotNotation(b ...bool)

LeafUseDotNotation sets a flag that list members in LeafNode paths should be identified using ".N" syntax rather than the default "[N]" syntax. Calling LeafUseDotNotation with no arguments toggles the flag on/off; otherwise, the argument sets the flag value 'true'/'false'.

func PrependAttrWithHyphen

func PrependAttrWithHyphen(v bool)

PrependAttrWithHyphen. Prepend attribute tags with a hyphen. Default is 'true'. (Not applicable to NewMapXmlSeq(), mv.XmlSeq(), etc.)

Note:
	If 'false', unmarshaling and marshaling is not symmetric. Attributes will be
	marshal'd as <attr_tag>attr</attr_tag> and may be part of a list.

func SetArraySize

func SetArraySize(size int) int

SetArraySize adjust the buffers for expected number of values to return from ValuesForKey() and ValuesForPath(). This can have the effect of significantly reducing memory allocation-copy functions for large data sets. Returns the initial buffer size.

func SetAttrPrefix

func SetAttrPrefix(s string)

SetAttrPrefix changes the default, "-", to the specified value, s. SetAttrPrefix("") is the same as PrependAttrWithHyphen(false). (Not applicable for NewMapXmlSeq(), mv.XmlSeq(), etc.)

func SetCheckTagToSkipFunc

func SetCheckTagToSkipFunc(fn func(string) bool)

SetCheckTagToSkipFunc registers function to test whether the value for a tag should be cast to bool or float64 when "cast" argument is 'true'. (Dot tag path notation is not supported.) NOTE: key may be "#text" if it's a simple element with attributes

or "decodeSimpleValuesAsMap == true".

NOTE: does not apply to NewMapXmlSeq... functions.

func SetFieldSeparator

func SetFieldSeparator(s ...string)

SetFieldSeparator changes the default field separator, ":", for the newVal argument in mv.UpdateValuesForPath and the optional 'subkey' arguments in mv.ValuesForKey and mv.ValuesForPath.

E.g., if the newVal value is "http://blah/blah", setting the field separator to "|" will allow the newVal specification, "<key>|http://blah/blah" to parse properly. If called with no argument or an empty string value, the field separator is set to the default, ":".

func SetGlobalKeyMapPrefix

func SetGlobalKeyMapPrefix(s string)

func SetPathSeparator added in v2.8.1

func SetPathSeparator(separator string)

func XMLEscapeChars

func XMLEscapeChars(b ...bool)

XMLEscapeChars(true) forces escaping invalid characters in attribute and element values. NOTE: this is brute force with NO interrogation of '&' being escaped already; if it is then '&amp;' will be re-escaped as '&amp;amp;'.

The values are:
"   &quot;
'   &apos;
<   &lt;
>   &gt;
&   &amp;

Note: if XMLEscapeCharsDecoder(true) has been called - or the default, 'false,' value has been toggled to 'true' - then XMLEscapeChars(true) is ignored. If XMLEscapeChars(true) has already been called before XMLEscapeCharsDecoder(true), XMLEscapeChars(false) is called to turn escape encoding on mv.Xml, etc., to prevent double escaping ampersands, '&'.

func XMLEscapeCharsDecoder

func XMLEscapeCharsDecoder(b ...bool)

XMLEscapeCharsDecoder(b ...bool) escapes XML characters in xml.CharData values returned by Decoder.Token. Thus, the internal Map values will contain escaped values, and you do not need to set XMLEscapeChars for proper encoding.

By default, the Map values have the non-escaped values returned by Decoder.Token. XMLEscapeCharsDecoder(true) - or, XMLEscapeCharsDecoder() - will toggle escape encoding 'on.'

Note: if XMLEscapeCharDecoder(true) is call then XMLEscapeChars(false) is called to prevent re-escaping the values on encoding using mv.Xml, etc.

func XmlCheckIsValid

func XmlCheckIsValid(b ...bool)

XmlCheckIsValid forces the encoded XML to be checked for validity.

func XmlDefaultEmptyElemSyntax

func XmlDefaultEmptyElemSyntax()

XmlDefaultEmptyElemSyntax() - <tag .../> rather than <tag ...></tag>. Return XML encoding for empty elements to the default package setting. Reverses effect of XmlGoEmptyElemSyntax().

func XmlGoEmptyElemSyntax

func XmlGoEmptyElemSyntax()

XmlGoEmptyElemSyntax() - <tag ...></tag> rather than <tag .../>.

Go's encoding/xml package marshals empty XML elements as <tag ...></tag>.  By default this package
encodes empty elements as <tag .../>.  If you're marshaling Map values that include structures
(which are passed to xml.Marshal for encoding), this will let you conform to the standard package.

Types

type LeafNode

type LeafNode struct {
	Path  string      // a dot-notation representation of the path with array subscripting
	Value interface{} // the value at the path termination
}

LeafNode - a terminal path value in a Map. For XML Map values it represents an attribute or simple element value - of type string unless Map was created using Cast flag. For JSON Map values it represents a string, numeric, boolean, or null value.

type Map

type Map map[string]interface{}

func New

func New() Map

Allocate a Map.

func NewMapGob

func NewMapGob(gobj []byte) (Map, error)

NewMapGob returns a Map value for a gob object that has been encoded from a map[string]interface{} (or compatible type) value. It is intended to provide symmetric handling of Maps that have been encoded using mv.Gob.

func NewMapJson

func NewMapJson(jsonVal []byte) (Map, error)

Just a wrapper on json.Unmarshal

Converting JSON to XML is a simple as:
	...
	mapVal, merr := mxj.NewMapJson(jsonVal)
	if merr != nil {
		// handle error
	}
	xmlVal, xerr := mapVal.Xml()
	if xerr != nil {
		// handle error
	}

NOTE: as a special case, passing a list, e.g., [{"some-null-value":"", "a-non-null-value":"bar"}], will be interpreted as having the root key 'object' prepended - {"object":[ ... ]} - to unmarshal to a Map. See mxj/j2x/j2x_test.go.

func NewMapJsonReader

func NewMapJsonReader(jsonReader io.Reader) (Map, error)

Retrieve a Map value from an io.Reader.

NOTE: The raw JSON off the reader is buffered to []byte using a ByteReader. If the io.Reader is an
      os.File, there may be significant performance impact. If the io.Reader is wrapping a []byte
      value in-memory, however, such as http.Request.Body you CAN use it to efficiently unmarshal
      a JSON object.

func NewMapJsonReaderRaw

func NewMapJsonReaderRaw(jsonReader io.Reader) (Map, []byte, error)

Retrieve a Map value and raw JSON - []byte - from an io.Reader.

NOTE: The raw JSON off the reader is buffered to []byte using a ByteReader. If the io.Reader is an
      os.File, there may be significant performance impact. If the io.Reader is wrapping a []byte
      value in-memory, however, such as http.Request.Body you CAN use it to efficiently unmarshal
      a JSON object and retrieve the raw JSON in a single call.

func NewMapStruct

func NewMapStruct(structVal interface{}) (Map, error)

DEPRECATED - import github.com/fatih/structs and cast result of structs.Map to mxj.Map.

import "github.com/fatih/structs"
...
   sm, err := structs.Map(<some struct>)
   if err != nil {
      // handle error
   }
   m := mxj.Map(sm)

Alernatively uncomment the old source and import in struct.go.

func NewMapXml

func NewMapXml(xmlVal []byte, cast ...bool) (Map, error)

NewMapXml - convert a XML doc into a Map (This is analogous to unmarshalling a JSON string to map[string]interface{} using json.Unmarshal().)

If the optional argument 'cast' is 'true', then values will be converted to boolean or float64 if possible.

Converting XML to JSON is a simple as:
	...
	mapVal, merr := mxj.NewMapXml(xmlVal)
	if merr != nil {
		// handle error
	}
	jsonVal, jerr := mapVal.Json()
	if jerr != nil {
		// handle error
	}

NOTES:
   1. Declarations, directives, process instructions and comments are NOT parsed.
   2. The 'xmlVal' will be parsed looking for an xml.StartElement, so BOM and other
      extraneous xml.CharData will be ignored unless io.EOF is reached first.
   3. If CoerceKeysToLower() has been called, then all key values will be lower case.
   4. If CoerceKeysToSnakeCase() has been called, then all key values will be converted to snake case.
   5. If DisableTrimWhiteSpace(b bool) has been called, then all values will be trimmed or not. 'true' by default.

func NewMapXmlReader

func NewMapXmlReader(xmlReader io.Reader, cast ...bool) (Map, error)

Get next XML doc from an io.Reader as a Map value. Returns Map value.

NOTES:
   1. Declarations, directives, process instructions and comments are NOT parsed.
   2. The 'xmlReader' will be parsed looking for an xml.StartElement, so BOM and other
      extraneous xml.CharData will be ignored unless io.EOF is reached first.
   3. If CoerceKeysToLower() has been called, then all key values will be lower case.
   4. If CoerceKeysToSnakeCase() has been called, then all key values will be converted to snake case.

func NewMapXmlReaderRaw

func NewMapXmlReaderRaw(xmlReader io.Reader, cast ...bool) (Map, []byte, error)

Get next XML doc from an io.Reader as a Map value. Returns Map value and slice with the raw XML.

NOTES:
   1. Declarations, directives, process instructions and comments are NOT parsed.
   2. Due to the implementation of xml.Decoder, the raw XML off the reader is buffered to []byte
      using a ByteReader. If the io.Reader is an os.File, there may be significant performance impact.
      See the examples - getmetrics1.go through getmetrics4.go - for comparative use cases on a large
      data set. If the io.Reader is wrapping a []byte value in-memory, however, such as http.Request.Body
      you CAN use it to efficiently unmarshal a XML doc and retrieve the raw XML in a single call.
   3. The 'raw' return value may be larger than the XML text value.
   4. The 'xmlReader' will be parsed looking for an xml.StartElement, so BOM and other
      extraneous xml.CharData will be ignored unless io.EOF is reached first.
   5. If CoerceKeysToLower() has been called, then all key values will be lower case.
   6. If CoerceKeysToSnakeCase() has been called, then all key values will be converted to snake case.

func (Map) Attributes

func (mv Map) Attributes(path string) ([]string, error)

If the path is an element with attributes, return a list of the attribute keys. (The list is alphabeticly sorted.) NOTE: Map keys that are not prefixed with '-', a hyphen, are not treated as attributes; see m.Elements(path). Also, if the attribute prefix is "" - SetAttrPrefix("") or PrependAttrWithHyphen(false) - then there are no identifiable attributes.

func (Map) Copy

func (mv Map) Copy() (Map, error)

Return a copy of mv as a newly allocated Map. If the Map only contains string, numeric, map[string]interface{}, and []interface{} values, then it can be thought of as a "deep copy." Copying a structure (or structure reference) value is subject to the noted restrictions.

NOTE: If 'mv' includes structure values with, possibly, JSON encoding tags
      then only public fields of the structure are in the new Map - and with
      keys that conform to any encoding tag instructions. The structure itself will
      be represented as a map[string]interface{} value.
Example
package main

import ()

func main() {
	/*
		// Hand-crafted Map values that include structures do NOT Copy() as expected,
		// since to simulate a deep copy the original Map value is JSON encoded then decoded.

		type str struct {
			IntVal   int     `json:"int"`
			StrVal   string  `json:"str"`
			FloatVal float64 `json:"float"`
			BoolVal  bool    `json:"bool"`
			private  string
		}
		s := str{IntVal: 4, StrVal: "now's the time", FloatVal: 3.14159, BoolVal: true, private: "Skies are blue"}
		m := make(map[string]interface{}, 0)
		m["struct"] = interface{}(s)
		m["struct_ptr"] = interface{}(&s)
		m["misc"] = interface{}(`Now is the time`)

		mv := mxj.Map(m)
		cp, _ := mv.Copy()

		fmt.Printf("mv:\n%s\n", mv.StringIndent(2))
		fmt.Printf("cp:\n%s\n", cp.StringIndent(2))

		// NoFail output:
		// mv:
		//     misc : [string] Now is the time
		//     struct : [mxj_test.str] {IntVal:4 StrVal:now's the time FloatVal:3.14159 BoolVal:true private:Skies are blue}
		//     struct_ptr : [*mxj_test.str] &{IntVal:4 StrVal:now's the time FloatVal:3.14159 BoolVal:true private:Skies are blue}
		// cp:
		//    misc : [string] Now is the time
		//    struct :
		//      bool : [bool] true
		//      float : [float64] 3.14159
		//      int : [float64] 4
		//      str : [string] now's the time
		//    struct_ptr :
		//      bool : [bool] true
		//      float : [float64] 3.14159
		//      int : [float64] 4
		//      str : [string] now's the time
		//
	*/
}
Output:

func (Map) Elements

func (mv Map) Elements(path string) ([]string, error)

If the path is an element with sub-elements, return a list of the sub-element keys. (The list is alphabeticly sorted.) NOTE: Map keys that are prefixed with '-', a hyphen, are considered attributes; see m.Attributes(path).

func (Map) Exists

func (mv Map) Exists(path string, subkeys ...string) (bool, error)

Checks whether the path exists. If err != nil then 'false' is returned along with the error encountered parsing either the "path" or "subkeys" argument.

func (Map) Gob

func (mv Map) Gob() ([]byte, error)

Gob returns a gob-encoded value for the Map 'mv'.

func (Map) Json

func (mv Map) Json(safeEncoding ...bool) ([]byte, error)

Just a wrapper on json.Marshal. If option safeEncoding is'true' then safe encoding of '<', '>' and '&' is preserved. (see encoding/json#Marshal, encoding/json#Encode)

func (Map) JsonIndent

func (mv Map) JsonIndent(prefix, indent string, safeEncoding ...bool) ([]byte, error)

Just a wrapper on json.MarshalIndent. If option safeEncoding is'true' then safe encoding of '<' , '>' and '&' is preserved. (see encoding/json#Marshal, encoding/json#Encode)

func (Map) JsonIndentWriter

func (mv Map) JsonIndentWriter(jsonWriter io.Writer, prefix, indent string, safeEncoding ...bool) error

Writes the Map as pretty JSON on the Writer. If 'safeEncoding' is 'true', then "safe" encoding of '<', '>' and '&' is preserved.

func (Map) JsonIndentWriterRaw

func (mv Map) JsonIndentWriterRaw(jsonWriter io.Writer, prefix, indent string, safeEncoding ...bool) ([]byte, error)

Writes the Map as pretty JSON on the Writer. []byte is the raw JSON that was written. If 'safeEncoding' is 'true', then "safe" encoding of '<', '>' and '&' is preserved.

func (Map) JsonWriter

func (mv Map) JsonWriter(jsonWriter io.Writer, safeEncoding ...bool) error

Writes the Map as JSON on the Writer. If 'safeEncoding' is 'true', then "safe" encoding of '<', '>' and '&' is preserved.

func (Map) JsonWriterRaw

func (mv Map) JsonWriterRaw(jsonWriter io.Writer, safeEncoding ...bool) ([]byte, error)

Writes the Map as JSON on the Writer. []byte is the raw JSON that was written. If 'safeEncoding' is 'true', then "safe" encoding of '<', '>' and '&' is preserved.

func (Map) LeafNodes

func (mv Map) LeafNodes(no_attr ...bool) []LeafNode

LeafNodes - returns an array of all LeafNode values for the Map. The option no_attr argument suppresses attribute values (keys with prepended hyphen, '-') as well as the "#text" key for the associated simple element value.

PrependAttrWithHypen(false) will result in attributes having .attr-name as terminal node in 'path' while the path for the element value, itself, will be the base path w/o "#text".

LeafUseDotNotation(true) causes list members to be identified using ".N" syntax rather than "[N]" syntax.

func (Map) LeafPaths

func (mv Map) LeafPaths(no_attr ...bool) []string

LeafPaths - all paths that terminate in LeafNode values.

func (Map) LeafValues

func (mv Map) LeafValues(no_attr ...bool) []interface{}

LeafValues - all terminal values in the Map.

func (Map) NewMap

func (mv Map) NewMap(keypairs ...string) (Map, error)

(Map)NewMap - create a new Map from data in the current Map.

'keypairs' are key mappings "oldKey:newKey" and specify that the current value of 'oldKey'
should be the value for 'newKey' in the returned Map.
	- 'oldKey' supports dot-notation as described for (Map)ValuesForPath()
	- 'newKey' supports dot-notation but with no wildcards, '*', or indexed arrays
	- "oldKey" is shorthand for the keypair value "oldKey:oldKey"
	- "oldKey:" and ":newKey" are invalid keypair values
	- if 'oldKey' does not exist in the current Map, it is not written to the new Map.
	  "null" is not supported unless it is the current Map.
	- see newmap_test.go for several syntax examples
- mv.NewMap() == mxj.New()

NOTE: "examples/partial.go" shows how to create arbitrary sub-docs of an XML doc.

func (Map) Old

func (mv Map) Old() map[string]interface{}

Cast a Map to map[string]interface{}

func (Map) PathForKeyShortest

func (mv Map) PathForKeyShortest(key string) string

PathForKeyShortest extracts the shortest path from all possible paths - from PathsForKey() - in Map, 'mv'.. Paths are strings using dot-notation.

func (Map) PathsForKey

func (mv Map) PathsForKey(key string) []string

PathsForKey returns all paths through Map, 'mv', (in dot-notation) that terminate with the specified key. Results can be used with ValuesForPath.

func (Map) Remove

func (mv Map) Remove(path string) error

Removes the path.

func (Map) RenameKey

func (mv Map) RenameKey(path string, newName string) error

RenameKey renames a key in a Map. It works only for nested maps. It doesn't work for cases when the key is in a list.

func (Map) Root

func (mv Map) Root() (string, error)

Return the root element of the Map. If there is not a single key in Map, then an error is returned.

func (Map) SetValueForPath

func (mv Map) SetValueForPath(value interface{}, path string) error

Sets the value for the path

func (Map) StringIndent

func (mv Map) StringIndent(offset ...int) string

Pretty print a Map.

func (Map) StringIndentNoTypeInfo

func (mv Map) StringIndentNoTypeInfo(offset ...int) string

Pretty print a Map without the value type information - just key:value entries.

func (Map) Struct

func (mv Map) Struct(structPtr interface{}) error

Marshal a map[string]interface{} into a structure referenced by 'structPtr'. Error returned if argument is not a pointer or if json.Unmarshal returns an error.

json.Unmarshal structure encoding rules are followed to encode public structure fields.
Example
package main

import ()

func main() {
	/*
	   	type str struct {
	   		IntVal   int     `json:"int"`
	   		StrVal   string  `json:"str"`
	   		FloatVal float64 `json:"float"`
	   		BoolVal  bool    `json:"bool"`
	   		private  string
	   	}

	   	mapVal := mxj.Map{"int": 4, "str": "now's the time", "float": 3.14159, "bool": true, "private": "Somewhere over the rainbow"}

	   	var strVal str
	   	mverr := mapVal.Struct(&strVal)
	   	if mverr != nil {
	   		// handle error
	   	}

	   	fmt.Printf("mapVal: %#v\n", mapVal)
	   	fmt.Printf("strVal: %#v\n", strVal)

	   // Unordered output for above:
	   // mapVal: mxj.Map{"int":4, "str":"now's the time", "float":3.14159, "bool":true, "private":"Somewhere over the rainbow"}
	   // strVal: mxj_test.str{IntVal:4, StrVal:"now's the time", FloatVal:3.14159, BoolVal:true, private:""}
	*/
}
Output:

func (Map) UpdateValuesForPath

func (mv Map) UpdateValuesForPath(newVal interface{}, path string, subkeys ...string) (int, error)

Update value based on path and possible sub-key values. A count of the number of values changed and any error are returned. If the count == 0, then no path (and subkeys) matched.

	'newVal' can be a Map or map[string]interface{} value with a single 'key' that is the key to be modified
	             or a string value "key:value[:type]" where type is "bool" or "num" to cast the value.
	'path' is dot-notation list of keys to traverse; last key in path can be newVal key
	       NOTE: 'path' spec does not currently support indexed array references.
	'subkeys' are "key:value[:type]" entries that must match for path node
            - For attributes prefix the label with the attribute prefix character, by default a
              hyphen, '-', e.g., "-seq:3". (See SetAttrPrefix function.)
            - The subkey can be wildcarded - "key:*" - to require that it's there with some value.
            - If a subkey is preceeded with the '!' character, the key:value[:type] entry is treated as an
	              exclusion critera - e.g., "!author:William T. Gaddis".

	NOTES:
		1. Simple elements with attributes need a path terminated as ".#text" to modify the actual value.
		2. Values in Maps created using NewMapXmlSeq are map[string]interface{} values with a "#text" key.
		3. If values in 'newVal' or 'subkeys' args contain ":", use SetFieldSeparator to an unused symbol,
	      perhaps "|".
Example
package main

import ()

func main() {
	/*

	   var biblioDoc = []byte(`
	   <biblio>
	   	<author>
	   		<name>William Gaddis</name>
	   		<books>
	   			<book>
	   				<title>The Recognitions</title>
	   				<date>1955</date>
	   				<review>A novel that changed the face of American literature.</review>
	   			</book>
	   			<book>
	   				<title>JR</title>
	   				<date>1975</date>
	   				<review>Winner of National Book Award for Fiction.</review>
	   			</book>
	   		</books>
	   	</author>
	   </biblio>`)

	   	...
	   	m, merr := mxj.NewMapXml(biblioDoc)
	   	if merr != nil {
	   		// handle error
	   	}

	   	// change 'review' for a book
	   	count, err := m.UpdateValuesForPath("review:National Book Award winner." "*.*.*.*", "title:JR")
	   	if err != nil {
	   		// handle error
	   	}
	   	...

	   	// change 'date' value from string type to float64 type
	   	// Note: the following is equivalent to m, merr := NewMapXml(biblioDoc, mxj.Cast).
	   	path := m.PathForKeyShortest("date")
	   	v, err := m.ValuesForPath(path)
	   	if err != nil {
	   		// handle error
	   	}
	   	var total int
	   	for _, vv := range v {
	   		oldVal := "date:" + vv.(string)
	   		newVal := "date:" + vv.(string) + ":num"
	   		n, err := m.UpdateValuesForPath(newVal, path, oldVal)
	   		if err != nil {
	   			// handle error
	   		}
	   		total += n
	   	}
	   	...
	*/
}
Output:

func (Map) ValueForKey

func (mv Map) ValueForKey(key string, subkeys ...string) (interface{}, error)

ValueForKey is a wrapper on ValuesForKey. It returns the first member of []interface{}, if any. If there is no value, "nil, nil" is returned.

func (Map) ValueForPath

func (mv Map) ValueForPath(path string) (interface{}, error)

ValueForPath wraps ValuesFor Path and returns the first value returned. If no value is found it returns 'nil' and PathNotExistError.

func (Map) ValueForPathString

func (mv Map) ValueForPathString(path string) (string, error)

ValuesForPathString returns the first found value for the path as a string.

func (Map) ValueOrEmptyForPathString

func (mv Map) ValueOrEmptyForPathString(path string) string

ValueOrEmptyForPathString returns the first found value for the path as a string. If the path is not found then it returns an empty string.

func (Map) ValuesForKey

func (mv Map) ValuesForKey(key string, subkeys ...string) ([]interface{}, error)

ValuesForKey return all values in Map, 'mv', associated with a 'key'. If len(returned_values) == 0, then no match. On error, the returned slice is 'nil'. NOTE: 'key' can be wildcard, "*".

'subkeys' (optional) are "key:val[:type]" strings representing attributes or elements in a list.
          - By default 'val' is of type string. "key:val:bool" and "key:val:float" to coerce them.
          - For attributes prefix the label with the attribute prefix character, by default a
            hyphen, '-', e.g., "-seq:3". (See SetAttrPrefix function.)
          - If the 'key' refers to a list, then "key:value" could select a list member of the list.
          - The subkey can be wildcarded - "key:*" - to require that it's there with some value.
          - If a subkey is preceeded with the '!' character, the key:value[:type] entry is treated as an
            exclusion critera - e.g., "!author:William T. Gaddis".
          - If val contains ":" symbol, use SetFieldSeparator to a unused symbol, perhaps "|".

func (Map) ValuesForPath

func (mv Map) ValuesForPath(path string, subkeys ...string) ([]interface{}, error)

ValuesForPatb retrieves all values for a path from the Map. If len(returned_values) == 0, then no match. On error, the returned array is 'nil'.

'path' is a dot-separated path of key values.
       - If a node in the path is '*', then everything beyond is walked.
       - 'path' can contain indexed array references, such as, "*.data[1]" and "msgs[2].data[0].field" -
         even "*[2].*[0].field".
'subkeys' (optional) are "key:val[:type]" strings representing attributes or elements in a list.
          - By default 'val' is of type string. "key:val:bool" and "key:val:float" to coerce them.
          - For attributes prefix the label with the attribute prefix character, by default a
            hyphen, '-', e.g., "-seq:3". (See SetAttrPrefix function.)
          - If the 'path' refers to a list, then "tag:value" would return member of the list.
          - The subkey can be wildcarded - "key:*" - to require that it's there with some value.
          - If a subkey is preceeded with the '!' character, the key:value[:type] entry is treated as an
            exclusion critera - e.g., "!author:William T. Gaddis".
          - If val contains ":" symbol, use SetFieldSeparator to a unused symbol, perhaps "|".
Example
package main

import ()

func main() {
	/*
	   	// a snippet from examples/gonuts1.go
	   	// How to compensate for irregular tag labels in data.
	   	// Need to extract from an XML stream the values for "netid" and "idnet".
	   	// Solution: use a wildcard path "data.*" to anonymize the "netid" and "idnet" tags.

	   	var msg1 = []byte(`
	   <?xml version="1.0" encoding="UTF-8"?>
	   <data>
	       <netid>
	           <disable>no</disable>
	           <text1>default:text</text1>
	           <word1>default:word</word1>
	       </netid>
	   </data>
	   `)

	   	var msg2 = []byte(`
	   <?xml version="1.0" encoding="UTF-8"?>
	   <data>
	       <idnet>
	           <disable>yes</disable>
	           <text1>default:text</text1>
	           <word1>default:word</word1>
	       </idnet>
	   </data>
	   `)

	   	// let's create a message stream
	   	buf := new(bytes.Buffer)
	   	// load a couple of messages into it
	   	_, _ = buf.Write(msg1)
	   	_, _ = buf.Write(msg2)

	   	n := 0
	   	for {
	   		n++
	   		// Read the stream as Map values - quit on io.EOF.
	   		// Get the raw XML as well as the Map value.
	   		m, merr := mxj.NewMapXmlReader(buf)
	   		if merr != nil && merr != io.EOF {
	   			// handle error - for demo we just print it and continue
	   			fmt.Printf("msg: %d - merr: %s\n", n, merr.Error())
	   			continue
	   		} else if merr == io.EOF {
	   			break
	   		}

	   		// get the values for "netid" or "idnet" key using path == "data.*"
	   		values, _ := m.ValuesForPath("data.*")
	   		fmt.Println("\nmsg:", n, "> path == data.* - got array of values, len:", len(values))
	   		for i, val := range values {
	   			fmt.Println("ValuesForPath result array member -", i, ":", val)
	   			fmt.Println("              k:v pairs for array member:", i)
	   			for key, val := range val.(map[string]interface{}) {
	   				// You'd probably want to process the value, as appropriate.
	   				// Here we just print it out.
	   				fmt.Println("\t\t", key, ":", val)
	   			}
	   		}
	   	}
	   	// NoFail output:
	   	// msg: 1 > path == data.* - got array of values, len: 1
	   	// ValuesForPath result array member - 0 : map[disable:no text1:default:text word1:default:word]
	   	//               k:v pairs for array member: 0
	   	// 		 disable : no
	   	// 		 text1 : default:text
	   	// 		 word1 : default:word
	   	//
	   	// msg: 2 > path == data.* - got array of values, len: 1
	   	// ValuesForPath result array member - 0 : map[disable:yes text1:default:text word1:default:word]
	   	//               k:v pairs for array member: 0
	   	// 		 disable : yes
	   	// 		 text1 : default:text
	   	// 		 word1 : default:word
	*/
}
Output:

func (Map) Xml

func (mv Map) Xml(rootTag ...string) ([]byte, error)

Encode a Map as XML. The companion of NewMapXml(). The following rules apply.

  • The key label "#text" is treated as the value for a simple element with attributes.
  • Map keys that begin with a hyphen, '-', are interpreted as attributes. It is an error if the attribute doesn't have a []byte, string, number, or boolean value.
  • Map value type encoding: > string, bool, float64, int, int32, int64, float32: per "%v" formating > []bool, []uint8: by casting to string > structures, etc.: handed to xml.Marshal() - if there is an error, the element value is "UNKNOWN"
  • Elements with only attribute values or are null are terminated using "/>".
  • If len(mv) == 1 and no rootTag is provided, then the map key is used as the root tag, possible. Thus, `{ "key":"value" }` encodes as "<key>value</key>".
  • To encode empty elements in a syntax consistent with encoding/xml call UseGoXmlEmptyElementSyntax().

The attributes tag=value pairs are alphabetized by "tag". Also, when encoding map[string]interface{} values - complex elements, etc. - the key:value pairs are alphabetized by key so the resulting tags will appear sorted.

func (Map) XmlIndent

func (mv Map) XmlIndent(prefix, indent string, rootTag ...string) ([]byte, error)

Encode a map[string]interface{} as a pretty XML string. See Xml for encoding rules.

func (Map) XmlIndentWriter

func (mv Map) XmlIndentWriter(xmlWriter io.Writer, prefix, indent string, rootTag ...string) error

Writes the Map as pretty XML on the Writer. See Xml() for encoding rules.

func (Map) XmlWriter

func (mv Map) XmlWriter(xmlWriter io.Writer, rootTag ...string) error

Writes the Map as XML on the Writer. See Xml() for encoding rules.

type MapRaw

type MapRaw struct {
	M Map
	R []byte
}

func NewMapsFromJsonFileRaw

func NewMapsFromJsonFileRaw(name string) ([]MapRaw, error)

ReadMapsFromJsonFileRaw - creates an array of MapRaw from a file of JSON values.

func NewMapsFromXmlFileRaw

func NewMapsFromXmlFileRaw(name string) ([]MapRaw, error)

NewMapsFromXmlFileRaw - creates an array of MapRaw from a file of XML values. NOTE: the slice with the raw XML is clean with no extra capacity - unlike NewMapXmlReaderRaw(). It is slow at parsing a file from disk and is intended for relatively small utility files.

type MapSeq

type MapSeq map[string]interface{}

MapSeq is like Map but contains seqencing indices to allow recovering the original order of the XML elements when the map[string]interface{} is marshaled. Element attributes are stored as a map["#attr"]map[<attr_key>]map[string]interface{}{"#text":"<value>", "#seq":<attr_index>} value instead of denoting the keys with a prefix character. Also, comments, directives and process instructions are preserved.

func NewMapFormattedXmlSeq

func NewMapFormattedXmlSeq(xmlVal []byte, cast ...bool) (MapSeq, error)

NewMapFormattedXmlSeq performs the same as NewMapXmlSeq but is useful for processing XML objects that are formatted using the whitespace character, " ". (The stdlib xml.Decoder, by default, treats all whitespace as significant; Decoder.Token() and Decoder.RawToken() will return strings of one or more whitespace characters and without alphanumeric or punctuation characters as xml.CharData values.)

If you're processing such XML, then this will convert all occurrences of whitespace-only strings into an empty string, "", prior to parsing the XML - irrespective of whether the occurrence is formatting or is a actual element value.

func NewMapXmlSeq

func NewMapXmlSeq(xmlVal []byte, cast ...bool) (MapSeq, error)

NewMapXmlSeq converts a XML doc into a MapSeq value with elements id'd with decoding sequence key represented as map["#seq"]<int value>. If the optional argument 'cast' is 'true', then values will be converted to boolean or float64 if possible. NOTE: "#seq" key/value pairs are removed on encoding with msv.Xml() / msv.XmlIndent().

  • attributes are a map - map["#attr"]map["attr_key"]map[string]interface{}{"#text":<aval>, "#seq":<num>}

  • all simple elements are decoded as map["#text"]interface{} with a "#seq" k:v pair, as well.

  • lists always decode as map["list_tag"][]map[string]interface{} where the array elements are maps that include a "#seq" k:v pair based on sequence they are decoded. Thus, XML like: <doc> <ltag>value 1</ltag> <newtag>value 2</newtag> <ltag>value 3</ltag> </doc> is decoded as: doc : ltag :[[]interface{}] [item: 0] #seq :[int] 0 #text :[string] value 1 [item: 1] #seq :[int] 2 #text :[string] value 3 newtag : #seq :[int] 1 #text :[string] value 2 It will encode in proper sequence even though the MapSeq representation merges all "ltag" elements in an array.

  • comments - "<!--comment-->" - are decoded as map["#comment"]map["#text"]"cmnt_text" with a "#seq" k:v pair.

  • directives - "<!text>" - are decoded as map["#directive"]map[#text"]"directive_text" with a "#seq" k:v pair.

  • process instructions - "<?instr?>" - are decoded as map["#procinst"]interface{} where the #procinst value is of map[string]interface{} type with the following keys: #target, #inst, and #seq.

  • comments, directives, and procinsts that are NOT part of a document with a root key will be returned as map[string]interface{} and the error value 'NoRoot'.

  • note: "<![CDATA[" syntax is lost in xml.Decode parser - and is not handled here, either. and: "\r\n" is converted to "\n"

    NOTES: 1. The 'xmlVal' will be parsed looking for an xml.StartElement, xml.Comment, etc., so BOM and other extraneous xml.CharData will be ignored unless io.EOF is reached first. 2. CoerceKeysToLower() is NOT recognized, since the intent here is to eventually call m.XmlSeq() to re-encode the message in its original structure. 3. If CoerceKeysToSnakeCase() has been called, then all key values will be converted to snake case.

    NAME SPACES: 1. Keys in the MapSeq value that are parsed from a <name space prefix>:<local name> tag preserve the "<prefix>:" notation rather than stripping it as with NewMapXml(). 2. Attribute keys for name space prefix declarations preserve "xmlns:<prefix>" notation.

    ERRORS: 1. If a NoRoot error, "no root key," is returned, check the initial map key for a "#comment", "#directive" or #procinst" key. 2. Unmarshaling an XML doc that is formatted using the whitespace character, " ", will error, since Decoder.RawToken treats such occurances as significant. See NewMapFormattedXmlSeq().

func NewMapXmlSeqReader

func NewMapXmlSeqReader(xmlReader io.Reader, cast ...bool) (MapSeq, error)

NewMpaXmlSeqReader returns next XML doc from an io.Reader as a MapSeq value.

NOTES:
   1. The 'xmlReader' will be parsed looking for an xml.StartElement, xml.Comment, etc., so BOM and other
      extraneous xml.CharData will be ignored unless io.EOF is reached first.
   2. CoerceKeysToLower() is NOT recognized, since the intent here is to eventually call m.XmlSeq() to
      re-encode the message in its original structure.
   3. If CoerceKeysToSnakeCase() has been called, then all key values will be converted to snake case.

ERRORS:
   1. If a NoRoot error, "no root key," is returned, check the initial map key for a "#comment",
      "#directive" or #procinst" key.

func NewMapXmlSeqReaderRaw

func NewMapXmlSeqReaderRaw(xmlReader io.Reader, cast ...bool) (MapSeq, []byte, error)

NewMapXmlSeqReaderRaw returns the next XML doc from an io.Reader as a MapSeq value. Returns MapSeq value, slice with the raw XML, and any error.

NOTES:
   1. Due to the implementation of xml.Decoder, the raw XML off the reader is buffered to []byte
      using a ByteReader. If the io.Reader is an os.File, there may be significant performance impact.
      See the examples - getmetrics1.go through getmetrics4.go - for comparative use cases on a large
      data set. If the io.Reader is wrapping a []byte value in-memory, however, such as http.Request.Body
      you CAN use it to efficiently unmarshal a XML doc and retrieve the raw XML in a single call.
    2. The 'raw' return value may be larger than the XML text value.
    3. The 'xmlReader' will be parsed looking for an xml.StartElement, xml.Comment, etc., so BOM and other
       extraneous xml.CharData will be ignored unless io.EOF is reached first.
    4. CoerceKeysToLower() is NOT recognized, since the intent here is to eventually call m.XmlSeq() to
       re-encode the message in its original structure.
    5. If CoerceKeysToSnakeCase() has been called, then all key values will be converted to snake case.

ERRORS:
    1. If a NoRoot error, "no root key," is returned, check if the initial map key is "#comment",
       "#directive" or #procinst" key.

func (MapSeq) StringIndent

func (msv MapSeq) StringIndent(offset ...int) string

Pretty print a Map.

func (MapSeq) StringIndentNoTypeInfo

func (msv MapSeq) StringIndentNoTypeInfo(offset ...int) string

Pretty print a Map without the value type information - just key:value entries.

func (MapSeq) Xml

func (mv MapSeq) Xml(rootTag ...string) ([]byte, error)

Xml encodes a MapSeq as XML with elements sorted on #seq. The companion of NewMapXmlSeq(). The following rules apply.

  • The "#seq" key value is used to seqence the subelements or attributes only.
  • The "#attr" map key identifies the map of attribute map[string]interface{} values with "#text" key.
  • The "#comment" map key identifies a comment in the value "#text" map entry - <!--comment-->.
  • The "#directive" map key identifies a directive in the value "#text" map entry - <!directive>.
  • The "#procinst" map key identifies a process instruction in the value "#target" and "#inst" map entries - <?target inst?>.
  • Value type encoding: > string, bool, float64, int, int32, int64, float32: per "%v" formating > []bool, []uint8: by casting to string > structures, etc.: handed to xml.Marshal() - if there is an error, the element value is "UNKNOWN"
  • Elements with only attribute values or are null are terminated using "/>" unless XmlGoEmptyElemSystax() called.
  • If len(mv) == 1 and no rootTag is provided, then the map key is used as the root tag, possible. Thus, `{ "key":"value" }` encodes as "<key>value</key>".

func (MapSeq) XmlIndent

func (mv MapSeq) XmlIndent(prefix, indent string, rootTag ...string) ([]byte, error)

XmlIndent encodes a map[string]interface{} as a pretty XML string. See MapSeq.XmlSeq() for encoding rules.

func (MapSeq) XmlIndentWriter

func (mv MapSeq) XmlIndentWriter(xmlWriter io.Writer, prefix, indent string, rootTag ...string) error

XmlIndentWriter writes the MapSeq value as pretty XML on the Writer. See MapSeq.Xml() for encoding rules.

func (MapSeq) XmlWriter

func (mv MapSeq) XmlWriter(xmlWriter io.Writer, rootTag ...string) error

XmlWriter Writes the MapSeq value as XML on the Writer. See MapSeq.Xml() for encoding rules.

type Maps

type Maps []Map

func NewMaps

func NewMaps() Maps

func NewMapsFromJsonFile

func NewMapsFromJsonFile(name string) (Maps, error)

NewMapsFromXmlFile - creates an array from a file of JSON values.

func NewMapsFromXmlFile

func NewMapsFromXmlFile(name string) (Maps, error)

NewMapsFromXmlFile - creates an array from a file of XML values.

func (Maps) JsonFile

func (mvs Maps) JsonFile(file string, safeEncoding ...bool) error

JsonFile - write Maps to named file as JSON Note: the file will be created, if necessary; if it exists it will be truncated. If you need to append to a file, open it and use JsonWriter method.

func (Maps) JsonFileIndent

func (mvs Maps) JsonFileIndent(file, prefix, indent string, safeEncoding ...bool) error

JsonFileIndent - write Maps to named file as pretty JSON Note: the file will be created, if necessary; if it exists it will be truncated. If you need to append to a file, open it and use JsonIndentWriter method.

func (Maps) JsonString

func (mvs Maps) JsonString(safeEncoding ...bool) (string, error)

JsonString - analogous to mv.Json()

func (Maps) JsonStringIndent

func (mvs Maps) JsonStringIndent(prefix, indent string, safeEncoding ...bool) (string, error)

JsonStringIndent - analogous to mv.JsonIndent()

func (Maps) XmlFile

func (mvs Maps) XmlFile(file string) error

XmlFile - write Maps to named file as XML Note: the file will be created, if necessary; if it exists it will be truncated. If you need to append to a file, open it and use XmlWriter method.

func (Maps) XmlFileIndent

func (mvs Maps) XmlFileIndent(file, prefix, indent string) error

XmlFileIndent - write Maps to named file as pretty XML Note: the file will be created,if necessary; if it exists it will be truncated. If you need to append to a file, open it and use XmlIndentWriter method.

func (Maps) XmlString

func (mvs Maps) XmlString() (string, error)

XmlString - analogous to mv.Xml()

func (Maps) XmlStringIndent

func (mvs Maps) XmlStringIndent(prefix, indent string) (string, error)

XmlStringIndent - analogous to mv.XmlIndent()

Directories

Path Synopsis
j2x.go - For (mostly) backwards compatibility with legacy j2x package.
j2x.go - For (mostly) backwards compatibility with legacy j2x package.
x2j - For (mostly) backwards compatibility with legacy x2j package.
x2j - For (mostly) backwards compatibility with legacy x2j package.
Unmarshal dynamic / arbitrary XML docs and extract values (using wildcards, if necessary).
Unmarshal dynamic / arbitrary XML docs and extract values (using wildcards, if necessary).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL