README
¶
Install
The main method of installation is through "go get" (provided in $GOROOT/bin)
go get github.com/djimenez/iconv-go
This both downloads from github and installs the package into your $GOPATH. To just recompile the package after you've already "get"ed (e.g. for a new go version), use "go install" instead.
go install github.com/djimenez/iconv-go
See documentation for "go get" and "go install" for more information.
PLEASE NOTE that this package requires the use of cgo, since it is only a wrapper around iconv - either provided by libiconv or glibc on your system. Attempts to build without cgo enabled will fail.
Usage
To use the package, you'll need the appropriate import statement:
import (
iconv "github.com/djimenez/iconv-go"
)
Converting string Values
Converting a string can be done with two methods. First, there's iconv.ConvertString(input, fromEncoding, toEncoding string)
output,_ := iconv.ConvertString("Hello World!", "utf-8", "windows-1252")
Alternatively, you can create a converter and use its ConvertString method. Reuse of a Converter instance is recommended when doing many string conversions between the same encodings.
converter := iconv.NewConverter("utf-8", "windows-1252")
output,_ := converter.ConvertString("Hello World!")
// converter can then be closed explicitly
// this will also happen when garbage collected
converter.Close()
ConvertString may return errors for the following reasons:
- EINVAL - when either the from or to encoding is not supported by iconv
- EILSEQ - when the input string contains an invalid byte sequence for the given from encoding
Converting []byte Values
Converting a []byte can similarly be done with two methods. First, there's iconv.Convert(input, output []byte, fromEncoding, toEncoding string). You'll immediately notice this requires you to give it both the input and output buffer. Ideally, the output buffer should be sized so that it can hold all converted bytes from input, but if it cannot, then Convert will put as many bytes as it can into the buffer without creating an invalid sequence. For example, if iconv only has a single byte left in the output buffer but needs 2 or more for the complete character in a multibyte encoding it will stop writing to the buffer and return with an iconv.E2BIG error.
in := []byte("Hello World!")
out := make([]byte, len(input))
bytesRead, bytesWritten, err := iconv.Convert(in, out, "utf-8", "latin1")
Just like with ConvertString, there is also a Convert method on Converter that can be used.
...
converter := iconv.NewConverter("utf-8", "windows-1252")
bytesRead, bytesWritten, error := converter.Convert(input, output)
Convert may return errors for the following reasons:
-
EINVAL - when either the from or to encoding is not supported by iconv
-
EILSEQ - when the input string contains an invalid byte sequence for the given from encoding
-
E2BIG - when the output buffer is not big enough to hold the full conversion of input
Note on E2BIG: this is a common error value especially when converting to a multibyte encoding and should not be considered fatal. Partial conversion has probably occurred be sure to check bytesRead and bytesWritten.
Note on Shift Based Encodings
When using iconv.Convert convenience method it will automatically try to append to your output buffer with a nil input so that any end shift sequences are appropiately written. Using a Converter.Convert method however will not automatically do this since it can be used to process a full stream in chunks. So you'll need to remember to pass a nil input buffer at the end yourself, just like you would with direct iconv usage.
Converting an *io.Reader
The iconv.Reader allows any other *io.Reader to be wrapped and have its bytes transcoded as they are read.
// We're wrapping stdin for simplicity, but a File or network reader could
// be wrapped as well
reader,_ := iconv.NewReader(os.Stdin, "utf-8", "windows-1252")
Converting an *io.Writer
The iconv.Writer allows any other *io.Writer to be wrapped and have its bytes transcoded as they are written.
// We're wrapping stdout for simplicity, but a File or network reader could
// be wrapped as well
writer,_ := iconv.NewWriter(os.Stdout, "utf-8", "windows-1252")
Documentation
¶
Overview ¶
Wraps the iconv API present on most systems, which allows for conversion of bytes from one encoding to another. This package additionally provides some convenient interface implementations like a Reader and Writer.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Converter ¶
type Converter struct {
// contains filtered or unexported fields
}
func NewConverter ¶
Initialize a new Converter. If fromEncoding or toEncoding are not supported by iconv then an EINVAL error will be returned. An ENOMEM error maybe returned if there is not enough memory to initialize an iconv descriptor
func (*Converter) Convert ¶
func (this *Converter) Convert(input []byte, output []byte) (bytesRead int, bytesWritten int, err error)
Convert bytes from an input byte slice into a give output byte slice
As many bytes that can converted and fit into the size of output will be processed and the number of bytes read for input as well as the number of bytes written to output will be returned. If not all converted bytes can fit into output and E2BIG error will also be returned. If input contains an invalid sequence of bytes for the Converter's fromEncoding an EILSEQ error will be returned
For shift based output encodings, any end shift byte sequences can be generated by passing a 0 length byte slice as input. Also passing a 0 length byte slice for output will simply reset the iconv descriptor shift state without writing any bytes.
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Source Files
Directories