optional

package module
v0.0.0-...-df395c8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 25, 2024 License: MIT Imports: 1 Imported by: 0

README

optional

Go library for working with optional values

What?

Have you ever needed to represent "something or nothing"? It's common in go to use a pointer for this, but in some situations that can add complexity or is otherwise undesirable.

This package aims to provide an alternative, which is the "Optional". An Optional puts another type in a box which can either be "Some" or "None". That, and a small handful of methods to interact with the inner value, is pretty much it.

The goal of this isn't to be particularly performant. This ain't no rust style "zero-cost abstraction". Don't use it in the hot paths of your code - that's where the pointer trick goes.

The goal is to provide the functionality that works in an intuitive way without any of the gotchas and questions that using a pointer would bring so that you can implement what you need quickly without any thought. If you find performance is an issue, then you can always replace these with pointers later.

Really, this is a generic solution to a common problem: How do you represent NOT a thing? Often we can find a particular "magic value" in the domain of our type which is not a valid value for our application and use that. Have a field for email address? An empty string can be used as your None value. Port number? Pick anything above 65,535 to represent "None selected". Sometimes, however, there is no obvious invalid value of your type in your use case. Moreover, you sometimes need to differentiate between "default" and "None", and in those cases you actually need two values which are both invalid for your application!

Disclamer: I'm sure you could find exciting new ways to shoot yourself in the foot if you tried to create an optional of a pointer type (or a struct which contains pointer fields). You can definitely do it, I just don't advise doing so unless you know what you are doing.

Where?

Where Optionals really shine, are situations which are not performance sensitive in the first place. Configuration is probably the best example, particularly if you are drawing configs from multiple sources and merging them together. I actually first wrote this in a fit of irritation after tracking down a nil-pointer deference in a configuration package for longer than it took to write the package.

How?

See the example/ directory for a setup for brain-dead config parsing. Sure, it's verbose for two real parameters and there are is a lot of boilderplate for what it does, but nothing is going to go wrong and it is fully extendable to a real world project without needing any complicated additional libraries involved. Don't get me wrong, Cobra and Viper are super powerful and well maintained, but 98% of the time I only really want one command per executable and every time I touch Cobra or Viper I spend at least half an hour reading through documentation. Why bother for the vast majority of my work what just involves stupid things like a slack-bot to render text as party parrots?

Note that while this looks like a lot of code for what it does, it does have a good set of functionality for a small project:

  • Clear precedence of config sources
  • Only basic flag library used
  • Flag for debug output
  • Flag to choose config file (including option to skip file loading without needing a separate flag!)
  • Annotations for env var mapping and file loading of the config are defined by the loader struct itself
  • Default values kept directly above config loader structs for easy comparison
  • No super ugly long parameter sets to pass from flag parsing to initialize structs (builder pattern preferred)
  • Reloadable, so if you create an init ConfigLoader from flags you can then do hot reloads in response to e.g. a SIGHUP (see example/main.go)
  • No magic hidden in a library you need to look up

If you want to try it, it was written so that the precedence is flags > file > env. The default values in the code are { Host: "localhost", Port: 1443 }.

Try running it a few different ways and seeing what happens!

go run ./example
PORT=3000 go run ./example
PORT=3000 go run ./example --port 5000
PORT=3000 go run ./example --port 5000 --host "example.com"
PORT=3000 HOST=host.from.env go run ./example
PORT=3000 HOST=host.from.env go run ./example --config alt.toml
HOST=host.from.env go run ./example --config none
PORT=3000 HOST=host.from.env go run ./example --config alt.toml --port 5000 --host host.from.flag
PORT=3000 HOST=host.from.env go run ./example --config alt.toml --port 5000 --host host.from.flag
PORT=3000 HOST=host.from.env go run ./example --debug --config alt.toml --port 5000 --host host.from.flag

State of the art

Currently, some other packages do provide a similar experience. There are a few issues that I have which are not addressed by the ones I am aware of, however.

markphelips/optional
Pros
  • should be somewhat performant due to just using a pointer under the hood
  • ergonomics pretty good
Cons
  • one optional type per underlying type
  • Requires a go generate script to support any non-implemented type
  • Doesn't support some useful features like applying transforms to the value inside the option
  • You could be surprised if you store one of these types in a struct since they are values wrapping pointers, and that would be a nasty debugging session.
leighmcculloch go-optional
Pros
  • Generic so it can be applied to any type
  • short and sweet
  • supports JSON and XML marshaling/unmarshaling
Cons
  • Generic over any, so things like equality can't be supported
  • I don't know if using an internal array is any more efficient than a boolean to represent Some/None
  • The Marshaling/Unmarshaling uses the zero values to represent None instead of null, removing much of the benefit there
  • Methods like String() uses Sprintf, which uses reflection. There is poor performance, then there is reflection performance.
Magic Values

This is just when you use a specific value to represent your None. Sometimes this makes sense, such as when you have a string field where an empty string would be meaningless.

Pros
  • Easy to use by just defining a const in your package
  • Possibly the most efficient way to do this
Cons
  • You don't always an invalid value to use as your magic value, so it's just impossible sometimes.
  • Ergonomics can get messy; different libraries may return different magic values which you have to translate between
Pointers
Pros
  • Effective and fast. Just a nil check tells you if a value is set or not.
Cons
  • Nil checks everywhere. It's on you to check for nil before every use, and the consequences of forgetting is a nil pointer dereference panic.
  • Makes for more difficult to read and reason about code at a surprisingly low level of complexity
  • If you every pass a struct by value your invariants can be broken by methods modifying some fields only in the copied struct and others in the copied and original struct. You need to be very careful if you do that.
This package
Pros
  • Ergonomics pretty good
  • No surprises
  • Generic implementation means all types can be used in the same way
  • Methods for performing transforms on data without extracting it first
  • Specialized optional types can be built on top to provide any needed functionality for specific use cases.
Cons
  • Inefficient in terms of space and performance
  • Core Option type is limited in implementing convenient stdlib interfaces due to the use of generics.
  • To make the thing more useful, only comparable values can be wrapped currently. This isn't usually too big of a deal for most values, but does mean that you cannot create an array option for example (but why would you do that?!? Just check for zero len!)

Why?

There are many great things about golang, but being spoiled by choice is not typically one of them.

Not that there is anything wrong with having a good stdlib implementation that can actually be used in prod or there is a lack of good libraries available; I mean there is literally no good way to represent "maybe a thing".

Sure, you can get by in most cases with a pointer. This is, in fact, the most performant way to do this. I do it all the time.

I kept hitting the same few situations, however, where this just caused me grief. The biggest was (is) configuration.

I get that many people feel strongly about configuration. Sometimes just having a quick TOML parser is all you need. Sometimes you are deploying to something like nomad or kubernetes and get better functionality by using env vars. Sometimes you are working at a company that has an in-house configuration system or has standardized on consul and you have no choice. Everyone knows junior devs love nothing more than pre-planning a future outage by using YAML configs and including a float field. And whatever you choose, you will probably also be accepting command-line flags. Flags are just more configuration!

Personally, I don't really care. I find that I have a few goals with configuration, and accomplishing them in the best way usually requires at least flags, env vars, and some kind of config file or networked configuration system.

Flags: Determines which entrypoint to use for code execution and also allows for a human override for any other configuration. Think about your favorite single-binary system like consul, k8s, etc. The flag is what determines if a given execution will run a dev server or production node. It is very clunky to use anything else to do this.

Env vars: Where your code runs is important. What datacenter are you in? What region? Is this production or dev? You could have some system merging the environment configuration with the application configuration, but why? I have seen more than enough puppet or chef configurations trying to handle application config to last a lifetime. But everyone uses orchestration systems like k8s now right? I'm tired of seeing a pod with a handful of side-cars that all have the same environment distilled into their own little files. Humans developed environment variables to solve a real problem, and they are really good at solving those problems!

The wild west: Use YAML, TOML, JSON, BSON, JSON5, Consul, Vault, Zookeeper, etcd, whatever bad distributed key/value store your company implemented on top of redis, or even DynoDB if you are a psycopath. I'm not gunna sweat it, because if you use flags and env vars properly the only things that cares about this stuff is your application and I, as an infrastructure engineer, am never going to have to interact with this. As long as you don't break your own config then ask me to fix it. You can do that, I suppose, but know that some poor person that is taking time out of keeping that old elasticsearch cluster alive for one more day is judging you hard.

Actually, on second thought just use TOML or whatever the company told you to use. You're probably just using it for boolean feature flags anyways.

FAQ's

Why do you even have Unwrap()? It seems like a less convenient version of Get().

Yep. On the one hand, I like the idea of options being values. They could be passed around and referenced easily without worrying about consequences.

In practice, I find I actually need pointers to options most of the time. They are things meant to be consumed rather than passed around a lot, and as such when the thing that actually needs them uses them, I don't really want them to stick around going forward.

As an example, consider a situation where you have a single config loader generating a nested config which includes pointers. Maybe one field of your config is a sub-config meant for one component or domain of your application. If that struct has pointer fields, then even if the struct is passed into your component by value there will be another pointer floating around and accessible in your other code. That should be okay, right? Surely nothing else will call a method on the main config struct that might modify its fields right?

Well, maybe we should just make that a pointer so that we can nil it out after the component consumes its config. That works, but now we have a nil pointer floating around and something else can still call methods on the main struct to modify fields. Did you check that pointer everywhere to make sure you are not dereferencing it when it's nil? Are you sure?

Well hey, just make the field an Optional and when the component consumes its part it can call Unwrap() to get the inner value and the rest of the struct just has a None Optional left. It has all the same methods and code has to handle the errors returned just like normal, but now it won't cause an unhandled panic if you lose focus for more than 10 seconds!

Why didn't you just wrap a pointer then do the right thing? Isn't copying things around by value all the time expensive?
  1. There is a whole world of hurt in golang around structs with pointer fields. If someone is so foolish as to blindly pass such a thing around by value, bad things can happen quickly.
  2. As such, I initially tried to only return pointers to optionals, but given the methods I wanted to provide this didn't always work well.
  3. You can always make an option with a pointer inner type if you want that. It probably isn't totally safe in all cases and there is a very real chance that it won't do what you want. YMMV!

Generating the keys and certs for testing

This is mostly a reminder for myself, given that the certs only have a lifetime of one year.

RSA
openssl genrsa -out tls/rsa/key.pem 4096
openssl rsa -in tls/rsa/key.pem -pubout -out tls/rsa/pubkey.pem
openssl req -new -key tls/rsa/key.pem -x509 -sha256 -nodes -subj "/C=US/ST=California/L=Who knows/O=BS Workshops/OU=optional/CN=www.whobe.us" -days 365 -out tls/rsa/cert.pem
ECDSA
openssl ecparam -name secp521r1 -genkey -noout -out tls/ecdsa/key.pem
openssl ec -in tls/ecdsa/key.pem -pubout > tls/ecdsa/pub.pem
openssl req -new -key tls/ecdsa/key.pem -x509 -sha512 -nodes -subj "/C=US/ST=California/L=Who knows/O=BS Workshops/OU=optional/CN=www.whobe.us" -days 365 -out tls/ecdsa/cert.pem
ED25519
openssl genpkey -algorithm ed25519 -out tls/ed25519/key.pem
openssl pkey -in tls/ed25519/key.pem -pubout -out tls/ed25519/pub.pem
openssl req -new -key tls/ed25519/key.pem -x509 -nodes -subj "/C=US/ST=California/L=Who knows/O=BS Workshops/OU=optional/CN=www.whobe.us" -days 365 -out tls/ed25519/cert.pem

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func And 

func And[T comparable, O Optional[T]](left, right O) O

And returns None if the first Optional is None, and the second Optional otherwise. Conceptually similar to left && right. This is a convenience function for Option selection. Convenient for merging configs, implementing builder patterns, etc.

func ClearIfMatch 

func ClearIfMatch[T comparable](opt MutableOptional[T], probe T)

ClearIfMatch calls clear if Optional.Match(probe) == true. This is a convenience for situations where you need to convert from a value of T with known "magic value" which implies None. A good example of this is if you have an int loaded from command line flags and you know that any flag omitted by the user will be assigned to 0. This can be done like this: o := Some(x) o.ClearIfMatch(0)

func Equal 

func Equal[T comparable, O Optional[T]](left, right O) bool

Equal is a convenience function for checking if the contents of two Optional types are equivilent. Note that Get() and Match() may be overridden by more complex types which wrap a vanilla Option. In these situations, the writer is responsible for making sure that the invariant Some(x).Match(Some(x).Get()) is always true.

func GetOr 

func GetOr[T comparable](opt Optional[T], val T) T

GetOr is the same as Get, but will return the passed value instead of an error if the Option is None. Another convenience function

func GetOrElse 

func GetOrElse[T comparable](opt Optional[T], f func() T) T

GetOrElse calls Get(), but run the passed function and return the result instead of producing an error if the option is None.

func GetOrInsert 

func GetOrInsert[T comparable](opt MutableOptional[T], val T) (T, error)

GetOrInsert calls Get, but will call Replace on the passed value then return it if the Option is None

func IsSomeAnd 

func IsSomeAnd[T comparable](opt Option[T], f func(T) bool) bool

IsSomeAnd returns true if the Option has a value of Some(x) and f(x) == true

func Must 

func Must[T comparable](opt Optional[T]) T

Must just calls Get, but panic instead of producing an error.

func Or 

func Or[T comparable, O Optional[T]](left, right O) O

Or returns the first Optional if it contains a value. Otherwise, return the second Optional. This is conceptually similar to left || right. This is a convenience function for situations like merging configs or implementing builder patterns.

func TransformOr 

func TransformOr[T comparable](opt MutableOptional[T], t Transformer[T], backup T) error

TransformOr just calls Transform(), except None values are mapped to backup before being transformed.

Types

type MutableOptional 

type MutableOptional[T comparable] interface {
	Optional[T]

	MutableClone() MutableOptional[T]
	Clear()
	Replace(T) (Optional[T], error)
	// Transform only applies the func to the values of Some valued Optionals. Any mapping of None is None.
	Transform(f Transformer[T]) error
	// Satisfies encoding.json.UnMarshaler
	UnmarshalJSON([]byte) error
}

MutableOptional is a superset of Optional which allows mutating and transforming the wrapped value.

type Option 

type Option[T comparable] struct {
	// contains filtered or unexported fields
}

Option is a generic way to make a field or parameter optional. Instantiating an Optional value through the Some() or None() methods are prefered since it is easier for the reader of your code to see what the expected value of the Option is, but to avoid leaking options across api boundries a FromPointer() method is given. This allows you to accept a parameter as a pointer and immediately convert it into an Option value without having to do the nil check yourself.

While this can be useful on its own, this can also be used to create a specialized option for your use case. See the config package for a comprehensive example of this.

Special care should be taken when creating Options of pointer types. All the same concerns around passing structs with pointer fields apply, since copying the pointer by value will create many copies of the same pointer. There is nothing to stop you from doing this, but I'm not sure what they use case is and it may lead to less understandable code (which is what this library was created to avoid in the first place!)

func FromPointer 

func FromPointer[T comparable](p *T) Option[T]

FromPointer creates an option with an inferred type from a pointer. nil pointers are mapped to a None value and non-nil pointers have their value copied into a new option. The pointer itself is not retained and can be modified later without affecting the Option value.

func None 

func None[T comparable]() Option[T]

None returns an Option with no value loaded. Note that since there is no value to infer type from, None must be instanciated with the desired type like optional.None[string]().

func Some 

func Some[T comparable](value T) Option[T]

Some returns an Option with an inferred type and specified value.

func (*Option[T]) Clear 

func (o *Option[T]) Clear()

Clear converts a Some(x) or None type Option into a None value.

func (Option[T]) Clone 

func (o Option[T]) Clone() Optional[T]

Clone creates a copy of the Option by value. This means the new Option may be unwrapped or modified without affecting the old Option. NOTE: the exception to this would be if you create an Option for a pointer type. Because the pointer is copied by value, it will still refer to the same value.

func (Option[T]) Get 

func (o Option[T]) Get() (T, error)

Get returns the current wrapped value of a Some value Option and returns an error if the Option is None.

func (Option[T]) IsNone 

func (o Option[T]) IsNone() bool

func (Option[T]) IsSome 

func (o Option[T]) IsSome() bool

func (Option[T]) MarshalJSON 

func (o Option[T]) MarshalJSON() ([]byte, error)

MarshalJSON implements the encoding.json.Marshaler interface. None values are marshaled to json null, while Some values are passed into json.Marshal directly.

func (Option[T]) Match 

func (o Option[T]) Match(probe T) bool

Match tests if the inner value of Option == the passed value

func (Option[T]) MutableClone 

func (o Option[T]) MutableClone() MutableOptional[T]

MutableClone creates a copy of the Option by value the same as Clone. The only differnce is that the returned type is a pointer cast as a MutableOptional so that the returned value can be further modified.

func (*Option[T]) Replace 

func (o *Option[T]) Replace(value T) (Optional[T], error)

Replace converts a Some(x) or None type Option into a Some(value) value. The base Option struct can never return an error from Replace, so it is generally safe to ignore the returned values from this, e.g. calling o.Replace() instead of _, _ = o.Replace().

func (*Option[T]) Transform 

func (o *Option[T]) Transform(t Transformer[T]) error

Transform applies function f(T) to the inner value of the Option. If the Option is None, then the Option will remain None.

func (*Option[T]) UnmarshalJSON 

func (o *Option[T]) UnmarshalJSON(data []byte) error

UnmarshalJSON implements the encoding.json.Unmarshaller interface. Json nulls are unmarshaled into None values, while any other value is attempted to unmarshal as normal. Any error encountered is returned without modification. There are some protections included to make sure that unmarshaling an uninitialized Option[T] does not break the Option invariants.

type Optional 

type Optional[T comparable] interface {
	IsSome() bool
	IsNone() bool
	Clone() Optional[T]
	Get() (T, error)
	Match(T) bool
	// Satisfies encoding.json.Marshaler
	MarshalJSON() ([]byte, error)
}

Optional defines the functionality needed to provide good ergonimics around optional fields and values. In general, code should not declare variables or parameters as Optionals and instead prefer using concrete types like Option. This interface is meant to ensure compatablility between different concrete option types and for the rare cases where the abstraction is actually necessary.

type OptionalError 

type OptionalError struct {
	// contains filtered or unexported fields
}

func (OptionalError) Error 

func (e OptionalError) Error() string

type Transformer 

type Transformer[T comparable] func(T) (T, error)

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.