Documentation
¶
Index ¶
- func Copy[T any](src T) T
- func GetCompletenessValidator(url string) *jsonschema.Schema
- func GetRenderedErrors(err error, byts []byte) []error
- func GetSanityValidator(url string) *jsonschema.Schema
- func IsComplete(schema Schema) error
- func JSONFromYaml(byts []byte) ([]byte, error)
- func JoinErrors(errs []error, joiner string) string
- func Merge[T any](obj T, src T) T
- func ParsedAdaptiveASHAConfigV0() interface{}
- func ParsedAdaptiveConfigV0() interface{}
- func ParsedAdaptiveSimpleConfigV0() interface{}
- func ParsedAsyncHalvingConfigV0() interface{}
- func ParsedAzureConfigV0() interface{}
- func ParsedBindMountV0() interface{}
- func ParsedBindMountsConfigV0() interface{}
- func ParsedCategoricalHyperparameterV0() interface{}
- func ParsedCheckGridHyperparameterV0() interface{}
- func ParsedCheckPositiveLengthV0() interface{}
- func ParsedCheckpointStorageConfigV0() interface{}
- func ParsedConstHyperparameterV0() interface{}
- func ParsedCustomConfigV0() interface{}
- func ParsedDeviceV0() interface{}
- func ParsedDevicesConfigV0() interface{}
- func ParsedDirectoryConfigV0() interface{}
- func ParsedDoubleHyperparameterV0() interface{}
- func ParsedEnvironmentConfigV0() interface{}
- func ParsedEnvironmentImageMapV0() interface{}
- func ParsedEnvironmentImageV0() interface{}
- func ParsedEnvironmentVariablesMapV0() interface{}
- func ParsedEnvironmentVariablesV0() interface{}
- func ParsedExperimentConfigV0() interface{}
- func ParsedGCSConfigV0() interface{}
- func ParsedGridConfigV0() interface{}
- func ParsedHyperparameterV0() interface{}
- func ParsedHyperparametersV0() interface{}
- func ParsedIntHyperparameterV0() interface{}
- func ParsedKerberosConfigV0() interface{}
- func ParsedLengthV0() interface{}
- func ParsedLogActionCancelRetriesV0() interface{}
- func ParsedLogActionExcludeNodeV0() interface{}
- func ParsedLogActionV0() interface{}
- func ParsedLogHyperparameterV0() interface{}
- func ParsedLogPolicyV0() interface{}
- func ParsedOptimizationsConfigV0() interface{}
- func ParsedPbsConfigV0() interface{}
- func ParsedProfilingConfigV0() interface{}
- func ParsedProxyPortV0() interface{}
- func ParsedProxyPortsConfigV0() interface{}
- func ParsedRandomConfigV0() interface{}
- func ParsedRegistryAuthV0() interface{}
- func ParsedReproducibilityConfigV0() interface{}
- func ParsedResourcesConfigV0() interface{}
- func ParsedS3ConfigV0() interface{}
- func ParsedSearcherConfigV0() interface{}
- func ParsedSearcherLengthV0() interface{}
- func ParsedSecurityConfigV0() interface{}
- func ParsedSharedFSConfigV0() interface{}
- func ParsedSingleConfigV0() interface{}
- func ParsedSlurmConfigV0() interface{}
- func ParsedSyncHalvingConfigV0() interface{}
- func ParsedTensorboardStorageConfigV0() interface{}
- func ParsedTestRootV0() interface{}
- func ParsedTestSubV0() interface{}
- func ParsedTestUnionAV0() interface{}
- func ParsedTestUnionBV0() interface{}
- func ParsedTestUnionV0() interface{}
- func SaneBytes(schema Schema, byts []byte) error
- func UnionMerge[T any](obj T, src T) T
- func WithDefaults[T any](obj T) T
- type JSON
- type JSONArray
- type JSONObject
- type Schema
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Copy ¶
func Copy[T any](src T) T
Copy is a reflect-based deep copy. It's only generally safe to use on schema objects.
func GetCompletenessValidator ¶
func GetCompletenessValidator(url string) *jsonschema.Schema
GetCompletenessValidator returns a jsonschema validator for bytes from a particular URL.
func GetRenderedErrors ¶
GetRenderedErrors takes a jsonschema valiation plus the bytes that caused it and returns user-facing errors.
func GetSanityValidator ¶
func GetSanityValidator(url string) *jsonschema.Schema
GetSanityValidator returns a jsonschema validator for bytes from a particular URL.
func IsComplete ¶
IsComplete ensures that the schema is totally valid, including eventuallyRequired fields.
func JSONFromYaml ¶
JSONFromYaml takes yaml-formatted bytes and converts them to json-format for the purpose of applying json-schema validation.
func JoinErrors ¶
JoinErrors is like strings.Join but for []error types.
func Merge ¶
func Merge[T any](obj T, src T) T
Merge will recurse through two objects of the same type and return a merged version (a clean copy).
The default behavior for merging maps is to include keys from both src and obj, while the default behavior for slices is to use one or the other. This is analgous to how json.Unmarshal treats maps and slices. However, the default merging behavior for an object can be overwritten by implementing the Mergable interface. An example of this is BindMountsConfig.
Example usage:
config, err := expconf.ParseAnyExperimentConfigYAML(bytes)
var cluster_default_storage expconf.CheckpointStorage = ...
// Use the cluster checkpoint storage if the user did not specify one.
config.RawCheckpointStorage = schemas.Merge(
config.RawCheckpointStorage, &cluster_default_storage
)
func ParsedAdaptiveASHAConfigV0 ¶
func ParsedAdaptiveASHAConfigV0() interface{}
func ParsedAdaptiveConfigV0 ¶
func ParsedAdaptiveConfigV0() interface{}
func ParsedAdaptiveSimpleConfigV0 ¶
func ParsedAdaptiveSimpleConfigV0() interface{}
func ParsedAsyncHalvingConfigV0 ¶
func ParsedAsyncHalvingConfigV0() interface{}
func ParsedAzureConfigV0 ¶
func ParsedAzureConfigV0() interface{}
func ParsedBindMountV0 ¶
func ParsedBindMountV0() interface{}
func ParsedBindMountsConfigV0 ¶
func ParsedBindMountsConfigV0() interface{}
func ParsedCategoricalHyperparameterV0 ¶
func ParsedCategoricalHyperparameterV0() interface{}
func ParsedCheckGridHyperparameterV0 ¶
func ParsedCheckGridHyperparameterV0() interface{}
func ParsedCheckPositiveLengthV0 ¶
func ParsedCheckPositiveLengthV0() interface{}
func ParsedCheckpointStorageConfigV0 ¶
func ParsedCheckpointStorageConfigV0() interface{}
func ParsedConstHyperparameterV0 ¶
func ParsedConstHyperparameterV0() interface{}
func ParsedCustomConfigV0 ¶
func ParsedCustomConfigV0() interface{}
func ParsedDeviceV0 ¶
func ParsedDeviceV0() interface{}
func ParsedDevicesConfigV0 ¶
func ParsedDevicesConfigV0() interface{}
func ParsedDirectoryConfigV0 ¶
func ParsedDirectoryConfigV0() interface{}
func ParsedDoubleHyperparameterV0 ¶
func ParsedDoubleHyperparameterV0() interface{}
func ParsedEnvironmentConfigV0 ¶
func ParsedEnvironmentConfigV0() interface{}
func ParsedEnvironmentImageMapV0 ¶
func ParsedEnvironmentImageMapV0() interface{}
func ParsedEnvironmentImageV0 ¶
func ParsedEnvironmentImageV0() interface{}
func ParsedEnvironmentVariablesMapV0 ¶
func ParsedEnvironmentVariablesMapV0() interface{}
func ParsedEnvironmentVariablesV0 ¶
func ParsedEnvironmentVariablesV0() interface{}
func ParsedExperimentConfigV0 ¶
func ParsedExperimentConfigV0() interface{}
func ParsedGCSConfigV0 ¶
func ParsedGCSConfigV0() interface{}
func ParsedGridConfigV0 ¶
func ParsedGridConfigV0() interface{}
func ParsedHyperparameterV0 ¶
func ParsedHyperparameterV0() interface{}
func ParsedHyperparametersV0 ¶
func ParsedHyperparametersV0() interface{}
func ParsedIntHyperparameterV0 ¶
func ParsedIntHyperparameterV0() interface{}
func ParsedKerberosConfigV0 ¶
func ParsedKerberosConfigV0() interface{}
func ParsedLengthV0 ¶
func ParsedLengthV0() interface{}
func ParsedLogActionCancelRetriesV0 ¶
func ParsedLogActionCancelRetriesV0() interface{}
func ParsedLogActionExcludeNodeV0 ¶
func ParsedLogActionExcludeNodeV0() interface{}
func ParsedLogActionV0 ¶
func ParsedLogActionV0() interface{}
func ParsedLogHyperparameterV0 ¶
func ParsedLogHyperparameterV0() interface{}
func ParsedLogPolicyV0 ¶
func ParsedLogPolicyV0() interface{}
func ParsedOptimizationsConfigV0 ¶
func ParsedOptimizationsConfigV0() interface{}
func ParsedPbsConfigV0 ¶
func ParsedPbsConfigV0() interface{}
func ParsedProfilingConfigV0 ¶
func ParsedProfilingConfigV0() interface{}
func ParsedProxyPortV0 ¶
func ParsedProxyPortV0() interface{}
func ParsedProxyPortsConfigV0 ¶
func ParsedProxyPortsConfigV0() interface{}
func ParsedRandomConfigV0 ¶
func ParsedRandomConfigV0() interface{}
func ParsedRegistryAuthV0 ¶
func ParsedRegistryAuthV0() interface{}
func ParsedReproducibilityConfigV0 ¶
func ParsedReproducibilityConfigV0() interface{}
func ParsedResourcesConfigV0 ¶
func ParsedResourcesConfigV0() interface{}
func ParsedS3ConfigV0 ¶
func ParsedS3ConfigV0() interface{}
func ParsedSearcherConfigV0 ¶
func ParsedSearcherConfigV0() interface{}
func ParsedSearcherLengthV0 ¶
func ParsedSearcherLengthV0() interface{}
func ParsedSecurityConfigV0 ¶
func ParsedSecurityConfigV0() interface{}
func ParsedSharedFSConfigV0 ¶
func ParsedSharedFSConfigV0() interface{}
func ParsedSingleConfigV0 ¶
func ParsedSingleConfigV0() interface{}
func ParsedSlurmConfigV0 ¶
func ParsedSlurmConfigV0() interface{}
func ParsedSyncHalvingConfigV0 ¶
func ParsedSyncHalvingConfigV0() interface{}
func ParsedTensorboardStorageConfigV0 ¶
func ParsedTensorboardStorageConfigV0() interface{}
func ParsedTestRootV0 ¶
func ParsedTestRootV0() interface{}
func ParsedTestSubV0 ¶
func ParsedTestSubV0() interface{}
func ParsedTestUnionAV0 ¶
func ParsedTestUnionAV0() interface{}
func ParsedTestUnionBV0 ¶
func ParsedTestUnionBV0() interface{}
func ParsedTestUnionV0 ¶
func ParsedTestUnionV0() interface{}
func SaneBytes ¶
SaneBytes will ensure that bytes for a given schema object are valid. Unlike IsComplete, SaneBytes operates on a byte array, because if you unmarshal into the object and then check, you will already have silently dropped the unrecognized fields.
The Schema object is only used for its type, and it may be nil.
func UnionMerge ¶
func UnionMerge[T any](obj T, src T) T
UnionMerge implments the typical Merge logic for union types. The key is to merge all the common fields unconditionally, but to only merge the src's union member into the obj's union member if they are the same member, or if obj has no member.
func WithDefaults ¶
func WithDefaults[T any](obj T) T
WithDefaults will recurse through structs, maps, and slices, setting default values for any struct fields whose struct implements the Defaultable pusedointerface. This lets us read default values out of json-schema automatically.
There are some forms of defaults which must be filled at runtimes, such as giving a default name to experiments with no name. This can be accomplished by implementing the RuntimeDefaultable interface for that object. See ExperimentConfig for an example.
Example usage:
config, err := expconf.ParseAnyExperimentConfigYAML(bytes)
// Use the cluster checkpoint storage if the user did not specify one.
config.RawCheckpointStorage = schemas.Merge(
config.RawCheckpointStorage, &cluster_default_storage
)
// Define any remaining undefined values.
config = schemas.WithDefaults(config)
Types ¶
type Schema ¶
type Schema interface {
ParsedSchema() interface{}
SanityValidator() *jsonschema.Schema
CompletenessValidator() *jsonschema.Schema
}
Schema defines some basic knowledge needed by helper functions like SaneBytes or IsComplete. Outside of testing, the Schema interface should always be defined in generated code.
Source Files
Directories