07-resource-management

README

Resource Management

This tour illustrates the basic contract to correctly work with closeable object references used in the library.

Many objects provided by the library offer some kind of resource management. In the first example, this is an OCM repository, the OCM component, component version and the access method. Another important kind of objects are the BlobAccess implementations.

Those objects may use external resources, like temporary file system content or caches. To get rid of those resources again, they offer a Close method.

To achieve the possibility to pass those objects around in non-functional call contexts they feature some kind of resource management. It allows to handle the life cycle of the resource in a completely local manner. To do so, a second method Dup is offered, which provides an independent reference to the original resources, which can be closed separately. The possible externally held resource are released with the close of the last reference.

This offers a simple contract to handle resources in functions or object methods:

1. a function creating such an object is responsible for the life cycle of its reference

   • if the object is returned, this responsibility is passed to its caller

     func f() (Object, error) {
         o, err:= Create()
         if err != nil {
             return nil, err
         }
         o.DoSomeThing()
         DoSomeThingOther(o)
         return o, nil
     }
     

   • otherwise, it must be closed at the end of the function (or if it is not used anymore)

     func f() error {
         o, err:= Create()
         if err != nil {
             return err
         }
         defer o.Close()
         o.DoSomeThing()
         DoSomeThingOther(o)
     }
     

   The object may be passed to any called function without bothering what this function does with this reference.

2. a function receiving such an object from a function as result it inherits the responsibility to close it again (see case 1)

3. a function receiving such an object as an argument can freely use it and a pass it around.

   func f(o Object) {
       o.DoSomeThing()
       DoSomeThingOther(o)
   }
   

   If it decides to store the reference in some state, it must use an own reference for this, obtained by a call to Dup. After obtaining an own reference the used storage context is responsible to close it again. It should never close the obtained reference, because the caller is responsible for this.

   func (r *State) f(o Object) (err error) {
       r.obj, err = o.Dup()
       return err
   }
   
   func (r *State) Close() error {
       if r.obj == nil {
           return nil
       }
       return r.obj.Close()
   }
   

Running the example

You can call the main program without any argument.

Walkthrough

The example is based on the initial getting started scenario. It separates the resource gathering from the handling of the found resources.

	// gathering resources, this is completely hidden
	// behind an implementation.
	resources, err := GatherResources(ctx, CachingFactory{})
	if err != nil {
		return err
	}

	var list errors.ErrorList

	list.Add(HandleResources(resources))

	// we are done, so close the resources, again.
	for i, r := range resources {
		list.Addf(nil, r.Close(), "closing resource %d", i)
	}
	return list.Result()

The resources are provided by an array of the interface Resource:

type Resource interface {
	GetIdentity() metav1.Identity
	GetType() string
	GetAccess() string
	GetData() ([]byte, error)

	SetError(s string)
	AddDataFromMethod(ctx ocm.ContextProvider, m ocm.AccessMethod) error

	Close() error
}

It encapsulates the technical resource handling and offers a Close method, also, to release potential local resources.

The example provides one implementation, using the original access method to cache the data to avoid additional copies.

// resource is a Resource implementation using
// the original access method to cache the content.
type resource struct {
	Identity     metav1.Identity
	ArtifactType string
	Access       string
	Data         blobaccess.BlobAccess
}

var _ Resource = (*resource)(nil)

func (r *resource) AddDataFromMethod(ctx ocm.ContextProvider, m ocm.AccessMethod) error {
	// provide an own reference to the method
	// to store this in the provided resource object.
	priv, err := m.Dup()
	if err != nil {
		return err
	}

	// release a possible former cache entry
	if r.Data != nil {
		r.Data.Close()
	}
	r.Data = priv.AsBlobAccess()
	// release obsolete blob access
	r.Access = m.AccessSpec().Describe(ctx.OCMContext())
	return nil
}

// Close releases the cached access.
func (r *resource) Close() error {
	c := r.Data
	if c == nil {
		return nil
	}
	r.Data = nil
	return c.Close()
}

The AddDataFromMethod uses Dup to provide an own reference to the access method, which is stored in the provided resource object. It implements the Close method to release this cached content, again. The responsibility for this reference is taken by the resourceobject.

In the GatherResources function, a repository access is created. It is not forwarded, and therefore closed, again, in this function.

	repo, err := ctx.RepositoryForSpec(spec)
	if err != nil {
		return nil, errors.Wrapf(err, "cannot setup repository")
	}

	// to release potentially allocated temporary resources,
	// many objects must be closed, if they should not be used
	// anymore.
	// This is typically done by a `defer` statement placed after a
	// successful object retrieval.
	defer repo.Close()

The same is done for the component version lookup.

	c, err := repo.LookupComponent("ocm.software/ocmcli")
	if err != nil {
		return nil, errors.Wrapf(err, "cannot lookup component")
	}
	defer c.Close()

Then the resource factory is used to create the Resource objects for the resources found in the component version.

	for _, r := range cv.GetResources() {
		res := factory.Create(
			r.Meta().GetIdentity(cv.GetDescriptor().Resources),
			r.Meta().GetType(),
		)
		acc, err := r.Access()
		if err != nil {
			res.SetError(err.Error())
		} else {
			m, err := acc.AccessMethod(cv)
			if err == nil {
				// delegate data handling to target
				// we don't know, how this is implemented.
				err = res.AddDataFromMethod(ctx, m)
				if err != nil {
					res.SetError(err.Error())
				}
				// release local usage of the access method object
				m.Close()
			} else {
				res.SetError(err.Error())
			}
		}
		resources = append(resources, res)
	}

Because the function cannot know what happens behind the call to AddDataFromMethod, it just closes everything what is created in the function, this also includes the access method (m).

Finally, it returns the resource array after all locally created references are correctly closed. The provided Resource objects have taken the responsibility for keeping their own references.

The resource handling function just uses the resources.

func HandleResources(resources []Resource) error {
	var list errors.ErrorList
	fmt.Printf("*** resources:\n")
	for i, r := range resources {
		fmt.Printf("  %2d: extra identity: %s\n", i+1, r.GetIdentity())
		fmt.Printf("      resource type:  %s\n", r.GetType())
		fmt.Printf("      access:         %s\n", r.GetAccess())
	}

	return list.Result()
}

The responsibility for closing the resources has been passed to the ResourceManagement functions, which calls the gather and the handling function. Therefore, it calls the Resource.Close function before finishing.

The final output of this example looks like:

versions for component ocm.software/ocmcli: ..., 0.9.0, 0.10.0, 0.11.0, 0.12.0, 0.12.1, 0.13.0, 0.14.0, 0.15.0, 0.17.0, 0.18.0, ...
looking up resources of the latest version:
  version:  0.17.0
  provider: ocm.software
*** resources:
   1: extra identity: "architecture"="amd64","name"="ocmcli","os"="linux"
      resource type:  executable
      access:         Local blob sha256:03a45dcde67ba565fe806cb5db67da3387f772f7c50af711a0edd6f802570c04[]
   2: extra identity: "architecture"="arm64","name"="ocmcli","os"="linux"
      resource type:  executable
      access:         Local blob sha256:5a622634ae43cf03eac91079389d83266891d1f9b2d8a3884cef6fe639180324[]
   3: extra identity: "architecture"="arm64","name"="ocmcli","os"="darwin"
      resource type:  executable
      access:         Local blob sha256:1482fe5b764e3a86cf96704d7a839ad7e53dcbfd4f5fce5405abffb1962153dd[]
   4: extra identity: "architecture"="amd64","name"="ocmcli","os"="darwin"
      resource type:  executable
      access:         Local blob sha256:805f181aff48511eea12c699ed1bbcee8bdc4c5168924e81058aff8715946875[]
   5: extra identity: "architecture"="amd64","name"="ocmcli","os"="windows"
      resource type:  executable
      access:         Local blob sha256:20839c68bf0c4cf99444d78ebb93f53358fa9e95fe806f186220bd21d520efa7[]
   6: extra identity: "name"="ocmcli-image"
      resource type:  ociImage
      access:         OCI artifact ghcr.io/open-component-model/ocm/ocm.software/ocmcli/ocmcli-image:0.17.0@sha256:16fb52a1cb11c867bd058f4124dea53fbab94229842cc14b52653c2e80b1cede