Common Optimization Interfaces¶

The protocols that underlie the Core Classes of This Package.

The core classes contain several convenience features that are not relevant to the API defined by this package. For example, a class need not have an attribute np_random in order to satistfy the SingleOptimizable interface. This means that the core classes themselves are not suited to check whether an object satisfies one of our interfaces.

Instead, this definition is done by the protocols defined in this module. They are derived from a subclass of typing.Protocol and are all runtime_checkable. This subclass is described in The Internal Machinery, but it extends the checks performed by standard runtime-checkable protocols.

This page lists the pure protocols and their properties. Their individual documentation is kept concise. For the full details, refer to the Core Classes of This Package.

class cernml.coi.protocols.Problem(*args, **kwargs)¶

Bases: AttrCheckProtocol, Protocol

Pure protocol version of coi.Problem.

As with the full class, this merely presents the commonalities between very different protocols such as Env and SingleOptimizable. On its own, it isn’t terribly interesting.

metadata: InfoDict = mappingproxy({'render_modes': [], 'cern.machine': <Machine.NO_MACHINE: 'no machine'>, 'cern.japc': False, 'cern.cancellable': False})¶

Capabilities and behavior of this optimization problem.

While the keys of this mapping are free-form, there is a list of Standard Metadata Keys.

Ideally, this is an immutable, class-scope attribute that does not get replaced after the problem has been instantiated. Unfortunately, there are various exceptions to this:

Wrappers often cannot define their metadata at the class scope. For them, metadata only makes sense at the instance level.
Some optimization problems can determine some of their metadata (e.g. "render_fps") only at instantiation time. In this case, metadata often exists both at the class and instance scope, but they disagree on some items.

So far, no optimization problem has been observed that modifies an existing metadata mapping. If any updates are necessary, it is considered best practice to deepcopy() the existing instance, modify the copy, and then replace the original.

The protocol defines metadata as a dict for compatibility reasons, but actually binds it to a types.MappingProxyType object. This is done as a safety measure, to prevent accidental overwrites of globally visible data.

property render_mode: str | None¶

The chosen render mode.

This is either None (no rendering) or an item from the list in the "render_modes" metadata. See also the list of Standard Render Modes.

This attribute is expected to be set inside __init__() and not changed again afterwards.

This is marked as read-only for compatibility with gymnasium.Wrapper, where it is not writable. However, implementors of this protocol are expected to replace this with a either a regular attribute or a writable property.

The default implementation of this property simply looks up render_mode in the instance dict. You can set this value with a line like the following:

vars(self)["render_mode"] = "human"

property spec: MinimalEnvSpec | None¶

Pre-instantiation metadata on how the problem was initialized.

This property is usually set by make(). You generally should not modify it yourself. Wrappers should deepcopy() the spec of the wrapped environment and make their modifications on the copy.

The type of this property is marked as a minimal compromise between cernml.coi.registration.EnvSpec and gymnasium.envs.registration.EnvSpec. This is to convince MyPy that Env implements this protocol.

This property is marked as read-only so that MyPy treats it as covariant. All subclasses replace it with a regular attribute, however. The function cernml.coi.make() circumvents the property and sets the spec directly in the instance dict:

vars(problem)["spec"] = the_spec

close() → None¶

Perform any necessary cleanup.

Note that the context manager protocol is not required by this protocol. However, because every problem has a close method, you can always use contextlib.closing() to emulate it, even if a concrete problem is not a context manager itself.

property unwrapped: Problem¶

Return the core problem.

This exists to support the Wrapper design pattern. Note that the type of this property is Problem, whereas that of gymnasium.Env.unwrapped is Env.

render() → Any¶

Render the environment.

This must act according to the render_mode passed to __init__(). All supported modes must be declared in the metadata item "render_modes".

The default implementation simply raises NotImplementedError. Implementors are encouraged to call it in case of an unknown render mode in order to fail in a conventionally understood manner:

class MyProblem:
    metadata = {
        "render_modes": ["human", "matplotlib_figures"]
    }

    def __init__(self, render_mode=None):
        self.render_mode = render_mode
        ...

    def render(self):
        if self.render_mode == "human":
            return ...
        if self.render_mode == "matplotlib_figures":
            return ...
        # Raise `NotImplementedError` on unknown render
        # mode.
        super().render()

Note

Even though this method raises NotImplementedError, it is not an abstract method. It need not be implemented by implementors who don’t support any rendering.

get_wrapper_attr(name: str) → Any¶

Gets the attribute name from the environment.

This exists to support the Wrapper design pattern.

class cernml.coi.protocols.SingleOptimizable(*args, **kwargs)¶

Bases: Problem, Protocol[ParamType]

Pure protocol version of coi.SingleOptimizable.

Minimal implementors who subclass this protocol directly must at least:

define optimization_space;
implement get_initial_params();
implement compute_single_objective(),

to qualify as a subclass.

render_mode: str | None = None¶: The render mode as determined during initialization. Unlike for Protocol, this is a regular attribute. Nonetheless, this attribute is expected to be set inside __init__() and then not changed again.

optimization_space: Space[ParamType]¶: The phase space of parameters returned by get_initial_params() and accepted by compute_single_objective(). This attribute is merely annotated, not defaulted. Implementors must provide it themselves.

constraints: Sequence[Constraint] = ()¶: The constraints that apply to this optimization problem. Not all optimization algorithms are able to handle constraints and they may be violated by an algorithm.

objective_name: str = ''¶: A custom name for the objective function.

param_names: Sequence[str] = ()¶: Custom names for each of the parameters of the problem. If not empty, this should have exactly as many items as the optimization_space.

constraint_names: Sequence[str] = ()¶: Custom names for each of the constraints of the problem. If not empty, this should have exactly as many items as the constraints.

abstractmethod get_initial_params( *, seed: int | None = None, options: InfoDict | None = None, ) → ParamType¶

Return an initial set of parameters for optimization.

Because the protocol version of this method cannot assume that the optimization problem has an RNG, it does not come with the same default implementation as coi.SingleOptimizable.get_initial_params().

Parameters:

seed – Optional. If passed and the problem contains a random-number generator (RNG), the RNG is re-seeded with this value.
options – A mapping of custom options that a problem may interpret as desired.

Returns:

A set of parameters suitable to be passed to compute_single_objective(). This should be within the bounds of optimization_space. If it isn’t, a host application may still choose to pass this value to compute_single_objective() in order to reset the problem’s internal state.

abstractmethod compute_single_objective( params: ParamType, ) → SupportsFloat¶

Perform an optimization step.

Parameters:: params – The parameters for which the loss shall be calculated. This must have the same structure, as optimization_space. It should be within the bounds of the space, but if this is exactly the value returned by get_initial_params(), it might exceed them.
Returns:: The objective associated with these parameters, also known as loss or cost. Numerical optimizers may want to minimize this value.

class cernml.coi.protocols.FunctionOptimizable(*args, **kwargs)¶

Bases: Problem, Protocol[ParamType]

Pure protocol version of coi.FunctionOptimizable.

Minimal implementors who subclass this protocol directly must at least implement the three abstract methods – get_optimization_space(), get_initial_params(), and compute_function_objective() – to qualify as a subclass.

render_mode: str | None = None¶: The render mode as determined during initialization. Unlike for Protocol, this is a regular attribute. Nonetheless, this attribute is expected to be set inside __init__() and then not changed again.

constraints: Sequence[Constraint] = []¶: Custom names for each of the constraints of the problem. If not empty, this should have exactly as many items as the constraints.

abstractmethod get_optimization_space( cycle_time: float, ) → Space[ParamType]¶

Return the optimization space for a given point in time.

This should return the phase space of the parameters, ideally a Box. While the bounds of the returned space may depend on cycle_time, its shape should not.

The default implementation raises NotImplementedError.

abstractmethod get_initial_params( cycle_time: float, *, seed: int | None = None, options: InfoDict | None = None, ) → ParamType¶

Return an initial set of parameters for optimization.

Because the protocol version of this method cannot assume that the optimization problem has an RNG, it does not come with the same default implementation as coi.FunctionOptimizable.get_initial_params().

Parameters:

cycle_time – The point in time at which the objective is being optimized. Typically measured in integer milliseconds from the start of a cycle in the machine.
seed – Optional. If passed and the problem contains a random-number generator (RNG), the RNG is re-seeded with this value.
options – A mapping of custom options that a problem may interpret as desired.

Returns:

A set of parameters suitable to be passed to compute_function_objective(). This should be within the bounds of the space returned by get_optimization_space(). If it isn’t, a host application may still choose to pass this value to compute_function_objective() in order to reset the problem’s internal state.

abstractmethod compute_function_objective( cycle_time: float, params: ParamType, ) → float¶

Perform an optimization step.

Parameters:

cycle_time – The point in time at which the objective is being optimized. Typically measured in integer milliseconds from the start of a cycle in the machine.
params – The parameters for which the loss shall be calculated. This must have the same structure as the space returned by get_optimization_space(). It should be within the bounds of the space, but if this is exactly the value returned by get_initial_params(), it might exceed them.

Returns:

The objective associated with these parameters, also known as loss or cost. Numerical optimizers may want to minimize this value.

get_objective_function_name() → str¶: Return the name of the objective function, if any.

get_param_function_names() → Sequence[str]¶: Return the names of the functions being modified.

override_skeleton_points() → Sequence[float] | None¶

Hook to let the problem choose the skeleton points.

Note that returning an empty sequence means that there are no suitable skeleton points. To leave the choice to the user, return the default value of None instead.

class cernml.coi.protocols.Env¶

Bases: Problem, Generic[ObsType, ActType]

See gymnasium.Env. Re-exported for convenience.

class cernml.coi.protocols.HasNpRandom(*args, **kwargs)¶

Bases: AttrCheckProtocol, Protocol

Pure protocol version of coi.HasNpRandom.

np_random: Generator¶

A random-number generator (RNG) for exclusive use by the class that owns it. It may be implemented as a property to support lazy initialization. If implemented as a property, it must be settable in order to support reseeding.

Start-up functions like SingleOptimizable.get_initial_params() or Env.step() should make sure to reseed it if required.

class cernml.coi.protocols.Space

Bases: Generic[T_co]

See gymnasium.spaces.Space. Re-exported for convenience.

cernml.coi.protocols.Constraint¶: alias of scipy.optimize.LinearConstraint | scipy.optimize.NonlinearConstraint

cernml.coi.protocols.InfoDict¶: alias of dict[str, Any]

cernml.coi.protocols.ParamType: TypeVar¶: The generic type variable for SingleOptimizable. This is exported for the user’s convenience.

Common Optimization Interfaces¶

Table of Contents

Previous topic

Next topic

This Page