Making an Optimization Configurable via GUI

This section introduces a few useful, but less common interfaces defined by Gym and the COI.

Fig. 1: Sequence diagram of the Configurable API.

Some Problem classes have several parameters that determine certain details of how they are solved. A typical configurable parameter of environments is the reward objective, i.e. the minimum reward for a step upon which an episode is considered solved.

While these parameters can be set through the initializer, this has the problem that it is difficult to annotate them with limits, invariants, etc.

The Configurable API provides a uniform way for problem authors to declare parameters of their class that can be modified ahead of an optimization run. It also allows specifying certain variants for each parameter. Host applications can use this interface to present a configuration dialog to the user.

Basic Usage

Implementing the Configurable interface generally follows a three-step process:

1. Define your configurable parameters in __init__().

2. Implement get_config() and return a Config object. This declares your configurable parameters and their invariants.

3. Implement apply_config(), which receives a ConfigValues object. Transfer each value into your object. Apply any further checks and raise an exception if any fail.

Take this class for example:

>>> from cernml import coi
...
>>> class ExampleEnv(coi.Configurable):
...
...     # Step 0: Optionally accept the parameter in __init__().
...     def __init__(self, *, action_scale=1.0):
...         # Step 1: Define parameter in __init__().
...         self.action_scale = action_scale
...
...     def get_config(self):
...         # Step 2: Add parameter to the `Config` returned from
...         #     `get_config()`.
...         config = coi.Config()
...         config.add(
...             'action_scale',
...             self.action_scale,
...             label='Action scale (mrad)',
...             range=(0.0, 2.0),
...             default=1.0,
...         )
...         return config
...
...     def apply_config(self, values):
...         # Step 3: Transfer it from `values` to your env. Apply any
...         #     additional checks that are necessary.
...         if 0.0 < values.action_scale < 0.1:
...             raise coi.BadConfig(
...                 f"config 'action_scale' must be either 0.0 or "
...                 f"greater than 0.1: {values.action_scale!r}"
...             )
...         self.action_scale = values.action_scale
...
>>> env = ExampleEnv()
>>> isinstance(env, coi.Configurable)
True

A host application may use this interface as follows:

>>> env = ExampleEnv()
>>> config = env.get_config()
>>> # Present configs to the user.
>>> config.get_field_values()
{'action_scale': 1.0}
>>> # Transfer a user choice back to the env.
>>> values = config.validate_all({"action_scale": 1.5})
>>> values
namespace(action_scale=1.5)
>>> env.apply_config(values)
>>> env.action_scale
1.5

If either the validation or the application step fails, the host receives an exception it can catch and show to the user:

>>> config.validate_all({"action_scale": 2.5})
Traceback (most recent call last):
...
BadConfig: invalid value for action_scale: 2.5
>>> values = config.validate_all({"action_scale": 0.05})
>>> env.apply_config(values)
Traceback (most recent call last):
...
BadConfig: config 'action_scale' must be either 0.0 or greater than 0.1:
0.05
>>> env.action_scale
1.5

Nested Usage

If your class consists of multiple configurable components, you can combine their individual configs via Config.extend() as long as the names don’t overlap:

>>> class Kicker(coi.Configurable):
...     def __init__(self) -> None:
...         self.scale = 0.1
...
...     def get_config(self) -> coi.Config:
...         return coi.Config().add("scale", self.scale)
...
...     def apply_config(self, values: coi.ConfigValues) -> None:
...         self.scale = values.scale
...
>>> class LossMonitor(coi.Configurable):
...     def __init__(self) -> None:
...         self.min_reading = 1.0
...
...     def get_config(self) -> coi.Config:
...         return coi.Config().add("min_reading", self.min_reading)
...
...     def apply_config(self, values: coi.ConfigValues) -> None:
...         self.min_reading = values.min_reading
...
>>> class Problem(coi.Configurable):
...     def __init__(self) -> None:
...         self.kicker = Kicker()
...         self.monitor = LossMonitor()
...
...     def get_config(self) -> coi.Config:
...         return (
...             coi.Config()
...             .extend(self.kicker.get_config())
...             .extend(self.monitor.get_config())
...         )
...
...     def apply_config(self, values: coi.ConfigValues) -> None:
...         self.kicker.apply_config(values)
...         self.monitor.apply_config(values)
...
>>> problem = Problem()
>>> config = problem.get_config()
>>> config
<Config: ['scale', 'min_reading']>
>>> values = {'scale': 0.0, 'min_reading': 0.0}
>>> values = config.validate_all(values)
>>> problem.apply_config(values)
>>> problem.kicker.scale
0.0
>>> problem.monitor.min_reading
0.0