# Optimizers Reference

**URL:** <https://guildai.org/t/optimizers-reference/43>\
**Category:** Reference\
**Created:** [June 7, 2020, 12:36am UTC](https://guildai.org/t/optimizers-reference/43 "2020-06-07T00:36:46Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![guildai](https://yyz1.discourse-cdn.com/flex031/user_avatar/guildai.org/guildai/32/103_2.png) [@guildai](https://guildai.org/u/guildai)\
**Post date:** [June 7, 2020, 12:36am UTC](https://guildai.org/t/optimizers-reference/43/1 "2020-06-07T00:36:46Z")

</div>

## Overview

An _optimizer_ is a Guild operation that runs a [_batch_](https://guildai.org/docs/runs#batches). A batch generates one or more _trial runs_ or _trials_. Optimizers are able to suggest flag values to minimize or maximize an objective.

Below is a list of supported optimizers.

| | |
| --- | --- |
| [gp](#gp) | Sequential optimizer using Gaussian processes. |
| [forest](#forest) | Sequential optimizer using decision trees. |
| [gbrt](#gbrt) | Sequential optimizer using gradient boosted regression trees. |
| [random](#random) | Batch processor using randomly selected values. |

Use the default optimizer for an operation by specifying the `--optimize` option with [`guild run`](https://guildai.org/commands/run). The default optimizer can be defined for an operation using the [`optimizers`](https://guildai.org/reference/guildfile#operation-optimizers) attribute. Guild uses the [`gp`](#gp) optimizer if one is not otherwise defined for an operation.

Specify a named optimizer with the `--optimizer` option to [`guild run`](https://guildai.org/commands/run). A name may be one of the optimizers below or may be the name of an optimizer defined for the operation.

Optimizer flags are set using `--opt-flag` or `-Fo`. Optimizer flags are specified like other flags using the format `NAME=VALUE`.

To run the default optimizer for `train`:

```command
guild run train --optimize

```

To use the `forest` optimizer:

```command
guild run train --optimizer forest

```

For more examples, see [_Guild File Cheatsheet_](https://guildai.org/cheatsheets/guildfile#optimizers).

## gp

Bayesian optimizer using Gaussian processes.

Refer to [skopt API documentation](https://scikit-optimize.github.io/stable/modules/generated/skopt.gp_minimize.html) for details on this algorithm and its flags.

Aliases: `gaussian`, `bayesian`

### gp Flags

#### acq-func

_Function to minimize over the gaussian prior (default is `gp_hedge`)_

Choices:

| | |
| --- | --- |
| `LCB` | Lower confidence bound |
| `EI` | Negative expected improvement |
| `PI` | Negative probability of improvement |
| `gp_hedge` | Probabilistically use LCB, EI, or PI at every iteration |
| `EIps` | Negative expected improvement per second |
| `PIps` | Negative probability of improvement per second |

#### kappa

_Degree to which variance in the predicted values is taken into account (default is `1.96`)_

#### noise

_Level of noise associated with the objective (default is `gaussian`)_

Use `gaussian` if the objective returns noisy observations, otherwise specify the expected variance of the noise.

#### random-starts

_Number of trials using random values before optimizing (default is `3`)_

#### xi

_Improvement to seek over the previous best values (default is `0.05`)_

## forest

Sequential optimization using decision trees. Refer to [skopt API documentation](https://scikit-optimize.github.io/stable/modules/generated/skopt.forest_minimize.html) for details on this algorithm and its flags.

### forest Flags

#### kappa

_Degree to which variance in the predicted values is taken into account (default is `1.96`)_

#### random-starts

_Number of trials using random values before optimizing (default is `3`)_

#### xi

_Improvement to seek over the previous best values (default is `0.05`)_

## gbrt

Sequential optimization using gradient boosted regression trees.

Refer to [skopt API documentation](https://scikit-optimize.github.io/stable/modules/generated/skopt.gbrt_minimize.html) for details on this algorithm and its flags.

### gbrt Flags

#### kappa

_Degree to which variance in the predicted values is taken into account (default is `1.96`)_

#### random-starts

_Number of trials using random values before optimizing (default is `3`)_

#### xi

_Improvement to seek over the previous best values (default is `0.05`)_

## random

Batch processor supporting random flag value generation.

Values are selected from the search space distribution specified for each flag value.

This optimizer does not attempt to optimize an objective.

The random optimizers does not support any flags.