EarlyStopping#

class ignite.handlers.early_stopping.EarlyStopping(patience, score_function, trainer, threshold=0.0, cumulative=False, threshold_mode='abs', mode='max', min_delta=None, min_delta_mode=None, cumulative_delta=None)[source]#

EarlyStopping handler can be used to stop the training if no improvement after a given number of events.

Parameters:

patience (int) – Number of events to wait if no improvement and then stop the training.
score_function (Callable) – It should be a function taking a single argument, an Engine object, and return a score float. An improvement is considered if the score is higher (for mode='max') or lower (for mode='min').
trainer (Engine) – Trainer engine to stop the run if no improvement.
threshold (float) – A minimum change in the score to qualify as an improvement. For mode='max', it is a minimum increase; for mode='min', it is a minimum decrease. An improvement is only considered if the change exceeds the threshold determined by threshold and threshold_mode.
cumulative (bool) – If True, threshold defines the change since the last patience reset, otherwise it defines the change after the last event. Default value is False.
threshold_mode (Literal['abs', 'rel']) –
Determines whether threshold is an absolute change or a relative change.
- In 'abs' mode:
  - For mode='max': improvement if score > best_score + threshold
  - For mode='min': improvement if score < best_score - threshold
- In 'rel' mode:
  - For mode='max': improvement if score > best_score * (1 + threshold)
  - For mode='min': improvement if score < best_score * (1 - threshold)
Possible values are "abs" and "rel". Default value is "abs".
mode (Literal['min', 'max']) – Whether to maximize ('max') or minimize ('min') the score. Default is 'max'.
min_delta (float | None) –
min_delta_mode (Literal['abs', 'rel'] | None) –
cumulative_delta (bool | None) –

Examples

from ignite.engine import Engine, Events
from ignite.handlers import EarlyStopping

def score_function(engine):
    val_loss = engine.state.metrics["nll"]
    return -val_loss

handler = EarlyStopping(
    patience=10,
    score_function=score_function,
    trainer=trainer,
)

# Note: the handler is attached to an *Evaluator*
evaluator.add_event_handler(Events.COMPLETED, handler)

Changed in version 0.6.0: Renamed min_delta_mode to threshold_mode. Renamed min_delta to threshold. Renamed cumulative_delta to cumulative. Added get_default_score_fn() and get_default_event_filter() static helpers.

Changed in version 0.5.4: Added mode parameter to support minimization in addition to maximization. Added min_delta_mode parameter to support both absolute and relative improvements.

Methods

`attach`	Attaches the early stopping handler to an engine and registers its reset callback.
`get_default_event_filter`	Build an event filter that delays early-stopping checks until the trainer has completed at least `after` epochs.
`get_default_score_fn`	Helper method to build a score function from an engine metric name.
`load_state_dict`	Method replace internal state of the class with provided state dict data.
`reset`	Reset the early stopping state, including the counter and best score.
`state_dict`	Method returns state dict with `counter` and `best_score`.

attach(engine, event=Events.COMPLETED, reset_engine=None, reset_event=Events.STARTED, *args, **kwargs)[source]#

Attaches the early stopping handler to an engine and registers its reset callback.

This method will: 1. Add the early stopping evaluation logic (self) to engine on the given event. 2. Add the reset method to reset_engine (or engine if not provided) on the given reset_event.

Parameters:

engine (Engine) – The engine to attach the early stopping evaluation to (typically an evaluator).
event (Any) – The event on engine that triggers the early stopping check. Default is COMPLETED.
reset_engine (Engine | None) – The engine to attach the reset callback to (typically the trainer). If None, defaults to engine.
reset_event (Any) – The event on reset_engine that triggers the handler state reset. Default is STARTED.
args (Any) –
kwargs (Any) –

Return type:

None

New in version 0.5.4.

get_default_event_filter(after)[source]#

Build an event filter that delays early-stopping checks until the trainer has completed at least after epochs.

This implements a warmup window for early stopping without coupling the warmup logic to the handler itself, so it composes with any event the handler is attached to (epoch, iteration, custom). The filter consults the trainer’s epoch counter (self.trainer.state.epoch) rather than the host engine’s event count, so the warmup is well-defined even when the handler is attached to an evaluator that re-runs from scratch each epoch.

Parameters:: after (int) – minimum number of trainer epochs that must have completed before the early-stopping handler is allowed to run. Must be a non-negative integer.
Returns:: A callable matching the event_filter signature (engine, event) -> bool.
Return type:: Callable[[Engine, int], bool]

Examples

from ignite.engine import Events
from ignite.handlers import EarlyStopping

handler = EarlyStopping(patience=5, score_function=score_fn, trainer=trainer)

# Skip the first 3 trainer epochs, then enforce early stopping.
evaluator.add_event_handler(
    Events.COMPLETED(event_filter=handler.get_default_event_filter(after=3)),
    handler,
)

New in version 0.6.0.

static get_default_score_fn(metric_name, score_sign=1.0)[source]#

Helper method to build a score function from an engine metric name.

The returned callable reads engine.state.metrics[metric_name] and multiplies it by score_sign. Use score_sign=-1.0 for error-like metrics (smaller is better) when the handler is configured with mode="max" so that decreases in the metric register as score improvements.

Parameters:

metric_name (str) – name of the metric in engine.state.metrics.
score_sign (float) – 1.0 (default) or -1.0. For error-like metrics where smaller is better, use -1.0.

Returns:

A callable taking an Engine and returning float.

Return type:

Callable

Examples

from ignite.handlers import EarlyStopping

# Validation accuracy: larger is better, default mode="max"
score_fn = EarlyStopping.get_default_score_fn("accuracy")
handler = EarlyStopping(patience=5, score_function=score_fn, trainer=trainer)

# Validation loss: smaller is better, flip the sign so larger is better
neg_loss_fn = EarlyStopping.get_default_score_fn("loss", -1.0)
handler = EarlyStopping(patience=5, score_function=neg_loss_fn, trainer=trainer)

New in version 0.6.0.

load_state_dict(state_dict)[source]#

Method replace internal state of the class with provided state dict data.

Parameters:: state_dict (Mapping) – a dict with “counter” and “best_score” keys/values.
Return type:: None

reset()[source]#

Reset the early stopping state, including the counter and best score.

New in version 0.5.4.

Return type:: None

state_dict()[source]#

Method returns state dict with counter and best_score. Can be used to save internal state of the class.

Return type:: OrderedDict[str, Any]

EarlyStopping#

Search Docs