IterationControl.jl

A lightweight package for controlling iterative algorithms, with a view to training and optimizing machine learning models.

Builds on EarlyStopping.jl and inspired by LearningStrategies.jl.

Other related software: DynamicIterators.jl.

Installation

using PkgPkg.add("IterationControl")Pkg.add("Tables") # optional

Quick Start

Suppose you have some kind of object SquareRooter(x), for iteratively computing approximations to the square root of x:

model = SquareRooter(9)model.root

1.0

train!(model, 2) # train for 2 iterationsmodel.root

3.4

train!(model, 1) # train for 1 more iterationmodel.root

3.023529411764706

Then you can replace the integer argument n in train!(model, n) with a number of more sophisticated controls by "lifting" the method train! to the IterationControl.train! method defined in this package:

using IterationControlIterationControl.train!(model::SquareRooter, n::Int) =  train!(model, n) # lifting

By definition, the lifted train! has the same functionality as the original one:

model = SquareRooter(9)IterationControl.train!(model, 2)model.root

3.4

But now we can also do this:

model = SquareRooter(9)IterationControl.train!(model, Step(2), NumberLimit(3), Info(m->m.root));

[ Info: 3.4
[ Info: 3.00009155413138
[ Info: 3.0
[ Info: final loss: 0.0
[ Info: final training loss: 1.3969838619232178e-9
[ Info: Stop triggered by NumberLimit(3) stopping criterion.

Here each control is repeatedly applied in sequence until one of them triggers a stop:

The first control Step(2) says, "Train the model two more iterations"
The NumberLimit(3) control asks, "Have I been applied 3 times yet?", signaling a stop (at the end of the current control cycle) if so
The Info control logs the value of the function m -> m.root, evaluated on model, to Info.

In this example only the NumberLimit(3) control can terminate model iteration.

Info

To try out the example above, you can use a built-in version of SquareRooter. Before executing the code above, run using IterationControl.SquareRooterExample.

The Controls section documents all the controls provided by IterationControl.jl, including those from EarlyStopping.jl.

Loss-based stopping criteria

If model admits a method returning a loss then we can lift that method to IterationControl.loss to enable control using a loss-based stopping criteria, such as a loss threshold. Let's demonstrate this for the built-in square rooting example, for which loss returns the difference between the root approximation and the actual 64-bit precision square root:

using IterationControlusing IterationControl.SquareRooterExamplemodel = SquareRooter(4)train!(model, 1)loss(model)

2.25

IterationControl.loss(model::SquareRooter) = loss(model) # liftingtraces = IterationControl.train!(    model,	Step(1),	Threshold(0.0001),	Callback(m->m.root),	WithLossDo(),);

[ Info: final loss: 3.716891878724482e-7
[ Info: final training loss: 0.0006096631752661175
[ Info: Stop triggered by Threshold(0.0001) stopping criterion.

julia> traces4-element Vector{Any}: ControlTrace{<:Step}, ending (new_iterations = 3, new_steps = 3) ControlTrace{<:Threshold}, ending (loss = 3.716891878724482e-7, excess = -9.962831081212756e-5) ControlTrace{<:Callback}, ending (called_back = 2.0000000929222947,) ControlTrace{<:WithLossDo}, ending (loss = 3.716891878724482e-7, called_back = 3.716891878724482e-7)

Control traces

In the demonstration above we used Callback instead of Info to suppress the logging of the root value to stdout, and we added a special callback, WithLossDo(), which calls the loss function on model. (More generally, WithLossDo(f) applies f to each loss.) The return values of callbacks can be inspected from the control traces returned by IterationControl.train!:

step, threshold, callback, withlossdo = traces;

julia> callbackControlTrace  control: Callback(Main.var"#5#6"(), false, nothing, false)  called_back: [2.05, 2.000609756097561, 2.0000000929222947]  table: (called_back = …)  triggered_stop: false

julia> withlossdoControlTrace  control: WithLossDo(identity, false, nothing)  loss: [0.20249999999999968, 0.002439396192741583, 3.716891878724482e-7]  called_back: [0.20249999999999968, 0.002439396192741583, 3.716891878724482e-7]  table: (loss = …, called_back = …)  triggered_stop: false

julia> withlossdo.loss3-element Vector{Float64}: 0.20249999999999968 0.002439396192741583 3.716891878724482e-7

Control traces have a property called table, returning a named tuple of vectors, but they also implement the Tables.jl interface, allowing conversion to common Julia table formats:

julia> threshold.table(loss = [0.20249999999999968, 0.002439396192741583, 3.716891878724482e-7], excess = [0.2023999999999997, 0.002339396192741583, -9.962831081212756e-5])

julia> using DataFramesjulia> DataFrame(threshold)3×2 DataFrame Row │ loss        excess           │ Float64     Float64     ─────┼─────────────────────────   1 │ 0.2025       0.2024   2 │ 0.0024394    0.0023394   3 │ 3.71689e-7  -9.96283e-5

It is possible to inspect traces "live" by running IterationControl.train! in an asynchronous task. See the document string for details.

Training losses

In many applications to machine learning, "loss" will be an out-of-sample loss, computed after some iterations. If model additionally generates user-inspectable "training losses" (one per iteration) then similarly lifting the appropriate access function to IterationControl.training_losses enables Prechelt's progress-modified generalization loss stopping criterion, PQ.

PQ is the only criterion from the EarlyStopping.jl package not otherwise enabled when IterationControl.loss is overloaded as above.

Reference. Prechelt, Lutz (1998): "Early Stopping - But When?", in Neural Networks: Tricks of the Trade, ed. G. Orr, Springer.

The interface just described is sufficient for controlling conventional machine learning models with an iteration parameter (see Early Stopping for a Tree Booster for a demo).

Online and incremental training

For online or incremental training, lift the method for ingesting data into the model to IterationControl.ingest!(model, datum) and use the control Data(data). Here data is any iterator generating the datum items to be ingested (one per application of the control). By default, the Data control becomes passive after data is exhausted. See Access to model through a wrapper below on dealing with any model wrapping necessary to implement data ingestion.

For a demonstration, see Controlling a Particle Tracker.

Access to model through a wrapper

Note that functions ordinarily applied to model by some control (e.g., a Callback) will instead be applied to IterationControl.expose(model) if this function has been appropriately overloaded.

IterationControl.train! — Function

train!(model, controls...; verbosity=1, compact=false, traces=[])

Train the iterative model, repeatedly applying the provided controls until one of the controls triggers a stop. The first control must be Step(k), for some integer k specifying the number of training iterations between control cycles; Step() is the same as Step(1).

Returns a vector of traces, one for each control. A trace records the evolving internal state of each control, unless compact=false, in which case only the current state is stored.

using IterationControlmodel = IterationControl.SquareRooter(4)controls = [Step(1), Threshold(1e-7), NumberLimit(100), InvalidValue()]julia> traces = IterationControl.train!(model, controls...)[ Info: final loss: 8.881784197001252e-15[ Info: final training loss: 9.292229252721995e-8[ Info: Stop triggered by Threshold(1.0e-7) stopping criterion.julia> model.root2.000000000000002julia> traces[2]ControlTrace  control: Threshold(1.0e-7)  loss: [ …, 0.002439396192741583, 3.716891878724482e-7, 8.881784197001252e-15]  excess: [ …, 0.002439296192741583, 2.716891878724482e-7, -9.99999911182158e-8]  table: (loss = …, excess = …)  message: Stop triggered by Threshold(1.0e-7) stopping criterion.  triggered_stop: truejulia> traces[2].loss5-element Vector{Float64}: 2.25 0.20249999999999968 0.002439396192741583 3.716891878724482e-7 8.881784197001252e-15

It is assumed that IterationControl.train!(model, n::Int) has been implemented (be sure to include the Int annotation here!) and that this trains the model for n more iterations. Specific controls may require implementations of IterationControl.loss(model), IterationControl.training_losses(model), or IterationControl.ingest!(model, datum). Refer to documentation for details.

Extended help

Appending to traces and live tracking

For "live" access to the traces, while train! is running as an asynchronous task, specify traces=v, where v is bound to some copy of Any[] to be populated by the traces once the first control cycle completes, for subsequent mutation as iteration proceeds:

model = IterationControl.SquareRooter(4)controls = [Step(1), Threshold(0.01), Callback(_->sleep(0.5))]traces = []Threads.@spawn IterationControl.train!(model, controls...; verbosity=0, traces=traces)length(traces) # 0sleep(2)length(traces) # 3julia> traces3-element Vector{Any}: ControlTrace{<:Step}, ending (new_iterations = 3, new_steps = 3) ControlTrace{<:Threshold}, ending (loss = 0.002439396192741583, excess = -0.007560603807258417) ControlTrace{<:Callback}, ending (called_back = nothing,)julia> traces[2].loss3-element Vector{Float64}: 2.25 0.20249999999999968 0.002439396192741583

Alternatively, one may specify traces=v, where v is the vector of traces returned by a previous call to train! with the same control types. In that v is extended, and matches the vector returned by the second call to train!. Continuing the above example:

controls[2] = Threshold(1e-100)IterationControl.train!(model, controls...; traces=traces)julia> traces[2].loss6-element Vector{Float64}: 2.25 0.20249999999999968 0.002439396192741583 3.716891878724482e-7 8.881784197001252e-15 0.0julia> traces[1].new_iterations6-element Vector{Int64}: 1 2 3 1 2 3

source

IterationControl.loss — Function

IterationControl.loss(model)

Return the current loss associated with model, which will be some measure of the quality of model, such as an out-of-sample statistical loss. The smaller the loss, the better the inferred quality.

This method must be overloaded for any model type wanting to support an iteration control with IterationControl.needs_loss(control) == true.

source

IterationControl.training_losses — Function

IterationControl.training_losses(model)

Return a vector of all, or recent, per-iteration "training" losses, since training of model began. Typically, a training loss is an in-sample statistical measure of the quality of model, as opposed to the (out-of-sample) loss returned by IterationControl.loss(model). The smaller the loss, the better the inferred quality.

Training losses are returned in chronological order.

This method must be overloaded for any model type wanting to support an iteration control with IterationControl.needs_training_losses(control) == true.

source

IterationControl.ingest! — Function

IterationControl.ingest!(model, datum)

Have model ingest new training datum, without actually training model.

This method must be overloaded for any model type wanting to support an iteration control with IterationControl.needs_ingest(control) == true.

source

IterationControl.expose — Function

IterationControl.expose(model)

Unwrap model to expose that part on which user-specified functions in iteration controls should act. For example, in IterationControl.train!(model, Step(), Info(model->print(model)), it is IterationControl.expose(model) that gets logged to Info, not model.

source

IterationControl.ControlTrace — Type

ControlTrace{C}

Type of object recording a trace for a control of type C.

All instances have the following public properties (accessed with dot syntax):

control
table: a named-tuple of vectors recording a history of the internal state of the control as iteration proceeded
triggered_stop: true if control was responsible for stopping training
message: The message that was logged to Info or Warn when IterationControl.train! completed (available even if a small verbosity level suppressed the logging)

Additional properties forward to the table property to access a particular component of state. Note here "state" means public state (i.e, what is returned by IterationControl.public_state(control, rawstate), where rawstate is the raw internal state).

A ControlTrace object implements the Tables.jl interface, so it can be converted to a dataframe, for example:

using IterationControlimport IterationControl.SquareRooterusing DataFramesmodel = SquareRooter(9)julia> traces = IterationControl.train!(    model,    Step(),    Threshold(0.01),    WithLossDo(x->x<0.01),)[ Info: final loss: 0.0005493331704400362[ Info: final training loss: 0.02343785763332562[ Info: Stop triggered by Threshold(0.01) stopping criterion.3-element Vector{Any}: ControlTrace{<:Step}, ending (new_iterations = 4, new_steps = 4) ControlTrace{<:Threshold}, ending (loss = 0.0005493331704400362, excess = -0.009450666829559964) ControlTrace{<:WithLossDo}, ending (loss = 0.0005493331704400362, called_back = true)julia> trace = traces[end]ControlTrace  control: WithLossDo{var"#2#3"}(var"#2#3"(), false, nothing)  loss: [ …, 2.5599999999999987, 0.1417301038062284, 0.0005493331704400362]  called_back: Bool[ …, 0, 0, 1]  table: (loss = …, called_back = …)  triggered_stop: falsejulia> trace.loss4-element Vector{Float64}: 16.0  2.5599999999999987  0.1417301038062284  0.0005493331704400362julia> DataFrame(trace)4×2 DataFrame Row │ loss          called_back     │ Float64       Bool─────┼───────────────────────────   1 │ 16.0                false   2 │  2.56               false   3 │  0.14173            false   4 │  0.000549333         true

source