Freezing Learned Parameters

If a model has a hyper-parameter called frozen, and that model is bound to data in a machine mach, then calling fit!(mach; kwargs...) has no effect on the learned parameters, unless mach has not yet been trained. This is true even if other hyper-parameters have changed, or one specifies new views of the data, as in fit(mach, rows=...). This can be useful for freezing one component model in a Pipeline or model Stack when retraining that component is expensive.

While most models don't have the frozen hyper-parameter, you can achieve the same effect by wrapping your model using Freezable as explained below.

By wrapping a self-tuning model, TunedModel(model; ...) in Freezable, you can evaluate its performance using evaluate without expensive nested cross-validation. The tuned parameters are then based just on the first evaluate training fold. So results must be interpreted carefully.

Experimental feature

The Freezable wrapper is not a mature feature. It is difficult to reason about in parallelized workflows (e.g., when an acceleration option is not the default CPU1()) and may have unexpected behaviour in those cases.

MLJBase.Freezable — Function

Freezable(model; frozen=true, cache=true)

Wrap model so fit! is a no-op after the first training pass. Place the wrapper inside a Pipeline, Stack, TunedModel, or any other NetworkComposite model, and the inner component skips retraining even when the parent rebuilds its learning network on a row change.

Set frozen=false to allow normal retraining. Use freeze! and thaw! to toggle after construction. Set cache=false to prioritize memory over speed.

Example 1: Freezing a single model

This example and the next assume you have MLJDecisionTreeInterface in your environment.

using MLJ    # or `using MLJBase, MLJModels`

X, y = make_regression(100)
DecisionTreeRegressor = @load DecisionTreeRegressor pkg=DecisionTree

model = Freezable(DecisionTreeRegressor())  # frozen=true by default
mach  = machine(model, X, y)

fit!(mach)                    # first fit trains
fit!(mach, rows=1:50)         # no-op while frozen
thaw!(model)
fit!(mach, rows=1:50)         # retrains

Example 2: Freezing a component inside a pipeline

using MLJ    # or `using MLJBase, MLJModels, MLJTransforms`

X, y = make_blobs(200)
DecisionTreeClassifier = @load DecisionTreeClassifier pkg=DecisionTree

pipe = Pipeline(
    scaler = Freezable(Standardizer()),
    clf    = DecisionTreeClassifier(),
)
mach = machine(pipe, X, y)
fit!(mach, rows=1:100)        # both components train
fit!(mach, rows=101:200)      # only clf retrains; scaler is frozen

source