API

LinearRegression(; fit_intercept)

Objective function: $|Xθ - y|₂²/2$.

RidgeRegression()
RidgeRegression(λ; lambda, fit_intercept, penalize_intercept, scale_penalty_with_samples)

Objective function: $|Xθ - y|₂²/2 + n⋅λ|θ|₂²/2$, where $n$ is the number of samples size(X, 1). With scale_penalty_with_samples = false the objective function is $|Xθ - y|₂²/2 + λ|θ|₂²/2$.

LassoRegression()
LassoRegression(λ; lambda, fit_intercept, penalize_intercept, scale_penalty_with_samples)

Objective function: $|Xθ - y|₂²/2 + n⋅λ|θ|₁$, where $n$ is the number of samples size(X, 1). With scale_penalty_with_samples = false the objective function is $|Xθ - y|₂²/2 + λ|θ|₁$

ElasticNetRegression()
ElasticNetRegression(λ)
ElasticNetRegression(λ, γ; lambda, gamma, fit_intercept, penalize_intercept, scale_penalty_with_samples)

Objective function: $|Xθ - y|₂²/2 + n⋅λ|θ|₂²/2 + n⋅γ|θ|₁$, where $n$ is the number of samples size(X, 1). With scale_penalty_with_samples = false the objective function is $|Xθ - y|₂²/2 + λ|θ|₂²/2 + γ|θ|₁$

HuberRegression()
HuberRegression(δ)
HuberRegression(δ, λ)
HuberRegression(δ, λ, γ; delta, lambda, gamma, penalty, fit_intercept, scale_penalty_with_samples, penalize_intercept)

Huber Regression with objective:

$∑ρ(Xθ - y) + n⋅λ|θ|₂²/2 + n⋅γ|θ|₁$

Where ρ is the Huber function ρ(r) = r²/2if|r|≤δandρ(r)=δ(|r|-δ/2)otherwise andnis the number of samplessize(X, 1). Withscalepenaltywith_samples = falsethe objective function is∑ρ(Xθ - y) + λ|θ|₂²/2 + γ|θ|₁`.

QuantileRegression()
QuantileRegression(δ)
QuantileRegression(δ, λ)
QuantileRegression(δ, λ, γ; delta, lambda, gamma, penalty, fit_intercept, scale_penalty_with_samples, penalize_intercept)

Quantile Regression with objective:

$∑ρ(Xθ - y) + n⋅λ|θ|₂²/2 + n⋅γ|θ|₁$

Where ρ is the check function ρ(r) = r(δ - 1(r < 0)) and $n$ is the number of samples size(X, 1). With scale_penalty_with_samples = false the objective function is $∑ρ(Xθ - y) + λ|θ|₂²/2 + γ|θ|₁$.

LADRegression()
LADRegression(λ)
LADRegression(λ, γ; lambda, gamma, penalty, scale_penalty_with_samples, fit_intercept, penalize_intercept)

Least Absolute Deviation regression with objective:

$|Xθ - y|₁ + n⋅λ|θ|₂²/2 + n⋅γ|θ|₁$ where $n$ is the number of samples size(X, 1). With scale_penalty_with_samples = false the objective function is $|Xθ - y|₁ + λ|θ|₂²/2 + γ|θ|₁$.

This is a specific type of Quantile Regression with δ=0.5 (median).

GeneralizedLinearRegression{L<:Loss, P<:Penalty}

Generalized Linear Regression (GLR) model with objective function:

$L(y, Xθ) + P(θ)$

where L is a loss function, P a penalty, y is the vector of observed response, X is the feature matrix and θ the vector of parameters. If scale_penalty_with_samples = true (default) the penalty is automatically scaled with the number of samples.

Special cases include:

• OLS regression: L2 loss, no penalty.
• Ridge regression: L2 loss, L2 penalty.
• Lasso regression: L2 loss, L1 penalty.
• Logistic regression: Logit loss, [no,L1,L2] penalty.

RobustRegression()
RobustRegression(ρ)
RobustRegression(ρ, λ)
RobustRegression(ρ, λ, γ; rho, lambda, gamma, penalty, fit_intercept, scale_penalty_with_samples, penalize_intercept)

Objective function: $∑ρ(Xθ - y) + n⋅λ|θ|₂² + n⋅γ|θ|₁$ where ρ is a given function on the residuals and $n$ is the number of samples size(X, 1). With scale_penalty_with_samples = false the objective function is $∑ρ(Xθ - y) + λ|θ|₂² + γ|θ|₁$.

LogisticRegression()
LogisticRegression(λ)
LogisticRegression(λ, γ; lambda, gamma, penalty, fit_intercept, penalize_intercept, scale_penalty_with_samples, multi_class, nclasses)

Objective function: $L(y, Xθ) + n⋅λ|θ|₂²/2 + n⋅γ|θ|₁$ where L is either the logistic loss in the binary case or the multinomial loss otherwise and $n$ is the number of samples size(X, 1). With scale_penalty_with_samples = false the objective function is $L(y, Xθ) + λ|θ|₂²/2 + γ|θ|₁$.

MultinomialRegression(a; kwa...)

Objective function: $L(y, Xθ) + n⋅λ|θ|₂²/2 + n⋅γ|θ|₁$ where L is the multinomial loss and $n$ is the number of samples size(X, 1). With scale_penalty_with_samples = false the objective function is $L(y, Xθ) + λ|θ|₂²/2 + γ|θ|₁$.

LinearRegressor

A model type for constructing a linear regressor, based on MLJLinearModels.jl, and implementing the MLJ model interface.

From MLJ, the type can be imported using

LinearRegressor = @load LinearRegressor pkg=MLJLinearModels

Do model = LinearRegressor() to construct an instance with default hyper-parameters.

This model provides standard linear regression with objective function

$|Xθ - y|₂²/2$

Different solver options exist, as indicated under "Hyperparameters" below.

Training data

In MLJ or MLJBase, bind an instance model to data with

mach = machine(model, X, y)

where:

• X is any table of input features (eg, a DataFrame) whose columns have Continuous scitype; check column scitypes with schema(X)

• y is the target, which can be any AbstractVector whose element scitype is Continuous; check the scitype with scitype(y)

Train the machine using fit!(mach, rows=...).

Hyperparameters

• fit_intercept::Bool

  whether to fit the intercept or not. Default: true

• solver::Union{Nothing, MLJLinearModels.Solver}

  "any instance of MLJLinearModels.Analytical. Use Analytical() for Cholesky and CG()=Analytical(iterative=true) for conjugate-gradient.

  If solver = nothing (default) then Analytical() is used. Default: nothing

Example

using MLJ
X, y = make_regression()
mach = fit!(machine(LinearRegressor(), X, y))
predict(mach, X)
fitted_params(mach)

RidgeRegressor

A model type for constructing a ridge regressor, based on MLJLinearModels.jl, and implementing the MLJ model interface.

From MLJ, the type can be imported using

RidgeRegressor = @load RidgeRegressor pkg=MLJLinearModels

Do model = RidgeRegressor() to construct an instance with default hyper-parameters.

Ridge regression is a linear model with objective function

$|Xθ - y|₂²/2 + n⋅λ|θ|₂²/2$

where $n$ is the number of observations.

If scale_penalty_with_samples = false then the objective function is instead

$|Xθ - y|₂²/2 + λ|θ|₂²/2$.

Different solver options exist, as indicated under "Hyperparameters" below.

Training data

In MLJ or MLJBase, bind an instance model to data with

mach = machine(model, X, y)

where:

• X is any table of input features (eg, a DataFrame) whose columns have Continuous scitype; check column scitypes with schema(X)

• y is the target, which can be any AbstractVector whose element scitype is Continuous; check the scitype with scitype(y)

Train the machine using fit!(mach, rows=...).

Hyperparameters

• lambda::Real

  strength of the L2 regularization. Default: 1.0

• fit_intercept::Bool

  whether to fit the intercept or not. Default: true

• penalize_intercept::Bool

  whether to penalize the intercept. Default: false

• scale_penalty_with_samples::Bool

  whether to scale the penalty with the number of observations. Default: true

• solver::Union{Nothing, MLJLinearModels.Solver}

  any instance of MLJLinearModels.Analytical. Use Analytical() for Cholesky and CG()=Analytical(iterative=true) for conjugate-gradient. If solver = nothing (default) then Analytical() is used. Default: nothing

Example

using MLJ
X, y = make_regression()
mach = fit!(machine(RidgeRegressor(), X, y))
predict(mach, X)
fitted_params(mach)

See also ElasticNetRegressor.

LassoRegressor

A model type for constructing a lasso regressor, based on MLJLinearModels.jl, and implementing the MLJ model interface.

From MLJ, the type can be imported using

LassoRegressor = @load LassoRegressor pkg=MLJLinearModels

Do model = LassoRegressor() to construct an instance with default hyper-parameters.

Lasso regression is a linear model with objective function

$|Xθ - y|₂²/2 + n⋅λ|θ|₁$

where $n$ is the number of observations.

If scale_penalty_with_samples = false the objective function is

$|Xθ - y|₂²/2 + λ|θ|₁$.

Different solver options exist, as indicated under "Hyperparameters" below.

Training data

In MLJ or MLJBase, bind an instance model to data with

mach = machine(model, X, y)

where:

• X is any table of input features (eg, a DataFrame) whose columns have Continuous scitype; check column scitypes with schema(X)

• y is the target, which can be any AbstractVector whose element scitype is Continuous; check the scitype with scitype(y)

Train the machine using fit!(mach, rows=...).

Hyperparameters

• lambda::Real

  strength of the L1 regularization. Default: 1.0

• fit_intercept::Bool

  whether to fit the intercept or not. Default: true

• penalize_intercept::Bool

  whether to penalize the intercept. Default: false

• scale_penalty_with_samples::Bool

  whether to scale the penalty with the number of observations. Default: true

• solver::Union{Nothing, MLJLinearModels.Solver}

  any instance of MLJLinearModels.ProxGrad. If solver=nothing (default) then ProxGrad(accel=true) (FISTA) is used. Solver aliases: FISTA(; kwargs...) = ProxGrad(accel=true, kwargs...), ISTA(; kwargs...) = ProxGrad(accel=false, kwargs...). Default: nothing

Example

using MLJ
X, y = make_regression()
mach = fit!(machine(LassoRegressor(), X, y))
predict(mach, X)
fitted_params(mach)

See also ElasticNetRegressor.

ElasticNetRegressor

A model type for constructing a elastic net regressor, based on MLJLinearModels.jl, and implementing the MLJ model interface.

From MLJ, the type can be imported using

ElasticNetRegressor = @load ElasticNetRegressor pkg=MLJLinearModels

Do model = ElasticNetRegressor() to construct an instance with default hyper-parameters.

Elastic net is a linear model with objective function

$|Xθ - y|₂²/2 + n⋅λ|θ|₂²/2 + n⋅γ|θ|₁$

where $n$ is the number of observations.

If scale_penalty_with_samples = false the objective function is instead

$|Xθ - y|₂²/2 + λ|θ|₂²/2 + γ|θ|₁$.

Different solver options exist, as indicated under "Hyperparameters" below.

Training data

In MLJ or MLJBase, bind an instance model to data with

mach = machine(model, X, y)

where:

• X is any table of input features (eg, a DataFrame) whose columns have Continuous scitype; check column scitypes with schema(X)

• y is the target, which can be any AbstractVector whose element scitype is Continuous; check the scitype with scitype(y)

Train the machine using fit!(mach, rows=...).

Hyperparameters

• lambda::Real

  strength of the L2 regularization. Default: 1.0

• gamma::Real

  strength of the L1 regularization. Default: 0.0

• fit_intercept::Bool

  whether to fit the intercept or not. Default: true

• penalize_intercept::Bool

  whether to penalize the intercept. Default: false

• scale_penalty_with_samples::Bool

  whether to scale the penalty with the number of observations. Default: true

• solver::Union{Nothing, MLJLinearModels.Solver}

  any instance of MLJLinearModels.ProxGrad.

  If solver=nothing (default) then ProxGrad(accel=true) (FISTA) is used.

  Solver aliases: FISTA(; kwargs...) = ProxGrad(accel=true, kwargs...), ISTA(; kwargs...) = ProxGrad(accel=false, kwargs...). Default: nothing

Example

using MLJ
X, y = make_regression()
mach = fit!(machine(ElasticNetRegressor(), X, y))
predict(mach, X)
fitted_params(mach)

See also LassoRegressor.

HuberRegressor

A model type for constructing a huber regressor, based on MLJLinearModels.jl, and implementing the MLJ model interface.

From MLJ, the type can be imported using

HuberRegressor = @load HuberRegressor pkg=MLJLinearModels

Do model = HuberRegressor() to construct an instance with default hyper-parameters.

This model coincides with RobustRegressor, with the exception that the robust loss, rho, is fixed to HuberRho(delta), where delta is a new hyperparameter.

Different solver options exist, as indicated under "Hyperparameters" below.

Training data

In MLJ or MLJBase, bind an instance model to data with

mach = machine(model, X, y)

where:

• X is any table of input features (eg, a DataFrame) whose columns have Continuous scitype; check column scitypes with schema(X)

• y is the target, which can be any AbstractVector whose element scitype is Continuous; check the scitype with scitype(y)

Train the machine using fit!(mach, rows=...).

Hyperparameters

• delta::Real

  parameterizes the HuberRho function (radius of the ball within which the loss is a quadratic loss) Default: 0.5

• lambda::Real

  strength of the regularizer if penalty is :l2 or :l1. Strength of the L2 regularizer if penalty is :en. Default: 1.0

• gamma::Real

  strength of the L1 regularizer if penalty is :en. Default: 0.0

• penalty::Union{String, Symbol}

  the penalty to use, either :l2, :l1, :en (elastic net) or :none. Default: :l2

• fit_intercept::Bool

  whether to fit the intercept or not. Default: true

• penalize_intercept::Bool

  whether to penalize the intercept. Default: false

• scale_penalty_with_samples::Bool

  whether to scale the penalty with the number of observations. Default: true

• solver::Union{Nothing, MLJLinearModels.Solver}

  some instance of MLJLinearModels.S where S is one of: LBFGS, IWLSCG, Newton, NewtonCG, if penalty = :l2, and ProxGrad otherwise.

  If solver = nothing (default) then LBFGS() is used, if penalty = :l2, and otherwise ProxGrad(accel=true) (FISTA) is used.

  Solver aliases: FISTA(; kwargs...) = ProxGrad(accel=true, kwargs...), ISTA(; kwargs...) = ProxGrad(accel=false, kwargs...) Default: nothing

Example

using MLJ
X, y = make_regression()
mach = fit!(machine(HuberRegressor(), X, y))
predict(mach, X)
fitted_params(mach)

See also RobustRegressor, QuantileRegressor.

QuantileRegressor

A model type for constructing a quantile regressor, based on MLJLinearModels.jl, and implementing the MLJ model interface.

From MLJ, the type can be imported using

QuantileRegressor = @load QuantileRegressor pkg=MLJLinearModels

Do model = QuantileRegressor() to construct an instance with default hyper-parameters.

This model coincides with RobustRegressor, with the exception that the robust loss, rho, is fixed to QuantileRho(delta), where delta is a new hyperparameter.

Different solver options exist, as indicated under "Hyperparameters" below.

Training data

In MLJ or MLJBase, bind an instance model to data with

mach = machine(model, X, y)

where:

• X is any table of input features (eg, a DataFrame) whose columns have Continuous scitype; check column scitypes with schema(X)

• y is the target, which can be any AbstractVector whose element scitype is Continuous; check the scitype with scitype(y)

Train the machine using fit!(mach, rows=...).

Hyperparameters

• delta::Real

  parameterizes the QuantileRho function (indicating the quantile to use with default 0.5 for the median regression) Default: 0.5

• lambda::Real

  strength of the regularizer if penalty is :l2 or :l1. Strength of the L2 regularizer if penalty is :en. Default: 1.0

• gamma::Real

  strength of the L1 regularizer if penalty is :en. Default: 0.0

• penalty::Union{String, Symbol}

  the penalty to use, either :l2, :l1, :en (elastic net) or :none. Default: :l2

• fit_intercept::Bool

  whether to fit the intercept or not. Default: true

• penalize_intercept::Bool

  whether to penalize the intercept. Default: false

• scale_penalty_with_samples::Bool

  whether to scale the penalty with the number of observations. Default: true

• solver::Union{Nothing, MLJLinearModels.Solver}

  some instance of MLJLinearModels.S where S is one of: LBFGS, IWLSCG, if penalty = :l2, and ProxGrad otherwise.

  If solver = nothing (default) then LBFGS() is used, if penalty = :l2, and otherwise ProxGrad(accel=true) (FISTA) is used.

  Solver aliases: FISTA(; kwargs...) = ProxGrad(accel=true, kwargs...), ISTA(; kwargs...) = ProxGrad(accel=false, kwargs...) Default: nothing

Example

using MLJ
X, y = make_regression()
mach = fit!(machine(QuantileRegressor(), X, y))
predict(mach, X)
fitted_params(mach)

See also RobustRegressor, HuberRegressor.

LADRegressor

A model type for constructing a lad regressor, based on MLJLinearModels.jl, and implementing the MLJ model interface.

From MLJ, the type can be imported using

LADRegressor = @load LADRegressor pkg=MLJLinearModels

Do model = LADRegressor() to construct an instance with default hyper-parameters.

Least absolute deviation regression is a linear model with objective function

$∑ρ(Xθ - y) + n⋅λ|θ|₂² + n⋅γ|θ|₁$

where $ρ$ is the absolute loss and $n$ is the number of observations.

If scale_penalty_with_samples = false the objective function is instead

$∑ρ(Xθ - y) + λ|θ|₂² + γ|θ|₁$.

Different solver options exist, as indicated under "Hyperparameters" below.

Training data

In MLJ or MLJBase, bind an instance model to data with

mach = machine(model, X, y)

where:

• X is any table of input features (eg, a DataFrame) whose columns have Continuous scitype; check column scitypes with schema(X)

• y is the target, which can be any AbstractVector whose element scitype is Continuous; check the scitype with scitype(y)

Train the machine using fit!(mach, rows=...).

Hyperparameters

See also RobustRegressor.

Parameters

• lambda::Real

  strength of the regularizer if penalty is :l2 or :l1. Strength of the L2 regularizer if penalty is :en. Default: 1.0

• gamma::Real

  strength of the L1 regularizer if penalty is :en. Default: 0.0

• penalty::Union{String, Symbol}

  the penalty to use, either :l2, :l1, :en (elastic net) or :none. Default: :l2

• fit_intercept::Bool

  whether to fit the intercept or not. Default: true

• penalize_intercept::Bool

  whether to penalize the intercept. Default: false

• scale_penalty_with_samples::Bool

  whether to scale the penalty with the number of observations. Default: true

• solver::Union{Nothing, MLJLinearModels.Solver}

  some instance of MLJLinearModels.S where S is one of: LBFGS, IWLSCG, if penalty = :l2, and ProxGrad otherwise.

  If solver = nothing (default) then LBFGS() is used, if penalty = :l2, and otherwise ProxGrad(accel=true) (FISTA) is used.

  Solver aliases: FISTA(; kwargs...) = ProxGrad(accel=true, kwargs...), ISTA(; kwargs...) = ProxGrad(accel=false, kwargs...) Default: nothing

Example

using MLJ
X, y = make_regression()
mach = fit!(machine(LADRegressor(), X, y))
predict(mach, X)
fitted_params(mach)

RobustRegressor

A model type for constructing a robust regressor, based on MLJLinearModels.jl, and implementing the MLJ model interface.

From MLJ, the type can be imported using

RobustRegressor = @load RobustRegressor pkg=MLJLinearModels

Do model = RobustRegressor() to construct an instance with default hyper-parameters.

Robust regression is a linear model with objective function

$∑ρ(Xθ - y) + n⋅λ|θ|₂² + n⋅γ|θ|₁$

where $ρ$ is a robust loss function (e.g. the Huber function) and $n$ is the number of observations.

If scale_penalty_with_samples = false the objective function is instead

$∑ρ(Xθ - y) + λ|θ|₂² + γ|θ|₁$.

Different solver options exist, as indicated under "Hyperparameters" below.

Training data

In MLJ or MLJBase, bind an instance model to data with

mach = machine(model, X, y)

where:

• X is any table of input features (eg, a DataFrame) whose columns have Continuous scitype; check column scitypes with schema(X)

• y is the target, which can be any AbstractVector whose element scitype is Continuous; check the scitype with scitype(y)

Train the machine using fit!(mach, rows=...).

Hyperparameters

• rho::MLJLinearModels.RobustRho

  the type of robust loss, which can be any instance of MLJLinearModels.L where L is one of: AndrewsRho, BisquareRho, FairRho, HuberRho, LogisticRho, QuantileRho, TalwarRho, HuberRho, TalwarRho. Default: HuberRho(0.1)

• lambda::Real

  strength of the regularizer if penalty is :l2 or :l1. Strength of the L2 regularizer if penalty is :en. Default: 1.0

• gamma::Real

  strength of the L1 regularizer if penalty is :en. Default: 0.0

• penalty::Union{String, Symbol}

  the penalty to use, either :l2, :l1, :en (elastic net) or :none. Default: :l2

• fit_intercept::Bool

  whether to fit the intercept or not. Default: true

• penalize_intercept::Bool

  whether to penalize the intercept. Default: false

• scale_penalty_with_samples::Bool

  whether to scale the penalty with the number of observations. Default: true

• solver::Union{Nothing, MLJLinearModels.Solver}

  some instance of MLJLinearModels.S where S is one of: LBFGS, IWLSCG, Newton, NewtonCG, if penalty = :l2, and ProxGrad otherwise.

  If solver = nothing (default) then LBFGS() is used, if penalty = :l2, and otherwise ProxGrad(accel=true) (FISTA) is used.

  Solver aliases: FISTA(; kwargs...) = ProxGrad(accel=true, kwargs...), ISTA(; kwargs...) = ProxGrad(accel=false, kwargs...) Default: nothing

Example

using MLJ
X, y = make_regression()
mach = fit!(machine(RobustRegressor(), X, y))
predict(mach, X)
fitted_params(mach)

See also HuberRegressor, QuantileRegressor.

LogisticClassifier

A model type for constructing a logistic classifier, based on MLJLinearModels.jl, and implementing the MLJ model interface.

From MLJ, the type can be imported using

LogisticClassifier = @load LogisticClassifier pkg=MLJLinearModels

Do model = LogisticClassifier() to construct an instance with default hyper-parameters.

This model is more commonly known as "logistic regression". It is a standard classifier for both binary and multiclass classification. The objective function applies either a logistic loss (binary target) or multinomial (softmax) loss, and has a mixed L1/L2 penalty:

$L(y, Xθ) + n⋅λ|θ|₂²/2 + n⋅γ|θ|₁$.

Here $L$ is either MLJLinearModels.LogisticLoss or MLJLinearModels.MultiClassLoss, $λ$ and $γ$ indicate the strength of the L2 (resp. L1) regularization components and $n$ is the number of training observations.

With scale_penalty_with_samples = false the objective function is instead

$L(y, Xθ) + λ|θ|₂²/2 + γ|θ|₁$.

Training data

In MLJ or MLJBase, bind an instance model to data with

mach = machine(model, X, y)

where:

• X is any table of input features (eg, a DataFrame) whose columns have Continuous scitype; check column scitypes with schema(X)

• y is the target, which can be any AbstractVector whose element scitype is <:OrderedFactor or <:Multiclass; check the scitype with scitype(y)

Train the machine using fit!(mach, rows=...).

Hyperparameters

• lambda::Real

  strength of the regularizer if penalty is :l2 or :l1 and strength of the L2 regularizer if penalty is :en. Default: eps()

• gamma::Real

  strength of the L1 regularizer if penalty is :en. Default: 0.0

• penalty::Union{String, Symbol}

  the penalty to use, either :l2, :l1, :en (elastic net) or :none. Default: :l2

• fit_intercept::Bool

  whether to fit the intercept or not. Default: true

• penalize_intercept::Bool

  whether to penalize the intercept. Default: false

• scale_penalty_with_samples::Bool

  whether to scale the penalty with the number of samples. Default: true

• solver::Union{Nothing, MLJLinearModels.Solver}

  some instance of MLJLinearModels.S where S is one of: LBFGS, Newton, NewtonCG, ProxGrad; but subject to the following restrictions:

  • If penalty = :l2, ProxGrad is disallowed. Otherwise, ProxGrad is the only option.

  • Unless scitype(y) <: Finite{2} (binary target) Newton is disallowed.

  If solver = nothing (default) then ProxGrad(accel=true) (FISTA) is used, unless gamma = 0, in which case LBFGS() is used.

  Solver aliases: FISTA(; kwargs...) = ProxGrad(accel=true, kwargs...), ISTA(; kwargs...) = ProxGrad(accel=false, kwargs...) Default: nothing

Example

using MLJ
X, y = make_blobs(centers = 2)
mach = fit!(machine(LogisticClassifier(), X, y))
predict(mach, X)
fitted_params(mach)

See also MultinomialClassifier.

MultinomialClassifier

A model type for constructing a multinomial classifier, based on MLJLinearModels.jl, and implementing the MLJ model interface.

From MLJ, the type can be imported using

MultinomialClassifier = @load MultinomialClassifier pkg=MLJLinearModels

Do model = MultinomialClassifier() to construct an instance with default hyper-parameters.

This model coincides with LogisticClassifier, except certain optimizations possible in the special binary case will not be applied. Its hyperparameters are identical.

Training data

In MLJ or MLJBase, bind an instance model to data with

mach = machine(model, X, y)

where:

• X is any table of input features (eg, a DataFrame) whose columns have Continuous scitype; check column scitypes with schema(X)

• y is the target, which can be any AbstractVector whose element scitype is <:OrderedFactor or <:Multiclass; check the scitype with scitype(y)

Train the machine using fit!(mach, rows=...).

Hyperparameters

• lambda::Real

  strength of the regularizer if penalty is :l2 or :l1. Strength of the L2 regularizer if penalty is :en. Default: eps()

• gamma::Real

  strength of the L1 regularizer if penalty is :en. Default: 0.0

• penalty::Union{String, Symbol}

  the penalty to use, either :l2, :l1, :en (elastic net) or :none. Default: :l2

• fit_intercept::Bool

  whether to fit the intercept or not. Default: true

• penalize_intercept::Bool

  whether to penalize the intercept. Default: false

• scale_penalty_with_samples::Bool

  whether to scale the penalty with the number of samples. Default: true

• solver::Union{Nothing, MLJLinearModels.Solver}

  some instance of MLJLinearModels.S where S is one of: LBFGS, NewtonCG, ProxGrad; but subject to the following restrictions:

  • If penalty = :l2, ProxGrad is disallowed. Otherwise, ProxGrad is the only option.

  • Unless scitype(y) <: Finite{2} (binary target) Newton is disallowed.

  If solver = nothing (default) then ProxGrad(accel=true) (FISTA) is used, unless gamma = 0, in which case LBFGS() is used.

  Solver aliases: FISTA(; kwargs...) = ProxGrad(accel=true, kwargs...), ISTA(; kwargs...) = ProxGrad(accel=false, kwargs...) Default: nothing

Example

using MLJ
X, y = make_blobs(centers = 3)
mach = fit!(machine(MultinomialClassifier(), X, y))
predict(mach, X)
fitted_params(mach)

See also LogisticClassifier.