GitHub

LinearBinaryClassifier

LinearBinaryClassifier

A model type for constructing a linear binary classifier, based on GLM.jl, and implementing the MLJ model interface.

From MLJ, the type can be imported using

LinearBinaryClassifier = @load LinearBinaryClassifier pkg=GLM

Do model = LinearBinaryClassifier() to construct an instance with default hyper-parameters. Provide keyword arguments to override hyper-parameter defaults, as in LinearBinaryClassifier(fit_intercept=...).

LinearBinaryClassifier is a generalized linear model, specialised to the case of a binary target variable, with a user-specified link function. Options exist to specify an intercept or offset feature.

Training data

In MLJ or MLJBase, bind an instance model to data with one of:

mach = machine(model, X, y)
mach = machine(model, X, y, w)

Here

  • X: is any table of input features (eg, a DataFrame) whose columns are of scitype Continuous; check the scitype with schema(X)
  • y: is the target, which can be any AbstractVector whose element scitype is <:OrderedFactor(2) or <:Multiclass(2); check the scitype with schema(y)
  • w: is a vector of Real per-observation weights

Train the machine using fit!(mach, rows=...).

Hyper-parameters

  • fit_intercept=true: Whether to calculate the intercept for this model. If set to false, no intercept will be calculated (e.g. the data is expected to be centered)
  • link=GLM.LogitLink: The function which links the linear prediction function to the probability of a particular outcome or class. This must have type GLM.Link01. Options include GLM.LogitLink(), GLM.ProbitLink(), CloglogLink(),CauchitLink()`.
  • offsetcol=nothing: Name of the column to be used as an offset, if any. An offset is a variable which is known to have a coefficient of 1.
  • maxiter::Integer=30: The maximum number of iterations allowed to achieve convergence.
  • atol::Real=1e-6: Absolute threshold for convergence. Convergence is achieved when the relative change in deviance is less than `max(rtol*dev, atol). This term exists to avoid failure when deviance is unchanged except for rounding errors.
  • rtol::Real=1e-6: Relative threshold for convergence. Convergence is achieved when the relative change in deviance is less than `max(rtol*dev, atol). This term exists to avoid failure when deviance is unchanged except for rounding errors.
  • minstepfac::Real=0.001: Minimum step fraction. Must be between 0 and 1. Lower bound for the factor used to update the linear fit.
  • report_keys: Vector of keys for the report. Possible keys are: :deviance, :dof_residual, :stderror, :vcov, :coef_table and :glm_model. By default only :glm_model is excluded.

Operations

  • predict(mach, Xnew): Return predictions of the target given features Xnew having the same scitype as X above. Predictions are probabilistic.
  • predict_mode(mach, Xnew): Return the modes of the probabilistic predictions returned above.

Fitted parameters

The fields of fitted_params(mach) are:

  • features: The names of the features used during model fitting.
  • coef: The linear coefficients determined by the model.
  • intercept: The intercept determined by the model.

Report

The fields of report(mach) are:

  • deviance: Measure of deviance of fitted model with respect to a perfectly fitted model. For a linear model, this is the weighted residual sum of squares
  • dof_residual: The degrees of freedom for residuals, when meaningful.
  • stderror: The standard errors of the coefficients.
  • vcov: The estimated variance-covariance matrix of the coefficient estimates.
  • coef_table: Table which displays coefficients and summarizes their significance and confidence intervals.
  • glm_model: The raw fitted model returned by GLM.lm. Note this points to training data. Refer to the GLM.jl documentation for usage.

Examples

using MLJ
import GLM ## namespace must be available

LinearBinaryClassifier = @load LinearBinaryClassifier pkg=GLM
clf = LinearBinaryClassifier(fit_intercept=false, link=GLM.ProbitLink())

X, y = @load_crabs

mach = machine(clf, X, y) |> fit!

Xnew = (;FL = [8.1, 24.8, 7.2],
        RW = [5.1, 25.7, 6.4],
        CL = [15.9, 46.7, 14.3],
        CW = [18.7, 59.7, 12.2],
        BD = [6.2, 23.6, 8.4],)

yhat = predict(mach, Xnew) ## probabilistic predictions
pdf(yhat, levels(y)) ## probability matrix
p_B = pdf.(yhat, "B")
class_labels = predict_mode(mach, Xnew)

fitted_params(mach).features
fitted_params(mach).coef
fitted_params(mach).intercept

report(mach)

See also LinearRegressor, LinearCountRegressor