Skip to content

Metrics

add_y_train_argument(func)

Add y_train argument to a function if it is not already present.

Parameters:

Name Type Description Default
func callable

Function to which the argument is added.

 required

Returns:

Name Type Description
wrapper callable

Function with y_train argument added.
Source code in skforecast\metrics\metrics.py 
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
def add_y_train_argument(func: Callable) -> Callable:
    """
    Add `y_train` argument to a function if it is not already present.

    Parameters
    ----------
    func : callable
        Function to which the argument is added.

    Returns
    -------
    wrapper : callable
        Function with `y_train` argument added.

    """

    sig = inspect.signature(func)

    if "y_train" in sig.parameters:
        return func

    new_params = list(sig.parameters.values()) + [
        inspect.Parameter("y_train", inspect.Parameter.KEYWORD_ONLY, default=None)
    ]
    new_sig = sig.replace(parameters=new_params)

    @wraps(func)
    def wrapper(*args, y_train=None, **kwargs):
        return func(*args, **kwargs)

    wrapper.__signature__ = new_sig

    return wrapper

mean_absolute_scaled_error(y_true, y_pred, y_train)

Mean Absolute Scaled Error (MASE)

MASE is a scale-independent error metric that measures the accuracy of a forecast. It is the mean absolute error of the forecast divided by the mean absolute error of a naive forecast in the training set. The naive forecast is the one obtained by shifting the time series by one period. If y_train is a list of numpy arrays or pandas Series, it is considered that each element is the true value of the target variable in the training set for each time series. In this case, the naive forecast is calculated for each time series separately.

Parameters:

Name Type Description Default
y_true pandas Series, numpy ndarray

True values of the target variable.

 required
y_pred pandas Series, numpy ndarray

Predicted values of the target variable.

 required
y_train list, pandas Series, numpy ndarray

True values of the target variable in the training set. If list, it is consider that each element is the true value of the target variable in the training set for each time series.

 required

Returns:

Name Type Description
mase float

MASE value.
Source code in skforecast\metrics\metrics.py 
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
def mean_absolute_scaled_error(
    y_true: Union[pd.Series, np.ndarray],
    y_pred: Union[pd.Series, np.ndarray],
    y_train: Union[list, pd.Series, np.ndarray],
) -> float:
    """
    Mean Absolute Scaled Error (MASE)

    MASE is a scale-independent error metric that measures the accuracy of
    a forecast. It is the mean absolute error of the forecast divided by the
    mean absolute error of a naive forecast in the training set. The naive
    forecast is the one obtained by shifting the time series by one period.
    If y_train is a list of numpy arrays or pandas Series, it is considered
    that each element is the true value of the target variable in the training
    set for each time series. In this case, the naive forecast is calculated
    for each time series separately.

    Parameters
    ----------
    y_true : pandas Series, numpy ndarray
        True values of the target variable.
    y_pred : pandas Series, numpy ndarray
        Predicted values of the target variable.
    y_train : list, pandas Series, numpy ndarray
        True values of the target variable in the training set. If `list`, it
        is consider that each element is the true value of the target variable
        in the training set for each time series.

    Returns
    -------
    mase : float
        MASE value.

    """

    if not isinstance(y_true, (pd.Series, np.ndarray)):
        raise TypeError("`y_true` must be a pandas Series or numpy ndarray.")
    if not isinstance(y_pred, (pd.Series, np.ndarray)):
        raise TypeError("`y_pred` must be a pandas Series or numpy ndarray.")
    if not isinstance(y_train, (list, pd.Series, np.ndarray)):
        raise TypeError("`y_train` must be a list, pandas Series or numpy ndarray.")
    if isinstance(y_train, list):
        for x in y_train:
            if not isinstance(x, (pd.Series, np.ndarray)):
                raise TypeError(
                    ("When `y_train` is a list, each element must be a pandas Series "
                     "or numpy ndarray.")
                )
    if len(y_true) != len(y_pred):
        raise ValueError("`y_true` and `y_pred` must have the same length.")
    if len(y_true) == 0 or len(y_pred) == 0:
        raise ValueError("`y_true` and `y_pred` must have at least one element.")

    if isinstance(y_train, list):
        naive_forecast = np.concatenate([np.diff(x) for x in y_train])
    else:
        naive_forecast = np.diff(y_train)

    mase = np.mean(np.abs(y_true - y_pred)) / np.nanmean(np.abs(naive_forecast))

    return mase

root_mean_squared_scaled_error(y_true, y_pred, y_train)

Root Mean Squared Scaled Error (RMSSE)

RMSSE is a scale-independent error metric that measures the accuracy of a forecast. It is the root mean squared error of the forecast divided by the root mean squared error of a naive forecast in the training set. The naive forecast is the one obtained by shifting the time series by one period. If y_train is a list of numpy arrays or pandas Series, it is considered that each element is the true value of the target variable in the training set for each time series. In this case, the naive forecast is calculated for each time series separately.

Parameters:

Name Type Description Default
y_true pandas Series, numpy ndarray

True values of the target variable.

 required
y_pred pandas Series, numpy ndarray

Predicted values of the target variable.

 required
y_train list, pandas Series, numpy ndarray

True values of the target variable in the training set. If list, it is consider that each element is the true value of the target variable in the training set for each time series.

 required

Returns:

Name Type Description
rmsse float

RMSSE value.
Source code in skforecast\metrics\metrics.py 
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
def root_mean_squared_scaled_error(
    y_true: Union[pd.Series, np.ndarray],
    y_pred: Union[pd.Series, np.ndarray],
    y_train: Union[list, pd.Series, np.ndarray],
) -> float:
    """
    Root Mean Squared Scaled Error (RMSSE)

    RMSSE is a scale-independent error metric that measures the accuracy of
    a forecast. It is the root mean squared error of the forecast divided by
    the root mean squared error of a naive forecast in the training set. The
    naive forecast is the one obtained by shifting the time series by one period.
    If y_train is a list of numpy arrays or pandas Series, it is considered
    that each element is the true value of the target variable in the training
    set for each time series. In this case, the naive forecast is calculated
    for each time series separately.

    Parameters
    ----------
    y_true : pandas Series, numpy ndarray
        True values of the target variable.
    y_pred : pandas Series, numpy ndarray
        Predicted values of the target variable.
    y_train : list, pandas Series, numpy ndarray
        True values of the target variable in the training set. If list, it
        is consider that each element is the true value of the target variable
        in the training set for each time series.

    Returns
    -------
    rmsse : float
        RMSSE value.

    """

    if not isinstance(y_true, (pd.Series, np.ndarray)):
        raise TypeError("`y_true` must be a pandas Series or numpy ndarray.")
    if not isinstance(y_pred, (pd.Series, np.ndarray)):
        raise TypeError("`y_pred` must be a pandas Series or numpy ndarray.")
    if not isinstance(y_train, (list, pd.Series, np.ndarray)):
        raise TypeError("`y_train` must be a list, pandas Series or numpy ndarray.")
    if isinstance(y_train, list):
        for x in y_train:
            if not isinstance(x, (pd.Series, np.ndarray)):
                raise TypeError(
                    ("When `y_train` is a list, each element must be a pandas Series "
                     "or numpy ndarray.")
                )
    if len(y_true) != len(y_pred):
        raise ValueError("`y_true` and `y_pred` must have the same length.")
    if len(y_true) == 0 or len(y_pred) == 0:
        raise ValueError("`y_true` and `y_pred` must have at least one element.")

    if isinstance(y_train, list):
        naive_forecast = np.concatenate([np.diff(x) for x in y_train])
    else:
        naive_forecast = np.diff(y_train)

    rmsse = np.sqrt(np.mean((y_true - y_pred) ** 2)) / np.sqrt(np.nanmean(naive_forecast ** 2))

    return rmsse