Skforecast in GPU¶
Traditionally, machine learning algorithms have been executed on CPUs (Central Processing Units)—general-purpose processors capable of handling a wide variety of tasks. However, CPUs are not optimized for the highly parallelized matrix operations that many machine learning algorithms rely on, often resulting in slower training times and limited scalability. In contrast, GPUs (Graphics Processing Units) are specifically designed for parallel processing, capable of performing thousands of simultaneous mathematical operations. This makes them particularly well-suited for training and deploying large-scale machine learning models.
Several popular machine learning libraries have implemented GPU acceleration, including XGBoost, LightGBM, CatBoost and CuML. By leveraging GPU capabilities, these libraries can dramatically reduce training times and enhance scalability. The following sections demonstrate how execute skforecast with GPU acceleration to build efficient and powerful forecasting models.
✎ Note
The performance advantage of using a GPU depends heavily on the specific task and the size of the dataset. Generally, GPU acceleration offers the greatest benefits when working with large datasets and complex models, where its parallel processing capabilities can significantly reduce training times.
In recursive forecasting (ForecasterRecursive
and ForecasterRecursiveMultiseries
) the prediction phase must be executed sequentially since each time step depends on the previous prediction. This inherent dependency prevents parallelization during inference, which explains why model fitting is substantially faster on a GPU, while prediction can actually be slower compared to using a CPU. To overcome this limitation, skforecast automatically switches the regressor to use the CPU for prediction, even if it was trained on a GPU.
In contrast, direct forecasters (ForecasterDirect
, ForecasterDirectMultivariate
) do not rely on previous predictions during inference. This lack of dependency allows both training and prediction to fully benefit from GPU acceleration.
✎ Note
Despite the significant advantages offered by GPUs (specifically Nvidia GPUs) in accelerating machine learning computations, access to them is often limited due to high costs or other practical constraints. Fortunatelly, Google Colaboratory (Colab), a free Jupyter notebook environment, allows users to run Python code in the cloud, with access to powerful hardware resources such as GPUs. This makes it an excellent platform for experimenting with machine learning models, especially those that require intensive computations. The following links provide access to Google Colab notebooks that demonstrate how to use skforecast with GPU acceleration.
# Libraries
# ==============================================================================
import numpy as np
import pandas as pd
import torch
import psutil
import xgboost
from xgboost import XGBRegressor
import lightgbm
from lightgbm import LGBMRegressor
import catboost
from catboost import CatBoostRegressor
import warnings
import skforecast
from skforecast.recursive import ForecasterRecursive
from skforecast.model_selection import backtesting_forecaster, TimeSeriesFold
print(f"skforecast version : {skforecast.__version__}")
print(f"xgboost version : {xgboost.__version__}")
print(f"lightgbm version : {lightgbm.__version__}")
print(f"catboost version : {catboost.__version__}")
skforecast version : 0.16.0 xgboost version : 2.1.2 lightgbm version : 4.5.0 catboost version : 1.2.8
# Print information abput the GPU and CPU
# ==============================================================================
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
print('Using device:', device)
if device.type == 'cuda':
print(torch.cuda.get_device_name(0))
print('Memory Usage:')
print('Allocated :', round(torch.cuda.memory_allocated(0) / 1024**3, 1), 'GB')
print('Reserved :', round(torch.cuda.memory_reserved(0) / 1024**3, 1), 'GB')
print(f"CPU RAM Free: {psutil.virtual_memory().available / 1024**3:.2f} GB")
Using device: cuda NVIDIA T1200 Laptop GPU Memory Usage: Allocated : 0.0 GB Reserved : 0.0 GB CPU RAM Free: 13.62 GB
# Data
# ==============================================================================
n = 1_000_000
data = pd.Series(
data = np.random.normal(size=n),
index = pd.date_range(start="1990-01-01", periods=n, freq="h"),
name = 'y'
)
data.head(2)
1990-01-01 00:00:00 1.500408 1990-01-01 01:00:00 -1.112868 Freq: h, Name: y, dtype: float64
XGBoost¶
To run an XGBoost model (version 2.0 or higher) on a GPU, set the argument device='cuda' during initialization.
# Suppress warnings
# ==============================================================================
warnings.filterwarnings(
"ignore",
message=".*Falling back to prediction using DMatrix.*",
category=UserWarning,
module="xgboost"
)
# Create and train forecaster with a XGBRegressor using GPU
# ==============================================================================
forecaster = ForecasterRecursive(
regressor = XGBRegressor(
n_estimators = 1000,
device = 'cuda',
verbosity = 1
),
lags = 50
)
start_time = pd.Timestamp.now()
forecaster.fit(y=data)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Training time using GPU: {elapsed_time}")
# Predict
# ==============================================================================
start_time = pd.Timestamp.now()
forecaster.predict(steps=100)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Prediction time using GPU: {elapsed_time}")
# Backtesting
# ==============================================================================
cv = TimeSeriesFold(
steps = 100,
initial_train_size = 990_000,
refit = False,
verbose = False
)
start_time = pd.Timestamp.now()
_ = backtesting_forecaster(
forecaster = forecaster,
y = data,
cv = cv,
metric = 'mean_absolute_error'
)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Backtesting time using GPU: {elapsed_time}")
Training time using GPU: 0 days 00:00:28.760755 Prediction time using GPU: 0 days 00:00:00.095367
0%| | 0/100 [00:00<?, ?it/s]
Backtesting time using GPU: 0 days 00:00:43.120299
# Create and train forecaster with a XGBRegressor using CPU
# ==============================================================================
forecaster = ForecasterRecursive(
regressor = XGBRegressor(n_estimators=1000),
lags = 50
)
start_time = pd.Timestamp.now()
forecaster.fit(y=data)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Training time using CPU: {elapsed_time}")
# Predict
# ==============================================================================
start_time = pd.Timestamp.now()
forecaster.predict(steps=100)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Prediction time using CPU: {elapsed_time}")
# Backtesting
# ==============================================================================
start_time = pd.Timestamp.now()
_ = backtesting_forecaster(
forecaster = forecaster,
y = data,
cv = cv,
metric = 'mean_absolute_error'
)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Backtesting time using CPU: {elapsed_time}")
Training time using CPU: 0 days 00:00:44.323256 Prediction time using CPU: 0 days 00:00:00.143112
0%| | 0/100 [00:00<?, ?it/s]
Backtesting time using CPU: 0 days 00:00:48.289869
LightGBM¶
# Suppress warnings
# ==============================================================================
warnings.filterwarnings(
"ignore",
message="'force_all_finite' was renamed to 'ensure_all_finite' in 1.6 and will be removed in 1.8.",
category=FutureWarning,
module="sklearn.utils.deprecation"
)
When using Google colab, run the following in a notebook cell to ensure LightGBM can utilize the NVIDIA GPU when executing in google colab.
!mkdir -p /etc/OpenCL/vendors && echo "libnvidia-opencl.so.1" > /etc/OpenCL/vendors/nvidia.icd
# Create and train forecaster with a LGBMRegressor using GPU
# ==============================================================================
forecaster = ForecasterRecursive(
regressor = LGBMRegressor(n_estimators=1000, device='gpu', verbose=-1),
lags = 50
)
start_time = pd.Timestamp.now()
forecaster.fit(y=data)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Training time using GPU: {elapsed_time}")
# Predict
# ==============================================================================
start_time = pd.Timestamp.now()
forecaster.predict(steps=100)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Prediction time using GPU: {elapsed_time}")
# Backtesting
# ==============================================================================
start_time = pd.Timestamp.now()
_ = backtesting_forecaster(
forecaster = forecaster,
y = data,
cv = cv,
metric = 'mean_absolute_error'
)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Backtesting time using GPU: {elapsed_time}")
Training time using GPU: 0 days 00:00:21.657198 Prediction time using GPU: 0 days 00:00:00.059898
0%| | 0/100 [00:00<?, ?it/s]
Backtesting time using GPU: 0 days 00:00:24.554291
# Create and train forecaster with a LGBMRegressor using CPU
# ==============================================================================
forecaster = ForecasterRecursive(
regressor = LGBMRegressor(n_estimators=1000, device='cpu', verbose=-1),
lags = 50
)
start_time = pd.Timestamp.now()
forecaster.fit(y=data)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Training time using CPU: {elapsed_time}")
# Predict
# ==============================================================================
start_time = pd.Timestamp.now()
forecaster.predict(steps=100)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Prediction time using CPU: {elapsed_time}")
# Backtesting
# ==============================================================================
start_time = pd.Timestamp.now()
_ = backtesting_forecaster(
forecaster = forecaster,
y = data,
cv = cv,
metric = 'mean_absolute_error'
)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Backtesting time using CPU: {elapsed_time}")
Training time using CPU: 0 days 00:00:27.196106 Prediction time using CPU: 0 days 00:00:00.044868
0%| | 0/100 [00:00<?, ?it/s]
Backtesting time using CPU: 0 days 00:00:25.313712
CatBoost¶
# Create and train forecaster with a CatBoostRegressor using GPU
# ==============================================================================
forecaster = ForecasterRecursive(
regressor = CatBoostRegressor(n_estimators=1000, task_type='GPU', silent=True, allow_writing_files=False),
lags = 50
)
start_time = pd.Timestamp.now()
forecaster.fit(y=data)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Training time using GPU: {elapsed_time}")
# Predict
# ==============================================================================
start_time = pd.Timestamp.now()
forecaster.predict(steps=100)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Prediction time using GPU: {elapsed_time}")
# Backtesting
# ==============================================================================
start_time = pd.Timestamp.now()
_ = backtesting_forecaster(
forecaster = forecaster,
y = data,
cv = cv,
metric = 'mean_absolute_error'
)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Backtesting time using GPU: {elapsed_time}")
Training time using GPU: 0 days 00:00:26.810713 Prediction time using GPU: 0 days 00:00:00.109821
0%| | 0/100 [00:00<?, ?it/s]
Backtesting time using GPU: 0 days 00:00:30.199580
# Create and train forecaster with a CatBoostRegressor using CPU
# ==============================================================================
forecaster = ForecasterRecursive(
regressor = CatBoostRegressor(n_estimators=1000, task_type='CPU', silent=True, allow_writing_files=False),
lags = 50
)
start_time = pd.Timestamp.now()
forecaster.fit(y=data)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Training time using CPU: {elapsed_time}")
# Predict
# ==============================================================================
start_time = pd.Timestamp.now()
forecaster.predict(steps=100)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Prediction time using CPU: {elapsed_time}")
# Backtesting
# ==============================================================================
start_time = pd.Timestamp.now()
_ = backtesting_forecaster(
forecaster = forecaster,
y = data,
cv = cv,
metric = 'mean_absolute_error'
)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Backtesting time using CPU: {elapsed_time}")
Training time using CPU: 0 days 00:01:13.747194 Prediction time using CPU: 0 days 00:00:00.101679
0%| | 0/100 [00:00<?, ?it/s]
Backtesting time using CPU: 0 days 00:01:06.052743
RAPIDS cuML¶
cuML is a library for executing machine learning algorithms on GPUs with an API that closely follows the scikit-learn API.
To use cuML with skforecast, you need to install the RAPIDS cuML library. The installation process may vary depending on your environment and the version of CUDA you have installed. You can find detailed installation instructions in the RAPIDS documentation.
# Libraries
# ==============================================================================
import cuml
from sklearn.ensemble import RandomForestRegressor
print(f"skforecast version : {skforecast.__version__}")
print(f"cuml version : {cuml.__version__}")
# Create and train forecaster with a RandomForestRegressor using GPU
# ==============================================================================
forecaster = ForecasterRecursive(
regressor = cuml.ensemble.RandomForestRegressor(
n_estimators=200,
max_depth=5,
),
lags = 20
)
start_time = pd.Timestamp.now()
forecaster.fit(y=data)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Training time using GPU: {elapsed_time}")
# Predict
# ==============================================================================
start_time = pd.Timestamp.now()
forecaster.predict(steps=100)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Prediction time using GPU: {elapsed_time}")
# Backtesting
# ==============================================================================
cv = TimeSeriesFold(
steps = 100,
initial_train_size = 90_000,
refit = False,
verbose = False
)
start_time = pd.Timestamp.now()
_ = backtesting_forecaster(
forecaster = forecaster,
y = data,
cv = cv,
metric = 'mean_absolute_error'
)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Backtesting time using GPU: {elapsed_time}")
# Create and train forecaster with a RandomForestRegressor using CPU
# ==============================================================================
forecaster = ForecasterRecursive(
RandomForestRegressor(n_estimators=200, max_depth=5),
lags = 20
)
start_time = pd.Timestamp.now()
forecaster.fit(y=data)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Training time using CPU: {elapsed_time}")
# Predict
# ==============================================================================
start_time = pd.Timestamp.now()
forecaster.predict(steps=100)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Prediction time using CPU: {elapsed_time}")
# Backtesting
# ==============================================================================
cv = TimeSeriesFold(
steps = 100,
initial_train_size = 90_000,
refit = False,
verbose = False
)
start_time = pd.Timestamp.now()
_ = backtesting_forecaster(
forecaster = forecaster,
y = data,
cv = cv,
metric = 'mean_absolute_error'
)
elapsed_time = pd.Timestamp.now() - start_time
print(f"Backtesting time using CPU: {elapsed_time}")