Note
Click here to download the full example code
Forecast One By One
A useful feature for short-term forecast in Silverkite model family is autoregression. Silverkite has an “auto” option for autoregression, which automatically selects the autoregression lag orders based on the data frequency and forecast horizons. One important rule of this “auto” option is that the minimum order of autoregression terms is at least the forecast horizon. For example, if the forecast horizon is 3 on a daily model, the minimum order of autoregression is set to 3. The “auto” option won’t have an order of 2 in this case, because the 3rd day forecast will need the 1st day’s observation, which isn’t available at the current time. Although the model can make predictions with an autoregression lag order less than the forecast horizon via simulations, it takes longer time to run and is not the preferred behavior in the “auto” option.
However, in many cases, using smaller autoregression lag orders can give more accurate forecast results. We observe that the only barrier of using an autoregression term of order 2 in the 3-day forecast model is the 3rd day, while we can use it freely for the first 2 days. Similarly, we are able to use an autoregression term of order 1 for the 1st day. In a 3 day forecast, if the accuracy of all 3 days are important, then replacing the first 2 days’ models with shorter autoregression lag orders can improve the accuracy. The forecast-one-by-one algorithm is designed in this context.
The observations above together bring the idea of the forecast-one-by-one algorithm. The algorithm allows fitting multiple models with the “auto” option in autoregression, when one is forecasting with a forecast horizon longer than 1. For each model, the “auto” option for autoregression selects the smallest available autoregression lag order and predicts for the corresponding forecast steps, thus improving the forecast accuracy for the early steps.
In this example, we will cover how to activate the forecast-one-by-one approach
via the ForecastConfig and the Forecaster classes.
For a detailed API reference, please see the
ForecastConfig and
OneByOneEstimator classes.
40 import warnings
41
42 warnings.filterwarnings("ignore")
43
44 import plotly
45 from greykite.common.data_loader import DataLoader
46 from greykite.framework.templates.autogen.forecast_config import ForecastConfig
47 from greykite.framework.templates.autogen.forecast_config import ModelComponentsParam
48 from greykite.framework.templates.forecaster import Forecaster
49 from greykite.framework.templates.model_templates import ModelTemplateEnum
50 from greykite.framework.utils.result_summary import summarize_grid_search_results
51
52 # Loads dataset into pandas DataFrame
53 dl = DataLoader()
54 df = dl.load_peyton_manning()
The forecast-one-by-one option
The forecast-one-by-one option is specified through the forecast_one_by_one parameter
in ForecastConfig.
63 config = ForecastConfig(
64 model_template=ModelTemplateEnum.SILVERKITE.name,
65 forecast_horizon=3,
66 model_components_param=ModelComponentsParam(
67 autoregression=dict(autoreg_dict="auto")
68 ),
69 forecast_one_by_one=True
70 )
The forecast_one_by_one parameter can be specified in the following ways
``True``: every forecast step will be a separate model. The number of models equals the forecast horizon. In this example, 3 models will be fit with the 3 forecast steps.
``False``: the forecast-one-by-one method is turned off. This is the default behavior and a single model is used for all forecast steps.
A list of integers: each integer corresponds to a model, and it is the number of steps. For example, in a 7 day forecast, specifying
forecast_one_by_one=[1, 2, 4]will result in 3 models. The first model forecasts the 1st day with forecast horizon 1; The second model forecasts the 2nd - 3rd days with forecast horizon 3; The third model forecasts the 4th - 7th days with forecast horizon 7. In this case, the sum of the list entries must equal the forecast horizon.an integer ``n``: every model will account for n steps. The last model will account for the rest <n steps. For example in a 7 day forecast, specifying
forecast_one_by_one=2will result in 4 models, which is equivalent toforecast_one_by_one=[2, 2, 2, 1].
Note
forecast_one_by_one is activated only when there are parameters in
the model that depend on the forecast horizon. Currently the only parameter
that depends on forecast horizon is autoreg_dict="auto". If you do not specify
autoreg_dict="auto", the forecast_one_by_one parameter will be ignored.
Note
Forecast-one-by-one fits multiple models to increase accuracy,
which may cause the training time to increase linearly with the number of models.
Please make sure your forecast_one_by_one parameter and forecast horizon
result in a reasonable number of models.
Next, let’s run the model and look at the result.
Out:
Fitting 3 folds for each of 1 candidates, totalling 3 fits
You may see a few warnings like “The future x length is 0, which doesn’t match the model forecast horizon 3, using only the model with the longest forecast horizon for prediction.” This is an expected behavior when calculating the training errors. Because the models are mapped to the forecast period only, but not to the training period. Therefore, only the last model is used to get the fitted values on the training period. You don’t need to worry about it.
Everything on the forecast_result level is the same as not activating forecast-one-by-one.
For example, we can view the cross-validation results in the same way.
126 # Summarizes the CV results
127 cv_results = summarize_grid_search_results(
128 grid_search=result.grid_search,
129 decimals=1,
130 # The below saves space in the printed output. Remove to show all available metrics and columns.
131 cv_report_metrics=None,
132 column_order=["rank", "mean_test", "split_test", "mean_train", "split_train", "mean_fit_time", "mean_score_time", "params"])
133 cv_results["params"] = cv_results["params"].astype(str)
134 cv_results.set_index("params", drop=True, inplace=True)
135 cv_results.transpose()
When you need to access estimator level attributes, for example, model summary or component plots, the returned result will be a list of the original type, because we fit multiple models. The model summary list can be accessed in the same way and you can use index to get the model summary for a single model.
143 # Gets the model summary list
144 one_by_one_estimator = result.model[-1]
145 summaries = one_by_one_estimator.summary()
146 # Prints the model summary for 1st model only
147 print(summaries[0])
Out:
============================ Forecast Model Summary ============================
Number of observations: 367, Number of features: 98
Method: Ridge regression
Number of nonzero features: 98
Regularization parameter: 0.3039
Residuals:
Min 1Q Median 3Q Max
-0.8742 -0.1628 -0.04904 0.09108 3.086
Pred_col Estimate Std. Err Pr(>)_boot sig. code 95%CI
Intercept 6.302 0.1605 <2e-16 *** (5.963, 6.591)
events_C...New Year 0.3751 0.2923 0.186 (-0.1245, 0.8328)
events_C...w Year-1 -0.1972 0.2271 0.376 (-0.6109, 0.2286)
events_C...w Year-2 -0.1852 0.1526 0.248 (-0.468, 0.1071)
events_C...w Year+1 -0.09128 0.212 0.620 (-0.4075, 0.4655)
events_C...w Year+2 -0.1268 0.2354 0.564 (-0.4417, 0.4624)
events_Christmas Day -0.4262 0.2446 0.036 * (-0.7246, 0.)
events_C...as Day-1 -0.545 0.2909 0.004 ** (-0.8045, 0.)
events_C...as Day-2 -0.1055 0.09123 0.184 (-0.2894, 0.02009)
events_C...as Day+1 -0.1499 0.1101 0.124 (-0.3483, 0.01239)
events_C...as Day+2 0.7339 0.3838 0.004 ** (0., 1.023)
events_E...Ireland] -0.01943 0.08299 0.540 (-0.2033, 0.1751)
events_E...eland]-1 -0.06712 0.06143 0.216 (-0.184, 0.0429)
events_E...eland]-2 -0.01041 0.04613 0.556 (-0.1154, 0.08672)
events_E...eland]+1 -0.07063 0.07951 0.288 (-0.2288, 0.06558)
events_E...eland]+2 0.0292 0.05921 0.418 (-0.08946, 0.1708)
events_Good Friday 0.11 0.1049 0.222 (-0.08342, 0.308)
events_Good Friday-1 -0.1989 0.1367 0.098 . (-0.4296, 0.)
events_Good Friday-2 -0.04079 0.07799 0.392 (-0.1872, 0.1476)
events_Good Friday+1 -0.01041 0.04613 0.556 (-0.1154, 0.08672)
events_Good Friday+2 -0.06712 0.06143 0.216 (-0.184, 0.0429)
events_I...ence Day 0.1638 0.1089 0.094 . (-0.03561, 0.3509)
events_I...ce Day-1 -0.2614 0.144 0.038 * (-0.5028, 0.)
events_I...ce Day-2 -0.01706 0.08844 0.854 (-0.1994, 0.1528)
events_I...ce Day+1 -0.2123 0.1058 0.012 * (-0.3594, 0.)
events_I...ce Day+2 -0.1669 0.1051 0.068 . (-0.3448, 0.)
events_Labor Day -0.07808 0.1711 0.578 (-0.444, 0.2382)
events_Labor Day-1 0.006859 0.1252 0.820 (-0.2589, 0.2693)
events_Labor Day-2 0.1052 0.09101 0.190 (-0.05062, 0.2862)
events_Labor Day+1 0.02279 0.08898 0.830 (-0.146, 0.2187)
events_Labor Day+2 0.05392 0.2149 0.706 (-0.3779, 0.4466)
events_Memorial Day -0.5431 0.3022 0.012 * (-0.8315, 0.)
events_M...al Day-1 -0.2853 0.1828 0.062 . (-0.5572, 0.)
events_M...al Day-2 0.02219 0.08681 0.520 (-0.1889, 0.1966)
events_M...al Day+1 -0.09167 0.09481 0.244 (-0.2891, 0.07072)
events_M...al Day+2 0.02977 0.0698 0.492 (-0.1238, 0.1847)
events_New Years Day -0.3642 0.2201 0.048 * (-0.643, 0.)
events_N...rs Day-1 -0.05085 0.1035 0.776 (-0.2923, 0.1162)
events_N...rs Day-2 -0.1679 0.1273 0.128 (-0.3948, 0.0276)
events_N...rs Day+1 0.3689 0.1987 0.010 * (0., 0.5371)
events_N...rs Day+2 -0.01825 0.1138 0.574 (-0.2592, 0.2486)
events_Other 0.1719 0.09994 0.066 . (-0.005852, 0.3734)
events_Other-1 -0.05274 0.06245 0.382 (-0.1641, 0.09075)
events_Other-2 0.02876 0.04972 0.544 (-0.06856, 0.1282)
events_Other+1 0.003735 0.06461 0.946 (-0.1164, 0.1272)
events_Other+2 -0.01036 0.06584 0.868 (-0.1385, 0.1318)
events_Thanksgiving -0.2246 0.1349 0.058 . (-0.4203, 0.)
events_T...giving-1 -0.07548 0.08681 0.272 (-0.2545, 0.05988)
events_T...giving-2 -0.1614 0.1086 0.086 . (-0.3207, 0.)
events_T...giving+1 0.06853 0.09691 0.368 (-0.1536, 0.2515)
events_T...giving+2 -0.08959 0.08891 0.230 (-0.2697, 0.06051)
events_Veterans Day -0.2094 0.149 0.104 (-0.4536, 0.)
events_V...ns Day-1 -0.2939 0.1669 0.018 * (-0.4572, 0.)
events_V...ns Day-2 0.09865 0.1274 0.328 (-0.1425, 0.3783)
events_V...ns Day+1 0.0236 0.06981 0.490 (-0.1395, 0.1769)
events_V...ns Day+2 -0.305 0.1781 0.024 * (-0.525, 0.)
str_dow_2-Tue 0.03979 0.03812 0.288 (-0.03074, 0.1252)
str_dow_3-Wed 0.06363 0.04144 0.108 (-0.02531, 0.1443)
str_dow_4-Thu 0.06377 0.05334 0.232 (-0.03574, 0.1801)
str_dow_5-Fri -0.01306 0.03505 0.734 (-0.0819, 0.05634)
str_dow_6-Sat -0.1248 0.03953 0.002 ** (-0.2028, -0.04826)
str_dow_7-Sun -0.07184 0.05146 0.150 (-0.166, 0.02596)
ct1 0.3804 0.1386 0.006 ** (0.1214, 0.6577)
is_weekend:ct1 0.05211 0.08319 0.544 (-0.121, 0.2172)
str_dow_2-Tue:ct1 -0.2006 0.1718 0.228 (-0.5533, 0.11)
str_dow_3-Wed:ct1 -0.07108 0.08229 0.424 (-0.211, 0.1025)
str_dow_4-Thu:ct1 -0.1686 0.1352 0.206 (-0.4631, 0.06617)
str_dow_5-Fri:ct1 0.06852 0.1159 0.552 (-0.1464, 0.3035)
str_dow_6-Sat:ct1 -0.1557 0.09606 0.120 (-0.3277, 0.03998)
str_dow_7-Sun:ct1 0.2074 0.1497 0.176 (-0.118, 0.4653)
ct1:sin1_tow_weekly -0.1741 0.06286 0.012 * (-0.306, -0.05449)
ct1:cos1_tow_weekly 0.4462 0.2406 0.072 . (-0.02801, 0.9079)
ct1:sin2_tow_weekly -0.1284 0.08107 0.104 (-0.2829, 0.02986)
ct1:cos2_tow_weekly 0.4423 0.1964 0.016 * (0.06371, 0.8168)
sin1_tow_weekly 0.1561 0.04486 <2e-16 *** (0.06501, 0.2456)
cos1_tow_weekly -0.005014 0.08808 0.952 (-0.1496, 0.1872)
sin2_tow_weekly -0.01693 0.04934 0.704 (-0.1111, 0.08364)
cos2_tow_weekly 0.07175 0.07239 0.326 (-0.06262, 0.2167)
sin3_tow_weekly -0.01231 0.02748 0.632 (-0.06064, 0.04727)
cos3_tow_weekly 0.01155 0.045 0.794 (-0.07853, 0.09719)
sin4_tow_weekly 0.01231 0.02748 0.632 (-0.04727, 0.06064)
cos4_tow_weekly 0.01155 0.045 0.794 (-0.07853, 0.09719)
sin1_toq_quarterly 0.0112 0.05824 0.860 (-0.1015, 0.1251)
cos1_toq_quarterly -0.09109 0.07053 0.224 (-0.2294, 0.03948)
sin2_toq_quarterly 0.007969 0.0619 0.918 (-0.1156, 0.1291)
cos2_toq_quarterly 0.009331 0.05595 0.856 (-0.1086, 0.1192)
sin3_toq_quarterly -0.08117 0.0523 0.114 (-0.192, 0.008878)
cos3_toq_quarterly -0.007711 0.06315 0.902 (-0.1301, 0.1209)
sin4_toq_quarterly -0.0479 0.05981 0.420 (-0.1645, 0.06953)
cos4_toq_quarterly -0.06148 0.05877 0.256 (-0.1863, 0.04531)
sin5_toq_quarterly -0.07068 0.07083 0.310 (-0.2124, 0.06965)
cos5_toq_quarterly 0.03178 0.04997 0.520 (-0.05985, 0.1341)
y_lag1 3.005 0.4741 <2e-16 *** (1.934, 3.711)
y_lag2 -0.3595 0.392 0.392 (-0.9389, 0.537)
y_lag3 0.1069 0.275 0.702 (-0.4461, 0.6108)
y_avglag_7_14_21 0.738 0.2775 0.006 ** (0.259, 1.318)
y_avglag_1_to_7 0.2454 0.3159 0.438 (-0.4112, 0.8298)
y_avglag_8_to_14 0.1943 0.2008 0.324 (-0.2095, 0.5909)
Signif. Code: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Multiple R-squared: 0.7859, Adjusted R-squared: 0.7358
F-statistic: 14.855 on 69 and 296 DF, p-value: 1.110e-16
Model AIC: 1504.1, model BIC: 1779.0
WARNING: the condition number is large, 9.03e+03. This might indicate that there are strong multicollinearity or other numerical problems.
WARNING: the F-ratio and its p-value on regularized methods might be misleading, they are provided only for reference purposes.
We can access the component plots in a similar way.
152 # Gets the fig list
153 figs = one_by_one_estimator.plot_components()
154 # Shows the component plot for 1st model only
155 plotly.io.show(figs[0])
Total running time of the script: ( 1 minutes 28.301 seconds)