Index of exogenous variable passed to .predict() #6874

amurthy-sunysb · 2024-08-02T02:07:08Z

amurthy-sunysb
Aug 2, 2024

I am trying to understand the index of the exogenous variable X that needs to be passed to the .predict() function. What explains the three outputs for the different exog dataframes passed into .predict functions below? Specifically:

Why does .predict() work when X_train is passed, which is missing the indices corresponding to fh ?
Why is the output same when X_test and X are passed?
What does this mean in the documentation: If self.get_tag("X-y-must-have-same-index"), X.index must contain fh index reference. This tag is True for the forecaster below.

y = load_airline()

# Create an exog variable
X = pd.DataFrame({'X1': np.random.rand(y.shape[0])}).set_index(y.index)

# Split the endog and exog variables into train and test
y_train, y_test, X_train, X_test = temporal_train_test_split(y, X, test_size=5)

# Create Linear Regression model with 6 lags
forecaster = make_reduction(LinearRegression(), strategy='recursive', window_length=6)
fh = ForecastingHorizon(y_test.index, is_relative=False)
forecaster.fit(y_train, X=X_train, fh=fh)

# Make prediction with exog
y_pred_X_test = forecaster.predict(X=X_test) # Pass x_test
y_pred_X_train = forecaster.predict(X=X_train) # Pass X_train
y_pred_X = forecaster.predict(X=X)  # Pass X

print(y_pred_X_test)
print(y_pred_X_train)
print(y_pred_X)

The outputs are as follows.
y_pred_X_test
1960-08 | 635.259370
1960-09 | 609.253811
1960-10 | 564.324084
1960-11 | 532.170112
1960-12 | 526.405133

y_pred_X_train
1960-08 | 635.259370
1960-09 | 612.309800
1960-10 | 564.309419
1960-11 | 519.339836
1960-12 | 513.604385

y_pred_X
1960-08 | 635.259370
1960-09 | 609.253811
1960-10 | 564.324084
1960-11 | 532.170112
1960-12 | 526.405133

fkiraly · 2024-08-04T17:41:24Z

fkiraly
Aug 4, 2024
Maintainer

I think what you are observing is explained by this particular estimator ignoring the index of the X passed in predict, and considering it as the start of the index in fh.

This is a minor inconsistency, it might be worth thinking about what should happen here, if the index passed does not match the fh index.

2 replies

amurthy-sunysb Aug 5, 2024
Author

Thanks @fkiraly!

Can you kindly clarify "...and considering it as the start of the index in fh." ?
fh is currently ['1960-08', '1960-09', '1960-10', '1960-11', '1960-12'].

The indices of the exog dataframes are as follows.
X: 1949-01 to 1960-12
X_train: 1949-01 to 1960-07
X_test: 1960-08 to 1960-12.

When we pass X, which values are used: the trailing 5 (1960-08 to 1960-12) or the leading 5 (1949-01 to 1949-05) ?
Based on the predictions, it seems like the trailing values are used, just want to confirm.

Why should this behavior be estimator-specific? Shouldn't this be a validation step independent of the regressor that is used?

I feel that we should raise an error when the indices of the exog dataframe do not align well with the forecasting horizon. If
we do not want to raise an error, it would be very helpful to at least clarify the behavior in the documentation. Currently, the sentence describing is X-y-must-have-same-index seems to be misleading.

fkiraly Aug 14, 2024
Maintainer

Hm, good point.
Sorry for the late answer, I had to think about this.

I agree with you - the boilerplate should deal with the indexing in the way you describe, it indeed should not be estimator specific.

The location of where you do it is not obvious though. What would help if we could write down some expected cases in relation to the forecaaters in question, e.g., index and error behaviour, and cases where this is currently not so. We can start with the examples in this discussion, could you kindly open an issue? (if no time, I will do it later)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Index of exogenous variable passed to .predict() #6874

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 1 comment 2 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

Index of exogenous variable passed to .predict() #6874

Uh oh!

Uh oh!

amurthy-sunysb Aug 2, 2024

Replies: 1 comment · 2 replies

Uh oh!

fkiraly Aug 4, 2024 Maintainer

Uh oh!

Uh oh!

amurthy-sunysb Aug 5, 2024 Author

Uh oh!

fkiraly Aug 14, 2024 Maintainer

amurthy-sunysb
Aug 2, 2024

Replies: 1 comment 2 replies

fkiraly
Aug 4, 2024
Maintainer

amurthy-sunysb Aug 5, 2024
Author

fkiraly Aug 14, 2024
Maintainer