8000 [BUG]: Longer reported times for initial execution of compare_models in regression experiment · Issue #4161 · pycaret/pycaret · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[BUG]: Longer reported times for initial execution of compare_models in regression experiment #4161

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
3 tasks done
danielelotito opened this issue Apr 16, 2025 · 0 comments
Open
3 tasks done

[BUG]: Longer reported times for initial execution of compare_models in regression experiment #4161

danielelotito opened this issue Apr 16, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@danielelotito
Copy link
danielelotito commented Apr 16, 2025 •
edited
Loading

pycaret version checks

Issue Description

When compare_models is run for the first time in a script, the time results of some of the tested models are not correct. Tested only in regression scenarios.
The 'lr' models shows biggest percentage difference, but other models are affected as well.

Reproducible Example

from pycaret.datasets import get_data
import pandas as pd
from pycaret.regression import *

data = get_data('insurance')
s = setup(data, target = 'charges', session_id = 123)

best = compare_models()
compare_models_results = pull()

s = setup(data, target = 'charges', session_id = 121)

best2 = compare_models()
compare_models_results2 = pull()
compare_models_results.rename(columns={'TT (Sec)': 'TT (Sec)1'}, inplace=True)
compare_models_results2.rename(columns={'TT (Sec)': 'TT (Sec)2'}, inplace=True)

series = [compare_models_results['TT (Sec)1'], compare_models_results2['TT (Sec)2']]
time_vs_time = pd.concat(series, axis = 1)

print(time_vs_time.loc['lr'])
print(time_vs_time)

Expected Behavior

TT (Sec)1: 0.006
TT (Sec)2: 0.006
Name: lr, dtype: float64

Model TT (Sec)1 TT (Sec)2
gbr 0.011 0.011
rf 0.018 0.018
lightgbm 0.463 0.463
et 0.017 0.017
ada 0.007 0.007
ridge 0.006 0.006
lar 0.006 0.006
llar 0.005 0.005
br 0.006 0.006
lasso 0.005 0.005
lr 0.006 0.006
huber 0.007 0.007
dt 0.006 0.006
par 0.006 0.006
en 0.005 0.005
knn 0.007 0.007
omp 0.006 0.006
dummy 0.009 0.009
I do not expect them to be exactly equal as above, but roughly equal. Probably this is not directly due to PyCaret, but from, e.g., Scikit-Learn, at the same time, PyCaret is reporting them without any warning.

Actual Results

TT (Sec)1    1.081
TT (Sec)2    0.006
Name: lr, dtype: float64
          TT (Sec)1  TT (Sec)2
gbr           0.010      0.011
rf            0.018      0.018
lightgbm      0.479      0.463
et            0.017      0.017
ada           0.007      0.007
lar           0.006      0.006
llar          0.006      0.005
br            0.006      0.006
ridge         0.076      0.006
lasso         0.083      0.005
lr            1.081      0.006
huber         0.007      0.007
dt            0.005      0.006
par           0.006      0.006
en            0.077      0.005
omp           0.007      0.006
knn           0.007      0.007
dummy         0.007      0.009

Installed Versions

System: python: 3.10.11 (main, Mar 5 2025, 14:09:57) [Clang 16.0.0 (clang-1600.0.26.6)] executable: [/Users/danielelotito/Documents/github/bug_pycaret/.venv/bin/python](https://file+.vscode-resource.vscode-cdn.net/Users/danielelotito/Documents/github/bug_pycaret/.venv/bin/python) machine: macOS-15.3.2-arm64-arm-64bit

PyCaret required dependencies:
pip: 25.0.1
setuptools: 65.5.0
pycaret: 3.4.0
IPython: 8.35.0
ipywidgets: 8.1.6
tqdm: 4.67.1
numpy: 1.26.4
pandas: 2.1.4
jinja2: 3.1.6
scipy: 1.11.4
joblib: 1.4.2
sklearn: 1.4.2
pyod: 2.0.4
imblearn: 0.13.0
category_encoders: 2.7.0
lightgbm: 4.6.0
numba: 0.61.2
requests: 2.32.3
matplotlib: 3.7.5
scikitplot: 0.3.12
yellowbrick: 1.5
plotly: 5.24.1
plotly-resampler: Not installed
kaleido: 0.2.1
schemdraw: 0.15
statsmodels: 0.14.4
sktime: 0.31.0
tbats: 1.1.3
pmdarima: 2.0.4
psutil: 7.0.0
markupsafe: 3.0.2
pickle5: Not installed
cloudpickle: 3.1.1
deprecation: 2.1.0
xxhash: 3.5.0
wurlitzer: 3.1.1

PyCaret optional dependencies:
shap: Not installed
interpret: Not installed
umap: Not installed
ydata_profiling: Not installed
explainerdashboard: Not installed
autoviz: Not installed
fairlearn: Not installed
deepchecks: Not installed
xgboost: Not installed
catboost: Not installed
kmodes: Not installed
mlxtend: Not installed
statsforecast: Not installed
tune_sklearn: Not installed
ray: Not installed
hyperopt: Not installed
optuna: Not installed
skopt: Not installed
mlflow: Not installed
gradio: Not installed
fastapi: Not installed
uvicorn: Not installed
m2cgen: Not installed
evidently: Not installed
fugue: Not installed
streamlit: Not installed
prophet: Not installed
5B9C
@danielelotito danielelotito added the bug Something isn't working label Apr 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@danielelotito
0