8000 Errors with WMT17 recipes and option flags overall · Issue #65 · eole-nlp/eole · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Errors with WMT17 recipes and option flags overall #65

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

8000 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
aaaallleen opened this issue Jul 11, 2024 · 7 comments
Closed

Errors with WMT17 recipes and option flags overall #65

aaaallleen opened this issue Jul 11, 2024 · 7 comments

Comments

@aaaallleen
Copy link
Contributor

I was testing out the recipes today and found that while running the commands for WMT17 recipe, errors were occurring due to the difference between the options between OMT and eole.
When I ran
eole build_vocab --config wmt17_ende.yaml --n_sample -1
which was provided in the README.md, an error of
pydantic_core._pydantic_core.ValidationError: 4 validation errors for BuildVocabConfig report_every Extra inputs are not permitted [type=extra_forbidden, input_value=100, input_type=int] For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden valid_metrics Extra inputs are not permitted [type=extra_forbidden, input_value=['BLEU'], input_type=list] For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden training Extra inputs are not permitted [type=extra_forbidden, input_value={'model_path': 'wmt17_en_...ormalization': 'tokens'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden model Extra inputs are not permitted [type=extra_forbidden, input_value={'architecture': 'transfo...sition_encoding': True}}, input_type=dict] For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
Some of these options were available for training and not build_vocab, causing this error.
Furthermore, I found out some flags that were visible in the code, eg. src_subword_model, dropout and src_subword_nbest weren't available as an option and would cause the same issue of validation errors for TrainConfig
@aaaallleen aaaallleen changed the title Errors with WMT17 recipes Errors with WMT17 recipes and option flags overall Jul 11, 2024
@francoishernandez
Copy link
Member
francoishernandez commented Jul 11, 2024 •
edited
Loading

Duplicate of #48
Fixed in #49
If you installed from source, you might want to pull the latest version. If using docker, you might be using the first version of the package which did include the proper fixes. Try removing the image and pulling it again.

Furthermore, I found out some flags that were visible in the code, eg. src_subword_model, dropout and src_subword_nbest weren't available as an option and would cause the same issue of validation errors for TrainConfig

See my reply here: #66 (comment)
Furthermore, specific settings are not really intended to be passed via the command line anymore. It's doable, but kind of a hassle (see test commands in pull_request_check.sh). The idea is to rely on clearer and more structured config files, with centralized validation logic with pydantic.

@aaaallleen
Copy link
Contributor Author

I'm using this docker image
ghcr.io/eole-nlp/eole:0.0.1-ubuntu22.04-cuda12.1
That is mentioned on the quickstart page. This image isn't updated yet since the code works after I rebuilt the code from a pulled version. The docker image might need an update. Furthermore, I don't see the use of recipes/wmt17/onmt.train.fp16.transformer.yaml, it seems like another demo of a configuration file. However, this config can't be run and the isn't structured properly according to the documentation page.

Furthermore, specific settings are not really intended to be passed via the command line anymore. It's doable, but kind of a hassle (see test commands in pull_request_check.sh). The idea is to rely on clearer and more structured config files, with centralized validation logic with pydantic.

I understand now, this makes a lot more sense now when I go through the code again. Thank you!

@francoishernandez
Copy link
Member

As mentioned in my previous reply, the 0.0.1 docker image has actually been rebuilt post-fix. See here: https://github.com/eole-nlp/eole/actions/workflows/build.yml
To retrieve the newest version you must remove the current one from your local setup and pull it again.
This was before we introduced proper versioning. It won't happen again.
The other config file you mention is an obsolete one, which should be removed from the repo. It's not mentioned in the readme for a reason.
You are just experiencing the life of an early adopter, reporting issues and unclear topics is part of the game, and thanks for that.

@aaaallleen
Copy link
Contributor Author

I noticed the issue when going through the images
image 0.0.1-ubuntu22.04-cuda12.1 isn't updated
image 0.0.1-torch2.3.0-ubuntu22.04-cuda12.1 is updated, the cuda11.8 version is updated too
However, the readme and the document site still shows
docker pull ghcr.io/eole-nlp/eole:0.0.1-ubuntu22.04-cuda12.1 which leads to pulling the unmodified version.
If this is the case, modifications to the Readme and documents might need to be maded

@francoishernandez
Copy link
Member

My bad, good catch! No time to fix right now but feel free to PR these small changes!

@aaaallleen
Copy link
Contributor Author

I'll open a PR and update the Readme! Also, I started a discussion topic, could you review it and see if it is a good idea?

@aaaallleen
Copy link
Contributor Author

Closing this as we figured the problem and came up with a fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@francoishernandez @aaaallleen
0