8000 Feature request: For guided alignment, allow empty alignment lines · Issue #222 · eole-nlp/eole · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Feature request: For guided alignment, allow empty alignment lines #222

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
Dagamies opened this issue Apr 1, 2025 · 1 comment
Open

Feature request: For guided alignment, allow empty alignment lines #222

Dagamies opened this issue Apr 1, 2025 · 1 comment
Labels
contribution welcome Feel free to PR enhancement New feature or request

Comments

@Dagamies
Copy link
Dagamies commented Apr 1, 2025

I have a use case, where some lines have guided alignments and due the nature of data for some it makes absolutely no sense to have it. I'm still forced to use OpenNMT-TF for this use case as it allows empty alignment lines.

It would be very good to have the same functionality in Eole.
@francoishernandez
Copy link
Member

Hi there,

Thanks for the request. This is not really in the scope right now, but we would probably accept a PR, as it would require relatively limited changes.

If anyone wants to try, I guess the main issues would be around:

  • loss computation, where we'd want to ignore such "empty alignment" lines --

    eole/eole/utils/loss.py

    Line 303 in d8fb4e5

    align_loss = self._compute_alignement_loss(align_head=align_head, ref_align=ref_align)
    ;
  • there might be some slight dataloading adaptations to allow such "no alignment" lines, but it seems mostly handled --

    eole/eole/inputters/text_corpus.py

    Lines 188 to 189 in d8fb4e5

    if align is not None:
    example["align"] = align
    ;

I don't have a very relevant setup to investigate this right now, but if you do, feel free to provide more details, e.g. error traces you encounter in your setup.

@francoishernandez francoishernandez added enhancement New feature or request contribution welcome Feel free to PR labels Apr 1, 2025
Sign up for fr 482C ee to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
contribution welcome Feel free to PR enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@francoishernandez @Dagamies
0