8000 Add default LoRA target modules for Phi-2 by arnavgarg1 · Pull Request #3911 · ludwig-ai/ludwig · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Add default LoRA target modules for Phi-2 #3911

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Jan 23, 2024
Merged

Add default LoRA target modules for Phi-2 #3911

merged 3 commits into from
Jan 23, 2024

Conversation

arnavgarg1
Copy link
Contributor

These modules maximize performance, also the current LoRA target modules for Phi-2 are wrong on the PEFT repo.

See #3910

@arnavgarg1 arnavgarg1 requested review from w4nderlust, tgaddair, justinxzhao, geoffreyangus, jeffkinnison, Infernaught and alexsherstinsky as code owners January 23, 2024 16:48
justinxzhao
justinxzhao reviewed Jan 23, 2024
View reviewed changes
@github-actions GitHub Actions
Copy link

Unit Test Results

  6 files  ±0    6 suites  ±0   14m 15s ⏱️ -11s
12 tests ±0    9 ✔️ ±0    3 💤 ±0  0 ❌ ±0 
60 runs  ±0  42 ✔️ ±0  18 💤 ±0  0 ❌ ±0 

Results for commit a1baa71. ± Comparison against base commit 91c8975.

@arnavgarg1 arnavgarg1 merged commit 9bb89c6 into master Jan 23, 2024
@arnavgarg1 arnavgarg1 deleted the phi2_modules branch January 23, 2024 17:52
vijayi1 pushed a commit to vijayi1/ludwig that referenced this pull request Jan 23, 2024
@arnavgarg1 @vijayi1
Add default LoRA target modules for Phi-2 (ludwig-ai#3911)
a77c754
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@justinxzhao justinxzhao justinxzhao approved these changes

@w4nderlust w4nderlust Awaiting requested review from w4nderlust w4nderlust is a code owner

@tgaddair tgaddair Awaiting requested review from tgaddair tgaddair is a code owner

@geoffreyangus geoffreyangus Awaiting requested review from geoffreyangus geoffreyangus is a code owner

@jeffkinnison jeffkinnison Awaiting requested review from jeffkinnison jeffkinnison is a code owner

@Infernaught Infernaught Awaiting requested review from Infernaught Infernaught is a code owner

@alexsherstinsky alexsherstinsky Awaiting requested review from alexsherstinsky alexsherstinsky is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@arnavgarg1 @justinxzhao
0