8000 perf(parser): speed up simple lookaheads by introducing `Lexer::peek_token` by ulrichstark · Pull Request #11358 · oxc-project/oxc · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

perf(parser): speed up simple lookaheads by introducing Lexer::peek_token #11358

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

perf(parser): speed up simple lookaheads by introducing Lexer::peek_token #11358

wants to merge 1 commit into from

Conversation

ulrichstark
Copy link
Contributor
@ulrichstark ulrichstark commented May 28, 2025 •
edited
Loading

This is my proof of concept function to implement @overlookmotel's tip:

We could also introduce a more slimmed-down (cheaper) version of lookahead when all you need is to get the next 1 or 2 tokens from lexer and then reset its pointer and errors_pos back to where they were before. Current lookahead does quite a lot of work which isn't required in these cases.

source: #11334 (comment)

@github-actions github-actions bot added A-parser Area - Parser C-performance Category - Solution not expected to change functional behavior, only performance labels May 28, 2025
@codspeed-hq CodSpeed HQ
Copy link
codspeed-hq bot commented May 28, 2025 •
edited
Loading

CodSpeed Instrumentation Performance Report

Merging #11358 will improve performances by 18.72%

Comparing ulrichstark:attempt-at-speeding-up-simple-lookaheads-in-parser (65f87d8) with main (b272b91)

Summary

⚡ 1 improvements
✅ 37 untouched benchmarks

Benchmarks breakdown

Benchmark BASE HEAD Change
⚡ mangler[cal.com.tsx] 3.5 ms 3 ms +18.72%

@ulrichstark ulrichstark marked this pull request as ready for review May 28, 2025 20:51
@ulrichstark

This comment was marked as outdated.

Sign in to view
@ulrichstark ulrichstark mentioned this pull request May 28, 2025
Open
@leaysgur
Copy link
Member

oxc/crates/oxc_parser/src/js/statement.rs

Lines 347 to 354 in adbb3a3

&& self.lookahead(|p| {
p.bump_any();
if !p.at(Kind::Using) || p.cur_token().is_on_new_line() {
return false;
}
p.bump_any();
!p.cur_token().is_on_new_line()
})

FYI: Not often, but in a few places this happens.

@Boshen
Copy link
Member
Boshen commented May 29, 2025

This looks very promising! Let's wait until we fix all the code around parser peek before going into this.

@Boshen Boshen marked this pull request as draft May 29, 2025 01:26
@ulrichstark ulrichstark force-pushed the attempt-at-speeding-up-simple-lookaheads-in-parser branch from 1a8ba85 to cf5ff60 Compare May 29, 2025 09:54
@ulrichstark
Copy link
Contributor Author
ulrichstark commented May 29, 2025 •
edited
Loading

Just rebased to include #11356 that was causing conflicts.

@ulrichstark ulrichstark force-pushed the attempt-at-speeding-up-simple-lookaheads-in-parser branch 2 times, most recently from c9f8278 to 0ec970e Compare June 6, 2025 20:55
@ulrichstark ulrichstark changed the title perf(parser): speed up simple lookaheads by introducing Lexer::lookahead_token perf(parser): speed up simple lookaheads by introducing Lexer::peek_token Jun 6, 2025
@ulrichstark
Copy link
Contributor Author
  • Rebased again
  • Changed api from Lexer::lookahead_token taking predicate to Lexer::peek_token returning peeked token
  • Found three more cases to use Lexer::peek_token

@Boshen Boshen self-assigned this Jun 7, 2025
@ulrichstark ulrichstark force-pushed the attempt-at-speeding-up-simple-lookaheads-in-parser branch from f247fc6 to 65f87d8 Compare June 15, 2025 15:04
@Boshen
Copy link
Member
Boshen commented Jun 16, 2025

I'll take a look at the remaining token peeks before this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@Boshen Boshen

Labels
A-parser Area - Parser C-performance Category - Solution not expected to change functional behavior, only performance
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ulrichstark @leaysgur @Boshen
0