8000 Add the Barefoot Tofino Assembler to the Tofino back end. by fruffy · Pull Request #5121 · p4lang/p4c · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Add the Barefoot Tofino Assembler to the Tofino back end. #5121

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Mar 18, 2025
Merged

Conversation

fruffy
Copy link
Collaborator
@fruffy fruffy commented Feb 7, 2025

The Tofino compiler is not really useful without the assembler. Hence we add it directly. Now the assembler is always active when we use bf-p4c.

The problem is that the assembler code is a little rough and does not pass many of the linting checks P4C has. This required a couple fixes and workarounds. I commented the remaining TODOs in the CMake files. Ideally, we gradually fix all the warnings that are thrown over time.

@fruffy fruffy added the tofino Topics related to the Tofino switch and back end. label Feb 7, 2025
@fruffy fruffy force-pushed the fruffy/assembler branch 12 times, most recently from b9b092b to 0abf784 Compare February 8, 2025 21:06
@fruffy fruffy force-pushed the fruffy/assembler branch 2 times, most recently from a28a715 to a2c1c66 Compare February 15, 2025 22:09
@fruffy fruffy marked this pull request as ready for review February 15, 2025 22:59
Copy link
Contributor
@jafingerhut jafingerhut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vlstill Feel free to give an actual detailed review (unlike this one, which is based on CI tests passing, the fact that these changes should affect ONLY the Tofino back end, and wanting to get the other parts of this working in the open-p4studio repo so we can leave it in a mostly-working state for others). However, as far as I can see right now, Fabian is the only one with the knowledge and interest to make these changes, and I'm the only other one interested in seeing that work move forward.

I have probably over-shared my qualifications on this approval :-)

@fruffy fruffy requested a review from ChrisDodd February 16, 2025 20:49
@ChrisDodd
Copy link
Contributor

Well, I guess as I originally wrote much of this code, I'm qualified to comment on it, if not review it.

The stuff in bf-asm/test is an old broken (non-functional) test suite that is based on running the even older python-based P4_14 compiler, and should probably just be deleted (nothing in it is used, and I don't think any of it is useful)

It looks like it does build fine (with just -DENABLE_TOFINO), but there aren't any tofino test cases? It seems like it should be possible to run some of the code in testdata through p4c --target tofino and then the assembler, but there's nothing set up to try to do that. When run manually, it looks ok, though. Currently ctest does run gtestasm, which runs some assembler internal unit tests, so there is that much.

It does have additional dependencies to build (I noticed at least pyyaml -- needed pip3 install pyyaml to get it to build), but the README.md is very out of date overall with respect to dependencies.

@fruffy
Copy link
Collaborator Author
fruffy commented Feb 18, 2025

The stuff in bf-asm/test is an old broken (non-functional) test suite that is based on running the even older python-based P4_14 compiler, and should probably just be deleted (nothing in it is used, and I don't think any of it is useful)

Okay, I will remove that code.

It looks like it does build fine (with just -DENABLE_TOFINO), but there aren't any tofino test cases? It seems like it should be possible to run some of the code in testdata through p4c --target tofino and then the assembler, but there's nothing set up to try to do that. When run manually, it looks ok, though. Currently ctest does run gtestasm, which runs some assembler internal unit tests, so there is that much.

Currently, compilation should be covered by the P4Testgen tests which runs the compiler with the following arguments: https://github.com/p4lang/p4c/blob/main/backends/p4tools/modules/testgen/targets/tofino/test/TestTemplate.cmake#L94

But it is not running packet-tests yet because the model isn't integrated.

One problem is the amount of warnings being thrown: https://productionresultssa18.blob.core.windows.net/actions-results/2f425077-cf0d-43ba-b2d1-299f3e6c0924/workflow-job-run-db3cdbd7-db2f-5c6d-ea50-8a2acf176300/logs/job/job-logs.txt?rsct=text%2Fplain&se=2025-02-18T13%3A36%3A26Z&sig=znJVJB1RLVnjLjLQb3KD732dHNwMwxMpYLNV7gyj%2FKg%3D&ske=2025-02-19T00%3A43%3A35Z&skoid=ca7593d4-ee42-46cd-af88-8b886a2f84eb&sks=b&skt=2025-02-18T12%3A43%3A35Z&sktid=398a6654-997b-47e9-b12b-9515b896b4de&skv=2025-01-05&sp=r&spr=https&sr=b&st=2025-02-18T13%3A26%3A21Z&sv=2025-01-05

Most of them seem related to particular macro usage in the tables file. The expansion causes additional ; being added.

@ChrisDodd
Copy link
Contributor

One problem is the amount of warnings being thrown:

Most of the warning seem to come from base class virtual methods that have unused arguments -- these arguments are named for documentation/clarity (basically noting what the derived classes will do with them) even though they are not used in the base class method. Fixing the warnings by removing the arguments will make the code less self-documenting and harder to read/understand. I guess the arg names should be replaced by comments to both keep the info and silence the warnings.

@fruffy fruffy force-pushed the fruffy/assembler branch 2 times, most recently from 097b730 to 7229ed3 Compare February 20, 2025 19:06
@fruffy
Copy link
Collaborator Author
fruffy commented Feb 20, 2025

One problem is the amount of warnings being thrown:

Most of the warning seem to come from base class virtual methods that have unused arguments -- these arguments are named for documentation/clarity (basically noting what the derived classes will do with them) even though they are not used in the base class method. Fixing the warnings by removing the arguments will make the code less self-documenting and harder to read/understand. I guess the arg names should be replaced by comments to both keep the info and silence the warnings.

Slowly working my way through the warnings. There are a lot of warnings I can not fix because they are generated using chip.schema. Is there any way we can influence the generation of that code?

@fruffy fruffy force-pushed the fruffy/assembler branch 3 times, most recently from 806c7a8 to d200785 Compare February 20, 2025 21:07
@fruffy fruffy force-pushed the fruffy/assembler branch 3 times, most recently from 4f23857 to fa1091b Compare March 15, 2025 01:09
@fruffy
Copy link
Collaborator Author
fruffy commented Mar 15, 2025

@ChrisDodd If you have some time, could you give the last couple commits a review? I tried to fix some warnings but some of the changes I am not 100% sure about.

@fruffy fruffy force-pushed the fruffy/assembler branch from fa1091b to e05cc32 Compare March 15, 2025 20:10
fruffy added 2 commits March 15, 2025 16:11
Signed-off-by: fruffy <fruffy@nyu.edu>
Signed-off-by: fruffy <fruffy@nyu.edu>
@fruffy fruffy force-pushed the fruffy/assembler branch from e05cc32 to 692528b Compare March 15, 2025 20:11
Signed-off-by: fruffy <fruffy@nyu.edu>
@fruffy fruffy requested a review from ChrisDodd March 16, 2025 17:35
@@ -551,7 +551,7 @@ Instruction *AluOP::pass1(Table *tbl_, Table::Actions::Action *act) {
if (k1 && (k1->value < Target::STATEFUL_ALU_CONST_MIN() ||
k1->value > Target::STATEFUL_ALU_CONST_MAX())) {
if (k1->value >= (INT64_C(1) << tbl->alu_size()) ||
k1->value < (INT64_C(~0u) << (tbl->alu_size() - 1))) {
k1->value < (~UINT64_C(0) << (tbl->alu_size() - 1))) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To do a signed comparison, you need to cast the value to a signed type (int64_t). I would expect this to give a signed vs unsigned comparison warning (and will do an unsigned comparison, which is wrong.)

Signed-off-by: fruffy <fruffy@nyu.edu>
Signed-off-by: fruffy <fruffy@nyu.edu>
Copy link
Contributor
@vlstill vlstill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it is a reasonably investment of time to go over the assembler implementation, so I skipped the bf-asm dir and looked at the changes around it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this using FetchContent if the dir is in repo (submodule?).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like it was contributed this way. I can change the installation direction to the cmake build folder instead.

Copy link
Contributor
@vlstill vlstill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

going over the changes after the initial bf-asm additon, part I

Co-authored-by: Vladimír Štill <git@vstill.eu>
Signed-off-by: Fabian Ruffy <5960321+fruffy@users.noreply.github.com>
Copy link
Contributor
@ChrisDodd ChrisDodd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to have some automated tests (in CI?) that ran the tofino backend compiler and assembler. Intel used to run everything in testdata/p4_14_samples and all the bmv2 tests in p4_16_samples this way.

@fruffy
Copy link
Collaborator Author
fruffy commented Mar 17, 2025

It would be good to have some automated tests (in CI?) that ran the tofino backend compiler and assembler.

We do have around 50 tests for TNA and T2NA as part of P4Testgen's workflow, which also invokes the compiler and assembler.

I can also add the P4-16 BMV2 tests?

@ChrisDodd
Copy link
Contributor

It would be good to have some automated tests (in CI?) that ran the tofino backend compiler and assembler.

We do have around 50 tests for TNA and T2NA as part of P4Testgen's workflow, which also invokes the compiler and assembler.

Ok, that is probably enough. As long as there is something that is executing at least some of the code.

It still would be nice to do some gcov (or similar) analysis of the compiler running the testsuite to see what code is (not) excercised anywhere in the testsuite.

@fruffy
Copy link
Collaborator Author
fruffy commented Mar 18, 2025

Merging this one. The other issues I can try to resolve in subsequent PRs.

@fruffy fruffy added this pull request to the merge queue Mar 18, 2025
Merged via the queue into main with commit fd4ae76 Mar 18, 2025
20 checks passed
@fruffy fruffy deleted the fruffy/assembler branch March 18, 2025 03:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tofino Topics related to the Tofino switch and back end.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants
0