8000 `import .zng` and merging Zngur files by nickpdemarco · Pull Request #43 · HKalbasi/zngur · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

import .zng and merging Zngur files #43

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

nickpdemarco
Copy link

Adds a gm command which takes arbitrarily many .zng files and a required --in flag which indicates where the generated files should be placed.

This work should be considered a draft for architecture review by @HKalbasi - before spending more time improving error messages and thinking about semantic edge cases, I wanted to see if this passes the smell test.

See examples/example-simple-merged to get a sense of how it works. Entities are semantically merged at the pa 8000 rsing stage - unless entities conflict (e.g different layout specs), their fields are merged in predictable ways: usually, that means a union of things like methods, constructors, fields, etc.

@nickpdemarco nickpdemarco marked this pull request as draft June 2, 2025 19:23
@HKalbasi
Copy link
Owner
HKalbasi commented Jun 2, 2025

The merge implementation looks good to me. I need to think a bit more about edge cases, but I guess raising error in case of conflict is safe at the start, and we can relax it later.

About the user interface, what about having an import command instead of/in addition to the cli merge command? That is, there would be a main.zng file:

import "./one.zng";
import "./two.zng";

// Maybe some things here as well

It seems like a useful feature for organizing big zng files, and its implementation is basically equivalent to this feature. Also, in the future, it might enable a way for resolving conflicts using the root file. What do you think?

@nickpdemarco
Copy link
Author

Sure - I think adding import is a great idea. Would you prefer that in this PR or a separate one? I worry this one is already getting a bit large.

@nickpdemarco
Copy link
Author
nickpdemarco commented Jun 3, 2025

Okay, that was pretty easy! I added a simple import directive (with pretty terrible error messages) in this PR. I don't think we need gm anymore, but I've kept it for now pending your review. If you'd like to remove gm and just use import, let me know. Note I've renamed ZngurFile to ZngurSpec because now there isn't a 1:1 correspondence between that entity and .zng files.

In general I'm having some issues getting good error messages for things like import and semantic conflicts because of how things are currently designed. ZngurSpec really only has semantic entities, no spans, so emitting a rich error is a bit tricky. I'm curious if you have a solution for this.

@nickpdemarco nickpdemarco changed the title Generate Merged import .zng and merging Zngur files Jun 3, 2025
@HKalbasi
Copy link
Owner
HKalbasi commented Jun 3, 2025

I added a simple import directive (with pretty terrible error messages) in this PR.

That's great!

If you'd like to remove gm and just use import, let me know.

I'm happy with removing the gm command if you don't need it.

ZngurSpec really only has semantic entities, no spans, so emitting a rich error is a bit tricky. I'm curious if you have a solution for this.

I guess we will need to pass spans in higher level representations as well. It would also be helpful for late stage errors that we currently don't have, like raising error about missing types.

For testing those errors, we will probably need a function that creates a virtual filesystem like tests in rust-analyzer (Using array of file contents instead of comments makes things cleaner and simpler I think, but exact details don't matter).

@nickpdemarco
Copy link
Author

I'm happy with removing the gm command if you don't need it.

Yeah, I'll pull it to keep the API as simple as possible.

I'm trying to figure out the semantics of cpp_value, cpp_ref, and layout_policy - it seems that cpp_ref implies !layout_policy, but I can't find any other currently-coded constraints. As I write the merge logic for these, is there anything else I should keep in mind?

Copy link
Contributor
@camio camio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making the example more relatable

To make this example a bit less contrived, you can create a couple C++ header/cpp pairs that make use of different rust entity subsets. You'd then create a corresponding .zng file for each.

foo.h
foo.cpp
foo.zng
bar.h
bar.cpp
bar.zng
main.cpp
main.zng
// main.zng
import "./one.zng";
import "./two.zng";

/* any entities `main.cpp` makes use of */

Ideally, "foo" and "bar" would be named after what functionality they each provide.

Import search path

There isn't a command-line interface to set the import search path, but I think we'll need this to avoid forcing a directory layout with multiple zng-containing dependencies. I think we should allow for multiple -I command-line options and, if there are none, default to the directory zng was invoked from. This is what protobuf does.

To run this example:

```
make
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to modify this command to make it work on my MacOS machine

Suggested change
make
make CXX="clang++ -std=c++11"

Comment on lines +1 to +10
a.out: main.cpp generated.h src/generated.rs src/lib.rs ../../target/release/libexample_simple_import.a
${CXX} -Werror main.cpp generated.cpp -g -L ../../target/release/ -l example_simple_import

../../target/release/libexample_simple_import.a: FORCE
cargo build --release

generated.h generated.cpp ./src/generated.rs: main.zng
cd ../../zngur-cli && cargo run g ../examples/simple_import/main.zng

FORCE: ;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The .PHONY target is the proper way to force rebuilds in makefiles

Suggested change
a.out: main.cpp generated.h src/generated.rs src/lib.rs ../../target/release/libexample_simple_import.a
${CXX} -Werror main.cpp generated.cpp -g -L ../../target/release/ -l example_simple_import
../../target/release/libexample_simple_import.a: FORCE
cargo build --release
generated.h generated.cpp ./src/generated.rs: main.zng
cd ../../zngur-cli && cargo run g ../examples/simple_import/main.zng
FORCE: ;
.PHONY: ../../target/release/libexample_simple_import.a
a.out: main.cpp generated.h src/generated.rs src/lib.rs ../../target/release/libexample_simple_import.a
${CXX} -Werror main.cpp generated.cpp -g -L ../../target/release/ -l example_simple_import
../../target/release/libexample_simple_import.a:
cargo build --release
generated.h generated.cpp ./src/generated.rs: main.zng
cd ../../zngur-cli && cargo run g ../examples/simple_import/main.zng

@nickpdemarco nickpdemarco force-pushed the nickpdemarco/generate-merged branch from 5e0f6a5 to 691edad Compare June 5, 2025 19:26
@nickpdemarco nickpdemarco force-pushed the nickpdemarco/generate-merged branch from 691edad to e22b9b6 Compare June 5, 2025 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0