8000 Improve file loading transparency by johschmitz · Pull Request #403 · esmini/esmini · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Improve file loading transparency #403

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

johschmitz
Copy link
Contributor

This is a preliminary pull request to potentially get some feedback. I yet have to add the same changes for other filetypes (not only the catalog files as it is now)

Provide more feedback about the loading process of the interlinked files (.xodr, .osgb, catalogs) to the user and avoid loading files relativ to the working directory which may result in undesired side effects since the working directory can be located anywhere. The user may use the --path commandline option in order to intentionally emulate the old behavior.

@eknabevcc
Copy link
Collaborator

Thanks for proposed changes! I think the code, as such, looks really good. But I have two comments:

1. Bump c++ version
I see you bumped c++ std version from 14 to 17, driven by std::filesystem I assume. We strive to keep as old as possible to maximize usability of esmini, not only portability in terms of various systems and platforms but also time dimension. It should be possible to compile esmini on older systems. It's balance. Not long ago we could skip 32bit Win 7.1 SDK support and C++ std 11, moving to 14. Which was very nice. But to move into 17 would require some investigations and tests which would require more incitements to perform.

So this part either has to be changed to comply with c++ 14, or we would need to hold this PR until we know the bump to c++ 17 is OK.

2. Missing case
The initial purpose discussed preceding this PR was to withdraw the search in current folder. If any user would need that search path it can be added by means of the --path argument (--path .), just as you mention 👍

However, to my understanding (and I might be wrong), there is another case being lost: Search for catalog directly in explicitly specified path (via --path argument).
Example:
Specified catalog path in xosc: ../my_typical_catalog_path/my_catalog.xosc
Specified search path with: --path ../my_experimental_catalog_path
Currently esmini would look in both:
../my_typical_catalog_path/../my_experimental_catalog_path/my_catalog.xosc
and
../my_experimental_catalog_path/my_catalog.xosc

Which has both pros and cons, I believe. But this is also a change that I'm not very comfortable about without some investigation and checking with users. Hence, either the current behavior should be maintained, or we need to put the PR on hold until investigated OK.

Conclusion
My recommendation would be to make this in two or three steps:

  1. Baby step change of behavior: Withdraw the current folder auto check (now)
  2. Limit use of explicitly specified path, only concatenate with specified catalog search path (later)
  3. Bump to c++ 17 and make use of the nice std::filesystem library (potentially together with step 2)

Let me know if I misunderstood the lost search path case.

@johschmitz johschmitz force-pushed the improve_loading_transparency branch from 129f138 to 6559591 Compare March 4, 2023 12:58
@johschmitz
Copy link
Contributor Author

@eknabevcc I did consider the --path requirement in my implementation but there was a bug so I made a slight change to the prove of concept such that the --path properly works now, see the new push.
The user will have to provide the additional resource paths like this:

--path /<user>/schmitz/bdsc/catalogs/vehicles --path /home/<user>/bdsc/catalogs/pedestrians

I tested it in the debugger (see screenshot) and it worked.
image
Regarding the C++ 17 upgrade I can only say we had a very smooth experience with that at my day job, basically it just worked. So I would strongly encourage you to take that step somewhat soon. I think C++ 20 is a bit more complicated and with that I would be a bit more careful.

@johschmitz johschmitz force-pushed the improve_loading_transparency branch from 6559591 to 30dadfe Compare March 4, 2023 14:14
@johschmitz
Copy link
Contributor Author

I also added the .xodr loading part. Now only the .osgb part is missing.

@johschmitz johschmitz force-pushed the improve_loading_transparency branch 2 times, most recently from 15f535c to 0f42fdd Compare March 4, 2023 16:10
@johschmitz
Copy link
Contributor Author

I have also added the .osgb part now and updated some formatting.

@johschmitz johschmitz changed the title [Work in progress] Improve file loading transparency Improve file loading transparency Mar 4, 2023
@eknabevcc
Copy link
Collaborator

As stated in first comment my recommendation would be to do a baby step first, changing only the actual behavior according to the initial issue (i.e. remove only current directory as pivot search folder and keep C++14).

Now we have instead extended changed behavior in addition to bumped C++ version.

It's perfectly fine to keep the change like this, as one larger step. However then it needs some considerations and checking (both behavior and C++ version) before it can be merged. So we simply need to put in on hold until that can be done.

Also before merge we should document the changed behavior. Preferably somewhere in the User guide. It's written in asciidoc format. Suggestion would be to edit it with Visual Studio Code. There is even an extension for previewing the result, AsciiDoc, although it's not perfect and usually not needed for simple text and lists. As complement I use the command line tool asciidoctor, from same developer, to generate HTML.

I understand that you might want to postpone the documentation until we have a "go" for merge. But at least a brief description of the changed behavior is needed for the investigation. So maybe you could provide that in a simple text format here, the briefer the better. That could potentially be re-used for the User guide in a next step. Of course, I could handle the actual update in User guide if you'd prefer that, as long as you provide the content.

Also, the commit don't pass the CI, failing on the compile step. That also needs to be looked into.

@johschmitz
Copy link
Contributor Author

Thanks for the preliminary review. Just to explain my thought process: I could add another non-C++ 17 pull request. But since I now understand the quirks of the current implementation and can avoid them as a user I am not in such a huge hurry to get it merged. You wrote "or we would need to hold this PR until we know the bump to c++ 17 is OK." and I decided aha that is an acceptable option for me, let's move forward. I definitely also did want to have a clean reference implementation using absolute and canonical loading paths for myself first to understand what is going on. All that absolute and canonical stuff is kind of hard to get right for all edge cases if you have to implement it yourself. People have long been using libraries for that.

We can still do the step by step thing if you want though. It is just that I started with the last step and will now work my way back. I can then rebase this PR on top of the other if that is needed. A rough estimate of the C++ 17 transition roadmap would surely be helpful in deciding if it makes sense to put in the extra work.

I can also edit the asciidoc documentation no problem. If you have some idea into which chapter this information needs to go let me know.

As for the failing builds, it seems to be related to the tests which I did not run locally. Need to look into this and fix them.

@eknabevcc
Copy link
Collaborator

I'll get back regarding C++17 latest end of next week. Awaiting some input on the topic. Let's wait with further steps until then.

Regarding failed CI: I just checked the first log file. Seems to be a compiler warning/error (we changed policy on CI to treat specified warnings as errors/failed build), so it could be a minor thing:
C:\projects\esmini\EnvironmentSimulator\Modules\ScenarioEngine\SourceFiles\ScenarioEngine.cpp(838,80): error C2665: 'roadmanager::Position::LoadOpenDrive': none of the 2 overloads could convert all the argument types

Provide more feedback about the loading process of the interlinked files
(.xodr, .osgb, catalogs) to the user and avoid loading files relativ to
the working directory which may result in undesired side effects since
the working directory can be located anywhere. The user may use the
--path commandline option in order to intentionally emulate the old
behavior.
@johschmitz johschmitz force-pushed the improve_loading_transparency branch from 0f42fdd to 17afd09 Compare March 7, 2023 19:16
@johschmitz
Copy link
Contributor Author

Alright, I will wait for an update. Take your time since it is not super critical. Meanwhile I took a stab at updating the documentation. What I am unsure about is how to run the tests locally. There also does not seem to be much information regarding that topic in the documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
0