[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DRAFT] Diz 3.0 rewrite - early pull request for comments / review #48

Draft
wants to merge 292 commits into
base: master
Choose a base branch
from

Conversation

binary1230
Copy link
Collaborator
@binary1230 binary1230 commented Apr 30, 2021

Diz 3.0

WARNING: This is super-early super-rough alpha-level code. Don't use yet unless you're ready to jump in with a C# debugger and hack the unit tests :) Do not even attempt to open prior projects yet. I am mostly opening this PR to start making my own release checklists. The code is now starting to cool down enough that it could be worth checking out now. The UI needs the most polish work, the underlying data model is starting to be pretty solid.

This is basically another major rewrite of Diz, which will be version 3.0. Major new features include:

  • Port to .NET5 !
  • Full overhaul of the data model (see below). Flexibility extends far beyond SNES now
  • Overhaul the UI and enable multiple simultaneous views of byte data from multiple projects and ROMs at once
  • Decoupling the UI from the mapping to the underlying data, making it more flexible
  • Easy to add more UI components and tie them into the underlying view architecture (i.e. do you want a graphics viewer or a hex editor? implement a new View interface and it's a piece of cake)
  • Rewrite of the SNES assembly output engine, easier to mod and test
  • Add Powershell command-line interface, to integrate Diz into your automation workflows (easy to port to other stuff like Python/etc)
  • Diz.Core is now a standalone library that can be included in other projects or emulators. Or, swap in your own CPU engines/etc or pipe data through from emulators/etc
  • Heavy unit test coverage on Diz.Core ByteSource/Storage
  • Flexible XML output to be able to accomodate future expansion
  • Use Github actions for builds / testing
  • CPU stepping functionality refactored into a way that's easier to control and subclass, so adding new CPUs or variants should be easy.
  • Diz's workhouse datagrid has been supercharged to allow displaying arbitrary columns. Just pass it a new class with different properties and it'll automatically change things up and format according to attributes
  • Datagrid has intelligent caching to preload and keep in memory the most recently viewed bytes. Should make dealing with the largest projects a breeze

Data Model

  • Probably the most exciting part: Diz now utilizes a series of arbitrarily nested regions of memory, called ByteSources, which overlay, combine, and resolve into a final view for the UI and assembly output
  • For example, for SNES, a typical setup will be: the root ByteSource is the SNES Address Bus, which contains a ChildSource
  • Customize as you like, for example, create a mapping that overrides underlying data
  • Mirroring should be handled well now: Map a region of WRAM into several mirrored regions. Tag bytes on WRAM with labels/annotations and they will magically show up in the correct mirrored addresses in the address space
  • The original RomByte class has been broken up into several "Annotation" classes. These can be attached to a byte on any ByteSource layer
  • Underlying storage can either be set to continuous (list based), great for quick and direct access in ROMs. or it can be set to sparse (dictionary based), which is great for large empty address spaces like CPU busses.
  • The view of the various byte regions largely "does the right thing" and is easy to consume from the SNES Address bus level. However, the underlying low-level data model is able to return a graph that lets you traverse through the layers and collect / aggregate the underlying data for more advanced usages.
  • Data can be setup with different policies. For example, a "patch" could be setup as a ByteSource that maps to 100 bytes of a bank of the SNES ROM, and anything in the path hides the data underneath. Or, take dumps of RAM at various points in the game and cycle through them.

Future stuff now easier to implement

  • Diz is currently setup just for SNES, but, the overhauls make it far easier to add in support for other systems and CPUs.
  • Memory mapping system was designed so it can map non-linearly if needed (i.e. map a compressed region of ROM into RAM via a custom algorithm)
  • Dependency injection support coming soon to add extra plugins/etc
  • Expression parsing for instructions that can be constrained to resolve back to the original ROM bytes (mirrored labels giving you the wrong offsets? it'll be no prob)
  • No reason this stuff couldn't be used to make a web-based visualizer via React/Blazor/etc.
  • It is my hope that the Diz XML file format can be used as an archival format for being both human-collaborator-friendly and a long-term storage friendly way to preserve videogame history

Checklist before putting this in:

  • Update credits to include @furious who contributed some of the .NET5 work / Github CI, and has a few more patches we need to integrate. also work done in Upgrade to .NET 5 (latest version) #42 was the foundation of this PR
  • Finish UI, it's a bit haggard right now after the data model
  • Probably merge this as a rebase and not the individual commits, my commits are kind of sloppy and not atomic up to this point
  • Finish polish on MVC architecture, it got a little beat up in the refactor
  • Backwards compatibility for previous version XML files
  • Optimize performance, if needed, for XML serialization
  • oy. so much more.
  • Get decent code coverage on remaining fringes of Data model (we're actually pretty decent right now)
  • Cleanup lots of commented and junk code leftover
  • Test some real-world .dizraw files I have
  • Notify the SNESlab discord of progress
  • Code review.
  • Consider un-renaming some of the files to make the diffs easier to look at?
  • IPropertyChanged stuff is horribly broken at the moment, re-enable and test. unit tests

Closes #46

…ents and labels. ready for integration with main form after a little more cleanup
- a bit too slow though, have to profile
- this is a giant mess right now, heavy WIP
- WORKS! messy as hell
- definitely breaking some stuff, this needs lots of testing
- starting to get there now
- DizApplicationContext
- controller/view interfaces
- initial work on "current selection"
- refactor INotifyPropertyChanged to be an interface (avoids needing to derive a concrete class)
- remove DizDocument (doesn't make a ton of sense in the context of multi-project editing)
- fix double buffering called @ wrong time
- add better table navigation functions
- remove BindingList (too slow for now)
- quick UX pass on start form
- remove BindingListView
- remove creation of RomDataGridRow() until it gets to the form control [much faster loading]
- databinding is broken for multiple forms open, but that's next
- (needs 2 other csprojs to work)
- enabled by default, need to disable
1. Don't double buffer if we're in remote desktop
2. Add SendMessage() so we can disable drawing for datagrid loads
@rdrpenguin04
Copy link

Will this be compatible cross-platform?

@binary1230
Copy link
Collaborator Author

Will this be compatible cross-platform?

I'm hoping it will. Everything already is except the GUI, and there's another branch off this branch that is attempting to replace the winforms GUI with AvaloniaUI, which is cross platform.

@rdrpenguin04
Copy link

Okay, thank you!

@rdrpenguin04
Copy link

Update: Tried on avalonia-ui-test2 (on Linux), and dotnet build still doesn't work, saying "Windows is required to build Windows desktop applications." If you want I can send you the full error here; otherwise, it may be better to open this as an issue on your repo.

@binary1230
Copy link
Collaborator Author

Update: Tried on avalonia-ui-test2 (on Linux), and dotnet build still doesn't work, saying "Windows is required to build Windows desktop applications." If you want I can send you the full error here; otherwise, it may be better to open this as an issue on your repo.

yea, unfortunately, it's still referencing winforms, the avalonia-ui-test2 branch is BOTH avalonia and winforms together, with the intention of slowly replacing one with the other.

also there's little avalonia UI actually written (I do have the main grid in a simplified form). the big thing going on first is to get the UI using ReactiveUI under the hood.

If you really did want to try it under linux, I -think- in that fork I had put all the winforms stuff in one project. you could modify the solution to not build that, and start avalonia.

all this is super-untested and very early, I'd have to look around and make sure it compiles/etc at the moment. I have though run it with the minimal avaloniaUI on linux. it might have been with mono, I forget (stock .net -should- work OK here now on non-windows)

@binary1230
Copy link
Collaborator Author

Update: Tried on avalonia-ui-test2 (on Linux), and dotnet build still doesn't work, saying "Windows is required to build Windows desktop applications." If you want I can send you the full error here; otherwise, it may be better to open this as an issue on your repo.

yea, unfortunately, it's still referencing winforms, the avalonia-ui-test2 branch is BOTH avalonia and winforms together, with the intention of slowly replacing one with the other.

also there's little avalonia UI actually written (I do have the main grid in a simplified form). the big thing going on first is to get the UI using ReactiveUI under the hood.

If you really did want to try it under linux, I -think- in that fork I had put all the winforms stuff in one project. you could modify the solution to not build that, and start avalonia.

all this is super-untested and very early, I'd have to look around and make sure it compiles/etc at the moment. I have though run it with the minimal avaloniaUI on linux. it might have been with mono, I forget (stock .net -should- work OK here now on non-windows)

the core of the project though IS separated out so it works without a gui (there's a powershell command-line-only project included that demonstrates using the Diz library functionality from non-gui). that means it can easily work inside another GUI or even in webassembly like on a webpage, or as a backend with a react-ui javascript frontend/etc.

sorry, I mostly ran out of time to keep working on this for the forseeable future. if you were interested in digging in, I'm down to help, a LOT of the backend datamodel overhaul has been completed in this branch already, the UI is just not caught up to it great.

- fixes IsoFrieze#62
- both 'Dword ptr' and 'Graphics' were using the character 'G' for their compressed encoding in the XML, must be unique
- I have... no idea why I didn't just use a Dictionary here in the first place. switched to that to enforce unique keys.
- change char for 32bit dword point from 'G' (conflict) to 'H' (unique)

(cherry picked from commit f1ce061)
- erratic builds, gotta figure out what's wrong it
- included automatically by every file in the solution
- this builtin include replaces our custom CommonSettings.targets file, more elegant
- for backporting to 2.0 branch
- moving stuff around so easier to merge / identical with 2.0 branch
- backporting lots of this stuff in small pieces since doing all of it at once in a huge pain
@zoncabe
Copy link
zoncabe commented Aug 13, 2022

hi guys, im really interested into making this work on linux
in what state is this project currently?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Request for comment: new data model proposal
3 participants