8000 Reorganize native hosting and service API · Issue #101 · atsushieno/aap-core · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Reorganize native hosting and service API #101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
atsushieno opened this issue Feb 25, 2022 · 3 comments
Closed

Reorganize native hosting and service API #101

atsushieno opened this issue Feb 25, 2022 · 3 comments

Comments

@atsushieno
Copy link
Owner
atsushieno commented Feb 25, 2022 •
edited
Loading

It is 50% design document, but tentative, also will be updated as it goes. Not a comprehensive list (at least yet).

Background

AAP had started with similar concept to desktop plugin APIs like LV2, which do not have cross-process client-service context. When implementing the framework, I used aap::PluginHost in plugins, not just hosts, because an AAP plugin is a service and needs hosting functionality. It then brought in confusion. It tries to support multiple instances from the same service, or client for different plugin services.

It's not just about those aap::PluginHost. For example, it affected how "known plugins" are managed there.

The hosting role and servicing role should be distinguished. And it would make sense to redesign all those related bits this time.

Problems

plugin management

In the first place, known plugins should not be stored in a global field as a fixed list. It makes sense when querying AudioPluginService packages takes significant amount of time, but so far it isn't.

  • AndroidPluginHostPAL::initializeKnownPlugins() is an entrypoint that sets plugin list cache.
    • It calls PluginHostPAL::setPluginListCache()
    • PluginHostPAL::getPluginListCache() retrieves the list. aap-juce is the only user so far.
    • PluginHostPAL::getInstalledPlugins() makes use of this list too.
  • PluginHostManager::updateKnownPlugins() sets the list of plugins, and getPluginInformation() (either by index or identifier) makes use of the list.
    • PluginHostPAL::getInstalledPlugins() mentioned above is used here.
  • AndroidPluginHostPAL holds a list of connections that contain AIBinder*, which means that the connections are statically global.

It should be also noted that PluginHostPAL::getInstalledPlugins() is implemented in somewhat complicated way, because it was meant to cover desktop hosting usages and was designed to work like JUCE AudioPluginFormatManager.

There is also no concept of plugin services apart from plugins.

instance management

  • There is no class for list of client connections that holds AIBinders in each element.
    • Surprisingly? they were stored globally and directly(!) at AndroidPluginHostPAL::serviceConnections.
    • And were directly added and/or removed in AudioPluginNatives_jni.cpp.

New design

non-public PAL

Any PAL layer e.g. aap::PluginHostPAL will not be public anymore. It was public due to aap-juce support requirements which was initially it was part of the same repo and split later (therefore confusion). We will dump getPluginHostPAL() global function. It is widely used so far, to retrieve plugin list, manage service connections, and create shared memory.

plugin information structures

There will be a new aap::PluginServiceInformation class, along with aap::PluginInformation and aap::PortInformation.

Later we will introduce aap::ParameterInformation (which is for another redesign issue).

Plugin list can be acquired by aap::PluginListSnapshot::queryServices(). We keep two-folds plugin query steps:

  • on Android, plugin services are queried list of aap::PluginServiceInformation (new), and then each aap_metadata.xml is parsed to turn it into list of aap::PluginInformation
  • on Desktop (not under actual development yet), they are queries per AAP_PATHs to query aap_metadata.xml, then follows the same steps as Android.

The list of aap::PluginServiceInformation should be retrieved on demand.

plugin instancing

  • aap::PluginInstance : one single instance of either a client instance or a service instance

    • aap::PluginServiceInstance
    • aap::PluginClientInstance
    • aap::PluginClientConnection : a new public API surface that represents a plugin client connection.
      • AIBinder* is stored as an opaque pointer.
      • It was aap::AudioPluginServiceConnection before (fun!)

  • aap::PluginClientConnectionList : connection list that should be managed per plugin client.

    • It cannot be instantiated without PAL aid
    • on Android, it is mapped to an org.androidaudioplugin.AudioPluginServiceConnector Kotlin object, and...
    • ... it needs to be a live list and its aap::PluginClientConnection entry has to be assigned an AIBinder* at JNI layer.

  • aap::PluginService : a native service instance that manages multiple instances for a binder connection

  • aap::PluginClientConnectionList : a list that manages multiple instances for different binder connections.

  • aap::PluginGraph : represents an audio plugin graph. May be optional single-instance entity at first.

  • aap::PluginHost : organizes aap::PluginClientConnectionList and aap::PluginGraph to function as a host.

Feature wise, aap::PluginClientConnectionList corresponds to part of aap::PluginHostManager features as and some from AndroidPluginHostPAL. It is designed to hold multiple instances as a client, and used by AAPMidiProcessor and AudioPluginInterfaceImpl, but never had complicated uses. They only deal with only one plugin.

Active Sensing

aap::PluginClientInstance::activate() starts active sensing, and aap::PluginClientInstance::deactivate() stops it. While processing audio, it is supposed to send active sensing messages. It should have no timestamp content as there may be time mismatch.

AAP aap::PluginServiceInstance::updateActiveSensingStatus() is called by the native binder based service implementation, and updates latest_active_sensing_time field (time is managed only within the service process). When aap::PluginServiceInstance::process() runs, it checks the time field and if it was way old (which TBD, but it's 300 msec. in MIDI 1.0 Standard), it does not call the plugin's process() function.

Mapping

old API new API
PluginHostPAL PluginClientConnectionManager
getPluginHostPAL() static function will not be provided anymore.
PluginHostPAL::getPluginPaths() PluginClientConnectionManager::getPluginServices()
PluginHostPAL::getPluginListCache() PluginClientConnectionManager::getPluginList()
PluginHostPAL::getInstalledPlugins() PluginClientConnectionManager::getPluginList()
PluginHostManager PluginListSnapshot
AudioPluginServiceConnection PluginClientConnection
atsushieno added a commit that referenced this issue Feb 26, 2022
@atsushieno
A big overhaul in native API (WIP).
de0dc71 
It would not work as it only compiles, no run-time verification done.

context: #101
atsushieno added a commit that referenced this issue Feb 27, 2022
@atsushieno
more native hosting API revamp
21754f7 
context: #101

AndroidPluginHostPAL does not store plugin_list_cache anymore.
Caching should be done as in PluginListSnapshot instead.

Live plugin service connections are now stored per PluginServiceConnection
Kotlin object. They are stored on a global map<K,V> instead.

AAPInstrumentSample is locally working now. Still not verified anything else.
atsushieno added a commit that referenced this issue Feb 28, 2022
@atsushieno
native host API refactoring contd: split host into client and service.
f9b0e96 
context: #101

There has been a bunch of changes:

- PluginHost has derived PluginClient and PluginService now.
- AudioPluginServiceConnector is mapped to aap::PluginClientConnectionList.
- binder extension now passes aap::PluginClientConnectionList.

Note that this changeset rather broke service implementation (which was
working in the previous rev.).
atsushieno added a commit that referenced this issue Mar 6, 2022
@atsushieno
more native API overhaul
444a103 
context: #101

- rename ambiguous audio-plugin-host-android.h/.cpp
  (we have the same filename in the public headers)
- remove extraneous JNI code bits.
- add some ASAN build simplification trick that we have in aap-lv2.
@atsushieno
Copy link
Owner Author

An emerging issue is that the reorganization did not weigh in how/where Binder instances are managed:

  • In the old implementation, AudioPluginHost instantiated ALL plugins on the device at startup time, which was no-brainer solution and very inefficient, but had no problem on retrieving them.
  • AudioPluginMidiDeviceService first acquired Binder implementation by running bindService() and then passed it to native code.
  • The new native PluginClient implementation begins with the state where no Binder is instantiated yet. It is possible to kick AudioPluginHostHelper.ensureBinderConnected() method, but needs some consistent connection up to aap::PluginClient.

In the current overhaul-native-api branch, aap::PluginClientConnection holds connection_data opaque pointer field which is AIBinder* on Android. When Kotlin AudioPluginServiceConnector.onBindAudioPluginService()method is called, viaAudioPluginNatives.addBinderForClient()`.

This Android client could just call AudioPluginHostHelper.ensureBinderConnected() before trying to retrieve AIBinder*, but not sure if it should be done all in Android-specific code region (binder-client-as-plugin.cpp) or up to platform agnostic level (aap::PluginClientConnection, aap::PluginClientConnectionList, and aap::PluginClient).

@atsushieno atsushieno mentioned this issue Mar 25, 2022
Closed
@atsushieno
Copy link
Owner Author

A big bunch of updates are done as 542ee7 (merged overhaul-native-api branch), which covers a lot of the changes I described above.

A missing piece is active sensing, which I may create a separate issue.

@atsushieno atsushieno mentioned this issue Mar 31, 2022
Closed
@atsushieno
Copy link
Owner Author

^ is copied into the existing issue. Now we can close this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@atsushieno
0