Static "reflection" capability: general parsed data from classes, methods or vars #701

icatalud · 2019-11-21T21:05:28Z

Updated

There is a number of issues (#251, sdk#39438, sdk#1150, sdk#39438) which are all related to the capacity of programmatically performing actions in the code according to parsed information from the code itself. If the parsed information was somehow available from within the code it would be possible to automate code patterns which would otherwise require to manually change the code in several places instead of just one and being also a convenience the static safety assurance that is provided by this parsed data instead of having to write a string manually (prone to error). Historically parsing has been used to execute code and “reading the code from within the code” has not been thought of as useful because it has no meaning from the perspective of code execution, however practice has shown referencing previously typed code is useful, especially in static typed languages.

Accessing the “code from within the code” could also provide the consistency convenience of static safety to certain code paths that have generally been accessible only through the means of what is called "reflection". The reflection capability could be separated as converting:

Class and methods to String data
String data to Class and methods

Number 1 is direct to approach as compile time constants.

Number 2 could be statically satisfied over the visible interface of a variable by creating a one line shortcut for a cumbersome code pattern.

Another way of understanding what accessing the parsed information means is: going back to the written definition of a variable. variables-classes-methods are fundamentally a reference to a pointer, being their names useless from the perspective of the machine. But practice has shown that sometimes developers do want to programmatically use the written definition of a var (or class or method, etc) as a string.

Proposal: `parse` keyword

Access the static “written code” data of any variable in the code (class, method or var) using the keyword-method parse:

class A { final bar = ‘bar’; foo(int x) {} }

const ParsedClass cls = parse(A);
const ParsedField field = cls.vars.bar;
const ParsedMethod method = cls.methods.foo;
const Map<String, ParsedMethod> methodsMap = cls.methodsMap;
const Map<String, ParsedField> varMap = cls.varMap;

class B {
  A a;
  foo() {
     const parsedB = parse();   // defaults to parse the enclosing class = parse(B)
     const ParsedVar parsedVarA = parse(a)
     const parsedA = parse(parsedVarA.type);   // = parse(A)
     return parsedB.methods.foo.name + parsedA.vars.bar.name;  // translates to: return 'foo' + 'bar'
  }
}

In general the Parsed type should expose all kind of data from the written code related to the variable-class-method, like names, parent class, fields, method arguments, types and so forth (even the method body). parse always creates compile time constant values and the declaration of any Parsed type vars should be enforced with the usage of an explicit const to make the constant nature of parse completely clear to the user (it can only be data inferred from the code statically).

Invoking methods sdk#45359

Invoking methods in a “reflective” way could be statically enabled by using invoke method directly on a ParsedClass and passing the object as the first parameter (for static methods the first parameter should be null). Example:

class B extends A {}

const ParsedClass cls = parse(A);

A a = A()
cls.invoke(a, 'foo', []);

Method invoke in ParsedClass could be translated by the compiler to the following code pattern (example):

/// invoke on parse(A).invoke translates to something like this
invoke(A object, String name, [Iterable args, Map namedArgs]) {
    switch (name) {
      case 'foo':
        parse(a.foo).checkValidArgs(args, namedArgs);   // throw exception if args are not valid
        return a.foo();
        break;
      case 'toString':
        parse(a.toString).checkValidArgs(args, namedArgs);
        return  a.toString();
        break;
      case 'noSuchMethod':
        parse(a.noSuchMethod).checkValidArgs(args, namedArgs);
        return a.noSuchMethod(args[0]);
        break;
      default:
        throw MethodNotInInterfaceException();
    }
  }

ParsedMethod.checkValidArgs resolves to code in a similar way as ParsedClass.invoke does. When invoking a method through "static reflection", noSuchMethod is never invoked by default, instead if the inputs are invalid, MethodNotInIterfaceException is thrown. Throwing an exception is necessary in case the underlying object did actually implement the method, in which case it would be inconsistent to invoke noSuchMethod.

First step

As first step I would suggest starting with parse(var-class-method) that only provides the name of the var as a symbol. This means that initially it would only be useful to do parse(myVar).name. The Parse type leaves the door open to keep extending the feature as proposed above.

Additionally the ParsedClass.invoke method could be considered to be implemented as mirrors support have been highly requested for Flutter.

The text was updated successfully, but these errors were encountered:

lrhn · 2019-11-25T07:04:09Z

It's not only the second point which affects tree-shaking.
Retaining even the name of functions, classes and parameters, as well as the underlying structure, is an overhead when compilation to JavaScript.

So, this is only free if the only valid use is compile-time constant, all names known at compile-time. That makes the feature much less powerful in practice. For example C.name is only ever equal to "C", and you already had to write the name. Similarly C.proto.foo will always be equal to "foo", which you just wrote.

That's not necessarily bad, it means that it's not reflection, it's just a cumbersome way to write a name as a string, but one which allows the system to detect and warn when the name changes from what you expected. In that sense, this request is very close to #251, except that it asks for strings instead of symbols.

The method invocation is not just about invoking methods, it's about invoking a specific superclass method even if a subclass may override it virtually. That breaks the virtual method abstraction.
And you do have to prevent the method from being called on an object which implements the interface, but which doesn't extend the class declaring the method. That's a breach of subclass substitutability.
So, the method invocation is squarely in reflection+reification land, and it breaks two of the most fundamental abstractions in object oriented programming. If the name or argument list can change between invocations, we need to dynamically generate invocations, which also breaks tree-shaking.
Most of the arguments for not allowing mirrors on some platforms will also apply to this invocation operation.

icatalud · 2019-11-26T03:53:43Z

There was a flaw in the proposal, I had forgotten that doing A.something is how the static members are accessed. I updated the proposal to access the Prototype of a class using protoOf(Class). protoOf could actually be generalized to be used on any type, not only classes.

I called it “static reflection” because it’s similar to reflection in that it returns the same values and allows the same operations, but yes as you stated it all the values can be obtained by writing them directly. Even then it’s quite useful, as many reflection uses cases are satisfied with that kind of “static” reflection. It can save code repetition, for example some time ago I had to “manually” create a list with the members of a flutter class that had over 100 members. That’s just one example. Yes #251 is more direct and if that is the main use case then it’s probably better to have that kind of approach. The advantage of the Prototype approach is that it is more general, it can be extended to provide any kind of parsed information about a class or method. Another very common example for this kind of “typed data” is in Models to convert to and from a JSON string, or generally converting a map into a static typed class. Usually code generation is used for this purpose.

class A extends Model {
  fromJson(Map<String> json) {
    final proto = protoOf();
    for(k in json.keys) {
      var field = proto.varsMap[k];
      if(field is Model) {
        proto.invokeGetter(this, k).fromJson(json[k]);   // assume the field starts initialized
      } else if(field is int) {
        proto.invokeSetter(this, k, int.parse(json[k]));
      } else {
        proto.invokeSetter(this, k, json[k]);
      }
    }
  }
}

About the treeshaking part I was not explicit enough, I was thinking that if the prototype invoke is typed then it must be assumed by the compiler that all the class methods are invoked and all members are retrieved.

In any case I think I prefer a general submirror invoke which fails if the internal class of the object has not been added to the submirrors list, which does not require typing the class, like the proposal on [sdk#45359]((flutter/flutter#45359).

About objects implementing the interface or child classes, I noted in the example and the explanation that the class types must match exactly, that's why the operation is not "safe" in a similar way how it is not safe to invoke a method on a dynamic type. It would need to be accepted that once a method is being invoked by its name (using invoke) and not by statically typing it, then it's not the encapsulated object oriented programming safe world anymore (it should be clearly be documented as advanced usage).

icatalud · 2019-11-26T19:30:47Z

There is actually a way to achieve this kind of “static reflection” in a complete way, without errors if the types do not match exactly. invoke could simply be a shortcut for a static typed switch statement pattern. What is currently being done when this kind of “reflection” is desired it's required to manually write the following:

  A a;
  invokeAny(name, args, namedArgs) {
    switch (name) {
      case 'foo':
        checkValidArgs(protoOf(a.foo), args, namedArgs);   // throw exception if args are not valid
        a.foo(args[0], args[1], protoOf(a.foo).namedArgs[0]: namedArgs[protoOf(a.foo).namedArgs[0]]);
        break;
      case 'toString':
        checkValidArgs(protoOf(a.toString), args, namedArgs);
        a.toString();
        break;
      case 'noSuchMethod':
        checkValidArgs(protoOf(a.noSuchMethod), args, namedArgs);
        a.noSuchMethod(args[0]);
        break;
      default:
        throw MethodNotInInterfaceError();
    }
  }

The shortcut would be:

protoOf(A).invoke(a, methodName, args, namedArgs);
// or alternatively
invoke methodName on a with args, namedArgs;

Would this generally be to far-fetched? Because it's a shortcut for quite a complex long code pattern, but I believe it has many use cases. It makes it trivial for a user to enable reflection on any class in a static type safe way.

icatalud · 2019-11-27T23:58:48Z

@lrhn I updated the proposal, it's now complete and consistent. Take a look.

icatalud · 2020-01-03T21:45:56Z

This feature by itself could enable generic Json support. An extension could be defined:

extension Json {
  factory fromJson<T>(String json) { ... }
  String toJson<T>() { ... }
}

If a type can be inspected and methods can be inspect-invoked, that is feasible. It requires unnecessary constant checks, but it a good initial solution. Optimizations explained in a proposal in #631 could be performed, specially if generic implementations are supported in which case the extension could be defined as extension Json<T> on T. Templating methods from #631 is a more versatile general feature, but this could be a first step.

icatalud added the feature Proposed language feature that solves one or more problems label Nov 21, 2019

icatalud changed the title ~~General "written" String data about classes and methods~~ Static "mirror" capability and general String parsed data from classes or methods Nov 26, 2019

icatalud changed the title ~~Static "mirror" capability and general String parsed data from classes or methods~~ Static "reflection" capability and general String parsed data from classes or methods Nov 26, 2019

icatalud changed the title ~~Static "reflection" capability and general String parsed data from classes or methods~~ Static "reflection" capability: general parsed data from classes, methods or vars Nov 27, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Static "reflection" capability: general parsed data from classes, methods or vars #701

Static "reflection" capability: general parsed data from classes, methods or vars #701

Static "reflection" capability: general parsed data from classes, methods or vars #701

Static "reflection" capability: general parsed data from classes, methods or vars #701

Comments

Proposal: parse keyword

Invoking methods sdk#45359

First step

Proposal: `parse` keyword