Wednesday, April 17, 2013

Goodbye InvocationMirror, Hello Invocation and Symbol


TL;DR: We added a class Symbol to dart:core, renamed the class InvocationMirror to Invocation (keeping it in dart:core), and moved Invoke.invokeOn to InstanceMirror.delegate (moving this functionality from dart:core to dart:mirrors). Furthermore, Function.apply and InstanceMirror.invoke take a Map<Symbol,dynamic> to represent named arguments. Finally, dart:mirror uses Symbol instead of String to represent names.






Previously, InvocationMirror looked like this:

abstract class InvocationMirror {
 String get memberName;
 List get positionalArguments;
 Map<String, dynamic> get namedArguments;
 bool get isMethod;
 bool get isGetter;
 bool get isSetter;
 bool get isAccessor => isGetter || isSetter;
 invokeOn(Object receiver);
}

This creates problems when minifying, both when compiling to JavaScript and when minifying Dart code.  For example, a method call like foo(), may get renamed to a() when minifying.  If this call ends up being handled by noSuchMethod, the invocation mirror passed to that method will have memberName == ‘a’, not memberName == ‘foo’.

Another problem with InvocationMirror is that using invokeOn turns off type inference in dart2js.  Type inference is an important tool for dart2js to generate efficient and compact JavaScript code.  Function.apply has the same problem.  This means that InvocationMirror.invokeOn is not only problematic when using minification.  It is a problem in general when compiling to JavaScript.

Returning to the problem of preserving names when minifying, a possible solution is to include a mapping between minified names and original names.  The downside to this is that this map is relatively large.

We are proposing another solution: do not use strings for names, instead use a new class Symbol:

/// Opaque name used by mirrors and invocations.
/// Use [MirrorSystem.getName] to obtain the underlying name as a String.
class Symbol {
 const Symbol(String name);
}

abstract class InvocationMirror {
 Symbol get memberName;
 List get positionalArguments;
 Map<Symbol, dynamic> get namedArguments;
 bool get isMethod;
 bool get isGetter;
 bool get isSetter;
 bool get isAccessor => isGetter || isSetter;
 invokeOn(Object receiver);
}

In addition, we propose to use Symbol instead of String to represent names in the library “dart:mirror” and to represent named arguments passed to Function.apply and InstanceMirror.invoke.

The advantage is that an expression like const Symbol(‘foo’) can also be minified so one can write the following without having a mapping between all mangled and unmangled names:

class Foo {
 noSuchMethod(InvocationMirror invocation) {
   if (invocation.memberName == const Symbol(‘foo’)) {
     print(‘Respond to foo()’);
     return invocation.invokeOn(someOtherObject);
   } else {
     return super.noSuchMethod(invocation);
   }
 }
}

It is possible to write:

...
   if (invocation.memberName == new Symbol(name)) {
...

We propose to warn in situations like this to alert the user to the fact that we have to include a mapping because a non-const symbol is created. In fact, we propose to emit similar warnings when using Function.apply and InvocationMirror.invokeOn.  We will also provide a tool that can display the cost of using these features.

In addition, we should consider if InvocationMirror has the right name.  A class prefixed with mirror belongs in the library dart:mirrors, not in the library dart:core. So we propose to change the name to Invocation.  In addition, the method invokeOn is really a reflective operation that belongs in dart:mirrors.  So we propose to move Invocation.invokeOn (nee InvocationMirror.invokeOn) to InstanceMirror.delegate.

In in this situation, the class Foo above would look like:

import ‘dart:mirrors’;

class Foo {
 noSuchMethod(Invocation invocation) {
   if (invocation.name == const Symbol(‘foo’)) {
     print(‘Respond to foo()’);
     return reflect(someOtherObject).delegate(invocation);
   } else {
     return super.noSuchMethod(invocation);
   }
 }
}

One question that arise is if InstanceMirror.delegate is “worse” than InvocationMirror.invokeOn in terms of impact on size and performance of generated JavaScript code.  Our estimate is that both options have equal impact if we use symbols, but this is a question that must be examined as we implement mirrors in dart2js.  However, the motivation for moving the method is not performance.

Interestingly, there are cases when InstanceMirror.invoke can be almost as fast as a regular call.  However, these cases are probably not interesting and a regular call would be a better option.  Mirrors are only interesting when the involved expressions are just so dynamic that they are not possible to analyze ahead of time.

InstanceMirror.delegate is assumed to be easier to analyze than having to extract the arguments from an Invocation and using InstanceMirror.invoke.  We also assume that there is less runtime overhead to forward a call using delegate versus invoke.

Restrictions on Symbol


The name argument to the Symbol constructor must not start with ‘_’ (underscore) and match one of these grammar rules:

  • identifier (“.” identifier)* ‘=’?
  • (identifier “.”)* operator (where operator is a user-defined operator)
  • ‘’ (an empty string)

Symbols like const Symbol(“foo.bar.baz”) is used to represent library names.  Furthermore, DeclarationMirror in dart:mirrors has a getter named qualifiedName and the operator[] in class “MyClass” in library “my_lib” would have the qualified name const Symbol(“my_lib.MyClass.[]”). The Dart Programming Language Specification states that:

The name of a setter is  obtained by appending the  string `='  to the identifier given in its signature.

So a setter “x” in “MyClass” would have the qualified name const Symbol(“my_lib.MyClass.x=”) and simple name const Symbol(“x=”).

Language Support for Symbol Literals


It would be possible to add another literal type to the language to represent instances of Symbol.  However, one should only add language features for commonly used constructs, and it is not currently clear that having another kind of literal is warranted since we are not aware of compelling use-cases outside the mirror API.  So we are not proposing to add a new kind of literal at this time.

Map Literals


We further propose to change the type of an untyped map literal from Map<String, dynamic> to Map<dynamic, dynamic>.  Also, the map literal syntax should be changed to allow at least Symbol keys.

FAQ


Q: Why is it problematic to have a getter to access the unmangled name of a Symbol?
A: Getting the unmangled name from a Symbol means that you have to include it in the generated JavaScript or Dart code. If the compiler can statically determine that you're trying to get the mangled name, it can warn and tell you the cost. If it is an accessor with a general name, for example, "name" then every time the compiler sees an untyped expression "e.name", it has to assume that you might be getting the unmangled name of a Symbol and warn. The problem is that there are many false positives on the getter, but no false positives on a static method. No matter what accessor name you choose, developers will tend to use the same name in their own classes. So you'll always have false positives. There are downsides to both solutions.  Using a static method is the conservative approach, and it would be possible to add a getter later.

Q: Why doesn’t Symbol.toString() return the unmangled name?
A: That would mean that you always have to include the unmangled name in the minified output (Dart or JavaScript), which might not be desirable.  See also the previous question.

Q: Is new Symbol(“foo”) really problematic for the compiler to deal with?
A: No, but it easier to understand a simple rule like “symbols should be const”, rather than, “after constant folding, inlining, etc., the compiler will warn if it cannot determine that the argument to Symbol is a constant string literal.”

Q: Why is Function.apply similar to using strings for member names? It doesn't seem to imply the use of a symbol except for named arguments.
A: When you use Function.apply, the compiler often cannot tell which function is being called and with how many arguments, or what their types are. This means that it has to assume that all closures (including tear-offs) may get invoked with arbitrary arguments. This makes it hard to optimize forEach, etc. A similar situation arises when the compiler cannot tell which methods are invoked through reflection, but in this case it is much worse than Function.apply, since all instance methods as well as closures become targets. This effectively turns off type inference.

Q: Why is Symbol not a subclass of String?
A: From a modelling perspective, there are two strings associated with a symbol.  A mangled (or minified) name and the original name written in source code (aka unmangled).  So rather than having an is-a relationship with String, it seems that the relationship is has-some.
Furthermore, if Symbol is a subclass of String, the Editor will not complain when you pass a string to, for example, InstanceMirror.invoke.

Q: Why can’t a symbol start with “_”?
A: In Dart, names starting with “_” are private to the library in which they’re used.  So you’ll have to use a LibraryMirror to create a private name.  The exact API is to be determined.

(Photo courtesy of NASA Webb Telescope under cc license.)