Tuesday, May 17, 2016

Unboxing Packages: vm_service_client

Three weeks ago, I wrote about the stream_channel package. Two weeks ago, I wrote about the json_rpc_2 package which is built on top of stream_channel. This week I’ll complete the trifecta by writing about the vm_service_client package, which uses json_rpc_2 in turn—and is a really cool package in its own right!

One of the lesser-known corners of the Dart VM is its service protocol, but it’s one of its most powerful components. It uses JSON-RPC 2.0 over WebSockets to allow clients to connect to the VM, inspect its internal state, set breakpoints, and all sorts of neat stuff. If you’ve ever used Observatory for debugging or profiling your Dart code, you’ve been using the service protocol under the hood: that’s how the Observatory web app talks to the VM’s internals.

Because the protocol is fully documented and based on a standard underlying protocol, it’s possible for anyone to use from their code. And the vm_service_client package makes it downright easy: it provides a Dart-y object-oriented API for (more or less) everything in the protocol. And that turns out to be a lot of stuff: I count 108 classes in the API documentation, with more added over time as the protocol adds new features.

Because the client’s surface area is so broad, I’m not even going to attempt to cover all of it. I’ll discuss the most important classes, of course, but I also want to touch on the broader picture of how we took a language-independent RPC protocol and created a native-feeling API to use it.

Connecting the Client

Before we go deep into the API, though, let’s start at the beginning: actually establishing a connection with a running VM instance. The first step is to get the VM to actually run the service protocol at all. If you run dart --enable-vm-service, it will listen for WebSocket connections on ws://localhost:8181/ws (by default).1 You can also force a running Dart process to start the VM service by sending it a SIGQUIT signal, as long as you’re not on Windows, but that’s a lot less reliable and customizable.

Once the service is running, you can connect a client to it using new VMServiceClient.connect(). This takes the service protocol’s WebSocket URL as either a string or a Uri—and, for convenience, it can also take Observatory’s HTTP URL, which it will use to figure out the corresponding WebSocket URL. If anything goes wrong, it’ll be reported through the done future.

import "package:vm_service_client/vm_service_client.dart";

main(List<String> args) async {
  var url = args.isEmpty ? "ws://localhost:8181/ws" : args.first;
  var client = new VMServiceClient.connect(url);

  // ...


Almost every piece of data the service protocol provides comes in two varieties: the reference and the full version. The reference contains a little bit of metadata, as well as enough information to make an RPC that will provide the full version, which contains all available information.

This split accomplishes two things. It makes the responses much more compact by avoiding unnecessary metadata. More importantly, though, it allows for circularity. A library can refer to the classes it contains, which can in turn refer back to the library that contains them.

In the client, all reference classes end in Ref, whereas their full versions do not. So VMLibraryRef is a reference to a VMLibrary. The full versions extend their corresponding references, so you can pass a VMLibrary to a method that expects a VMLibraryRef. Every reference has a load() method that returns its full version.

This is an important place where the client makes the API feel more native. Because the service protocol can’t really overload an RPC based on its argument type, it has to have different calls for resolving references to different types of objects. But as a Dart client, we can provide an extra degree of uniformity. This is a pattern that comes up several times throughout the client.

Runnable Isolates

Isolates are a special case: they have three states instead of just two. There’s VMIsolateRef and VMIsolate, but neither of these will provide all the isolate’s information. For that you need VMRunnableIsolate.

The extra layer exists because the VM loads an isolate in stages. First it creates the isolate with simple metadata like its name and its start time. But it needs to do a bunch of additional work, loading libraries and classes and stuff like that, before it can actually run any code in the isolate. Only once all that work is done is a VMRunnableIsolate available.

You can use the VMServiceClient.onIsolateRunnable stream to get a notification when the isolate you care about is runnable, but if you already have a reference to an unrunnable version there’s an easier way. VMIsolateRef.loadRunnable() returns the runnable version once it’s available, and does so in a way that’s guaranteed to be safe from race conditions.

Once you have a connection to the service, you need to be able to find the part you want to interact with. I don’t mean looking up its API docs, I mean actually getting to it from your VMServiceClient object. For the most part, the VM service is organized hierarchically:

Note that many of these getters are maps. This comes entirely from the client: the service protocol always sends collections down as lists for compactness. But it’s useful for users to be able to look up libraries by their URIs or classes by their names, so the client converts the lists into the data structure that best fits with how we expect users to interact with the data.

Let’s take a look at how you’d add a breakpoint at the beginning of a call to File.open() from dart:io:

var vm = await client.getVM();
var isolate = await vm.isolates.first.loadRunnable();
var library = await isolate.libraries[Uri.parse("dart:io")].load();
var file = await library.classes["File"].load();


Code deals with data, so the VM service needs a way to represent data, and that way is the Instance class. Like most types, Instances have references and full values—these are represented in the client as VMInstanceRef and VMInstance.

On its own, VMInstance only provides two pieces of information. Its klass getter returns the VMClassRef representing the instance’s class, and its fields getter returns its fields and their values. In practice, though, many instances include more information—the service provides extra information for many core library types, which the client represents as classes like VMIntInstance and VMListInstance.

The client also provides VMInstanceRef.getValue(), a convenience method for converting instances to local Dart objects. Every VMInstanceRef subclass overrides this to recreate their particular type of object. It also takes an optional onUnknownValue() callback to which plain instances—including those in data structures—are passed to be converted into local values based on the caller’s logic.

Evaluating Code

The VM service doesn’t just let you look at the VM’s state, it lets you run code on it as well! A few different classes have evaluate() methods that take strings to evaluate and return VMInstanceRefs. Where you evaluate the code determines what names are accessible.

  • VMLibraryRef.evaluate() runs code in the context of a library. This lets the code access anything the library has imported, as well as any of its private names.
  • VMClassRef.evaluate() runs code in the context of a class. This is mostly the same as the library context, except that the code can refer to static class members without needing to prefix them with the class’s name.
  • VMInstanceRef.evaluate() runs code in the context of an instance. This means the code can refer to the instances fields and to this.
  • VMFrameRef.evaluate() can be used when an isolate is paused to run code in the same context as one of the current stack frames.

This is another example of the client using the same API to draw similarities between different parts of the underlying protocol. Because the client has the well-known unifying metaphors of objects and methods, it’s able to take disparate APIs and expose them in a consistent way that isn’t possible using raw RPCs.

Go Forth and Make Something Cool

In some ways, the VM service client is the most exciting package I’ve written about yet. It’s designed to make available a whole bunch of internal VM functionality, and the only limit on what can be done with that functionality is your imagination. So take this package and make something cool. Make a REPL or a visual object inspector. Heck, make an entirely new debugger!

Join me again in two weeks when I write about one of the oldest and most fundamental packages in the entire Dart ecosystem.

  1. Humans interacting directly with Observatory usually pass --observe flag instead of --enable-vm-service. This will also enable the service, but it also turns on a handful of other options, the exact set of which is subject to change. It’s much safer to use --enable-vm-service when writing code to interact with the VM.

Wednesday, May 4, 2016

Unboxing Packages: json_rpc_2

Last week I wrote about the stream_channel package for two-way communication, so this week it seemed natural to move to a package that uses it: json_rpc_2. This is an implementation of the JSON-RPC 2.0 specification, which is a popular protocol for providing structure and standardization to WebSocket APIs.

Although it’s most commonly used with WebSockets, the protocol itself is explicitly independent of the underlying transport mechanism. This makes it a great fit for stream channels, which can be used to represent a two-way stream of JSON objects in a way that works with any underlying mechanism. Thanks to stream channels, JSON-RPC 2.0 can be used across WebSockets, isolates, or any channel a user chooses to wrap.

Shared APIs

There are three main classes in json_rpc_2: Client makes requests and receives responses, Server handles requests and returns responses, and Peer does both at once. Because all of these involve two-way communication, they all have the same two constructors. The default constructor takes a StreamChannel<String> where each string is an encoded JSON object, and automatically decodes incoming objects and encodes outgoing ones. On the other hand, if you want to communicate using decoded maps and lists, you can use the withoutJson() constructor, which only requires that the objects be JSON-compatible.

The three classes also have the same lifecycle management. In order to give the user time to set up request handlers or enqueue request batches, they don’t start listening to the stream channel until listen() is called. Once it is, it returns a future that completes once the channel has closed—also accessible as the done getter. And if the user wants to close the channel themselves, they can call close().


The Client class is in charge of making requests of a server. The core method for this is sendRequest(), which takes a method (the name of the remote procedure to call) and parameters to pass to that method.

The structure of these parameters depends what the server accepts. JSON-RPC 2.0 allows both positional parameters, which are passed as an Iterable of JSON-safe objects, and named ones, which are passed as a Map from string names to JSON-safe values. The parameters can also be omitted entirely if the method doesn’t take any.

The call to sendRequest() returns a future that completes with the server’s response. The protocol defines two types of response: “success” and “error”. On a success, the server returns a JSON-safe object which the sendRequest() future emits. On a failure, the server returns an error object with associated metadata. This metadata is wrapped up as an RpcException and thrown by the future.

import 'package:json_rpc_2/json_rpc_2.dart' as rpc;

/// Uses the VM service protocol to get the Dart version of a Dart process.
/// The [observatoryUrl] should be a `ws://` URL for the process's VM service.
Future<String> getVersion(Uri observatoryUrl) async {
  var channel = new WebSocketChannel.connect(observatoryUrl);
  var client = new rpc.Client(channel);

  // getVM() returns an object with a bunch of metadata about the VM itself.
  var vm = await client.sendRequest("getVM");
  return vm["version"];

If you don’t care whether the request succeeds, you can also call sendNotification(). JSON-RPC 2.0 defines a notification as a request that doesn’t require a response, and a compliant server shouldn’t send one at all. Notifications are commonly used by peers for emitting events, but I’ll get to that later.

JSON-RPC 2.0 also has a notion of batches, where a bunch of requests are sent as part of the same underlying message. The server is allowed to process batched requests in whatever order it wants, but it’s required to send the responses back as a single message as well. This can use less bandwidth if you have a bunch of requests that don’t have strong ordering needs.

The json_rpc_2 client lets the user create batches using the withBatch() method. This takes a callback (which may be asynchronous), and puts all requests that are sent while that callback is running into a single batch. This batch is sent once the callback is complete.


The Server class handles requests from one or more clients. Its core API is registerMethod(), which controls how those requests are handled. It just takes a method name and a callback to run when that method is called. The value returned by that callback becomes the result returned to the client.

import "package:json_rpc_2/json_rpc_2.dart" as rpc;
import "package:shelf/shelf_io.dart" as io;
import "package:shelf_web_socket/shelf_web_socket.dart";

var _i = 0;

main() async {
  io.serve(webSocketHandler((webSocketChannel) {
    var server = new rpc.Server(webSocketChannel);

    // Increments [_i] and returns its new value.
    server.handleMethod("increment", () => ++_i);
  }), 'localhost', 1234);

The server presents an interesting API design challenge. Most methods require certain sorts of parameters—one might need exactly three positional parameters, one might need two mandatory named and one optional parameter, and another might not allow any parameters at all. JSON-RPC 2.0 is pretty clear about how to handle this at the protocol level, but how do we let the user specify it?

We could have users manually validate the parameters—and in fact, for complex validations we do. Users can always manually throw new RpcException.invalidParams() based on whatever logic they code. But it’s a huge pain to manually validate the presence and type of every parameter, so Server uses a couple clever tricks to figure out requirements with minimal user code.

The first trick is that the callback passed to registerMethod() can take either zero or one parameters. This is how Server figures out whether the method allows parameters at all. In the example above, if a client tried to call increment with parameters of any kind, they would get an “invalid parameters” error. But the most clever trick is how parameters that are passed are parsed, and it involves an entirely new class.


The Parameters class wraps a JSON-safe object and provides methods to access it in a type-safe way that will automatically throw RpcExceptions if the object isn’t the expected format. It’s what gets passed to the registerMethod() callback, if it takes a parameter at all.

If you call asList and the caller passed the parameters by name, it’ll throw an RpcException. If you call asMap and the parameters were passed by position? RpcException as well. Or you can just call value and get the underlying parameter no matter what form it takes.

Parameters also lets you verify the parameter values themselves. The [] operator can be used for either positional parameters (with int arguments) or named parameters (with string arguments), and returns a Parameter object which extends Parameters with a bunch of methods for validating types beyond just lists and maps.

All of the native JSON types have getters like asString, asNum, and similar. Just like asList and asMap, these getters return the parameter values if they’re the correct types and throw RpcExceptions if they aren’t. There are also derived getters like asDateTime and asUri which ensure that the value can be parsed as the appropriate type, and asInt which ensures that a number is an integer.

// Sets [_i] to the given value.
server.handleMethod("set", (parameters) {
  _i = parameters[0].asInt;
  return _i;

It’s important to note that the [] operator will return a parameter even if it doesn’t exist, either because there weren’t enough positional parameters passed or because a parameter with that name wasn’t passed. This makes it easy to support optional parameters.

A parameter that doesn’t exist will always throw an RpcException for its asType methods, and even for value. But there are methods where it won’t throw. If you call asStringOr() for a parameter that exists, it behaves just like asString, but for a non-existent parameter it’ll return the defaultValue parameter. Every asType getter has a corresponding asTypeOr() method. Even value has valueOr().

// Returns the logarithm of [_i].
// If the `"base"` named parameter is passed, uses that as the base. Otherwise,
// uses `e`.
server.handleMethod("log", (parameters) {
  return math.log(_i)/math.log(parameters["base"].asNumOr(math.E));


The Peer class works as both a Server and a Client over the same underlying connection. In terms of API, it’s exactly the sum of those two classes. It adds no methods of its own, so in that sense you already know everything about it. But it’s still instructive to talk about why it exists.

While I can easily imagine a structure where two endpoints are truly peers, each invoking methods on the other and receiving results, in practice most of the time I’ve seen peer-structured protocols has been for the sake of event dispatch. You see, JSON-RPC 2.0 doesn’t include an explicit mechanism for the server pushing events to the client. It can only respond to requests made by the client. This is intentional, since it makes the protocol much simpler, and the peer structure is the standard way around it.

To support server events, both the client and server must act as peers, able to send and receive requests. In this world, events are modeled as requests sent from the server to the client—or more specifically, notifications, since the server doesn’t expect a response. The client registers a method for each type of event it wants to handle, and the server sends a request for every dispatch.

/// Uses the VM service protocol to print the VM name.
/// Prints the VM name again every time it's changed.
void printVersions(Uri observatoryUrl) async {
  var channel = new WebSocketChannel.connect(observatoryUrl);
  var peer = new rpc.Peer(channel);

  peer.registerMethod("streamNotify", (parameters) async {
    if (parameters["streamId"].asString != "VMUpdate") {
      throw new rpc.RpcException.invalidParams(
          "Only expected VMUpdate events.");

    print("VM name is ${await peer.sendRequest("getVM")["version"]}.");

  print("VM name is ${await client.sendRequest("getVM")["version"]}.");

RPC Home

Next time you need to communicate with a JSON-RPC 2.0 server, you know where to turn. Next time you need to create an RPC server, I hope you look to JSON-RPC 2.0 as the underlying protocol. It’s clean and straightforward, and best of all, it’s got a great implementation already written and ready to use.

I wrote about stream_channel in my last article. In this article, I wrote about json_rpc_2, which uses stream_channel. Join me in two weeks when I build this layer cake a little higher and write about a package that uses json_rpc_2!