file:URIs, Dart would support a new
package:scheme by looking for packages in a
packages/directory next to the entrypoint, wherever that happened to be.
This worked pretty well—well enough to form the foundation for the pub package manager, written by Bob and myself. But it also had flaws. A lot of symlinks had to be generated all over every package—the
testpackage currently contains about four hundred. Not only was this messy, many tools ended up following the symlinks and corrupting the package cache. So the team came together again and came up with the idea of a package configuration file that lists available packages as well as the concrete URIs where they can be found.
Support for package config was quickly added to all Dart implementations. In fact, pub has been generating them since Dart 1.12. The only task remaining was to gracefully turn off pub’s support for generating the
packages/directory and all the symlinks that come with it… but that turned out to be easier said than done.
The thing is, for all its flaws, the
packages/directory was pretty easy to work with from Dart code. You just had to do some path manipulation with
Platform.scriptto figure out where the
packages/directory was, and from there you could do whatever you wanted. It wasn’t clean, but it worked most of the time, and there was no way to do the same stuff with
Once again conversation ensued. APIs were proposed, rejected, redesigned, tweaked, implemented, rolled back, iterated on, landed, stabilized1, and verified. It took seven releases to get everything just right, but we did it, and we’re ready for you to try it out. And I’m here to tell you how.
Disabling The Packages Directory
The first step on your journey to package enlightenment is to abandon your attachment to worldly directories. Relinquish your need for the comfort of familiarity by passing
downgrade). Pub won’t generate a
packages/directory, nor of course will it symlink to the directory that it doesn’t generate. If you already had a
packages/directory, pub will get rid of it as well as any symlinks pointing its way.
For most packages and applications, this is all you’ll need to do! The latest versions of core Dart tools like pub and
testfully support running with a package config. As long as your code isn’t manually dealing with packages, you’re good to go. And if it is, keep reading!
Core Package APIs
The most broadly-applicable library package API is
Isolate.resolvePackageUri(). It converts a
package:URI into a
file:3 URL that can be easily converted to a path. This is the easiest way to find resources in your package, or even public resources in your dependencies.
It’s important to note that this API is asynchronous, though: it returns a
Future<Uri>rather than just a
Uri. This is because the VM loads package information lazily, and the load process is asynchronous. If you have any
package:imports in your code, the information will already be loaded, but it’s not safe to assume that—even code that uses lots of different packages may be run from a snapshot every now and then.
The other important core APIs tell you where the package resolution comes from.
Isolate.packageConfigreturns the URL of the package config file used for the current isolate, and
nullif a package root is being used. Its twin,
Isolate.packageRoot, returns the URL of the current isolate’s package root, and
nullif a package config is being used. These getters are useful for forwarding the current resolution scheme—for example, if you want to start a Dart process that uses the same resolution as the current isolate, you can use these to figure that out.
I should probably also mention the
Platform.packageRootgetters as well, but I do not recommend using them. They return exactly the text passed on the command-line for the
--package-rootflag, respectively. This means they’re unreliable: if the package resolution information was auto-detected, they’ll return
null. If the current Isolate uses a different resolution scheme than the main isolate, they could return something completely irrelevant. Avoid them.
If you want to go beyond the basics in your package code,
package_resolveris the package for you. It provides classes that encapsulate strategies for package resolution—using a package config, using a root, or using nothing at all. Where code used to just pass around a
packageRootstring, it can now pass a
PackageResolverthat says how to resolve URIs.
You can construct a
PackageResolverby passing a package config map to
new PackageResolver.config(), or by passing a package root to
new PackageResolver.root(). If all you have is the URI for a package config, you can also use
PackageResolver.loadConfig(). But most of the time, all you need is information about the current isolate’s package resolution, which you can get from the
The resolver supports more or less the same APIs that the Isolate does:
resolveUri()is equivalent to
Isolate.packageRoot. But it builds other useful methods on top of these as well.
packagePath()method returns the root of a package on the local filesystem. It assumes that you’re using pub, since it needs to look above packages’
lib/directories, but this works in practice for essentially all Dart code.
packageUriFor()method is the reverse of
Isolate.resolvePackageUri(): it converts a
file:URL into a
package:URI that refers to the same file, if such a URI exists. This is useful for reporting nice-looking messages; it’s much easier to read a short
package:URI than a long absolute path.
processArgumentgetter is useful when running Dart processes or compiling Dart to JS. It returns an argument that can be passed directly to the
dartdevcexecutables that will cause them to use the resolver’s resolution strategy. If you passed using a custom package config map, it’ll even manufacture a
data:URI for you4.
It even exposes the package config map directly through the
packageConfigMapgetter. Note that since the isolate API doesn’t yet provide this information directly, this may require re-loading it from disk.
But I Want It Synchronous!
PackageResolverAPIs are asynchronous because they might be implemented using the asynchronous Isolate APIs. But asynchrony can be annoying to work with, and sometimes it’s not necessary. That’s what the
SyncPackageResolverclass is for. It exposes pretty much the same API as
PackageResolver, but with no futures to be found.
The catch is that if you want to get a
SyncPackageResolver, all the information—the package config map or the package root URI—needs to be available up-front. So
Future<SyncPackageResolver>, since they have to wait for Isolate API futures and/or load information from disk.
You can convert back and forth between synchronous and asynchronous package resolvers easily using the
asAsyncAPIs. This makes it possible for packages to take whichever one they work best with without causing pain for their users. For example,
SyncPackageResolverso it can work synchronously, but
PackageResolversince serving handling requests is already inherently async.
Why Bother With The Package Root?The
package_resolverpackage goes to pains to work well with both package configs and package roots, and to make it easy for users to support both. This raises a question: if pub is moving away from generating a
packages/directory, what’s the point of supporting it?
First of all, there’ll be a transition period where both modes will be fairly common. Pub still generates the
packages/directory by default for now, and the language implementations prefer it if it exists. A smooth transition relies on code’s ability to be used in both old and new settings.
But even in the future when
packages/files aren’t generated by default, support for package roots will be sticking around, and it can still be useful. It may not be the best way to run code manually, but a lot of code isn’t run manually—Pub’s transformer support, for example, loads isolates over HTTP, where it serves virtual
In addition to supporting current tools, we want leave the door open for future tools to innovate as-yet-undreamed-of ways to handle Dart code. So we choose flexibility and continue to support package roots.
On Into The FutureOnce there’s been a release or two with the
--no-package-dirflag and users have a chance to make sure their packages work with only a package config, we’ll disable the flag by default. You’ll need to pass
--packages-direxplicitly if you want it generated. A while after that, we’ll hide the flag and stop officially supporting it. We may get rid of it entirely one day, but there’s no hurry—it’s not that hard to support.
So go forth and update your packages! Use the core library APIs and the
package_resolverpackage, or one of the helpful packages I didn’t have a chance to get to:
resourceprovides an abstraction for loading resources, and
shelf_packages_handlermakes it easy for a Shelf server to support a package root. Or write your own! The underlying tools are powerful, and I’m excited to see what you can do with them.
- Well, mostly stabilized. There are still a few corners of the API that are still in the process of being thoroughly tested. Bear with us if you run into them, and maybe wait another release cycle before you depend too heavily on running without the
- This flag is new in 1.19, but some readers may notice its similarity to the undocumented
--no-package-symlinksthat pub used to support.
--no-package-symlinkswas added ages ago to let the Dart analyzer team experiment with their package config support, and we decided to tweak it a bit when making it public. ↩
- Since the Dart VM supports loading packages over HTTP, it’s theoretically possible for it to return an
http:URL. This doesn’t happen that often in practice, but it is a possibility—for example, the
testrunner can load test suites over HTTP from a
pub serveinstance. ↩
- Note that there are usually limits on how long command-line arguments can be, so this may not work if the package config is too big. ↩