Friday, October 19, 2012

More Dart libraries ready for use, thanks to Pub



One of the big features of the Dart M1 release is our package manager, pub. You can discover, install, and manage 3rd party Dart libraries with pub. Pub is a command-line app with Dart Editor integration, as well as a hosting service for Dart packages. We've been uploading select packages, but soon you'll be able to upload your own packages to pub.

I sat down with two of the pub engineers, Bob Nystrom and Nathan Weizenbaum, to learn more about pub and what's on the roadmap.


Q) What exactly is a package?

Bob: It’s a directory containing Dart code and all of the other stuff that goes along with it: tests, docs, examples, resources. It also has metadata that describes the package and its dependencies–the other packages it relies on.

Nathan: It’s important to note that Dart applications can be packages even if they don’t export any libraries and they’re not going to be re-used anywhere. As long as it declares its dependencies in a pubspec, pub considers it a package.



Q) I see some import statements like 'package:foo/foo.dart'. What is the package: there for?

B: At some point, I’ll write down the full explanation for why that’s needed. The short answer is that it “flattens” your import paths so that imports across transitive dependencies work. The simple answer is “it’s how you import code that’s in a package.”

N: “package:” is the magic that lets pub do its work. It tells Dart to look for imports in the “packages” directory, rather than just looking relative to the importing file. Pub generates this directory and fills it with your dependencies, allowing them to be imported.



Q) What happens when I run 'pub install' ?

B: In short, it makes sure all of your package’s dependencies are installed and wired up so that you can use them. I wrote up a more detailed description of what happens from pub install.

N: If a “pubspec.lock” file exists, “pub install” will install the locked versions of all the packages. This is very useful for application packages, since they can make sure that all their developers -- and more importantly, the deployment servers -- are using exactly the same versions of all their dependencies. This helps prevent unexpected breakages.



Q) What package managers did you study when you designed pub?

Bob: For the pub app, the main ones I looked at were RubyGems, bundler, and npm. I also did a bit of reading on maven, apt-get, easy_install and probably a couple of others I’m forgetting. I briefly checked out LuaRocks, Chicken Scheme’s package management system and a bunch of other random ones too.

For pub.dartlang.org, I continually had rubygems.org, npmjs.org, pypi.python.org, and cpan.org open in other tabs for design ideas.

Nathan: The biggest inspiration for pub’s installation model was definitely Bundler, from which we picked up the install/update dichotomy. The version constraints follow in the footsteps of many major package systems for other languages, and make use of the semantic versioning spec.



Q) How can someone publish a package to pub.dartlang.org?

B: Right now, there’s no way to upload your own package. We’re building out a user experience to do that now and should have that in place relatively soon. In the meantime, file a bug with the title “Request to host package - <your package name>” and we’ll upload it for you. Make sure to include a path to a repository (usually github) where we can download it.

N: Once user uploading is allowed, it’ll be very easy to publish a package. You’ll just need to run a single command, either through the Dart editor or from the command line, and pub will make sure your package is in good shape and then push it up to pub.dartlang.org.



Q) What do you have planned for pub?

B: The next big thing will be letting you upload your own package and getting us out of that loop. After that, there’s a long list of features we want to add. For pub itself, we want commands for linting your package to see if it’s following our guidelines, building and compiling your package to something deployable, running your tests, and other stuff.

For the site, there’s a bunch of discovery things we want to do now that we’re starting to get some data. Stuff like “here’s the most downloaded packages”, “here are the packages that are the most depended on”. We want to make the site automatically show you which packages are the most prominent and loved in the community.

Also, there’s a bunch of stuff we can do to make your life as a package author better: things like automatically running your tests, generating reference docs from your code and showing it on the site, displaying your README, etc.

N: In the more distant future, we’re planning to make pub work really well for deploying applications, especially to browsers. We want to have a single command that will grab all your Dart libraries and their dependencies, compile them to Javascript, throw away any unused code and merge the rest into a single minified JS file that’s ready to be served.



Q) What advice do you have for new package authors?

B: Follow the guidelines but don’t feel your package needs to be “done” before you put it out there. Everything is versioned specifically to enable things to grow and evolve over time.

N: Make use of other packages! Pub makes it so easy to depend on other packages, there’s no reason not to use them if they provide useful functionality.



Q) I have a good idea for pub. How can I send feedback?

B: Bring it up on the mailing list or file a bug. Filing a bug will make sure we look at, bringing it up on the list will solicit more discussion from the entire community.

N: Keep in mind that there are a lot of features we want to add to Pub, and only two of us working on it. If we don’t immediately address your suggestion, don’t despair; we’ve seen it, and we’ll keep it in mind.



Q) How can I learn more about pub?

B: Read the docs! We have lots.

N: If the docs don’t fully answer your questions, feel free to ask them on the mailing list. If we don’t answer it, one of the other community members will.


Thanks to Bob and Nathan, and thanks to all our Dart library authors that are already getting their code ready for pub.