Much ado about scripting, Linux & Eclipse: card subject to change


EMF 2.3 & EMF-QTV 1.1 released

Along with the rest of Europa, EMF 2.3 was released yesterday. Additionally, three new components, Model Query, Model Transaction, and Validation Framework (collectively "EMF-QTV") have graduated from EMFT and published their 1.1 releases.

What's new in the EMF project this year? See below.

Thanks to the whole EMF team for a great release and for making my life interesting, to say the least. Thanks also to all those who helped by providing ideas, suggestions, bug reports (especially those with patches & JUnits!), wiki documentation, FAQ entries & articles.


So long, and whoops for all the spam

After zx challenged me to start talking more about what I do, I created a second blog -- this time at I did this for three reasons:

  1. To provide a place to talk about release engineering away from the other clutter on this blog;
  2. To encourage other release engineering folks to provide posts here as well; and
  3. Because I'm a little sick of Blogger (preview sucks, no html shortcuts) and am ready to try WordPress for something new

Unfortunately, while importing old releng-related entries to the new site, I seem to have accidentally caused some old posts to be syndicated on Planet Eclipse, and so for everyone who's just been inundated with my old pre-syndication posts, my apologies.

Anyway, please watch the new releng blog for stuff about my work. If you want to be an author/contributor on the new blog, sign up, then drop me a line and I'll add you.


And Now For Something Completely Different...

Much like Ian's post the other day, I've always wanted to blog that line. Anyway, it's been brought to my attention that of late I've been apparently posting less-than-positive things about Eclipse, so to balance that out, here's my top three cool UI features in Eclipse 3.3, in order from oldest to newest:

#3: Use PDE UI to generate an Ant build.xml script for a plugin project

This has been around for years, true, but it's certainly top on my list of time-savers. Here's a quick HOWTO for taking a plugin and ending up with a versioned, Update Manager-compatible OSGi bundle.

  1. Create, import, or extract a plugin project from CVS.
  2. Right-click the plugin's plugin.xml and select PDE Tools > Create Ant Build File.
  3. Open the Ant view (CTRL-3, Ant or SHIFT-ALT-Q,Q > Ant) then drag the newly-generated build.xml into that view to see all the available targets. It's like the Outline view, but you can launch targets by double-clicking them.
  4. Double-click the build.update.jar target to run that build. You should see a new jar created in the root of your plugin project. (If you don't, select the plugin project's folder in the Package Explorer, Project Explorer, or Navigator view, and hit F5 to refresh.)
  5. You can run other targets, like to create a source zip, or edit them to suit your needs. If you delete the build.xml you can always regenerate it from the latest template.

#2: Use Package Explorer + Working Sets + Filters to clean up your view

I've been using these for at least a year by now, but only recently discovered that they behave differently in the Package Explorer than in the other filesystem views, like Navigator and Project Explorer. The best by far, for me, is the way the Package Explorer does things. Here's a shot of how one of my rather project-heavy workspaces looks in the Navigator view, compared to the Package Explorer with Working Sets set as my Top Level Elements:

To make this work, open the Package Explorer view, then click the menu icon (an arrow pointing down in the top right corner of the view -- I'll just call it Menu for simplicity). Here's a few things you can do to alter your view:
  • Toggle Working Sets / Projects:
    Menu > Top Level Elements > {Working Sets | Projects}
  • Select what Working Set(s) to show (when TLE = Projects):
    Menu > Top Level Elements > Projects, then
    Menu > Select Working Sets...
  • Select, create, edit, or delete Working Sets (when TLE = Working Sets):
    Menu > Top Level Elements > Working Sets, then
    Menu > Configure Working Sets...
  • Hide closed projects, empty packages, libraries, non-Java files, etc.:
    Menu > Filters

#1: Use Autopin UI Tweaklet to manage open editor tabs

This is my most recent discovery, and once you get used to it, a great new way to manage editor tabs.

First, download & install the Tweaklet plugin(s) from either the Platform UI Incubator downloads or updates site.

Then, marvel as a whole new way of working with tabs is revealed: you only ever have one tab in use until you start editing that file. When you do, the Autopin tweaklet automatically pins the tab, and subsequent files are opened in a second one, then a third, and so on.

Like Working Sets, this is also fantastic when dealing with massive workspaces or when you have a lot of files that reference other files. Eg., when drilling down from one Java class into another and into another to find the method you need, or for PHP scripts that include() or require() others, again, looking for methods or variable declarations, or even in concert with the full-text search (CTRL-H, first tab) to find that elusive build-related metafile which needs to be fixed, without ending up with 20 open files. Kick ass.

The bottom line here is that there's lots of hidden features in Eclipse to make your life easier and your work more efficient. And because we all work a little differently, there's lots of ways to tweak the way the UI behaves to accomodate all our differences. Eclipse is you? Aye!


I did it all for the cookie...

No, seriously, I don't test for schwag, or for the cookie, though some muppets do. I test because I like to work on and with tools, sites, projects, frameworks... that just work, without jumping through hoops or needing to recite black magic incantations. (Which doesn't mean I won't accept schwag if people want to send it to me. So far the best schwag-like mail I've gotten has been a clown nose from Kim, but that's another story.)

Anyway, inspired by Ian's bad experiences today with Europa, I thought I'd try to reproduce them. I couldn't, but I got something IMHO worse. While there's ways to work around the problems I hit, the issue here is Newbie Usability. I've been hacking w/ Eclipse for 4 years. The point of Europa is to be easy to use for the first-timers, who don't know about things like the eclipse.ini file (I've never looked at that file until today) or any/all of the OSGi startup flags you can use.

So, to synthesize the Newbie experience further, I googled for "-Dosgi" and searched the Eclipse wiki, but the best I could find is passing mention of these flags in some 145 Eclipse bugs. Another usability issue, IMHO -- insufficiently quick-findable documentation.

Oh, wait, I think (while still wearing the Newbie hat), what about Sure enough, it's documented as Eclipse runtime options. OK, that's cool, and that also means if I installed the SDK I have those docs in my Europa install, but if the problem is starting Europa in the first place, I still have to go hunting outside the Eclipse Help system. At least I've found what I wanted.

Back to the original problem... installing 121 out of 122 Europa features and watching as my Eclipse install crashes and burns on the first startup. Not cool. Second startup is better, but has problems on shutdown. Third time's the charm, apparently.

This whole experience brings to mind five, no, three questions, which I'll pose here and cross-post to the cross-project-issues-dev list too.

  1. Should the Buckminster SVN support be contributed to Europa, or should it ONLY be on Buckmister's own Discovery or Update site, in order to avoid a config problem right out of the box? I'd suggest it should not be in Europa because of the negative perception it might cause both Europa overall and Buckminster in particular. It portrays two issues: (i) the full suite (122 features) doesn't work 100% out of the box, and (if you'll forgive the rather damning phrase), (ii) "it's all Buckminster's fault". I don't mean to offend the good folks on the Buckminster team, only to suggest what *some* might say, and how to avoid that perception from ever being voiced anywhere but this rant. ;-)
  2. Since those who want SVN support need to add another Update site anyway (or install SVN features manually some other way), is it so onerous to ask those people to jump thru one extra hoop and remove the hoop from everyone else who doesn't need/want SVN?
  3. Who in their right mind will need to install 121 features (451M) on day one of their Europa experience? I, for example, only really need about 75M of features on a regular basis (EMF, XSD, Mylyn, phpeclipse, PDE, JDT, CVS) on top of the base 40M platform (68M unpacked).

I agree with Michael Scharf (who, incidentally, was the only person who could follow instructions) that Eclipse can be compared to ketchup in that it's:

[U]sed everywhere. It often does not fit. ... It comes in massive amounts (like the 18 million lines of code of the Europa release). It's often hard not to use it.

But that said, I don't think installing Europa should be harder than banging on the bottle to shake the last dregs of ketchup down the neck and onto my fries. It's hard *not* to use Eclipse, true, but should it be this hard to *use* it too?

Blogger + Italic Tag = No RSS

I blogged on Saturday, and decided to use italics (<i> tags) to emphasize some information. Unfortunately, this seems to have prevented Blogger from syndicating the post into my Eclipse Atom RSS feed. Strange. The problem doesn't seem to manifest with <em> tags and <span style="font-style: italic"> tags.

Thirteenth Step

Ye gods. As of last week, my little release engineering world is responsible for a total of 13 builds, with two more on the way. Time to crank up a little APC and have a quick look back...

  1. 1Q2004: pre-PDE build system migrated to PDE thanks to Marcelo Paternostro.
  2. 2Q2004: EMF 2.0 released, using Eclipse 3.0 releng.basebuilder. [Builds/branches/codebases: 1/1/1]
  3. 2Q2004: UML2 1.0 released, using cloned/simplified version of EMF builder. [2/2/2]
  4. 2Q2005: EMF 2.1 released, updated to Eclipse 3.1 builder. [2/3/2]
  5. 2Q2005: UML2 1.1 released. [2/4/2]
  6. 3Q2005-4Q2005: EMF 2.2 and UML2 2.0 start development, adding two new branches.[2/6/2]
  7. 4Q2005: first EMF Technologies project starts releasing builds, just before Xmas. Before long, the infant project has crossed the Rubicon, with 4 components: OCL, Query, Validation, and Transaction. [Builds/branches/codebases: 6/10/3]
  8. 1Q2006: EODM and JET join the EMFT party, solidifying the 'cross-project' nature of the EMFT build. 8 different builds supported by 3 code bases, across 1 to 3 active CVS branches. [8/12/3]
  9. 2Q2006: JET Editor, Net4j, and CDO are added to EMFT, and begin releasing builds. Callisto released, including EMF 2.2. Also along for the ride as part of GMF are OCL 1.0 and QTV 1.0 (Query, Transaction, Validation). System updated to use 3.2 basebuilder. [11/15/3]
  10. 3Q2006: Teneo joins EMFT. [12/16/3]
  11. 4Q2006: EMF, UML2, OCL, QTV (3) and JET/JET Editor (2) start work on new branches, in time for M3. Common Modeling Build started, adding new MDT component UML2 Tools. System updated to use 3.3 basebuilder. [13/25/4]
  12. 1Q2007: EODM branches for M4. [13/26/4]
  13. 2Q2007: EODM, OCL, QTV, UML2, JET & JET Editor migrated to Modeling Build and moved into EMF, MDT and M2T websites, making the old UML2 build obsolete and merging the old JET and JET Editor builds into a single website/build. EMF Compare build added to new Modeling.EMFT build. Common Modeling Build now supports multiple branches (Eclipse 3.2/3.3) and nine different component builds (across 4 different projects: EMF, EMFT, MDT, M2T), with varying numbers of upstream dependencies (from 1 to 9) & support for bundling or build-time use of 3rd party code (Orbit and non-Orbit). Original EMF build that started it all supports 4 branches and several non-JUnit test types. Three semi-dormant Technology.EMFT projects left to migrate (Teneo, CDO, Net4j). [Builds/branches/codebases: 13/27/3]

And there's so much still to do! No wonder I'm so tired these days...


EMF: Tastes great, less killing

If you google, er, 'do a Google search' for "EMF", you'll find lots of articles decrying the dangers of Electromagnetic fields (EMFs).

However, just last week, IBM released a tool called Spatio-Temporal Epidemiological Modeler (STEM), which is built using our very own EMF (that's the modeling framework, not the toxic substance). STEM, a part of the Open Healthcare Framework (OHF), is "designed to enable the rapid creation of epidemiological models for how an infectious disease, such as avian influenza or dengue fever, is likely to geographically spread over time." [1]

How's that for a carbon offset?


Eclipse: Have it your way

With mere weeks to go before Europa, and inspired by Kevin's comments in bug 192180, I'd like to post a question of the Eclipse community at large.

Q: If Eclipse was a condiment, which one would it be, and why?

Some ideas:


That Ol' PDE Black Magic

... or more reasons why features still suck.

In the past 24hrs, I've encountered two feature-related PDE issues, which I think warrant discussion as they are, to the uninitiated, seemingly black magic. The first of these was discovered and fixed last night while watching The Dresden Files in ten-minute chunks while waiting for builds to complete, but I'm sure that's just a coincidence.

Bug The Firste: feature.xml should not specify version numbers when including required plugins like org.eclipse.test [192231].

This issue manifests when your build stops working one day and complains, "eclipse/test.assembly/eclipse/plugins/${org.eclipse.test} not found." The fix is simple: make sure you set version="0.0.0" in your feature.xml when including org.eclipse.test or other plugins referenced in your map file and required by your build. See also Build Problems.

Bug The Seconde: PDE will build features in the order they're listed in feature.xml [192292].

This issue manifests when your source feature, for example, builds OK but contains no source because PDE is building it BEFORE your runtime feature(s). Once again, the fix is simple. You just have to reorder your feature.xml to put your source feature(s) last.

Bug The Thirde (Bonus Bug!): If your plugin or feature doesn't have the right Nature and Builder in its .project file, it won't behave the way you'd expect [192247] in Eclipse.

This one's a no-brainer, but also a handy tip when working with PDE. If your plugin project doesn't have the Plugin nature, you can't open a MANIFEST.MF file with the Manifest Editor; if your feature doesn't have the Feature nature, PDE won't give you warnings/errors if your feature is misconfigured (eg., if you have a version set to "o.o.o" instead of "0.0.0"). Just add something like this:



Request For Comment: Will anyone miss the EMF Standalone Zip?

With the advent of jarred plugins, improvements to Update Manager, and all the new smaller EMF 2.3 features (including 7 core "runtime" ones in Europa's Enabling Features category -- see bugs 106804 and 189295), the EMF team is evaluating if the Standalone Zip introduced in EMF 2.1 is still meaningful and useful to consuming teams, projects, and products.

For more on this topic and to learn what's actually in this zip, see:

To voice your opinion (be it 'I need that zip!' or 'not useful to me'), please post your feedback in bug 191837.



Managing Plugins and Features with Link Files and Extension Locations

After loitering and latering (but no lootering!) in IRC for a few hours today, I've come to the conclusion that not enough people know about .link files. I guess I could wiki this, but I thought I'd blog it instead.

Scenario 1: You're an Eclipse user. You use Eclipse a lot. In fact, you use multiple versions of Eclipse for testing, debugging, and even *gasp* developing. You waste a ton of time unpacking zips and running Update Manager.

Scenario 2: You use Eclipse and occasionally do something silly like deleting files outside of Eclipse or updating CVS files using commandline CVS instead of the click-click-wait method in Eclipse. (This is usually not a problem, but can sometimes lead to ...)

Scenario 3: Something Wicked This Way [Came] and your Eclipse has become corrupt. It's time to blow away your config files, .metadata files, or run with -clean. If that doesn't work you probably have to reinstall (oh crap) everything -- Eclipse + all your trusty plugins. You could also copy carefully from the busted Eclipse to the minty-fresh one, but that's both time consuming and possibly dangerous. Is there a better way?

Scenario 4: You produce a product based on Eclipse but want to keep your stuff and Eclipse's stuff separate when you bundle it all up for easier digestion by your users. Maybe for license reasons, maybe for legal reasons, maybe just for file system hygiene or to make support easier.

Scenario 5: You're a control freak and just like to keep stuff in different places according to your own particular... um... idiom.

In all of these scenarios, .link files can help. Here's how to set one up:

  1. In your Eclipse install folder, create a links/ folder next to features/ and plugins/.
  2. In your eclipse/links/ folder, create a textfile called This file need only be one line, with newline and no trailing slash(es), and can be called anything ending with .link.
  3. Unpack a zip full of plugins and features into the path specified in your .link file.
  4. Start up Eclipse. Check Help > About to see if all your plugins and features are listed.
  5. Rinse, lather, repeat.

You can have lots of .link files and lots of install locations, which allows you to easily swap between different versions of plugins and features for testing purposes, or to share an install location between multiple Eclipse installs. And, when you upgrade Eclipse, you can just copy the old links/ folder into the new Eclipse and voom, all your old plugins and features are there. Please note that when changing between versions of Eclipse, sometimes things will end up disabled, if the new Eclipse won't work with the old plugins. If this happens, you can easily switch back to the old one, look for an update, or download a new zip.

You can also manage your plugins inside Eclipse. So, if you prefer clicking to typing, you can convert a .link file location into a Update Manager (UM) install location, so that you can use UM to install updates to plugins that you created by unpacking a zip, or just to reuse an existing folder for later UM installs. Here's how:

  1. Complete steps above. Verify all your plugins & features are properly available in Eclipse.
  2. To convert a linked folder to an "Extension Location" as used by Update Manager, you just have to put a single file in the eclipse/ folder inside the linked folder. Here's a sample:
    $ cat /home/nickb/eclipse/eclipse-plugins-europa/eclipse/.eclipseextension
    name=Eclipse Platform
  3. Remove or rename the .link file that references the folder you just converted so that you're not trying to add the same plugins twice.
  4. Start up Eclipse. Check Help > About to make sure you no longer have the plugins & features you just disabled. Then launch Help > Software Updates > Manage Configuration > Add an Extension Location to re-add those plugins & features.
  5. Restart Eclipse.