I got the chance to play with debugging a running WebObjects app today, with the added fun of having a roomful of 20 users of the app taking turns to mention “did you know that [X|Y|Z] isn’t working?”

Long story short, if you need to get the status of threads of a running WebObjects app, jdb provides some great tools. I have only scratched the surface of it, thanks entirely to the great intro document by Andrew Lindesay. (Andrew recently moved his website to .Mac from some New Zealand host, so I’m linking to help throw some Google Juice his way so others can find his article)

I’ve put a wikified crib sheet together to make it easier to get set up.

Apple released an update to WebObjects - WO 5.3.1 Developer - which includes these changes:

This update addresses the issues and adds support for the features listed below: (emphasis mine)

• Adds a modified Developer tools license that allows WebObjects applications developed with Xcode 2.2 to be deployed on any compatible platform
• Adds better SQL Generation in the EOModeler Plug-in design tool in Xcode
• Improved FetchSpecification building in the EOModeler Plugin design tool in Xcode
• Adds a “components and elements” window for improved workflow in WebObjects Builder
• Addresses incompatibilities with Xcode 2.2 Developer tools on Mac OS X 10.4
• Bug fixes

Way to go, Apple! You had us all a bit nervous after the switch in licensing scheme at WWDC2005, but it’s sooooo good to see the decision come down on the right side of the fence. This will go a long way toward helping retain/grow WebObjects usage.

I’ve personally been asked by several people about WebObjects’ cross-platformedness (specifically by people interested in running their own Pachyderm server when it’s available), and my previous “well, officially not so much, but technically - it works just fine” response was something less than optimal. Now it’s a wholehearted “development is MacOSX (but free), and deployment runs on anything that can handle an app server - even open source stuff is cool”

Downloading the update now…

Update: Not downloading now… It’s not showing up in Software Update yet, and the link on the Apple downloads site is for Server… Grabbing XCode 2.2 from connect.apple.com now, to see if it’s got it included (or triggers a WO update after installing)

Update: Nope it ain’t broken. It was purely operator error on my part - I missed an “=” sign in the .html for a component, and that was borking file uploads. It was weird, because Safari and IE would upload just fine, but Firefox would barf.

Josh just discovered a weird behaviour in Pachyderm, where uploading files was breaking. But only on Firefox. WTF?

A quick google brought up this post on NSLog.org, via ryfar.com:

NSLog(@”Guido’s blog”);: WOFileUpload problem
WOFileUpload problem

I implemented today a generic component for file uploads to my application and got “no form data left” errors from WOFileUpload.

To solve this, I added:

System.setProperty("WOUseLegacyMultipartParser", "YES");

to my application constructor. Works, but I’m not happy with it. Have to check back later on that topic.

Update: Well, that didn’t fix the problem, but it may come in handy some day…

Update 2: Looks like the code mentioned above may break things rather than fixing them. We’ve removed it, and things are sorta working again. And, King found one cause of the exception we were seeing - a missing = sign in the name="myButtonName" for the submit button was causing FF to barf violently. But, we’re still seeing a separate upload problem…

This message from Jeremy Matthews just came across the WO-Deploy list, and I wanted to hurl a copy here to remind me to check it out. Looks like the latest stable build of Tomcat 5.5 may include some love for WebObjects… (emphasis mine)

FYI,

We just released Tomcat 5.5.12 installer for Mac OS X 10.2-10.4.2

What’s New:
more than 100+ changes from the apache group; view changelogs at:
http://jakarta.apache.org/tomcat/tomcat-5.5-doc/changelog.html

Changed to the newest stable distro (5.5.12)
Rewrote everything to work with the newest version (lots o’ changes
here…)
Updated Autostart script to declare WebObjects variables
Included JDBC adaptors from: Oracle, MySQL, FrontBase, OpenBase, and
PostgreSQL
Tons of maintenance changes
Resolved some mis-declared variables

One note: If you are building the app you may need to edit
permissions (left locked out by default for security), by running:
(I believe)
sudo chmod -R a+rwx /Library/Tomcat/

Download pages:
http://www.versiontracker.com/dyn/moreinfo/macosx/27151
http://www.macupdate.com/info.php/id/18567
http://mac.softpedia.com/get/Development/Java/Tomcat-by-SW.shtml

I played around with Rails a bit today. I contrived an excuse to attempt to create a generic “event management” application. I’m familiar with the basic requirements of the app, and have a need to have a generalizable app available to share with groups and departments on campus. And, I want to play around with Rails and ruby to get a feel for what it can do.

Well, I had better luck than Stephen did, but still not quite the magic “30 seconds from concept to production app” wonder that has been whipped up around Rails. I was following the “ToDo” howto and going off in my own direction. It didn’t work too well, and I’ve just realized that’s because the ToDo chapter is written to Rails 0.9, and I’ve got Rails 0.8.something installed. Apparently, Big Changes were involved going to 0.9. OK - so that should discount some of these issues/hangups.

At first, I was stumped trying to get Apache 1.3 (the stock install on MacOSX 10.4.2) playing with Rails. Thought I was following the instructions, but no joy. Gave up, and settled for the WEBrick server bundled with Rails. I got hung up for about 20 minutes because I didn’t realize that ./script/server needs to be run as root. I could have sworn that wasn’t required the last time I played with Rails. Once I figured that out, I got the free “hello world” page. Great. On to application creation… It would be cool(er) if the Hello World page gave me an interface to query any models/tables I’ve got configured. Kinda like D2W does…

I created a simple “event” model, with an “events_controller.rb”, and associated “events” views. I used the rails scripts to generate controllers and scaffolds, and ran rake to make sure it all checked out. I spent a lot of time trying to figure out which controllers should be created (Event vs. Events, and Person vs. Persons vs. People) but once I finally got the right controllers and scaffolds set up (and trimmed the excess tests from the various abortive attempts), then it was pretty clear sailing. The scaffolded app isn’t extremely high end - and as I’ve mentioned before, WebObjects DirectToWeb does a better job in some ways. But, the Rails scaffolded app is nice and clean, and looks rather straightforward to customize/extend. “Out of the box,” it gives you a barebones index (list all records), an inspect page, an edit page (with appropriate widgets - textfields, textareas, date selectors, checkboxes…), an add/insert page, and a delete function. Pretty straightforward - but still no Search. wtf. D2w does that really really well, even letting you pick which field to search on, and providing the appropriate widgets to help you enter search parameters.

I haven’t started looking at relations/joins - tying People to Events as either presenters or attendees - but that shouldn’t be too difficult. So far, I haven’t had to “write” any code, but invoking the various generation scripts should probably count - I didn’t get functionality for free, since I had to teach it via generation. D2W gives you lots “for free” by reverse engineering a database schema automatically. Generated code is still code - it’s just written by a script.

I’m going to read up on these before digging back in, though…

• manuals.rubyonrails.com
• documentation.rubyonrails.com
• Ruby on Rails manual - chapter 38: ToDo App
• Raible’s notes on the RoR session at OSCON2005
• … and any others I find…

I have to wonder, though… If O’Reilly is such a supporter of Rails, touting it as one of the Next Big Things, wtf isn’t there a book in The O’Reilly Safari???

The Pachyderm project uses jGenerator to wrap images in a flash .swf container for display in the final product. That process does a few things that are pretty handy:

• Makes loading the images into flash easy - it’s just loading more flash…
• Lets us embed metadata in a “tombstone” display field, much like the cards displayed in a museum. These tombstones travel with the asset, and can be displayed automagically wherever appropriate.
• Provides a lightweight DRM - the images are useless outside of the finished Pachyderm presentation (unless you’re able to decompile flash, or take screenshots) - it’s not an overbearing DRM, just a way to make it easy to be honest.

(left) tombstone in “closed” state. Click the arrow widget dealie to expand it to view the full tombstone (right)

But, we’ve reached a point where a bit of a dilemma has been forced upon it. Josh was describing it as Gordian today, and I think the solution might be as radical as that.

The jGenerator library that we use to wrap images in .swfs has been acting more and more flakey over the last few weeks. Likely a result of increased load, we’re seeing what may be some kind of funky threading or deadlock issues deep in java.awt classes, which are relied upon for jGenerator to do its magic.

So, here’s our dilemma:

1. Keep on using jGenerator, Hoping For The Best™ - we’d add some debug/babysitting code to detect the deadlock issue, and attempt to recover from it.
2. Switch to the OpenLaszlo fork of jGenerator, hoping that they may have resolved whatever issues are plaguing it. That’s kind of a blind faith option, since we don’t know if/how the Laszlo folks have modified jGenerator in their fork.
3. Take the sword to the knot, and dump our legacy tombstone/drm/swf-wrapping implementation. Build a new one, from scratch, waaaay past the 11th hour. We know what we would need to do in the authoring application to support a much more robust and flexible metadata-embedding-and-display strategy. The idea that we’ve come up with is actually more useful in many ways, as it can be applied to any media type - not just .swf-wrapped-images. We could easily create an XML-based lookup table that the flash templates would have to consult to gather metadata about assets in a presentation. That’s actually pretty straightforward to do in the authoring app, but every flash template file will need to be modified to teach it about the tombstone xml lookup…

I’m leaning quite strongly toward the third option. Let’s dump the bottomless pit of jGenerator, and focus on the future. There may be a short-term solution - drop tombstones altogether for awhile, taking some time to design the new solution without rushing it. We’d have to keep a snapshot of Pachyderm running, since Mavericks relies pretty heavily on tombstones - but even that needs to be fixed, since jGenerator barfs on characters like accents…

Basically, nobody’s comfortable with the status quo - it’s not stable, is unreliable, and can lock up publishing altogether. The Laszlo option might work, or it might not, leaving us no further ahead. The xml-lookup option is the most solid design, but would take more time than we have. Stupid dilemmas…

Update: Chatting with Josh last night, and he came up with a potentially simpler solution. No need for a lookup table, just have one xml file per media asset - change the filename from .jpg or .mov or whatever to .xml and you have the xml definition of the tombstone for that asset. No xml file, no tombstone. Actually, that could give us a bunch of flexibility - different types of tombstones for different sizes of an asset, for instance. This would be relatively trivial to implement in the authoring app, but we need to figure out what it would take to implement something like this in the flash templates…

We’re using jGenerator for the Pachyderm project - to replace the abandoned Macromedia Generator product - for wrapping images in .swf files for display within Pachyderm presentations. The .swf files provide value-add stuff like “tombstone” data, and a lightweight, unobtrusive form of DRM.

However, jGenerator has been rather neglected for 3 years now, and as a result it’s starting to show cobwebs etc… Remember my friend Murphy? Largely a result of these cobwebs.

Today, days after we thought we nuked the jGenerator problem, it came back in full force. Building presentations would occasionally hang (sometimes not reliably, or sometimes very reliably). We tracked down the problem thanks to the handy dandy java debugger jdb (handy document on using jdb in a running WebObjects app), and it was the result of jGenerator using java.awt.Rectangle classes rather than java.awt.geom.Rectangle2D.

The methods had been altered to use Rectangle2D in the signatures, but were actually passing around Rectangle objects instead. This apparently causes problems on MacOSX and other Unix-like systems, as was discovered by the OpenLaszlo folks. They had to start their own fork of jGenerator 2.2 to fix this (and potentially other) problems. We just did the same.

We had another round of Team Programming at a Distance - the three of us brainstorming and hacking and researching. That is soooo much more fun and rewarding than copying and pasting…

After what felt like a very long weekend of debugging jGenerator, I finally figured out what was going wrong, and how to fix it. It didn’t make sense at all, which is why I didn’t pursue that line of investigation earlier. The library worked fine before. It hadn’t changed. Then it didn’t work. I was figuring that meant something outside of the library had changed, and had borked something used by jGenerator.

Whatever the cause, I wound up using a java decompiler (jad) to figure out exactly where the problem was, while comparing to the versions of available jGenerator source code. I removed what appeared to be the offending line (calling Log.logRB() - which is a utility log method within jGenerator), and recompiled a fresh copy of jgen.jar using the Ant build script.

Deployed that sucker, restarted the Mavericks instance of the Pachyderm authoring app, and cést voila! I had it working early this morning, then published fresh copies of all of the Mavericks sections for review. *touches wood* It’s working great now!

So… in the future, when debugging something that isn’t behaving as expected… throw away all assumptions - no matter how “sure” you are about something, test and verify each and every line of investigation.

OK. Murphy, you can go to hell now…

For the Pachyderm project, we wanted a way to automatically update, build and deploy a WebObjects application and its supporting framework. The initial reaction was to just use a shell script, with xcodebuild running on the server to build the appropriate projects.

That didn’t work for us, because our server is still running 10.3 (with the appropriate older XCode dev. kit), while we’ve moved on to 10.4 and XCode 2.1 for development- so our server can’t understand the .xcodeproj files and barfs appropriately. Doh.

So, King and I whipped up an Automator-based workflow that runs on one of our dev. boxes. It first runs a shell script to update the source code from subversion, then builds the framework and application. It then runs a shell script to record the subversion revision number in a file in the compiled application (so we can display the revision number for bug reports etc… and not pollute our source code with revision numbers). Then, it connects to the server over afp, moves the old build products out of the way, and copies the new ones into place. About an hour later, WOMonitor comes through and cycles the app.

Basically, there is one “master” workflow that runs several nested workflows. This “master” is saved as an application, and I’ve set iCal to launch it every morning at 1:00am.

Yes. Source code management and enterprise application deployment with Automator and iCal

It seems to work well, but we initially had some permissions problems. It will also make it trivial for use to update the app at will.

Update: Added some details of the build settings for converting an existing WebObjects Application project to a tidy servlet deployment package.

King, Josh and I spent the morning in the WebObjects lab trying to figure out a way to get Pachyderm deployed on non-MacOSX boxes under the new regime. We’ve now got a spanky new Tomcat-enabled build, that sucks in all of the supporting WebObjects, APOLLO, and Pachyderm jars into a single .war - topping 112MB of compiled goodness.

The good news is that it works, and seems to work well (once you have identified and addressed the mines laid in your path). The bad news is that form of deployment requires a valid WebObjects deployment license - which is no longer available separately from MacOSX Server 10.4 (for WO 5.3 at least - separate deployment licenses are still available for 5.2.4, but they are refused by apps compiled on 5.3)

A very helpful Apple engineer in the lab (who will remain nameless to protect him/her from any flak for going out on a limb to help us with this) was able to obtain a WO 5.3 deployment license string for us to test our Tomcat deployment. It looks like it worked perfectly. I had to delete the license so this engineer could keep their job, so I can’t test again, or on another platform at the moment.

One strategy we may have/get to adopt is to keep a WO 5.2.4 development system around, so that it can compile the .war SSD bundle with a 5.2.4 platform-agnostic deployment license - That’s plan B, though, since I’m really (futilely?) hoping that we can convince Apple to reconsider the MacOSX-only licensing. (I’m even 100% OK with non-MacOSX Server deployments being unsupported - just don’t make the app refuse to light up because of a license string!)

I’ll post a little write-up on the steps we had to take, and the build settings that need twiddling - and how they differ from the documented bits. That’s currently locked on my Powerbook, which is now a paperweight after sucking the life-sustaining juices from 2 separate batteries. I’ll plug it in when I get back to the hotel.

Oh, and to the person that IMed me to ask me about this - your timing was absolutely perfect! We were just getting to the solution when your IM came up. I’m very sorry that I was such a rude prick to you, but I didn’t want to be distracted right then, and we were using iChat to pass config files and data around.

Update: Here’s some info about the settings you need to tweak to have a project build as a nice tidy servlet doowackie.

First, you need to add JavaWOJSPServlet.framework to your project. That seems to automagically do some stuff to start building a .war file. Yay.

But, you also need to tell WebObjects more explicitly how you want your servlet compiled. These are the settings that worked for me. No warrantee, no guarantee, YMMV, IANAL, etc… If it borks something for you, well you probably should have backed up first…

Replace the fuzzy part with your WebObjects deployment license (5.2.4 would be a good bet, if you’re still on it - otherwise, you may want to take a look in /System/Library/Frameworks/JavaWebObjects.framework/Resources/ for some random ASCII that might do the trick - just be sure to abide by the EULA, and don’t break your license agreement. I’m only mentioning this so you can test on your own machine, and replace the license with one you purchase for deployment), and replace the SERVLET_WEBAPPS_DIR with the path to the directory you want to build this sucker. I just built into a quick/fresh download of Tomcat 3.3.2, still living in my ~/Downloads directory. Not likely the final deployment directory, but you get the idea. Also, this could be any directory you like. It’s just the build location - it doesn’t talk to the Tomcat installation or anything.

In my case, this built a directory that contained 112MB of our code, as well as the Entire WebObjects System, and any other dependent .jars.

Update the second: On a lark, I thought I’d try deploying under Tomcat 5.5.9 - the latest build. Out of the .tar, it requires java 1.5, so until you install the 1.4 compatability stuff it’s no joy on MacOSX. But adding the compatability stuff is trivial. But, it still doesn’t get my app deployed - as expected, Tomcat 5.x appears to have difficulty hosting a WebObjects application servlet, since it’s supporing a newer rev. of the servlet spec (2.4, IIRC), when WO spits out servlets matching version 2.2 of the servlet spec. Not a critical thing at this point. Just gathering data points for discussions with the PHBs.

I’d never tried deploying a WebObjects app under Tomcat (or JBoss, or any other J2EE container) before. I’d always just stuck with the built-in WebObjects appserver, and did the config/management in JavaMonitor. Well, it’s now a Really Good Idea™ to make sure any app you want to deploy on non-MacOSX boxes will work in the servlet engines.

So, I did a quick RTFM. (twice, because it’s taking longer for it to soak in after midnight after a busy day of WWDC) Then I twiddled the appropriate bits in my XCode project build settings. Built the .war bundle. Downloaded Tomcat (for servlet spec 2.2), copied the .war into the webapps directory, and lit it up. It just worked. Mostly. Except for funky CSS stuff (because I don’t have any freaking idea how to deploy a Tomcat/WO app, and stuff that’s normally vended via Apache is now just firing 404s).

But, what I care about is that it works!!! We’ve now got (ok. it’s been in WO for a long time now, but I just took advantage of it - cut me some slack) what appears to be an easy way to deploy a WebObjects app on any platform that can run Tomcat. That’s pretty sweet. We’ll be hitting the WebObjects Lab this morning to get some more stuff figured out, but this is looking promising now!

The WWDC session on deploying WO apps on JBoss was pert’ near useless. I did manage to make a bee-line to the mike for Q&A to ask wtf the deal was with cross-platform deployment.

Just to document the event:

OK. Now to sleep.

Someone on the WebObjects-Dev list, known only as “LD” just posted a tip about getting CodeSense completion working in XCode2 for WebObjects code. I took his tip and wrote an easy-peasy shell script that got the job done on my 2 machines.

If you need to enable CodeSense completion for WO, just grab a copy of this script, and run it in Terminal.

$ ./xcode_webobjects_codesense

You may first need to chmod +x xcode_webobjects_codesense first, to get it executable, but that’s about all.

Thanks to LD for the detailed description!

David just published a great post on developing with WebObjects. He really boils down the WebObjects development experience into a few bullet points, which I will copy/paste here for searchability…

The last point is really the best one - I’ve come across this so many times on CAREO and Pachyderm. We’ll be building away and I’ll just stop and say something lame like “Wait - this is getting too complicated. There has to be an easier way…”

Lazy binding is the smarter way to initialize variables only once, and only when needed. David LeBer just posted a couple of articles showing how lazy binding is useful in WebObjects apps.

• Lazy WOComponent ivar initialization
• Programatically creating a WODisplayGroup

Well, not exactly alive, but it’s working. It’s freaking working! I can now insert new records (creating new xml documents) and save them, run queries, edit records, and save those changes. Whew.

There is some final cleanup I want to do (finish migrating from DOM4J to JDOM, clean out all of the debugging spaghetti outputs, and comment the heck out of it), then I’ll package it up and drop it on Sourceforge.

This has been one of the most frustrating projects I’ve ever done (and I’ve been involved with some doozies). It was frustrating simply because I wasn’t able to grok some of the finer points of EOF’s behind-the-scenes magic. Thanks to King for providing insight (it wouldn’t have ever worked without his guidance!)

I’m completely sure there is some stuff I’ll need to modify, and it’s entirely possible I’m doing stuff the Wrong Way, but it’s working well enough to be used, and that’s what counts…

UPDATE: Just updated the adaptor to use an EO_PK root of the current database, which will increment primary keys in a predictable way. Very cool. Works quite nicely. Now to generate some metrics on the whole operation. Tomorrow…