Archive for the 'Programming' Category

Refactoring to Seam by Brian Leonard

Friday, July 27th, 2007

The Poll:

The thrust of the talk: take a typical Java EE 5 application (JavaServer Faces -> managed session bean -> entity class -> database) and refactor it to Seam. This is interesting, but we don’t even have any Java EE 5 web apps to refactor, so both the Java EE and the Seam stuff are new to me.

Java EE: lots of config and the syntax looks nasty after Ruby, though a big improvement over J2EE of years past, for sure.

EJB 3 and EE5

  • Dramatic simplification of all bean types
  • POJOs + annotations
  • Dependency injection

You know, I looked around at the audience at this point, and noticed that the room full of big, dumb-looking people. Not a good sign.

OK, here’s what I learned.

JBoss Seam is an application framework for JSF and EJB; it brings them together. Must be good: it has AJAX! Also has improved state management. The presenter showed this at the end and, yeah, this is good.

Seam eliminates the managed bean, reduces the XML and provides “bijection” for stateful components: dynamic, contextual, bidirectional. What? Yeah, I couldn’t figure it out either.

We get a live demo where the presenter troubleshoots various problems. Never a good idea to expect that you’ll work through issues live in presentation. Buzz kill for the audience.

Ahhh, a choice J2EE moment. The framework needs a blank property file, but if it’s not there, you get a cryptic error message that doesn’t mention property files.

We see pattern-matching to look up EJBs. Can’t recall now if that’s Java EE or Seam.

Seam replaces XML with annotations. Good. The more annotations the better I think. Looks like you can inject dependencies via annotations:
@In private User user;

Pimp My Build by Josh Cronemeyer, Andy Slocum

Friday, July 27th, 2007

This was a talk from a couple of guys from ThoughtWorks about CruiseControl, one of the first CI (Continuous Integration) tools. Martin Fowler works at ThoughtWorks! We are not worthy.

Audience Pool

90% use CI
I’m the only one who doesn’t use CruiseControl, and the presenter wants to know what I do use. “Ruby” seems to be a worthy answer. Wheh.

There are several implementations of CruiseControl — Java, .Net, Ruby — and it can be customized plugins. This talk is actually about customizing CruiseControl.

The presenters use their own ThoughtWorks project as an example. It’s a big, old Java project with 8,000 class files. So, in the ballpark of the entire Con-way application, er, applications. It takes over thirty minutes to build and test. They have about 6,000 tests, and are slightly embarrassed by how light their test coverage is.

The central problems with their CI is the volume of information and correlating it together. They solved these problems with the practical application of brute force by using Lucene to index the CruiseControl build logs (this includes source control commit comments and bug tracking numbers). It has a simple web front-end, so there’s one place to figure out which bug is fixed in which build. It was initially for developers, but business analysts like it, too.

We had a nice chat about build lights (build broke, light goes red). Apparently, x10.com is the place to go to buy one. You may remember them from endless, annoying banner ads. Other sources:

  • NaBazTag
  • Ambient Orb
  • Lava

I liked the presentation’s funky diagrams. Think Scooby Doo’s Mystery Machine van.

Suggestion: build a CI build dashboard on an old PC and monitor in your bullpen. Hey, the ARAOT is ahead of the curve!

High-performance JavaScript: Why Everything You’ve Been Taught is Wrong by Joseph Smarr

Thursday, July 26th, 2007

Javascript and performance tuning seems to pack ‘em in theses days, so this talk was standing room only. It also had the highest percentage of Macs I’ve seen at OSCON — must have been refugees from RailsConf. This was a talk about lessons learned from developing Plaxo — a very Web 2.0 community site. This talk rocked — the presenter was energized, organized, and it contained all sorts of good nuggets. My experience with AJAX and Web 2.0 mirrored Plaxo’s, and I was happy to learn how to keep my user’s browser from locking up on all the nifty drag-and-drop pages. Some of the accepted Javascript wisdom hasn’t panned-out too well for me, either. Plaxo almost never shipped, and they had to work really hard to make it fast. What happened?

  • didn’t take performance seriously from day one
  • didn’t think browser limits were significant
  • didn’t use the app daily as they were developing it
  • didn’t push back on feature requests for performance
  • didn’t value perceived performance/responsiveness

They needed to unlearn a lot of standard assumptions.Why everything you’ve been taught (AJAX euphoria! Web browsers can do anything now!) is wrong: Browsers are pushed out of their comfort zone; Javascript is parsed and interpreted every time you load the page.

The Plaxo mantra: Be lazy, responsive, pragmatic, and vigilant.

Lazy

  • Noting is faster than doing nothing
  • Write less code
    • Javascript is parsed every time it’s loaded, and Javascript is downloaded a lot, so keep it under 500K of code — this seems to be a standard best practice. Be careful of third-party libraries — they can add a lot of code.
  • Draw the UI as late as possible
    • Never pre-draw hidden UI elements
    • Cache HTML if appropriate
    • Don’t keep hidden UI up-to-date
    • Consider just redrawing everything versus carefully updating parts of the page

Responsive: When the user says, “jump,” you jump!

  • The initial page load is very important to user
    • CSS at top of page and Javascript at bottom. (News to me. Good idea.)
  • Draw placeholders right away
  • Yield early and often
    • use setTimeout(0) to yield
    • use a closure
    • use onmousedown() instead of onclick()
  • Cache backend responses
    • send all data requests through data manager code
    • use range caches and fill in missing pieces
    • balance local updates vs. re-fetching from APIs

At this point, it really struck me that that a lot of these points are basic UI practices from any UI toolkit like Swing, VB, etc. It’s not like caching and quickly responding to user requests with some UI feedback are revolutionary ideas. I’m all in favor of leaving out optimizations like caching until you really need them, but I’m, uh, surprised that this was such a surprise for our friends at Plaxo. Same goes for practices like eating your own dog food.

Pragmatic: Don’t make things even harder

  • Play to the browser’s strengths
    • avoid DOM, user innerHtml
    • avoid dynamic CSS-class definitions
    • avoid reflow when possible
    • avoid memory allocation like string splitting
    • do DOM manipulation off-DOM, then reinsert at the ends
  • Cheat when you can
    • Use global functions when reasonable (sacre bleu!)
    • Finding by CSS class is slow
    • Directly attach onclick, etc. handlers instead of using event listeners
  • If you have to do it every time, do it statically

Vigilant: Only you can prevent slow web apps

  • Profile
  • Use Firebug (I agree, it rocks!)
  • Measure with a consistent environment
    • browsers bog down over time, you need to restart them for accurate numbers.
  • Consider performance from day one
    • apps get slow when you make them do a lot of things
    • consider how much code is needed
    • make sure everyone remembers how important speed is

Summary: Make the browser happy. Web browsers are more like mobile phones, not a desktop. They are a flimsy, temperamental platform, but everyone’s got one, so it’s the best place to be. Don’t push the limits unless you have to.

The Coming XQuery + XForms XPlosion by Kurt Cagle

Wednesday, July 25th, 2007

Now before I pan this talk, let me say that Kurt strikes me as one of those guys whose hard work with open source and standards bodies makes things go. OK?

Things started out just fine, with some rambling and ranting about how the open source software developer crowd is faddish. “XML is staid? I’m a graybeard because I work in XML? What?” Funny. True!

News flash: everything is more complicated now, mainly because we aren’t just running on a disconnected box under the desk.

Halfway through the talk, though, the presenter is still ranting! Wait, the point? Maybe we need to simplify the models we use to combat complexity.

  • Web enforces simplicity
  • Traditional mid-tier apps break down on the web because they assume synchronous interactions
  • Simplest arch pattern is an old one — MVC

Ten minutes later, a question from the audience: are we going to talk about XQuery? Oh, yeah, OK.

There are many emerging XML standards:

  • XHTML + XForms
  • XSLT
  • XQuery
  • Schematron
  • XBL

XForms started as replacement for HTML forms, but became more. They create an XML model that uses XPath to “bind” control to various parts of model. XForm is sort of like a VB form. Did you just cringe? I did.

XForm implementations to watch

  • Mozilla (presenter’s favorite)
  • Orbeon
  • Chiba
  • Formsplayer
  • PicoForm
  • OpenOffice

Maybe XForms will be the Next Big Thing, but I’m not holding my breath.

Performance Whack-a-Mole by Josh Berkus

Wednesday, July 25th, 2007

This turned out to be a good talk, chock-full of concrete practical advice.

Here’s our audience poll:
95% use DB apps
15% are consultants (like the presenter)

Clients just say: “the database is slow,” but anything in the big application stack (application, middleware, database, operating system, hardware_ can make things slow. Every layer in the stack needs do its job to scale the application. Otherwise, we are leaving significant performance gains on the table. In fact, “database” problems are usually somewhere else — like in the middleware.

You have many performance problems in your application, but most aren’t significant. Performance is usually degraded by a few problems — 10% of problems cause 90% of slowdown. (And big problems mask smaller problems.)

Apps tune differently. The presenter uses three rough categories: web, online transaction processing, data warehouse.
Web

  • Entire database can fit in RAM
  • 90% or more of database queries are simple reads
  • CPU-bound
  • Performance moles: caching, pooling, connection time

OLTP

  • Database slightly larger than RAM to 1TB
  • Many small data write queries
  • CPU or I/O bound
  • Moles: locks, cache, transactions, write speed, log

Data Warehouse

  • 100GB to 100TB database
  • Large complex reporting queris
  • Large bulk loads of data
  • Also called “business intelligence”
  • Moles: Sequential scans, resources, bad queries

Your first step should be to measure a baseline. Check basic setup (versions, configurations) for sanity. Then measure each layer.

Tools

  • OS tools are simple, easy to use, and non-invasive
    • ps, mpstat
  • Benchmark. Invasive
    • micorbenchmarkL bonnie++ for files system perf
  • DB
    • DBTrace
    • DB query log
    • Query analysis (troublehoot “bad” queries)
    • ogbench, Wisconsin, TPCB, OSDB, PolePosition
  • Application
    • app server tools
    • workload simulation and screen scraping
  • lwp and log replay tools
  • bug detectors
  • valgrind, MDB, GDB, DTrace

OK, we’ve got the baseline, now what? What are the symptoms; when do they occur?

The Performance Moles

I/O

  • Symptoms: one device saturating I/O — other resources are OK
  • heavy writes or very large DB
  • causes
    • bad I/O hardware, software or configuration
    • not enough RAM
    • too much data requested
    • bad schema

CPU

  • Symptoms: maxed-out CPU, but RAM available
  • mostly-read loads or complex calculations
  • causes
    • too many queries
    • insufficient caching or pooling
    • too much data requested
    • bad queries or schema
  • Note: most DB servers should be CPU-bound at max load

Locking Mole

  • Symptoms: Resource aren’t maxed-out
  • lots of pessimistic locking
  • causes
    • long-running transaction or stored procedures
    • longly held open cursors
    • unneeded pessimistic locking
    • poor transaction management

App Mole

  • Symptoms: DB server maxed, app server is OK
  • common in J2EE
  • causes
    • not enough app servers
    • too much data/ too many queries
    • bad caching.pooling
    • driver issues
    • ORM frameworks

A Taste of Haskell

Tuesday, July 24th, 2007

Back in the Oregon Convention Center. Pretty handy that two of the biggest open source get-togethers are a short ride up the Esplanade. Con-way even ponied-up for the OSCON tutorials.I went to the Haskell talk because all the cool kids are were talking about it. (Gerry tells me that they are on to Erlang now.) Anyway, Simon Peyton-Jones from Microsoft Research (yes, that’s right) gave an energized talk to all of us later-to-the-party programmers.Things didn’t look too promising at the start, though. We had some presenter equipment set-up hijinks: “Any Ubuntu experts here? How do you increase the font size in Emacs?” Once that was all squared-away, I noticed that the presenter was barefoot.Audience poll! 30% have written a Haskell program.OK, what is Haskell? A functional, lazy, (really) strongly-typed, compiled, general purpose language. It’s about value, not state … what? Yeah, I know, that doesn’t make much sense. It did make sense to me about three hours later, but it’s beyond my ability to enlighten you via this blog. Sorry! Try http://haskell.org/haskellwiki/Haskell or http://en.wikipedia.org/wiki/Haskell_%28programming_language%29.It’s a research language, and the presenter is one of the inventors. He defines a successful research language as one that dies a slow death rather than an immediate death, so Haskell is successful.We’re using xmonad as an example of a Haskell application. It’s an X11 window manager. It’s not exactly Enlightenment. And the first big diagram is a state machine! Uh, isn’t Haskell about values, not state? We find out later that since we’re dealing with I/O — the outside world — we can’t keep everything pure. Turns out that, for quite a while, Haskell was a nifty language that couldn’t do any I/O, which rather limited its practical value.OK, here’s a code sample. The syntax seems reasonable, but everything looks verbose to me after Ruby. Functions are defined by pattern matching. In some sense, they are a “collection of equations.” The code below basically says: for an empty list, return an empty list; for a list of one, return a list of one followed by an empty list; for more than one, return the same list with the first and second elements swapped.swap :: w -> w -> wswap [] = []swap (w : []) = w : []swap (w1 : w2 : ws) = w2 : w1 : wsThis is pretty cool. Pattern matching forces us to think of all cases. The presenter didn’t mention this, but I wonder if they are driving at the goal of mathematically-provable programs. It reminds me of Eiffel and its design by contract.Haskell has a lot of the things you’d exepect. There’s a library called Test.QuickCheck that passes in 100 (random?) values into each of your functions to ensure that you’ve covered all the cases. (As you might imagine, functions are easy to test.) There’s a documentation generator, a package repository and general-purpose libraries.We dived a whole level deeper into Haskell, but I’m not sure I can do it justice. So, the big picture?

  • No side effects at all
  • Purity makes the interface explicit
  • Types are everywhere

Haskell is trying to be useful and safe (safe == no side effects). It is is very safe, but needs to become more useful. From Haskell’s perspective, Java, C#, and Ruby are very useful languages that are becoming safer. Apparently Haskell’s not useless, as there’s a company in Portland that employs twenty Haskell coders and would like to hire some more.Did the presenter just say, “The nettle must be grasped?”

Final Keynote by Dave Thomas

Wednesday, May 30th, 2007

I have to say that all the RailsConf keynotes I went to were good, and I heard that the others were good, too. Is that level of quality sustainable? I hope so.

Dave noted that EuroRailsConf was the most profane conference ever. He thought this was a bad thing.

We all were getting a bit loopy at this point, so my notes are short and fragmented. Here’s what I can piece back together.

Ze Frank talked about what to do when you have many amateur authors. This was an interesting problem that we wrestled with back in the day at Gap Inc.’s intranet. You can ignore them (obvious bad consequences). You can try and control them, but this doesn’t scale. You can shut them down (draconian). He suggests that you instead facilitate conversation. Which sounds fine, but fuzzy now that I write it down here. Should have gone to his talk, I guess.

Dave warned about Cargo Cult programming — definitely a danger in the Rails community. He then listed some assumptions we should question:
– The web browser: it’s a freakin’ full-duplex mainframe terminal!
– Object-oriented programming. The world is not hierarchial, it’s chaotic. Why are we so class-centric? Try to write a program without classes and you might be surprised by how it works.
– Relational databases
– REST

Laying Tracks: How to Contribute to the Ruby on Rails Open Source by Josh Susser

Wednesday, May 30th, 2007

This talk was practical advice on how to do just what the title says: contribute to Rails.

Josh comes from a Smalltalk background. He also worked on OpenDoc, “A technology that no one has heard of it.” You are wrong, my friend — I remember when it was all the buzz at MacWorld Boston ‘96, and my boss was wondering if we should become OpenDoc developers. Ah, CyberDog, we shed a tear for you.

Anyway, Josh now works at Java Card.

The session’s audience:
15% use the Rails Trac
10-15% opened a ticket
10% contributed a patch
5% had patch accepted

Prerequisites for contributing: Rails Trac account, Subversion, subscribe to the Rails Core discussion list.

First steps:
1. Create a local, vanilla Rails projects for this work
2. svn co to vendor
3. Set up tests
4. Ensure tests run

You need to use TDD, follow code style, write docs, write tests, and ensue that the rdoc builds. Use SVN diff to generate patch. Create a Trac ticket with [PATCH] in the subject.

How to get some patch accepted
– Make tiny changes
– Discuss big patches first on Rails Core
– Need good code
– Need tests
– Get buy in for Rails developers

“Get used to disappointment. “There’s a backlog.

JRuby on Rails: A Taste of Honey (for the Enterprise) by Charles Nutter, Thomas Enebo

Wednesday, May 30th, 2007

Here’s a real advantage of a conference over email: I learned that Charles Nutter has an awesome Midwestern accent! I’ve got a whole new voice in my head while I read his posts, and there are a lot of them. Does Charles Nutter sleep? He seems to churn out emails and blog comments about JRuby 24/7.

On to the talk. For the record, I am not sure how I feel about the whole Rails running on JRuby inside a WAR on Glassfish stack at the enterprise data center. Sounds like a tall, tipsy stack of dinnerware to me. But I do like JRuby. It’s been really handy at C o n – w a y for calling EJBs, using mainframe proxies, and running JUnit test suites. And there’s a real possibility that the Java VM could become a great VM for Ruby.

This session’s poll:
25% of the audience are Java developers
25% ex-Java developers
15% ex-Perl users

Who uses which database?
40% MySQL
5% SQLite
10% MS SQL Server
20% Oracle
1% DB2

Why use JRuby instead of C Ruby?
– Unicode support
– Native JVM threads
– The big pain point …

What are the biggest differences?

1) Database support. Ruby MySQL works about the same. JRuby can use JDBC, of course, and JNDI to look up connection pools. DB2 support is not 100%? (Works pretty good for me.) Oracle ActiveRecord tests are not 100%

2) Native Ruby extensions don’t work out of the box. Need porters.

3) Command-line performance because Java start-up time is slow.

Charles and Thomas demoed Goldspike, a way to package a Rails app into a J2EE WAR:
rake war:standalone:create

And they demoed a Glassfish gem! Hm, well, theoretically, if all this just works, it could be pretty cool. Though I dread digging through the stack trace.

RESTful Modeling and Active by Record F. Morgan Whitney

Wednesday, May 30th, 2007

Vonage seems to have done a really nice job of integrating Rails with their legacy J2EE apps (cracks me up to write “J2EE apps”). This talk was somewhat of a rehash of Mapping Rails to Legacy Systems, with more details about the REST bits.

Turns out that Vonage essentially implemented Rails 2.0’s ActiveRecord six months before the Rails team.

gears