Archive for the 'OSCON' Category

Hack Your Manager by Peter Scott

Thursday, August 9th, 2007

I kept waiting for the point of this talk. It reminds me of other “business” presentations that were well-executed, and held my attention, but that never actually communicated concrete information.So, here’s the point of this talk, I think: how we deal with people are more important in our business than technical issues, and empathy is the key.Not many audience polls, but 80% of the audience gets a Dune reference.Questions:How many projects are there without people: 0%How many of you could guarantee failure of project without anyone noticing: 20%. How do you know this hasn’t happened at your company? Well, you can’t know.Managers, budgets, etc. — yeah they are lame, but you need them. You can just complain about it … or deal with it.How do you relate to managers on their level? (You don’t need to sell out.) You’ve got emotions at work, too. Most common mistake is to assume managers are interested in technology for its own sake. Don’t just hunker down. (I know this paragraph doesn’t hang together. Sorry! I really was trying to make coherent sense of all this.)This is all just about natural language processing (NLP), but the speaker asks us to not prejudge it. So:

  • You need to translate between geek speak and manager speak
  • No one really knows what anyone else is talking about
  • You have to keep asking questions

Advanced Spring by Rod Johnson

Wednesday, August 8th, 2007

The Spring man himself, Rod Johnson, gave this talk. He looks better in person than he does on this book cover.

O’Reilly-mandated audience poll:
Using Spring: 90%
Strong knowledge: 50%
Written a BeanPostProcesser: Me + someone else (not that I remember what it was for)
33% of the audience use Spring for data access.
Use Spring version 2: 40%
Use Spring version 1: 10%

Rod says: Spring is easy to customize because of its component model.

What is Spring? Well, it’s a Java framework that makes it easier to work with J2EE, and offers alternatives to the weak spots or omissions in J2EE. That’s my description. Since this was an “advanced” talk, the presenter offered:
Core components with Inversion of Control (IoC), but IoC is really just an enabler.
The triad:

  1. Consistent patterns
  2. Integration
  3. Portable (decouples your business logic from infrastructure)

Gives you: Universal POJO programming model

Spring uses good APIs abstractions like DataSource, and replace bad ones like JTA. and Spring uses configuration meta-data to wire together POJOs.

Rod shows us an example of using AOP pointcut using matching rule. Man, I don’t know. Pointcuts makes sense when I really stare at them, but they are nasty looking.

With Spring and AOP, you can add remoting to any POJO without forcing the POJO to know about it. Likewise, you are not forced to make all service classes remote. Nasty magic AOP and XML config aside, this really is pretty cool. A big practical advantage when you are working with code or writing tests.

Now we talk about FactoryBeans. Rod asks: “who can tell me what a Spring FactoryBean does?” Weak audience response. Uh oh, Rod can tell that we’re not really advanced Spring users. Contempt. (A FactoryBean creates other beans.)

Back to our feature tour programming. You can add the JMX API to you Spring beans. “Very cool.” Of course, only 5% of the audience use JMX.

Spring can use weave any bean with aspects. Example: can audit calls to particular method. Yikes, another pointcut example. I still find the syntax confusing as hell. But, the EJB interception model “sucks” because it’s not typesafe.

We start to pick up to whirlwind tour pace. Rod is cramming his 90-minute JavaOne presentation into 45 minutes.

There are (will be?) new extension points:

  • New XML tags to produce zero or note Spring bean definitions
  • Properties
  • Tags for transactions

What’s coming in Spring 2.1;

  • increased use of annotations for config. and can mix XML and annotations
  • JCA 1.5
  • Improved JPA support
  • Aims: Make Spring still easier to use

Bayeux and CometD: Open Source Comet Infrastructure by Alex Russell

Wednesday, August 8th, 2007

Bayeux and CometD: Open Source Comet Infrastructure
Alex Russell, Project Lead, SitePen

OK, this is a hard one to explain without going into a lot of background. And I felt a bit sorry for the presenter, because it appeared that half of the (small) audience was already familiar with the topic. Part of the social networking/CometD/Dojo “community.”

The presenter works on Dojo, and he’s got a great summary of CometD and Bayeaux.

Essentially, he’s trying to solve a fundamental problem of social networking websites. Each users has a page from the website in their browser. They need to send messages to each other — to each others’ browsers — but they can’t do this directly with peer-to-peer networking. Everything must go through a central server and the browsers must poll for messages.

This network architecture is difficult to scale and has high latency. High latency is bad. Users notice latency and they stop using the site.

So, what to do? Responsiveness matters, but the total amount of data isn’t a big deal.

There’s a protocol called Comet that keeps an HTTP connection open, but it relies on “brittle hacks.” (I think. Sorry if I muffed this bit.)

Bayeux is a new replacement protocol. It’s goals:

  • Browser native, no plugins
  • Easy to digest (simplicity is key, so that anybody can do use it)

What it is:

  • JSON-based
  • Hierarchical channels with simple globbing
  • Transport negotiated in handshake
  • Explicitly extensible
  • Stateless
  • Small number of verbs: handshake, connect, …

What it is not:

  • Authentication
  • Guranteed delivery

The client API is simple:
dojox.comet.publish("topic", { /* payload */});

CometD is the reference server implementation of Bayeaux, but there are more:

  • Jetty
  • WebSphere
  • Tomcat

Refactoring to Seam by Brian Leonard

Friday, July 27th, 2007

The Poll:

The thrust of the talk: take a typical Java EE 5 application (JavaServer Faces -> managed session bean -> entity class -> database) and refactor it to Seam. This is interesting, but we don’t even have any Java EE 5 web apps to refactor, so both the Java EE and the Seam stuff are new to me.

Java EE: lots of config and the syntax looks nasty after Ruby, though a big improvement over J2EE of years past, for sure.

EJB 3 and EE5

  • Dramatic simplification of all bean types
  • POJOs + annotations
  • Dependency injection

You know, I looked around at the audience at this point, and noticed that the room full of big, dumb-looking people. Not a good sign.

OK, here’s what I learned.

JBoss Seam is an application framework for JSF and EJB; it brings them together. Must be good: it has AJAX! Also has improved state management. The presenter showed this at the end and, yeah, this is good.

Seam eliminates the managed bean, reduces the XML and provides “bijection” for stateful components: dynamic, contextual, bidirectional. What? Yeah, I couldn’t figure it out either.

We get a live demo where the presenter troubleshoots various problems. Never a good idea to expect that you’ll work through issues live in presentation. Buzz kill for the audience.

Ahhh, a choice J2EE moment. The framework needs a blank property file, but if it’s not there, you get a cryptic error message that doesn’t mention property files.

We see pattern-matching to look up EJBs. Can’t recall now if that’s Java EE or Seam.

Seam replaces XML with annotations. Good. The more annotations the better I think. Looks like you can inject dependencies via annotations:
@In private User user;

Pimp My Build by Josh Cronemeyer, Andy Slocum

Friday, July 27th, 2007

This was a talk from a couple of guys from ThoughtWorks about CruiseControl, one of the first CI (Continuous Integration) tools. Martin Fowler works at ThoughtWorks! We are not worthy.

Audience Pool

90% use CI
I’m the only one who doesn’t use CruiseControl, and the presenter wants to know what I do use. “Ruby” seems to be a worthy answer. Wheh.

There are several implementations of CruiseControl — Java, .Net, Ruby — and it can be customized plugins. This talk is actually about customizing CruiseControl.

The presenters use their own ThoughtWorks project as an example. It’s a big, old Java project with 8,000 class files. So, in the ballpark of the entire Con-way application, er, applications. It takes over thirty minutes to build and test. They have about 6,000 tests, and are slightly embarrassed by how light their test coverage is.

The central problems with their CI is the volume of information and correlating it together. They solved these problems with the practical application of brute force by using Lucene to index the CruiseControl build logs (this includes source control commit comments and bug tracking numbers). It has a simple web front-end, so there’s one place to figure out which bug is fixed in which build. It was initially for developers, but business analysts like it, too.

We had a nice chat about build lights (build broke, light goes red). Apparently, is the place to go to buy one. You may remember them from endless, annoying banner ads. Other sources:

  • NaBazTag
  • Ambient Orb
  • Lava

I liked the presentation’s funky diagrams. Think Scooby Doo’s Mystery Machine van.

Suggestion: build a CI build dashboard on an old PC and monitor in your bullpen. Hey, the ARAOT is ahead of the curve!

High-performance JavaScript: Why Everything You’ve Been Taught is Wrong by Joseph Smarr

Thursday, July 26th, 2007

Javascript and performance tuning seems to pack ‘em in theses days, so this talk was standing room only. It also had the highest percentage of Macs I’ve seen at OSCON — must have been refugees from RailsConf. This was a talk about lessons learned from developing Plaxo — a very Web 2.0 community site. This talk rocked — the presenter was energized, organized, and it contained all sorts of good nuggets. My experience with AJAX and Web 2.0 mirrored Plaxo’s, and I was happy to learn how to keep my user’s browser from locking up on all the nifty drag-and-drop pages. Some of the accepted Javascript wisdom hasn’t panned-out too well for me, either. Plaxo almost never shipped, and they had to work really hard to make it fast. What happened?

  • didn’t take performance seriously from day one
  • didn’t think browser limits were significant
  • didn’t use the app daily as they were developing it
  • didn’t push back on feature requests for performance
  • didn’t value perceived performance/responsiveness

They needed to unlearn a lot of standard assumptions.Why everything you’ve been taught (AJAX euphoria! Web browsers can do anything now!) is wrong: Browsers are pushed out of their comfort zone; Javascript is parsed and interpreted every time you load the page.

The Plaxo mantra: Be lazy, responsive, pragmatic, and vigilant.


  • Noting is faster than doing nothing
  • Write less code
    • Javascript is parsed every time it’s loaded, and Javascript is downloaded a lot, so keep it under 500K of code — this seems to be a standard best practice. Be careful of third-party libraries — they can add a lot of code.
  • Draw the UI as late as possible
    • Never pre-draw hidden UI elements
    • Cache HTML if appropriate
    • Don’t keep hidden UI up-to-date
    • Consider just redrawing everything versus carefully updating parts of the page

Responsive: When the user says, “jump,” you jump!

  • The initial page load is very important to user
    • CSS at top of page and Javascript at bottom. (News to me. Good idea.)
  • Draw placeholders right away
  • Yield early and often
    • use setTimeout(0) to yield
    • use a closure
    • use onmousedown() instead of onclick()
  • Cache backend responses
    • send all data requests through data manager code
    • use range caches and fill in missing pieces
    • balance local updates vs. re-fetching from APIs

At this point, it really struck me that that a lot of these points are basic UI practices from any UI toolkit like Swing, VB, etc. It’s not like caching and quickly responding to user requests with some UI feedback are revolutionary ideas. I’m all in favor of leaving out optimizations like caching until you really need them, but I’m, uh, surprised that this was such a surprise for our friends at Plaxo. Same goes for practices like eating your own dog food.

Pragmatic: Don’t make things even harder

  • Play to the browser’s strengths
    • avoid DOM, user innerHtml
    • avoid dynamic CSS-class definitions
    • avoid reflow when possible
    • avoid memory allocation like string splitting
    • do DOM manipulation off-DOM, then reinsert at the ends
  • Cheat when you can
    • Use global functions when reasonable (sacre bleu!)
    • Finding by CSS class is slow
    • Directly attach onclick, etc. handlers instead of using event listeners
  • If you have to do it every time, do it statically

Vigilant: Only you can prevent slow web apps

  • Profile
  • Use Firebug (I agree, it rocks!)
  • Measure with a consistent environment
    • browsers bog down over time, you need to restart them for accurate numbers.
  • Consider performance from day one
    • apps get slow when you make them do a lot of things
    • consider how much code is needed
    • make sure everyone remembers how important speed is

Summary: Make the browser happy. Web browsers are more like mobile phones, not a desktop. They are a flimsy, temperamental platform, but everyone’s got one, so it’s the best place to be. Don’t push the limits unless you have to.

Network Monitoring with Nagios by Taylor Dondich

Thursday, July 26th, 2007

Nagios is a network monitor. I use it, and I like it better than others that I’ve tried — BigSister and some nasty Zope-based thing. By “using it,” I mean installing the BSD port and editing the config files in Emacs so that I would get an email when Apache fell over. I figure that there is probably more to Nagios.

Today’s audience poll:
50% use Nagios
10% are C programers
40% use other scripting languages

The presenter is the author of the O’Reilly Nagios book, and also wrote the Groundwork Fruity configuration plugin.

So, Nagios is about state monitoring: What’s about to break? What’s broke?

Can monitor network services (SMTP, POP, HTTP) and host resources (CPU load, disk/memory usage). It also does notification, escalation, has event handlers, and has a web interface.

10% of the audience monitor 100+ hosts; 5% monitor 300+ hosts. I monitor five servers. Apparently, the web UI falls down around several hundred hosts.

The presenter made a point of listing Nagios’ weak points, and said the he wished that other presenters would point out weak points, too. I agree — we seem to focus on features when we talk about open source. The Nagios weak points:

  • initial config
  • scalability
  • web interface
  • notifications/escalations

80% of users in the audience configured Nagios by hand, like me. This is hard! There are better ways:

  • Monarch (Perl)
  • Fruity
  • Nagat

And some recommended viewer plugins:

  • Nagcon (console based)
  • Nagios Checker (Firefox ext)
  • Groundwork Solutions (wrapped)

The Coming XQuery + XForms XPlosion by Kurt Cagle

Wednesday, July 25th, 2007

Now before I pan this talk, let me say that Kurt strikes me as one of those guys whose hard work with open source and standards bodies makes things go. OK?

Things started out just fine, with some rambling and ranting about how the open source software developer crowd is faddish. “XML is staid? I’m a graybeard because I work in XML? What?” Funny. True!

News flash: everything is more complicated now, mainly because we aren’t just running on a disconnected box under the desk.

Halfway through the talk, though, the presenter is still ranting! Wait, the point? Maybe we need to simplify the models we use to combat complexity.

  • Web enforces simplicity
  • Traditional mid-tier apps break down on the web because they assume synchronous interactions
  • Simplest arch pattern is an old one — MVC

Ten minutes later, a question from the audience: are we going to talk about XQuery? Oh, yeah, OK.

There are many emerging XML standards:

  • XHTML + XForms
  • XSLT
  • XQuery
  • Schematron
  • XBL

XForms started as replacement for HTML forms, but became more. They create an XML model that uses XPath to “bind” control to various parts of model. XForm is sort of like a VB form. Did you just cringe? I did.

XForm implementations to watch

  • Mozilla (presenter’s favorite)
  • Orbeon
  • Chiba
  • Formsplayer
  • PicoForm
  • OpenOffice

Maybe XForms will be the Next Big Thing, but I’m not holding my breath.

Performance Whack-a-Mole by Josh Berkus

Wednesday, July 25th, 2007

This turned out to be a good talk, chock-full of concrete practical advice.

Here’s our audience poll:
95% use DB apps
15% are consultants (like the presenter)

Clients just say: “the database is slow,” but anything in the big application stack (application, middleware, database, operating system, hardware_ can make things slow. Every layer in the stack needs do its job to scale the application. Otherwise, we are leaving significant performance gains on the table. In fact, “database” problems are usually somewhere else — like in the middleware.

You have many performance problems in your application, but most aren’t significant. Performance is usually degraded by a few problems — 10% of problems cause 90% of slowdown. (And big problems mask smaller problems.)

Apps tune differently. The presenter uses three rough categories: web, online transaction processing, data warehouse.

  • Entire database can fit in RAM
  • 90% or more of database queries are simple reads
  • CPU-bound
  • Performance moles: caching, pooling, connection time


  • Database slightly larger than RAM to 1TB
  • Many small data write queries
  • CPU or I/O bound
  • Moles: locks, cache, transactions, write speed, log

Data Warehouse

  • 100GB to 100TB database
  • Large complex reporting queris
  • Large bulk loads of data
  • Also called “business intelligence”
  • Moles: Sequential scans, resources, bad queries

Your first step should be to measure a baseline. Check basic setup (versions, configurations) for sanity. Then measure each layer.


  • OS tools are simple, easy to use, and non-invasive
    • ps, mpstat
  • Benchmark. Invasive
    • micorbenchmarkL bonnie++ for files system perf
  • DB
    • DBTrace
    • DB query log
    • Query analysis (troublehoot “bad” queries)
    • ogbench, Wisconsin, TPCB, OSDB, PolePosition
  • Application
    • app server tools
    • workload simulation and screen scraping
  • lwp and log replay tools
  • bug detectors
  • valgrind, MDB, GDB, DTrace

OK, we’ve got the baseline, now what? What are the symptoms; when do they occur?

The Performance Moles


  • Symptoms: one device saturating I/O — other resources are OK
  • heavy writes or very large DB
  • causes
    • bad I/O hardware, software or configuration
    • not enough RAM
    • too much data requested
    • bad schema


  • Symptoms: maxed-out CPU, but RAM available
  • mostly-read loads or complex calculations
  • causes
    • too many queries
    • insufficient caching or pooling
    • too much data requested
    • bad queries or schema
  • Note: most DB servers should be CPU-bound at max load

Locking Mole

  • Symptoms: Resource aren’t maxed-out
  • lots of pessimistic locking
  • causes
    • long-running transaction or stored procedures
    • longly held open cursors
    • unneeded pessimistic locking
    • poor transaction management

App Mole

  • Symptoms: DB server maxed, app server is OK
  • common in J2EE
  • causes
    • not enough app servers
    • too much data/ too many queries
    • bad caching.pooling
    • driver issues
    • ORM frameworks

Open Source Geospatial Boot Camp by Schuyler Erle

Wednesday, July 25th, 2007

Holy fast talking presenter, Batman!

Today’s audience poll:
Used open source software: 95%
Comfortable with the command line: 90%
Used open source GIS software: 20%

We ran though some GIS/cartography terms that are handy to know when doing GIS.

Did you know? The Earth is not a sphere. In fact, It’s an irregular ellipsoid. There are many different standards for modelling the Earth. You should use WGS 1984 as your ellipsoid model.

Projection: trade-off between conformal (preserve angle) and equidistant (distance) and equivalent (area). Got it? We have to make trade-offs because we are translating a sphere, er, ellipsoid onto a flat surface.

    There are lots of GIS data formats

  • OS tools help hide this
  • GEOTiff (raster)
  • ESRI Shapefile (vector)
  • GML (XML)

Open source tool: Quantum GIS — useful for browsing GIS data. Looks like PostgreSQL, at least, has good GIS support.

You’d think I’d have gained more information from this talk. Unfortunately, the presenter spent lots of time fiddling with stuff to make it work. This “show you how it is with GIS!” “It’s fun!” Well … I don’t know. I do think that this is how it with GIS. It’s certainly been my experience with my GPS and mapping software. Almost the antithesis of what I like: polished, clean interfaces, adherence to conventions. Makes me think that GIS is not for me.