Dumb Thing I Did Recently With FreeBSD

October 30th, 2007

The point: if you have odd problems with your new computer, check for IRQ conflicts.

That’s been good advice for, what, twenty years? Anyway, I forgot it. Here’s what happened in case you’re as clueless as me.

I put together a new server for my “data center” — in the corner of the basement near the washer and the spare bicycle wheels. It’s spec’ed more or less from the Ars Technica mid-level system guide: Asus motherboard, AMD 64 CPU, SATA drives. I put FreeBSD on it.

Or, I tried to. Everything worked fine until I rsync’ed backup files from the old server, then boom! kernel panic and reboot. OK, I Googled about and learned that the ‘nfe’ network driver is recommended over the default ‘nve’ driver. Fair enough. That seemed to work better, though I still had many Ethernet transaction errors and watchdog timeouts. My connectivity was laggy and prone to drops.

My hub was 10BaseT only. I think I bought it right after I cancelled my Compuserve account. I replaced the hub with a nice Gigabit switch, and everything was great.

Well, better. It bugged me that my NIC only worked without the reccomended patch. I still saw an occasional warning in the log. And my SSH sessions dropped more than they should. I tried an old 100 MHz PCI NIC, and that worked, so I figured it was just bad hardware. I looked for something better than the onboard Intel NIC. Turns out, the onboard NIC is regarded as a good one, and when I tried another brand, it didn’t work at all.

At this point, I decided to live with the situation. That was OK until I installed Gnome so that I could run nightly Selenium tests. Boom. Panic every night when network backups started. I switched out the DSL router. I moved the server next to the router and replaced the 50-foot “data center” cable with a new short one. I removed all other computers from the wired network.

More panics. Hundreds of oversize frame errors n startup. Not just slightly above MTU size, but like 10,000. Huh?

At this point, I considered donating the motherboard and RAM to FreeGeek and just sucking it up and replacing all of it. I when through the messages log carefully and there it was: USB and NIC on the same IRQ.

My theory is that some Gnome-related daemons probe the USB ports, and this was interpreted as Ethernet traffic. I obvioulsy don’t really know crap about hardware though. In any case, I disabed USB in the BIOS and nary a problem now.

Ignite Portland

October 26th, 2007

OK, full disclosure: I’m just bumping the previous post (irritable OSCON talk summary) down. Tired of it.

I checked out Ignite Portland last night because Mike Jones was giving a talk about unicycles. I now want to work in the Wieden+Kennedy office, if not Wieden+Kennedy.

The whole night was slick and well-run, and drew a big crowd. So funny what kind of geeky thing will be a hit in Portland.

I liked the idea of Rewilding. Does that include removing all the dams on the Columbia and reintroducing megafauna to the Great Plains? It should.

Not sure if I’m going to pick up knitting, Audrey, but Cheryl will like the knitting sites. Halfway through the Chickens talk, I was convinced that I am depriving my daughter of a full life because we aren’t raising poultry in the backyard. I will try and make amends by cooking here an authentic Southern dinner of collard greens and tofu with gravy.

Hack Your Manager by Peter Scott

August 9th, 2007

I kept waiting for the point of this talk. It reminds me of other “business” presentations that were well-executed, and held my attention, but that never actually communicated concrete information.So, here’s the point of this talk, I think: how we deal with people are more important in our business than technical issues, and empathy is the key.Not many audience polls, but 80% of the audience gets a Dune reference.Questions:How many projects are there without people: 0%How many of you could guarantee failure of project without anyone noticing: 20%. How do you know this hasn’t happened at your company? Well, you can’t know.Managers, budgets, etc. — yeah they are lame, but you need them. You can just complain about it … or deal with it.How do you relate to managers on their level? (You don’t need to sell out.) You’ve got emotions at work, too. Most common mistake is to assume managers are interested in technology for its own sake. Don’t just hunker down. (I know this paragraph doesn’t hang together. Sorry! I really was trying to make coherent sense of all this.)This is all just about natural language processing (NLP), but the speaker asks us to not prejudge it. So:

  • You need to translate between geek speak and manager speak
  • No one really knows what anyone else is talking about
  • You have to keep asking questions

Advanced Spring by Rod Johnson

August 8th, 2007

The Spring man himself, Rod Johnson, gave this talk. He looks better in person than he does on this book cover.

O’Reilly-mandated audience poll:
Using Spring: 90%
Strong knowledge: 50%
Written a BeanPostProcesser: Me + someone else (not that I remember what it was for)
33% of the audience use Spring for data access.
Use Spring version 2: 40%
Use Spring version 1: 10%

Rod says: Spring is easy to customize because of its component model.

What is Spring? Well, it’s a Java framework that makes it easier to work with J2EE, and offers alternatives to the weak spots or omissions in J2EE. That’s my description. Since this was an “advanced” talk, the presenter offered:
Core components with Inversion of Control (IoC), but IoC is really just an enabler.
The triad:

  1. Consistent patterns
  2. Integration
  3. Portable (decouples your business logic from infrastructure)

Gives you: Universal POJO programming model

Spring uses good APIs abstractions like DataSource, and replace bad ones like JTA. and Spring uses configuration meta-data to wire together POJOs.

Rod shows us an example of using AOP pointcut using matching rule. Man, I don’t know. Pointcuts makes sense when I really stare at them, but they are nasty looking.

With Spring and AOP, you can add remoting to any POJO without forcing the POJO to know about it. Likewise, you are not forced to make all service classes remote. Nasty magic AOP and XML config aside, this really is pretty cool. A big practical advantage when you are working with code or writing tests.

Now we talk about FactoryBeans. Rod asks: “who can tell me what a Spring FactoryBean does?” Weak audience response. Uh oh, Rod can tell that we’re not really advanced Spring users. Contempt. (A FactoryBean creates other beans.)

Back to our feature tour programming. You can add the JMX API to you Spring beans. “Very cool.” Of course, only 5% of the audience use JMX.

Spring can use weave any bean with aspects. Example: can audit calls to particular method. Yikes, another pointcut example. I still find the syntax confusing as hell. But, the EJB interception model “sucks” because it’s not typesafe.

We start to pick up to whirlwind tour pace. Rod is cramming his 90-minute JavaOne presentation into 45 minutes.

There are (will be?) new extension points:

  • New XML tags to produce zero or note Spring bean definitions
  • Properties
  • Tags for transactions

What’s coming in Spring 2.1;

  • increased use of annotations for config. and can mix XML and annotations
  • JCA 1.5
  • Improved JPA support
  • Aims: Make Spring still easier to use

Bayeux and CometD: Open Source Comet Infrastructure by Alex Russell

August 8th, 2007

Bayeux and CometD: Open Source Comet Infrastructure
Alex Russell, Project Lead, SitePen

OK, this is a hard one to explain without going into a lot of background. And I felt a bit sorry for the presenter, because it appeared that half of the (small) audience was already familiar with the topic. Part of the social networking/CometD/Dojo “community.”

The presenter works on Dojo, and he’s got a great summary of CometD and Bayeaux.

Essentially, he’s trying to solve a fundamental problem of social networking websites. Each users has a page from the website in their browser. They need to send messages to each other — to each others’ browsers — but they can’t do this directly with peer-to-peer networking. Everything must go through a central server and the browsers must poll for messages.

This network architecture is difficult to scale and has high latency. High latency is bad. Users notice latency and they stop using the site.

So, what to do? Responsiveness matters, but the total amount of data isn’t a big deal.

There’s a protocol called Comet that keeps an HTTP connection open, but it relies on “brittle hacks.” (I think. Sorry if I muffed this bit.)

Bayeux is a new replacement protocol. It’s goals:

  • Browser native, no plugins
  • Easy to digest (simplicity is key, so that anybody can do use it)

What it is:

  • JSON-based
  • Hierarchical channels with simple globbing
  • Transport negotiated in handshake
  • Explicitly extensible
  • Stateless
  • Small number of verbs: handshake, connect, …

What it is not:

  • Authentication
  • Guranteed delivery

The client API is simple:
dojox.comet.publish("topic", { /* payload */});

CometD is the reference server implementation of Bayeaux, but there are more:

  • Jetty
  • WebSphere
  • Tomcat

Refactoring to Seam by Brian Leonard

July 27th, 2007

The Poll:

The thrust of the talk: take a typical Java EE 5 application (JavaServer Faces -> managed session bean -> entity class -> database) and refactor it to Seam. This is interesting, but we don’t even have any Java EE 5 web apps to refactor, so both the Java EE and the Seam stuff are new to me.

Java EE: lots of config and the syntax looks nasty after Ruby, though a big improvement over J2EE of years past, for sure.

EJB 3 and EE5

  • Dramatic simplification of all bean types
  • POJOs + annotations
  • Dependency injection

You know, I looked around at the audience at this point, and noticed that the room full of big, dumb-looking people. Not a good sign.

OK, here’s what I learned.

JBoss Seam is an application framework for JSF and EJB; it brings them together. Must be good: it has AJAX! Also has improved state management. The presenter showed this at the end and, yeah, this is good.

Seam eliminates the managed bean, reduces the XML and provides “bijection” for stateful components: dynamic, contextual, bidirectional. What? Yeah, I couldn’t figure it out either.

We get a live demo where the presenter troubleshoots various problems. Never a good idea to expect that you’ll work through issues live in presentation. Buzz kill for the audience.

Ahhh, a choice J2EE moment. The framework needs a blank property file, but if it’s not there, you get a cryptic error message that doesn’t mention property files.

We see pattern-matching to look up EJBs. Can’t recall now if that’s Java EE or Seam.

Seam replaces XML with annotations. Good. The more annotations the better I think. Looks like you can inject dependencies via annotations:
@In private User user;

Pimp My Build by Josh Cronemeyer, Andy Slocum

July 27th, 2007

This was a talk from a couple of guys from ThoughtWorks about CruiseControl, one of the first CI (Continuous Integration) tools. Martin Fowler works at ThoughtWorks! We are not worthy.

Audience Pool

90% use CI
I’m the only one who doesn’t use CruiseControl, and the presenter wants to know what I do use. “Ruby” seems to be a worthy answer. Wheh.

There are several implementations of CruiseControl — Java, .Net, Ruby — and it can be customized plugins. This talk is actually about customizing CruiseControl.

The presenters use their own ThoughtWorks project as an example. It’s a big, old Java project with 8,000 class files. So, in the ballpark of the entire Con-way application, er, applications. It takes over thirty minutes to build and test. They have about 6,000 tests, and are slightly embarrassed by how light their test coverage is.

The central problems with their CI is the volume of information and correlating it together. They solved these problems with the practical application of brute force by using Lucene to index the CruiseControl build logs (this includes source control commit comments and bug tracking numbers). It has a simple web front-end, so there’s one place to figure out which bug is fixed in which build. It was initially for developers, but business analysts like it, too.

We had a nice chat about build lights (build broke, light goes red). Apparently, x10.com is the place to go to buy one. You may remember them from endless, annoying banner ads. Other sources:

  • NaBazTag
  • Ambient Orb
  • Lava

I liked the presentation’s funky diagrams. Think Scooby Doo’s Mystery Machine van.

Suggestion: build a CI build dashboard on an old PC and monitor in your bullpen. Hey, the ARAOT is ahead of the curve!

High-performance JavaScript: Why Everything You’ve Been Taught is Wrong by Joseph Smarr

July 26th, 2007

Javascript and performance tuning seems to pack ‘em in theses days, so this talk was standing room only. It also had the highest percentage of Macs I’ve seen at OSCON — must have been refugees from RailsConf. This was a talk about lessons learned from developing Plaxo — a very Web 2.0 community site. This talk rocked — the presenter was energized, organized, and it contained all sorts of good nuggets. My experience with AJAX and Web 2.0 mirrored Plaxo’s, and I was happy to learn how to keep my user’s browser from locking up on all the nifty drag-and-drop pages. Some of the accepted Javascript wisdom hasn’t panned-out too well for me, either. Plaxo almost never shipped, and they had to work really hard to make it fast. What happened?

  • didn’t take performance seriously from day one
  • didn’t think browser limits were significant
  • didn’t use the app daily as they were developing it
  • didn’t push back on feature requests for performance
  • didn’t value perceived performance/responsiveness

They needed to unlearn a lot of standard assumptions.Why everything you’ve been taught (AJAX euphoria! Web browsers can do anything now!) is wrong: Browsers are pushed out of their comfort zone; Javascript is parsed and interpreted every time you load the page.

The Plaxo mantra: Be lazy, responsive, pragmatic, and vigilant.


  • Noting is faster than doing nothing
  • Write less code
    • Javascript is parsed every time it’s loaded, and Javascript is downloaded a lot, so keep it under 500K of code — this seems to be a standard best practice. Be careful of third-party libraries — they can add a lot of code.
  • Draw the UI as late as possible
    • Never pre-draw hidden UI elements
    • Cache HTML if appropriate
    • Don’t keep hidden UI up-to-date
    • Consider just redrawing everything versus carefully updating parts of the page

Responsive: When the user says, “jump,” you jump!

  • The initial page load is very important to user
    • CSS at top of page and Javascript at bottom. (News to me. Good idea.)
  • Draw placeholders right away
  • Yield early and often
    • use setTimeout(0) to yield
    • use a closure
    • use onmousedown() instead of onclick()
  • Cache backend responses
    • send all data requests through data manager code
    • use range caches and fill in missing pieces
    • balance local updates vs. re-fetching from APIs

At this point, it really struck me that that a lot of these points are basic UI practices from any UI toolkit like Swing, VB, etc. It’s not like caching and quickly responding to user requests with some UI feedback are revolutionary ideas. I’m all in favor of leaving out optimizations like caching until you really need them, but I’m, uh, surprised that this was such a surprise for our friends at Plaxo. Same goes for practices like eating your own dog food.

Pragmatic: Don’t make things even harder

  • Play to the browser’s strengths
    • avoid DOM, user innerHtml
    • avoid dynamic CSS-class definitions
    • avoid reflow when possible
    • avoid memory allocation like string splitting
    • do DOM manipulation off-DOM, then reinsert at the ends
  • Cheat when you can
    • Use global functions when reasonable (sacre bleu!)
    • Finding by CSS class is slow
    • Directly attach onclick, etc. handlers instead of using event listeners
  • If you have to do it every time, do it statically

Vigilant: Only you can prevent slow web apps

  • Profile
  • Use Firebug (I agree, it rocks!)
  • Measure with a consistent environment
    • browsers bog down over time, you need to restart them for accurate numbers.
  • Consider performance from day one
    • apps get slow when you make them do a lot of things
    • consider how much code is needed
    • make sure everyone remembers how important speed is

Summary: Make the browser happy. Web browsers are more like mobile phones, not a desktop. They are a flimsy, temperamental platform, but everyone’s got one, so it’s the best place to be. Don’t push the limits unless you have to.

Network Monitoring with Nagios by Taylor Dondich

July 26th, 2007

Nagios is a network monitor. I use it, and I like it better than others that I’ve tried — BigSister and some nasty Zope-based thing. By “using it,” I mean installing the BSD port and editing the config files in Emacs so that I would get an email when Apache fell over. I figure that there is probably more to Nagios.

Today’s audience poll:
50% use Nagios
10% are C programers
40% use other scripting languages

The presenter is the author of the O’Reilly Nagios book, and also wrote the Groundwork Fruity configuration plugin.

So, Nagios is about state monitoring: What’s about to break? What’s broke?

Can monitor network services (SMTP, POP, HTTP) and host resources (CPU load, disk/memory usage). It also does notification, escalation, has event handlers, and has a web interface.

10% of the audience monitor 100+ hosts; 5% monitor 300+ hosts. I monitor five servers. Apparently, the web UI falls down around several hundred hosts.

The presenter made a point of listing Nagios’ weak points, and said the he wished that other presenters would point out weak points, too. I agree — we seem to focus on features when we talk about open source. The Nagios weak points:

  • initial config
  • scalability
  • web interface
  • notifications/escalations

80% of users in the audience configured Nagios by hand, like me. This is hard! There are better ways:

  • Monarch (Perl)
  • Fruity
  • Nagat

And some recommended viewer plugins:

  • Nagcon (console based)
  • Nagios Checker (Firefox ext)
  • Groundwork Solutions (wrapped)

The Coming XQuery + XForms XPlosion by Kurt Cagle

July 25th, 2007

Now before I pan this talk, let me say that Kurt strikes me as one of those guys whose hard work with open source and standards bodies makes things go. OK?

Things started out just fine, with some rambling and ranting about how the open source software developer crowd is faddish. “XML is staid? I’m a graybeard because I work in XML? What?” Funny. True!

News flash: everything is more complicated now, mainly because we aren’t just running on a disconnected box under the desk.

Halfway through the talk, though, the presenter is still ranting! Wait, the point? Maybe we need to simplify the models we use to combat complexity.

  • Web enforces simplicity
  • Traditional mid-tier apps break down on the web because they assume synchronous interactions
  • Simplest arch pattern is an old one — MVC

Ten minutes later, a question from the audience: are we going to talk about XQuery? Oh, yeah, OK.

There are many emerging XML standards:

  • XHTML + XForms
  • XSLT
  • XQuery
  • Schematron
  • XBL

XForms started as replacement for HTML forms, but became more. They create an XML model that uses XPath to “bind” control to various parts of model. XForm is sort of like a VB form. Did you just cringe? I did.

XForm implementations to watch

  • Mozilla (presenter’s favorite)
  • Orbeon
  • Chiba
  • Formsplayer
  • PicoForm
  • OpenOffice

Maybe XForms will be the Next Big Thing, but I’m not holding my breath.