Wednesday, August 31, 2005

I moved to a much faster server

A few people noted that our recipe web portal occasionally ran slowly; it was running on a very low end virtual linux server during development. I just switched the DNS records so now points to a much, much faster server.


Tuesday, August 30, 2005

MySQL optimizations; free Opera browser license

OK, I know that there is a large overhead for doing single table inserts, even maintaining a pool of connections. This little fact was very obvious this morning: I had a one-shot little program that generates data for about 5 million rows in a database table. The calculations themselves take a while (about 5 minutes on my PC), ignoring database access. Last night, right before I went to bed, I added a few lines of ad-hoc JDBC calls to write out my 5 million rows after everything was calculated in memory. I woke up this morning and it was still running.

So much for being a little sloppy, even for a one-shot (throw it away after it runs) program. I re-wrote it batching up just 2000 inserts at a time (insert into... values (..), (..), (..)...) and the database access code ran in just 20 minutes. After the data was added, it took another two minutes to index the table (obviously you want to do this *after* adding the data). I made a note to myself to review my production code - I think that I almost always batch up inserts when appropriate, but I want to make doubly sure!

re: free Opera browser license: as seen on Slashdot: a short term free license giveaway. I grabbed a free license so I have one more popular browser to test my web applications against.

My IntelliJ review was not well received

Oh well. An editor at DevX asked me to write a review of IntelliJ 5.0. I had never written a product review before but I do *like* IntelliJ, so I thought 'why not?'. Anyway, the reader ranking for the review is low - I wish that people could leave comments because it would be good to get feedback.

Monday, August 29, 2005

Great book on DHTML/Javascript

One of my customers asked me to "beef up" on using Javascript, AJAX, etc. I have been reading a great book: DHTML Utopia.

Usually, I dislike working in Javascript but this book has really turned me on to some good techniques. If your web applications are running on a local area network (low latency), doing every thing on the server is fairly OK but it is better to do some things locally on the client browser when running over the internet. Human usability studies have shown how productivity-destroying it is to make users wait for more than a second or so when they are working.

I especially like the way Stuart sticks to 'the standards'. Personally, I refuse to spend too much time worrying about Internet Explorer strangeness (unless someone is paying me to - even then I don't like it).

We are rebranding our new web portal to

CJ are my wife's first and middle initials, and a nick name. For about 7 years, I used my site for technology demos - a low level of effort. I recently decided to invest in the creation of three public service (i.e., free, but I will probably add a few Adsense ads eventually) web portals:
  • - our knowledge based cooking and recipe portal. Until this morning, this was our site. My wife likes the name change :-)
  • - a web portal (under construction) for announcing local events, searching for events by ZIP code and distance from where you live, etc. This is also an experiment in architectures for scaling for large numbers of users: I have a budget for how much server costs I am willing to donate for this.
  • - a news clipping service that generates customized RSS feeds. (under construction)
I love working on interactive web portals; good thing since this is also how I spend most of my time when consulting for customers.

I mostly enjoy the rapid feedback: sometimes I can almost have a change to a web application done before I get off of the telephone with someone requesting an enhancement or bug fix. While I sometimes think that I would rather be working in a more dynamic environment (like Ruby on Rails, or UnCommon Web for Common Lisp), my development environment rocks: IntelliJ working with JavaBeans, JSPs, custom tag libraries, and some Javascript and CSS. My edit/build/redeploy cycle is so quick that very little of my working time is spent waiting.

The other great thing about web portals is the low cost of hosting. Running Tomcat or JBoss is fairly light weight and efficient. Eventually, all dynamic code gets compiled and run through HotSpot, so the performance is good also (even on the really low end servers that I usually deploy to).

Sunday, August 28, 2005

The "Mom software modification"

My Dad (interesting guy: in his mid 80's and still a member of the US National Academy of Sciences) called with the bad news: my Mom did not like the way food ingredients were showing up on the knowledge-intensive recipe web portal that my wife and I are working on.

Apparently my Mom did *not* like seeing "0.5 cups" instead of "1/2 cup".

Fortunately, I did not have to change anything in the model or controller layers - just one small bit of code in my presentation layer had to be changed.

BTW, for people interested in artificial intelligence: check out the "AI agent" page at

Great George Orwell quotes

Thanks to reddit I discovered the site Here are some great quotes from George Orwell (about 5% of those on the linked site):

Every war when it comes, or before it comes, is represented not as a war but as an act of self-defense against a homicidal maniac.

In a time of universal deceit - telling the truth is a revolutionary act.

All political thinking for years past has been vitiated in the same way. People can foresee the future only when it coincides with their own wishes, and the most grossly obvious facts can be ignored when they are unwelcome.

All the war-propaganda, all the screaming and lies and hatred, comes invariably from people who are not fighting.

The nationalist not only does not disapprove of atrocities committed by his own side, but he has a remarkable capacity for not even hearing about them.

He was an embittered atheist, the sort of atheist who does not so much disbelieve in God as personally dislike Him.

War against a foreign country only happens when the moneyed classes think they are going to profit from it.

When it comes to the pinch, human beings are heroic.

Saturday, August 27, 2005

Fast progress on

I am jazzed: except for the "AI agent recipe creator", the software for the KBrecipe knowledge intensive recipe portal is just about done. Mix a little Java, little AI...

My wife and I have a strong interest in both cooking and healthy food; this portal is our gesture at doing a public service :-)

If you have a few minutes, please try it out - I am interested in feedback. The implementation just uses some standard stuff: JSPs, struts, and a custom database layer (no object relational mapping). My wife and I will hopefully get more recipes entered into the system in the next few days - right now there are only 8 recipes in the system for software testing.

Friday, August 26, 2005

Great (and free) Lisp programming environment

It is good to know more than one programming language: Common Lisp is one of my favorite languages and I often encourage other programmers to learn it. Peter Seibel (author of " Practical Common Lisp" and well known in the Lisp community) has put together a complete programming environment LispBox that integrates a free Lisp development system (CLisp, OpenMcl, or AllegroLisp) for Windows, OS X, and Linux. He included Emacs, Slime, and ASDF (a package system for automatically loading systems from the web).

Neat stuff and a huge time saver if you want to set up your own Lisp development system.

Efficient capital markets

With only a few caveats, I believe in globalization and free market economics. Production and all work should be done in the most efficient way possible, factoring in long term environmental damage to production costs. I just read an interesting guest editorial in the Asia Times (taken from the author's informative and useful blog) that makes a good point about efficiency in international markets: that a global economy driven by tough efficiency will naturally evolve away from using the US dollar for oil and other commodity trading when using the dollar adds cost to all involved (except for the US).

Unfortunately, I think that my country (the US) will continue to take strident and largely ineffective steps to try to prop up the dollar as the primary currency for buying and selling commodities. I believe that it is better for us to concentrate on efficiency through an educated work force and investments in infrastructure than to try to sustain a very good deal that we have had since the early 1970s (i.e., getting off the gold standard, float the dollar, and deals to make the dollar the currency for buying and selling oil). This was a great deal, we should appreciate our good fortune, but realize that this good deal will not last forever.

Even with the current tragedy in Iraq, I am as hopeful for the future of the world as I have been in a long time. I believe that efficient world markets with tightly intertwined economies will bind people together and hopefully lead to more peace and world wide prosperity.

Monday, August 22, 2005

beta version of is available

My wife and I have been working on a public service 'healthy recipes' web portal. For the very brave, a beta version is available at:

I just updated my database schema and data to incorporate some of the USDA nutritional database. After the next software update in a few days you will see nutritional data associated with each recipe.

You can use the portal without logging in, but you will be missing a lot of features: you can maintain a private database of food ingredients that you have on hand and optionally only request recipes for which you can make with what you have in your kitchen.

There are few recipes available right now - I am mostly working on the infrastructure before my wife and I start entering more recipe data into the system.

Google: Desktop 2 beta, $4B for implementing my idea

I tried out Google Desktop version 2 beta after getting home from a friend's birthday dinner late Sunday night: it has a high 'cool factor', but I might tire of the sidebar (that said, toggling between sidebar and taskbar placement is just a mouse click - so, you can get it out of sight quickly while working). Anyway, I think that having their own desktop fits in really well with their information management (and probably in the future knowledge management) business model.

With their new stock sale, Google has plenty of money for internal development and for purchasing tiny technology companies (mostly to get the people).

Like many people, I have an idea for how Google should spend their money :-)

I think that it would be very interesting (but a huge risk) to try writing a Windows/Linux replacement desktop that would act the same on both systems.

For Windows, they would provide a new shell (other people have done this) that would integrate file browsing, search, program launch, command shell (of course!!), FireFox based browsing, etc.

For Linux, the implementation would be a new Window Manager.

In addition to an expensive development effort, there are a few other large problems to be solved:
  • Microsoft would try to torpedo the Windows shell replacement - it would be very bad for their business model.
  • It would be difficult to support the multitude of Linux distributions
One approach for might be to have a Google Linux distribution, but that is really moving away from their information management business model. To make this work, their complete desktop replacement would have to work really well in corporate environments.

Sunday, August 21, 2005

Windows vs. Linux on Slashdot; plain text

I have a little free time before a friend's (who has a picture of himself on stage at Woodstock with Hendrix :-) birthday party tonight - so, I treated myself to some low productivity time on Slashdot.

Another re-hashing of problems with Linux and Windows, but there was something interesting to me: the standard complaints about Microsoft's technically weak, but money making shifting file formats.

Time to step back: why do we write down information? Mostly to record thoughts and information we or other people will need later. I think that formatted text is way overrated.

Notes, memos, short users' guides, email, source code, design notes, requirements, use cases, etc. can all be expressed quite well as plain text files.

Plain text files are easier to store in CVS (even with subversion's binary diff'ing, plain text works best in subversion also).

I have never had problems reading a ten year old plain text file.

Sometimes a little formatting is good - XHTML (or HTML) with limited markup is easy to edit and can be presented nicely with CSS.

I am not just bashing Microsoft Office here: how many people or organizations really need things like DocBook?

If you must write formatted text artifacts, at least use for the transparent file formats.

If someone sends you memos as Word documents, try asking for plain text - convert someone to a better way of working.

Saturday, August 20, 2005

I upgraded to version 5 of IntelliJ

As I just wrote in a review for DevX: JetBrains is very smart to give a short term price promotion for the update because they are their own best competitor: I found IntelliJ 4.5 just about perfect, so without the deep discount (until the end of August) I probably would not have bought the upgrade. For me, the nicest improvements are in Javascript editor support.

There are better programming languages than Java (pick a dynamic language: Common Lisp, Ruby, Smalltalk, Python Scheme, etc.) but there are three things that keep me focused on Java:
  • The quality of IDEs like IntelliJ, Eclipse, and NetBeans
  • The great (and free!) infrastructure software for web portals (Tomcat, JBoss, etc.)
  • Most of my customers insist that I use Java

Friday, August 19, 2005

OK, HSQLDB is cool after all

I complained a week ago about stability problems when using HSQLDB as an embedded database engine. By using a JVM shutdown hook, I totally solved my file locking problems.

Using JVM shutdown hooks is easy; first define a class that extends the Thread class:
class MyShutdown extends Thread {
public MyShutdown() { super(); }
public void run() {
System.out.println("\n\n*** MyShutDown thread started ***\n");
try {
} catch (Exception ee) { ee.printStackTrace(); }
Here I am assuming that my JDBC access utilities are in a class DbUtil that has a static method close_connections() for sending HSQLDB a "SHUDOWN COMPACT" command.

Registering a JVM shutdown hook is also easy:
   MyShutdown sh = new MyShutdown();
Make sure you only register the shutdown hook exactly one time.
When the JVM that is running my application terminates, my shutdown hook is called. This works well for both running unit tests and when used inside a Tomcat container.

I use JVM shutdown hooks fairly often - it is useful to be able to perform cleanup work as a JVM terminates. On Linux servers, be sure to shutdown Tomcat (or JBoss, etc.) cleanly. Shutdown hooks are obviously not called if there is a server crash.

Wednesday, August 17, 2005

For Java developers: case study of the utility of dynamic languages

As seen on Planet Lisp:

Joe Marshall's great writeup on the big wins from using Common Lisp to develop an application.

I do spend most of my time designing and writing Java applications, but I also believe in keeping an open mind about recognizing Java's weaknesses (mostly that it is not a dynamic language).

Ruby and Python are also dynamic languages, but lack the blazingly fast performance of natively compiled Common Lisp code.

Saturday, August 13, 2005

I am still looking for the best database solution for web apps

I know that different applications have different database requirements but I still like to have standard tools that I use for most jobs.

For many years the words "database" and "PostgreSQL" were synonomous for me :-)

Still, my usual deployment pattern is to help a customer rent a cheap dedicated Linux server and set it up for them so that it supports the web application that I wrote for them.

PostgreSQL is not often installed out of the box in leased Linux servers, so using it is a little extra overhead. MySQL is almost always installed, but I really don't get the same joy from MySQL that I get from PostgreSQL.

I have used HSQLDB (an embedded Java database), but little problems like sometimes locks not getting reset have kept me from using it in production environments.

I am experimenting with using IBM Cloudscape (or the open source Derby version) as an embedded database in Tomcat (using Tomcat server events to support smooth Cloudscape initialization and shutdown). I am using this for my own web portal (a business for my wife and I) before trying it on any customer jobs.

Ideally, I would like to have one standard bundle of Tomcat + an embedded database server that is preconfigured for fast deployments to new servers.

Free (and good!) picture story for young kids

My wife Carol wrote (and took the pictures for) this picture story for our young grandchildren. They liked it a lot, so your children or grandchildren might also like it.

Tuesday, August 09, 2005

Not sure about Sun's Glassfish J2EE stack project

I wrote a book a few years ago about Sun One Services (the name for their J2EE stack when it was an expensive product). Opening up Sun's stack for free use and making the source code available through the Glassfish project is a good thing, but there are already so many great free and open source J2EE stacks: Jonas, JBoss, Apache Geronimo, and Gluecode. Glassfish binaries are released weekly, but with limited testing. Who would have thought 4 years ago that J2EE stacks would be a free commodity item?

Sunday, August 07, 2005

JBoss vs. GlueCode, web frameworks

I have been working on the open source Jaffa web application framework (as a paid consultant) and we use JBoss as a platform. Sure there is a learning curve for building on JBoss, but it is a stable and flexible platform.

I think that IBM's Gluecode web app stack seems to fill a different need: Gluecode is very polished and a turnkey solution for development and deployment: a better solution (I think) for some web applications because of the the simple administration functionality. I would feel better about delivering a web application to semi-technical customers that was deployed on Gluecode.

BTW, you might want to give Jaffa a try: there is a steep learning curve but it includes builtin business patterns, automatically generated CRUD backed web forms (generated from domain models expressed in an XML notation), etc. Cool stuff - check it out.

Thursday, August 04, 2005

ancient software

The year is 2560. Most software that people use is ancient - hundreds of years old with no changes except for a possible bug fix every 50 years or so. Use a word processor that has not been updated in 150 years (but is bug free)? Use an operating system that is as stable, secure, and unchanging as a cut diamond? You bet! Sure there will be new software written, and eventually it will either become ancient and rock solid, or fall into disuse.

Yes, this might be a crazy idea, but for some types of software it makes sense. Even with programming languages: one of the things that I like about Common Lisp is the maturity of the language and standard library. Smalltalk is another good example of an older and more stable language. I would like to see languages like Ruby, Java, and Python also stabalize at the language and core library level (and what is up with removing mapping functions from Python?) Oh, and how about freezing Word(tm) except for bug fixes?