Saturday, December 31, 2005

A trip down memory lane: I have a Prolog programming job

I just accepted a small consulting job that is to be written in Prolog. Ten or fifteen years ago, I was pretty much a Prolog guru (I once rewrote a planning system that had taken 6 weeks to write in Common Lisp to a Prolog version - took just 4 or 5 days -- and the Prolog version had better graphics than the Common Lisp version).

Fortunately, I worked through the excellent book Art of Prolog a few years ago, so I am not that rusty at Prolog programming. When Prolog is an appropriate for a problem domain, probably no other programming language is as productive. Good applications of Prolog include search, circuit layout, planning, and natural language processing (but not statistical natural language processing).

Friday, December 30, 2005

A novel search engine application

During a couple hour conversation today with my publisher, he posed the problem of determining what word sequences can logically go together in natural language. Examples would be:
  • the cow is brown - OK
  • the cow is green - probably does not make sense
  • the recess was one hour - OK
  • the recess is brown - makes no sense if recess is used as a school break, could make sense if the recess is the (Wordnet #2) sense "an enclosure that is set back or indented"
I would normally think of this as a sort-of hidden Markov model problem: look for frequencies of words appearing together, allowing a few wild card intermediate words.

However, a better solution came to me in about 15 seconds: use a search engine like Nutch to index a reasonably large part of the web. To test word sequences like "the cow is brown" and "the cow is green", we would need to look at the number of times the words [cow, brown] appeared in that sequence close to together; same for [cow, green]. This approach would even give reasonable answers for sequences like [recess, brown] that might make sense for some Wordnet senses of the words, but would likely not occur in actual use.

Thursday, December 29, 2005

The world is getting more peaceful

You would not know it from most shoddy "20 second sound bite" news services but this Washington Post article clearly states otherwise. I quote:
A major study by the Rand Corp. published this year found that U.N. peace-building operations had a two-thirds success rate. They were also surprisingly cost-effective. In fact, the United Nations spends less running 17 peace operations around the world for an entire year than the United States spends in Iraq in a single month.
The reality is that, since the end of the Cold War, armed conflict and nearly all other forms of political violence have decreased. The world is far more peaceful than it was.
It turns out that terrorism kills very few people, even small wars kill many people, and as the world becomes more tightly linked economically, war makes even less sense to sane and rational people.

Wednesday, December 28, 2005

New open source project: Ruby Message System

I have relied on guarenteed delivery asynchronous messaging to build
large scale systems for 20 years. I was surprised when I could not find
something simular to Java's JMS for Ruby so I decided to build my own
and release the server under a GPL license and the client libraries
under a LGPL license. When I have code to release it will be in the
usual place (

If I am reinventing the wheel, please let me know! I only plan on
implementing what I need, but maybe when there is a public code base
other people might contribute. This project is in the planning stage
right now. Here are some rough notes:

1. unlike Java JMS, there is currently no planned support for publish
and subscribe

2. the primary data structure of RMS is a shared collection of named
message queues

3. there is currently no planned support for security: it is
anticipated that enterprise applications will use RMS behind a
firewall. very limited security will be provided by an optional
configuration file that specifies allowed IPs for clients.

4. operations supported:

send_message(queue, message)
register_listener(queue, a_listener)

note: a_listener object must be able to respond to the message:


5. sent messages are persisted using a database or a flat file and must
be serializable

6. once a message is delivered to all registered listeners for a queue
the message is deleted from persistent store

7. eventually, I would like to support transparent interoperability
with ActiveMQ via stump so Ruby code could interoperate with systems
written in different languages that use ActiveMQ.

I would appreciate any suggestions, ideas, etc.

PS. I was planning on using the stump system for accessing the Java ActiveMQ JMS server but after receiving a recommendation from someone for the stump ruby bindings and ActiveMQ I think that I will experiment with ActiveMQ - it is not too much trouble to keep ActiveMQ running on one of my private servers.

Tuesday, December 27, 2005

Great commentary by Mark Shields

Mark Shields is more than a bit more liberal ("to the left") than I am, but he gets it absolutely correct in this good opinion piece.

I am a "fiscal conservative": I don't believe in large government expenditures in anything but education, defense (but to not use our armed forces in foolish ways), maintaining roads, etc. I voted for Bush in 2000 but I now feel that he is clearly the worst president that the U.S. has had in my lifetime. No one even comes close to both his failed policies that damage our country everyday and his thouroughness in protecting his own privacy but eagerness to strip away privacy and civil liberties of U.S. citizens. He gets away with it the way that corrupt elite-backed rulers have throughout history: dehumanize desired adversaries and do everything that you can to scare your own population instead of making them feel secure like a good leader tries to do. Think of Winston Churchill: "We have nothing to fear but fear itself." Now think how the radical neoconservatives who seem to have hijacked our government do everything that they can to scare people in order to get away with spoiling what our great country stands for.

Monday, December 26, 2005

Java and Ruby integration

One of the chapters of the new book that I am writing is on Java and Ruby integration, mostly through cooperating web services. The idea for this chapter came from a small customer job that I did 18 months ago: integrate a CRM system written in PHP with a document work flow web portal that I had written for them using Tomcat+JSP+custom tag libraries+Prevayler. Sure, one of the great things about Ruby Rails is that the whole stack fits so well together - you lose some of the advantages of Rails when you start mixing in other technologies. Still, at this time a subset of J2EE and Ruby on Rails look to me like the two best tools for building web applications, so it makes sense to have strategies for using both technologies in a company or organization.

Thursday, December 22, 2005

flickr: coolest web portal?

A good friend of mine has been recommending for a while and I set up a site with public pictures of Sedona, my family, etc..

Flickr is close to "desktop application functionality". Since I use so many different computers for my work, I like relying on web apps like flickr, gmail,, etc. And with all of my work and writing materials stored one of my servers with CVS, I have the freedom to work just about anywhere.

For just about anything I do on the internet I have sufficient bandwidth. I do look forward to improvements in reducing network latency that will make using web portals and web applications even more compelling. There are drawbacks: for example, had a recent outage that affected some customers for a while - that is the drawback to outsourcing your business infrastructure. Still, I think that the convenience of well implemented web portals trumps any sporadic access problems.

Wednesday, December 21, 2005 is done (for now)

I ended up spending almost 6 hours writing and deploying the RubyPlanet.NET web portal. I would have preferred to write the portal software in Ruby but for cost reasons I used Java: I have a leased server running a single Tomcat instance. With Tomcat, it is trivial to add new virtual hosts so deploying this new web application only took a few minutes and costs me nothing extra. I currently do not have a leased server running Apache and FastCGI but that situation will change in the next week.

There are other features I could add to this web portal, but I basically did it to "scratch an itch": I wanted a single page summary of what people are writing on their blogs about Ruby - this "blog scraping" is automatic so will take no effort on my part. I did add sections "News" and "Resources" that I have to manually maintain but I have a slick administration web interface that lets me effortlessly modify these sections.

Anyway, I keep my new web portal bookmarked and check it myself a few times a day to get Ruby news. Other people are also welcome to use it :-)

Moving to more collaborative web applications

While I was listening to a 90 minute presentation on the Croquet project, I started thinking of what lessons can web portal developers learn from the croquet developers who have no constraints of supporting legacy technology. Croquet, still in early development, uses peer to peer protocols to allow small numbers of users to share a 3D simulation space - 3D graphics, naive physics, IP telephony, etc. are built in.

Users of a web portal do not know who the other users are and can not interact with them. Now, to a limited degree, we could support some interaction between users who have an active session using AJAX: user modifications to a shared data store could be broadcast back out and a Javascript could listen for broadcast changes. However, this requires a central server - Croquet sends changes to an environment (actually, it only broadcasts user inputs and counts on the environment simulation being deterministic - same precision math used on all connected computers, etc.) using peer to peer protocols.

Tuesday, December 20, 2005

Efficient use of time, getting things done

After coming back from a 20 day vacation in Belize and Guatemala, I have been revisiting making my work flow as efficient as possible.

The first thing that I am doing (a "New Year Resolution" made several months ago) is to concentrate on mastering just two technologies: Java and Ruby server side development. I find it fun to spend about 10 hours a week spread out over research and learning about AI in general, trying many software frameworks written in Lisp, Smalltalk, Python, Ruby, and Java - a New Year Resolution is to limit myself to a few hours only to Ruby and Java, and a few hours on AI research.

In the last few years I have tried to streamline my development work flow to minimize time wasted: use CVS for everything, use standard web server setups and automate deployments, etc.

For Ruby development, I just came up with another developer's trick that is saving me time: I set up a Ruby standard class library project with Eclipse (and RDT and Rad Rails) than has all of the source code in $RUBY/lib/ruby (standard Ruby libraries, gem installed libraries, and my local site installed libraries). Now, in a few seconds I can look at the source code for Ruby class libraries that I am using - very handy!

In streamlining my Java server side development, I have a fairly standard setup with IntelliJ and Tomcat (I have for all intents and purposes given up on EJBs) so the startup time on a new project is about 1 minute.

Monday, December 19, 2005

RubyPlanet.NET - all things Ruby

I was looking for a Ruby oriented blog feed aggregator and did not find one so I just created the domain name RubyPlanet.NET and I am currently implementing the site.

I will automatically filter out non-Ruby blog posts so feel free to submit your blog feed URL to me even if you only occasionally post about Ruby programming.

Saturday, December 17, 2005

World peace: yes it is possible

I just returned from a trip to Belize and Guatemala. Visiting other countries always reminds me of how similar people are. I believe that spiritually we are all bound tightly together and on a practical day to day level we all want the same thing: a better life for our children, a safe environment to live in, and time to spend with people we love.

So why do we have wars? The answer has been the same for all of known history: a small group of elites profits from war, whether they are the people behind George W. Bush, Muslim extremist leaders, the Nazis, wealthy Romans, etc. Normal people do not profit from war but often their leaders and the elites in their countries do. The mechanism of going to war has also always been the same: dehumanize the opponent and spread fear among your own population.

The more I travel the more optimistic I get that with better communication, human consciousness will reach a higher point where self-serving leaders will loose the power to spread fear and dehumanize fellow human beings.

Friday, December 16, 2005

I have a new book contract

I need to check with my publisher before talking too much about my new project, but the book will focus on advanced development techniques using Ruby and Ruby on Rails. I assume that readers already have knowledge of Ruby, Ruby on Rails, and some enterprise development experience. Anyway, I love to write and I am very much looking forward to working on this project.

I find myself using Ruby more than Java now. I still think that Java is great for huge enterprise projects but most of my work is for smaller deployments and mid-size projects where Ruby and Ruby on Rails simply seems like a better fit. Since I have a Lisp (I have written 2 Springer-Verlag Lisp books and many years of Lisp use) and Smalltalk background, Ruby is a very comfortable language for me to use. I find it interesting that Ruby might end up being the vehicle for introducing Lisp-like and Smalltalk type programming paradigms to mainstream programmers. Really, the only problem that I have with Ruby is that it is slow - not a high perfomance compiled language like Common Lisp.

Back from Belize

Carol, our friends Tim, Kathleen, Tom, and Cheryl, and I all had a great time. We did 5 days of beach time, then rented a sailboat for a week to explore the second largest coral reef in the world, then spent 4 days inland to the north.

On the spur of the moment, two days ago we rented a car and driver to go into Guatemala to see the ruins of the Mayan super-city at Tikal - awesome! When I was in high school, I took about a 3000 mile train ride down to Merida to see the Mayan ruins at Chichen Itza. A very long day driving to Tikal was easier than that train ride!

Sunday, November 27, 2005

I am going to be offline for a few weeks

Later this week, I will be in the Caribbean (mostly diving) - so no blogging for a while. Carol and I are going with two other couples - all dear friends. I will post a few pictures when I get back. I have two very exciting projects starting up, so it is a little difficult to break away right now for a vacation - but the work will be here when I get back and I am looking forward to it.

Wednesday, November 23, 2005

Best paper on Ruby on Rails

Rick Bradley wrote Evaluation: moving from Java to Ruby on Rails for the CenterNet rewrite as an internal company strategy document - a great read! I have only been using Ruby on Rails for relatively small (low traffic) projects so I was most interested in Rick's take on RoR for enterprise class systems.

This guy gets it right: Java language extensions hurt the platform

I don't usually just write blog entries linking to other people's writing, but Ed Burnette gets it totally right: adding new language features to the Java language hurts the Java platform. Look, there are two huge advantages to the Java platform:
  • The JVM is supported on many platforms
  • The wealth of Java libraries, tools, and applications
As Ed points out so well, language extensions hurt efforts to get JVMs universally available on all platforms. Writing Java code that only targets version 1.5 language features prevents it from running on GNU gcj/CLASSPATH and on operating systems that do not have a reliable 1.5 JVM. Except for a few projects for my own use only, I make a point of setting IntelliJ to target version 1.4.x JVMs. I obviously like to run 1.5 JVMs to pick up the runtime improvements, but for serious work, I still target the 1.4.x language.

Saturday, November 19, 2005

Cornucopia complex, Ruby on Rails vs. Java and Lisp

"Cornucopia Complex" is an old joke of mine: fear of too many good things :-)

I claim to be program language agnostic (best tool for the job, etc.) but that is not strictly true because given sufficient libraries of pre-built code, I would just about always choose to program in Lisp. The point here is that existing infrastructure software continually drives me to one language or another. Java usually wins big here with tools like Tomcat, Lucene (including that fantastic Nutch project - I have been using Nutch in the last week - great stuff!), Hibernate, etc.

But Java does not win hands down: Ruby on Rails gets more compelling the more I use it. Seriously, compare using one of the Java AJAX frameworks with the simplicity of in RoR just including the prototype Javascript library in a RHTML template, and using link_to_remote, form_remote_tag, etc. On the Ruby side, a controller class action simply returns HTML with full access to form values using params[], etc. The first time I saw this in Curt Hibb's "Ajax on Rails" tutorial I couldn't believe how simple AJAX could be. Amazing what a single clean architectural framework will do for you.

The real deal would be to easy integration between Java for infrastructure code and RoR for web application user interfaces, and there are certainly easy ways to do this with plain old sockets, XML-RPC, SOAP (uh, a little too heavyweight for me usually), REST, shared database access, etc.

A few years ago, I got enthusiastic about Python and was initially excited by the prospect of natively compiling libraries like Lucene using the GNU gcj compiler, CLASSPATH, etc. and linking with Python (I used this for a while on OS X, but the shared library for natively compiled Lucene and all required libraries was huge). Same could be done for Ruby. However, that just seems wrong to me - I would rather have a Java JVM process and a Ruby (or Python, or Common Lisp) process and use socket, RPC, etc. Socket I/O on the same server is efficient to the point of just about being free anyway. Platform selection seems to be getting easier: if I need a major piece of Java infrastructure software (like Lucene), then I would choose Java. For most other web applications, I would choose RoR. If I had lots of time, I might choose Lisp and build up my own platform - but I think that this would be cost effective in few projects.

Friday, November 18, 2005

Updated version of my "Practical AI Programming in Java" free web book

A download link is on the Open Content page on my main web site.

I re-worked the chapter on Hidden Markov Models, added a chapter based on Paul Graham's 'Plan for SPAM' ideas (with his permission), and converted the manuscript to Open Document Format (using OO2 version 2).

As always, please email me if you see any typos or other errors.

I just added PDF and RTF export to

I have been writing the online word processor for my own use, but it is also freely available to anyone who wants to create a login account for themselves.

I found the old export as a HTML fragment file to be fairly useful for exporting from the web site to a local word processor, but RTF is a little more convenient. While I was adding RTF export functionality I decided to also implement PDF file generation.

Thursday, November 17, 2005

MIT's $100 laptop

I have been following this story for a long time, so it feels good to see pictures of a mock-up unit in the news today. The unit will only be made available to schools and NGOs, but I have a few suggestions:
  • Have a program where people can purchase 10 units for donation to schools in poor countries (or their own country for that matter). Kids getting the units could optionally get a picture of the donor's family so they know who gave them the computer. Donors would receive one machine for themselves for 'geek value'.
  • Computer geeks could purchase a unit for software development testing, or whatever, for double the price, plus shipping so one extra student would also get a computer.
Universal improvement in education will help make the world a better place, for sure.

Wednesday, November 16, 2005

Tomcat, Ruby on Rails, and Araneida

While I do most of my work using Tomcat (mostly JSPs, POJOs, a few custom tag libraries, and a few persistence strategies) I still devote a fair amount of time covering two other great server side platform options: Ruby on Rails and Common Lisp Araneida and cl-http web servers.

For server side Lisp, I think that I am going to start favoring Araneida over cl-http because Araneida (with LispWorks Professional) only uses about 20 megabytes of working set memory while cl-http uses much more memory. Most people might find Lisp a strange language choice, but not only have I written two Springer-Verlag Lisp books so my Lisp skills are fairly good, I find the ultra-high performance of natively compiled Lisp refreshing after both java (which also has great performance on the server side once HotSpot has had a good chance to work) and Ruby (which only has OK performance while executing built in methods that are compiled C code).

Because good Lisp programmers are difficult to find, I tend to stay away from Lisp for work performed for clients and just use it for my own business. This is required because I usually build systems and then hand them off to my customers to run and support and they are unlikely to have Lisp programmers on staff.

Another very good Lisp server package is the DrScheme/MzScheme web server that supports continuations (like Avi's Smalltalk Seaside project) and also seems to have low memory use and good runtime performance. The Common Lisp based UnCommon Web package also supports a modular/component style of web application development - it looks promising but so far I have only read papers and documentation on it. Continuation based web application development looks very interesting because the programmer does not have to work so hard maintaining navigational state, etc. and also allows a more component based development style.

Tuesday, November 15, 2005

Partnering and forming a Corporation

My wife and I are planning on forming a corporation around our business ideas. Our business plan is simple (at least in concept): slowly build up a collection of knowledge-intensive web portals that provide free services with some income generation from advertisements. If appropriate business-wise, package web portals as commercial products for deployment inside customers' local area networks and provide consulting services for customization.

The great thing about this business plan is that except for our time, it is a low-cost business to startup and run. A difficult decision for us is whether or not we want to actively persue finding partners for our business. It would be great to have a graphics artist/web designer, a marketing person, someone with better hosting bandwidth, etc. Since our goal is more to create a growing sustainable business than to make a quick buck, sharing ownership in the corporation is not a problem. We are more concerned with finding people who share our goals and ideas.

Friday, November 11, 2005

Interesting artificial intelligence problem

I have been working on a very interesting AI problem today. (This is not for a customer, rather it is for my wife's and my web portal so I can talk about it :-)

A bit of background before discussing the problem itself: users of our web portal are encouraged to maintain a simple database on our server of what ingredients that they have on hand in their kitchens. This data is stored per user in a database on my server and is easy to keep up to date. Users can search/view all recipes or just recipes that they have the ingredients on hand to make.

The AI problem is this: given (in a case based reasoning sense) a set of recipes, the ingredients the user has on hand, user preference for cooking style (American Comfort Food, Italian, Chinese, or Japanese) and how spicy they want the dish to be, the problem is this: create a good recipe using only the ingredients the user has in their kitchen. As a bonus, I offer advice, based on the requested cooking style, on spices to buy the next time the user is at the store.

For me, this is a "perfect" AI problem because my wife and I are ourselves the domain experts (we are excellent cooks and very creative in making up new recipes) and I am the "knowledge engineer" so the work is progressing fairly fast (but still a work in progress and incomplete).

Thursday, November 10, 2005

Possible privacy concerns, but I like Google's personal search

A few years ago when I was in California I spent a fun afternoon at the sci-fi writer and physicist David Brin's house - he had a Java related business idea that we were talking about. Anyway, he has an interesting take on privacy: reduced privacy is OK if everyone gets the same power of "surveillance" (hopefully, I am paraphrasing him OK on this). I thought about David's take on privacy when playing with Google's new beta (of course :-) personal search tonight: you login with your Google account and it remembers your search queries, what links you follow, etc. This is all searchable and convenient so I think that I am going to use it frequently in the future.

Since nothing you do on the internet is really private without taking some effort, I just accept that. Now, in practice, only software agents are likely to be monitoring what we do on the web, whether it is Google, Yahoo, or one of dozens (or more!) companies that make our business their business. US, UK, etc. governments have had the ability to track internet and telephone traffic for a long time, so why not private companies.

Back to privacy: it is easy to encrypt email, etc. and I do do that for some customers who request it. I also have one old (but with all security patches applied!) computer that I use for nothing but online banking. Sometimes privacy and security are important and sometimes they are not.

Another way I voluntarily give up privacy (again, for much convenience) is by using to organize all of my bookmarks which mostly have to do with research, my business, and self-study educational resources. At the end of the day, I don't feel like I give up any competitive advantage by sharing my bookmarks.

I am working (so far for only about 12 hours) on - it is still in beta, with many features planned. Anyway, users of this system trust me to respect their privacy just the way that I expect Google, Yahoo, etc. to respect my privacy. At the end of the day, just deal with people and companies that you feel that you can trust.

Intel Macs running OS X, Linux, and Windows

This is big, at least to me. I use two laptops and two desktop PCs (and some leased hosted servers) for my business: 2 Macs and 2 dual boot Linux/Windows boxes. I don't like clutter in my home office, and the idea of one high end Intel Mac laptop (with an external monitor for desktop use) that is OS X/Linux/Windows capable sounds great to me.

I absolutely don't mind rebooting since I usually need to use one OS for a while, depending on what I am working on. As I have written before, the secret to no hassle switching computers and operating systems is a CVS server so you can sync up development assets easily on any computer (cvs or svn update is your friend :-)

Wednesday, November 09, 2005

Excellent: official Java support for system tray functionality

This is excellent! I use a very good 3rd party library to add system tray functionality to JFC Java applications when running under Windows and quietly fail when running on Linux. Sun's JDK 6.0 (Mustang) support covers Windows, KDE Linux, and Gnome Linux desktops in a portable way.

BTW, I do wish that Linux distribution creators and Sun could get together to have a JRE installed by default on Linux desktops, but I hold out little hope of this. GNU Java (Classpath, gcj, etc.) are coming along nicely but I find it much less facile to develop with than Sun's JDK.

A la cart media purchases will lead to higher quality in TV and movies

Apple's iTunes music store has changed the way I buy music: preview before I buy and only buy what I really like - a great combination where I spend more money on music than I used to but my satisfaction as a consumer is much greater than when I used to buy music CDs. I hope for the same kind of flexibility in purchasing video content: movies, short "Indie" movies, and selected TV shows. The improved quality will certainly be greatest for TV, which has the most room for improvement.

The current TV network system works against the success of smaller niche market programs that have small but loyal followings. We spend a lot of money on cable TV each month and I am not really a happy consumer. There is just too much stuff offered that my family has no interest in what so ever.

I don't mind paying for TV shows, movies, and "Indie" films that I really like, but I want control and selection. DRM that does not threaten the security of my computers is fine with me - I would be happy enough "re-paying" for future viewing of the same material and I really don't want the hassle of storing programs myself (my brother owns about 1200 DVDs and has a physical storage problem).

I think that there are a lot of consumers like myself who would be happy enough paying more money to producers of content that is what I want when I want it.

Sunday, November 06, 2005

What a difference a day makes

Yesterday afternoon, I was home in the mountains of Northern Arizona hiking to Indian ruins with 8 friends. This afternoon I arrived in California on business - I did get in a good walk on the beach today as soon as I arrived. Arizona rocks, California rocks less - but change and variety are good.

Friday, November 04, 2005

Latest Java Trails version 0.8 looks very cool

Congratulations to Chris Nelson! Trails is looking better and better. I had a chance to grab the new version released yesterday and went through the first few tutorials: 1 and 2.

I have little spare time right now, so I just quickly zoomed through the tutorials. When I get a few spare hours, I want to dig into the Trails code itself.

I like Ruby on Rails because of the ease of customizing the auto-generated scaffold code. I spent a little time looking at the generated Trails configuration files, etc. and I am not sure about customizing as far as changing global appearance, etc. but the second tutorial shows how to use Java 5 annotations to change generated form labels, data formats, etc. The second tutorial does show how to change page templates, which is probably what I want: a functional equivalent to global changes of style sheets.

Wednesday, November 02, 2005


Since I used Microsoft's new beta portal on my Linux laptop running FireFox, I may not have had the "full experience" :-) That said, looks well done: the client side Javascript is slick and the interface is simple and clean.

I was not so happy with the requirement to login with Passport (otherwise, all your customizations go away when your session ends). Anyway, I think that competition between Google, Yahoo, and Microsoft in the web application area is great. Because I mostly use Linux I use Google's and Yahoo's web applications but Microsoft has announced that they will support other platforms - I am at least trying to keep an open mind about using in the future, depending on what useful functionality that they provide.

Tuesday, November 01, 2005

Poor product: Sony Network Walkman NW-E95: poor MP3 support

This is just my personal opinion: I wanted to get my wife a portable MP3 player for her birthday - she enjoys downloading free MP3s (mostly talks people give, etc.) from the internet. We went out and bought a Sony Network Walkman NW-E95 today only to discover that while it works beautifully for transferring music from an audio CD collection it seems to be (again this is just my opinion) purposefully crippled for playing arbitrary MP3 files. There is a lot of great free content on the internet in MP3 format and I think that Sony made a really bad decision to not support a wide variety of MP3 formats. I found a web blog where someone suggested taking MP3 files and using them to create audio CDs - then load these CDRs to the Sony Network Walkman NW-E95; this is more trouble than my wife wants to go to listen to audio lectures (she has little interest in listening to music on a portable player). BTW, we live in a very small town, so we have to drive 25 miles to get to an electronics store - so, returning this device will be an additional hassle. This is also why we did not go to the store, write down product names, and go home first to research the available players on the web.

I bought a Sony Portable Playstation several months ago - love that. Sony gets somethings right, at least.

Monday, October 31, 2005

Great Java open source project: Nutch search engine

Nutch is an Apache licensed open source search engine project that I have been keeping an eye on for a while. One thing that makes this project especially compelling is that the author of the (fabulous) Lucene search library Doug Cutting is also a principle designer and implementer of Nutch. You can grab the source code using subversion:
svn co
Nutch now contains two new modules: the Nutch Distributed File System (patterned after the Google File System) and a Java version of MapReduce (patterned after Google's MapReduce). So far, I have only been looking at the source code (no builds and playing with it yet!) but this stuff looks really good. Anyone want to start a search engine company? :-)

Good update: RadRails 0.4 released today

RadRails version 0.4 was released today. It is a good upgrade: RHTML files are now color syntax hilighted (but with a default black background color that can be easily changed - what is up with that :-) The code assist support is also useful.

Sunday, October 30, 2005

Efficiency (or lack of) in Java reflection: glad we have it anyway

It is good that the java.lang.reflect package exists. This interesting read (a PDF file on shows a great example of using a decorator/proxy/delegation pattern for implementing a logging property across multiple classes. Neat stuff and I like this better than, for example, using Aspect Oriented Programming (AOP) to get the same effect: add logging without touching original POJO classes.

However, Java reflection is not very efficient and I have seen at least one very large Java application that has poor performance because it performs lots of reflection. Other languages like Ruby have better support. Common Lisp Object System (CLOS) with support for things like before/after methods, class slot introspection, etc. also is very effective for applications requiring more flexibility than Java.

I add spell checking/correcting to online word processor

I had a little time this weekend to add spell checking/correcting to my online word processor.

There are some general directions on the "About" page.

This system is still in beta (I have just written it in the last week and a half in my spare time) so please email me with any bug reports. Thanks!

Saturday, October 29, 2005

WebSphere Community Edition: not available until the end of the year

I was enthusiastic about the announcement from IBM: bundling WebSphere Community Edition with Eclipse development plugins etc. sounded great but after a fair amount of searching I saw that it will not be available until the end of the year.

I don't intend to be critical of IBM: I appreciate their Java SDK for the PowerPC architecture, their support of Linux, etc., etc. However, it seems like this early announcement is just to generate some buzz for IBM's purchased Gluecode and perhaps to rain on JBoss's parade.

While I am very happy with my development environment for simple JSP, JavaBeans, custom tag libraries, etc. I would like a super productive environment for use when I want to use more of the J2EE stack. The JBoss Eclipse IDE looks like it may be a possibility and IBM's WebSphere Community Edition looks like another possibility - I think that it is worth waiting for this to sort itself out.

Meanwhile, I find it ironic that my favorite Ruby Rails development system is based on Eclipse (and RDT and RadRails - new but promising, BTW).

Friday, October 28, 2005

Classic computer science text and papers on the web

I have seen the old Frank Sinatra movie that my wife is watching so I decided to read tonight. I still appreciate my rather large personal library because physical books, journals, and reprints are more fun to read than balancing a laptop. That said, I thought that it would be interesting to share some of my favorite links to classic computer science texts and papers:This is just a sampling taken from my bookmarks. The ACM has a program to make classic texts available. Other must-study 'classics' like "Introduction to Algorithms" (Cormen, et. al.) are still too new (first edition 1991) to be freely available.

I find it interesting to re-read material that I used many years ago - you get a different take on papers and texts after years of experience, making them interesting in new ways.

Thursday, October 27, 2005 updated: added document export and search (or: sometimes it is good to be lazy)

I updated my new web portal to support search and exporting all of a user's documents compressed into a ZIP file.

I only had an hour or so free today to work on this, so I got lazy: I had intended to export the rich text documents that a user keeps on the web portal as OpenDocument formatted files for, AbiWord, etc. I often read (programatically, that is) OpenDocument files: easy: it is a ZIP file, so just grab the contents, style, whatever that you need as ZIP file entries.

The problem was that it looked like a multi-hour task to take the internal rich text format used on the web portal (which is really just HTML snippets) and generate equivalent looking OpenDocument formatted document files. So, I got lazy and just bundle all of a user's documents in one ZIP file for users to download. It turns out that imports these ill-formed HTML files just fine - so, I am happy that I did not spend the time right now generating OpenDocument files.

I usually use Lucene for full text search - I love it because it so easy to work with and customize (see my KBtextmaster GPLed project for utilities to search Word, PDF,, AbiWord, etc. files with Lucene). For now, I am just using a database text search for the web portal - good enough for the short term.

Wednesday, October 26, 2005

New web app for editing rich text documents (or: Rails vs. Java saga continues)

I have been experimenting with parallel development of a web app in both Ruby on Rails and Java/JSPs. I just hosted the Java version at if you want to try it.

I did diverge a lot in the two parallel implementations: I ended up making the Java version for single users while the Ruby on Rails version has groups and access control lists. For fun, I also used different JavaScript rich text editing libraries for the two projects. I used a very light weight library for the Java version at because I intend to leave that web application running and let people use it for online document editing. One word of warning: I don't have the export to OpenDocument format implemented yet - until I do, there will be no way to transfer your online writing to your local word processor.

Anyway, I started this project for my own use, so the Java single user (vs. groupware) version meets my needs and will be the version that I continue to work on and maintain. Anyone else is also welcome to use this web app.

Monday, October 24, 2005

Rife with automatic CRUD scaffolding worth looking at

I have been looking at Rife (a component based Java server side framework) for a while, but have not invested the time to do anything other than read though tutorial examples. has released an automatic CRUD scaffolding framework that at least on the surface is a little like Ruby Rails scaffolding. You can read about it here.

Rife CRUD scaffolding customization does not look nearly as flexible as Ruby Rails scaffolding, but that is fair: Rife does its scaffolding at runtime so there is no code to tweak - different than Ruby Rails where you can edit the generated .rhtml, Ruby controller, etc. files.

I like the superb runtime performance of simple JSPs (with custom tag libraries) but there is no reason to believe that the runtime overhead of Rife and Rife CRUD scaffolding would be too bad -- servers are cheaper than programmers (usually, except for very large scale deployments) so the agile nature of Rife looks good. I am hoping to have time to download Rife CRUD later today or tonight to kick the tires.

PS. I just took the time to download the calendar example - looked through the source + configuration and ran the example. I did *not* take the time to write my own example from scratch. Anyway, Rife + CRUD scaffolding looks cool, but I think that I will stick with Ruby Rails and for Java development: plain old JSPs and custom tag libraries (mostly).

PSS. Geert posted a response to this blog in which he points out that the generated code is compiled on the fly so there is no performance penalty and that the ability to customize CRUD scaffolding is more flexible than I thought.

Great interview with Brent Scowcroft

I have always admired Brent Scowcroft. He was the first President Bush's national security advisor, and is well known as a realist - I think that he is the kind of guy who looks first to what is good for our country which is why he is widely admired by both democrats and republicans. This is a great article. Everyone should enjoy this and gleam more insight into Washington politics and power. Great read!

Sunday, October 23, 2005

Getting back to simplicity

There is a trend in computer usage that I like: getting back to simplicity. Unless you want them to for entertainment, computers should simplify your life, not make things more complex. I think that this goes for developing software also.

As a Java developer, I used to want to know and understand every nook and cranny of the language, standard libraries, the JVM, and the entire J2EE stack. Now, I find myself concentrating just on those technologies that are most useful for getting work done: on the server side JavaBeans, JSPs, custom tag libraries and a small number of persistence strategies. On the fortunately rare occasions that I need to write JFC based clients, I simply use the NetBeans UI designer to bang out the framework with event handlers stubbed out. Simplicity of tools and approach, and concentrate on problem solving...

I have also been really getting into the simplicity and consistency of the Ruby on Rails framework. Except for very large scale web application deployments, I now think that RoR is a very viable option to server side Java. Again, easy to master tools let you concentrate more on problem solving.

In my work I have tasks that need to be done on Linux, Windows, and OS X systems - not the best for striving for simplicity! I manage this extra complexity having a beefed-up PC configured for both Linux and Windows XP. The default boot is Ubuntu Linux and I have nothing extra installed on the system except for what I need to work: IntelliJ, Tomcat, JBoss, Ruby, Ruby on Rails, Python, the gcc/g++ toolchain and OpenOffice. I don't boot up to "have fun" (well, work is fun, but you know what I mean).

I separately have Windows and Mac OS X systems configured with both entertainment software (video editing on the Mac, etc.), and development tools. Except for work for 2 customers, I generally don't boot OS X or Windows to work.

As a computer user, I really like Ubuntu Linux: I use it and don't really have to think about it. Updates are easy and transparent. I wanted to burn a data DVD-R backup a little while ago: pop in a blank DVD-R, drag the files to backup (everything on /home :-) and burn it. At least as easy as OS X, and easier than with Windows XP. The Ubuntu desktop seems to not offer as many options (without digging), but just presents functionality that you need - they do a very good job of putting together a user-oriented Linux distribution, BTW.

My next move to simplicity is moving the data for most of my writing off of my computers and use one of my rented servers and the document management application that I am now developing using Ruby on Rails - I expect to be done with that in another free evening or two. I decided to put together this web based content management system for myself after realizing how nice it is to just use GMail,, Yahoo Calendars, and other web application to organize my stuff.

Friday, October 21, 2005

Side by side comparison of Ruby on Rails vs. JSP + JavaBeans

As both a research project and to build something for my own use, I have been (as time permits) developing a web app that lets me use JavaScript on the client side to do styled/rich text editing and to save edited files on one of my servers.

What is different here is that I do a little development on one platform, then on the other - but being sure to use both RoR and JSPs+JavaBeans for new coding (vs. simply porting what I have already done).

I am finding Ruby on Rails to be a faster prototype/development environment, mostly because it takes over 10 seconds to re-test under Tomcat (remote access via IntelliJ) and re-testing under RoR is almost instantaneous. Ruby also is a little faster to code in.

No way can I put an accurate number on how much faster Ruby on Rails development is, but I would estimate that I save about 20% of development time - not a huge deal, but nice. This comparison is actually comforting from a Java perspective: for customers who prefer Java on the server, the extra overhead is not too bad.

There have been some comments that Ruby and RoR development is more "fun". I don't know about that: I really enjoy writing Java web applications with just JSPs, JavaBeans, and perhaps a few custom tag libraries. I must admit though that the "fun level" decreases when I have to mix in JMS, EJBs, etc., but that is just because I can't code from memory and have to occasionally stop to look stuff up.

Anyway, now that I am adding classes and behavior for groups, access control lists, etc., I am droping the Java half of this experiment. This is not saying anything against Java: I just want to beef up my Ruby on Rails experience and now get something done for my own use.

PS. a little off topic: I always think that I am a little bit more productive when using Linux or OS X than when I develop under Windows XP. This is a small effect though, and just a feeling, really.

The more things stay the same: distributed tuple space toolkits like Jini

Long ago at SAIC, my old Friend Tim Kraft (now doing great things at Overture/Yahoo) and I did a commercial product that was loosely based on David Gelernter's Linda data model. About 5 years ago, at Intelligenesis, we looked seriously at Jini-type models for distributed processing.

Anyway, I noticed that the new Jini starter kit is now Apache licensed which should allow what is a great technology (for some distributed data problems :-) to be used easily by commercial application developers. Worth looking at if you have never experimented with Linda, JavaSpaces, or Jini before.

Wednesday, October 19, 2005

Code generation wizards: Ruby on Rails vs. Microsoft (and dynamic programming languages)

Please excuse me in advance if this is not a fair comparison: the last time that I used Microsoft VisualStudio code generating wizards was about 8 years ago at Angel Studio where I wrote a 'network play' library for multi-user PC racing games. The generated code was so obtuse that no human being could really understand it (well, I tried). I ended up writing my own equivalent library using UDP (using the same API so we could build against either library).

Anyway, that gave me a feeling of bad aji (Japanese word for taste - used in playing Go and refers to good or bad shape of stones played - good aji or bad aji). Needless to say, VisualStudio's code generator generated code with bad aji!

Ruby on Rails is really three things: a code generating wizard, utility class libraries, and a runtime component. The RoR code generator produces easy-to-read Ruby code and .rhtml templates (look like JSPs except for embedded Ruby code rather than Java code). The generated code is so readable that it is a pleasure to modify as required.

Now to be fair, Ruby is a dynamic language that has great introspection capabilities (i.e., code can look at itself, modify itself, etc.) This makes all the difference in the quality of generated code. Dynamic languages like Ruby, Lisp/Scheme, Python, and Smalltalk will always win over languages like C++, Java, and C# in flexibility of generated code, ORM, etc., etc.

What I think is interesting is that there has never been a mainstream computer programming language that is dynamic! I believe that this is because dynamic languages are favored by the "very high end of the programmer food chain"; for example: software architects who write custom domain specific languages to bring domain experts and programmers closer together in the development process. While some computer science text books (like "Structure and Interpretation of Computer Programs", "On LISP", etc.) cover meta-level programming very well, there are way too many programmers who have not been exposed to higher level programming techniques. I think that someday, a dynamic language will become the "new C++", or the "new Java" - hopefully sooner than later.

My notes on using RadRails for Eclipse + Ruby on rails development

I have not seen (yet) very much online documentation for RadRails so it is worth noting a few things that I do:
  • Start by installing (as needed) Eclipse, RDT, and RadRails
  • Use RadRails to setup a new blank Ruby on rails project
  • In Eclipse, make sure that you switch to the "Rails perspective" (instead of the default "Java perspective")
  • Design the database schema for your application and be sure to follow the RoR table naming conventions (e.g., a table named "users" for an automatically generated model for class "User", table "doc_acls" for model "DocAcl", etc.)
  • Rails will want 3 databases with names ending in _development, _test, and _production). Create your development database with your application tables. Then, in Eclipse/RadRails, edit in the database information
  • Outside of Eclipse, generate models and scaffolds; for example: "ruby script/generate scaffold User Admin", "ruby script/generate model Group", "ruby script/generate model GroupAcl", etc.
  • Return to Eclipse/RadRails and do a "refresh" on your project so that Eclipse 'notices' all of the Ruby source files, .rhtml files, etc. that you just generated
  • RadRails has a view (lower right part of Eclipse window) with a 'Server" tab - use this to start and stop WEBrick for testing. Point a web browser at
  • Enjoy...
P.S. a reader (thanks Mel!) just sent me this link showing how to do RoR development on Eclipse - I especially like Brian Hogan's hint for editing rhtml files with syntax highlighting.
P.S.S. reader Murphee (thanks!) pointed out that you can use the New menu (in the Rails perspective)to create scaffoldsand models - no need to create these externally to Eclipse.

Our Java portal for recipes is almost feature complete

I am still working on the AI agent for custom recipe creation at our Java portal for recipes but otherwise the site is just about feature complete. Our site has a unique feature: users create free login accounts and then they can maintain a private database for ingredients on hand. They can then search for recipes and optionally specify that only recipes for which the user has the ingredients for will be shown.

The AI agent functionality is (or will be :-) also cool: the user can specify which ingredient that they have on hand that they would most like to use up. The also specify a cooking style (e.g., American comfort food, Japanese, etc.) and how spicy they want it. The AI agent will then search for recipes that the use the specified ingredient and for which the user has many of the ingredients for - then the recipe is customized for cooking style and for only using ingredients in the user's "on hand database".

Tuesday, October 18, 2005

Sharing and the gift economy

Carol and I decided to license the recipes on our new healthy foods and recipes web portal under a Creative Commons license that allows people to re-publish our stuff with attribution and a reference to our web site. For us, this makes sense: we want people to use our web portal and sharing recipes is one way to promote it.

I do the same thing with my free web books and open source software projects: by sharing I get good feedback and improvements from many people and because hundreds of other web sites reference my main web site, when anyone searches for a "Java consultant", there I am :-)

I frequently push back a little with some of my customers who want to keep everything they do closed and proprietary. Unless you are helping a competitor put you out of business, sharing has many rewards.

Monday, October 17, 2005

Nice business model: Intergalactic Medicine Show

I saw this on Slashdot this morning and signed up for the first issue. Well worth $2.50 for 2 reasons: the SciFi short stories (so far) are good and I like to spend small amounts of money to support business models that I like (usually decentralized, small businesses that make good use of the web). It looks like Orson Scott Card is providing a good venue for SciFi writers to get published. I especially like SciFi short stories because you can take a short break and enjoy an entire story at once.

Saturday, October 15, 2005

Really bad decision over NASA's funding

I think that it was the current science-ignoring administration and Congress that made the really bad call to reduce funding for unmanned robotic space exploration because of an emphasis on entertainment over science. Let's be clear on one thing: we get far more scientific value per dollar by sending huge numbers of tiny devices throughout our solar system (and beyond) than putting humans just outside the earths gravitational well.

I believe that funding for unmanned missions historically have cost just a few percent of what manned missions cost.

If it were my call, I would fund both manned and unmanned exploration, but definitely not cut funding for the high value relatively low cost unmanned missions. Absolute stupidity.

Wednesday, October 12, 2005

Easy update of Ubuntu Linux to new release

Why can't Microsoft make upgrades this easy. A few caveats: Ubuntu is not officially releasing "Breezy" until tomorrow, so I did this on my laptop (which is not my main Linux development system):

In the Synaptic package manager, under Settings -> Repository, I manually edited my repositories changing all occurrences of "hoary" to "breezy" and I removed the install CDROM as a repository source. I then clicked the "Mark All Upgrades" taskbar icon and then clicked "Apply" - when asked, I chose the "Smart Mode" upgrade that apparently is meant for upgrading to new releases.

One particularly great thing: under "hoary", I had to build and install my own driver for the RT2500 wifi device in my laptop and manually start it. After the upgrade, wireless is on with no manual operations. Note that with the RT2500, when booting Windows XP, I have to manually start wireless.

I am having zero problems with my laptop since upgrading so after waiting a few days I will do the same painless process for installing the new release on my main development computer. Usually, I back up my home directory and do a complete reinstall for new Linux disto releases but Ubuntu has always been so smooth and painless to use I decided to try an upgrade and save all my settings, etc.

Another nice thing is the automatic upgrading of to 2.0 pre-release and the latest GNome stuff. Sweet. The upgrade took a lot of wall clock time but only a few minutes of my time (I was working on another computer during the upgrade).

I like the improvements Ubuntu has made since the last release 6 months ago - small steady improvements that would not confuse a non-technical user. If both of my parents were not on OS X, I would set them up with Ubuntu Linux (which incidently also runs very well on Mac hardware).

If I ran Microsoft, I would do what Ubuntu does: bring out a steady stream of small tweaks and improvements and concentrate on stability and reliability. I think that most Windows users would happily pay a small yearly per PC license fee in return for no new gratuitous (market driven to help force upgrades) OS changes and an emphasis on stability and quality.

PS. I am typing this on my Desktop Linux computer - my laptop just started a new screen saver that I have never seen before: 3D ants walking around on a 3D mobius strip -- way too cool :-)

Tuesday, October 11, 2005

Ruby tools improving

Anyone who reads my blog knows that I love to write Java code using IntelliJ. As much as I prefer the Ruby language to Java, better IDEs and more available infrastructure software keeps me (for now) firmly in the Java development camp.

However, Ruby tools are definitely improving. The Ruby Development Tools for Eclipse keep getting better and the new RAD IDE for developing Ruby on Rails applications (RadRails) is looking very promising.

Besides congratulating the RDT and RadRails developers for some cool work, I actually have a point to make:

In the new IT world, developers concentrate on providing services to users that are based on centralized data stores like existing relational databases, generated data from web services (via XML-RPC, REST, SOAP, etc.), semantic data (RDF, OWL, etc.) repositories, document stores (e.g., Microsoft's SharePoint, WebDAV repositories of documents), etc.

Choice of programming language and runtime platform seems to be less important than it used to be as long as there is a good supply of developers for a language and the language is stable. Developers are free to use the best language and tools for each job and in my experience the availability of existing software that can be reused is the most important thing to take into consideration when selecting languages and tools. That said, system maintenance is much less expensive when an entire SOA or web portal is written for one language/platform.

SOA and

While reading an interview with OOo developer Florian Reuter I kept thinking of two very different possible future IT worlds:
  • Microsoft continues to dominate the market and business processes continue to be hobbled by closed (or at least opaque) file formats.
  • The Open Document standard becomes a (close to) universal standard with many interoperating software systems that work nicely with each other to facilitate "knowledge flow". People and organizations get the maximum benefit from their data and information assets for the lowest cost.
I really wish that Microsoft would get on board with the Open Document specification. As I have written recently, I believe that Microsoft's best strategy is to switch to a subscription based licensing for Windows and Office so that they get a yearly fee from users - this frees them from having to add features that few people need and concentrate on quality. If they play fair in supporting the Open Document standard, they will be in a good position to compete.

In any case, to update Sun Microsystems' saying: "The internet is the computer". Call it Web 2.0 if you wish, but I see the trend to SOA and interactive web portals as a natural evolution of the internet.

Monday, October 10, 2005

User's perspective on web services like and GMail: running stateless

Sometimes I am questioned on my preference for using web applications over local application programs. For me, it is largely about "running stateless": any slight loss of efficiency over using local applications and data is more than compensated by having all of my stuff available no matter where I am or which computer I am using.

Of course, I am not really running stateless, but someone else is maintaining state for me.

An alternative is keeping all work on a networked file server but then there is the problem of having the right application handy.

Using XWindows is also a good alternative: I remember in the mid 1980s having to do a lot of work on servers located in Norway, my office was in La Jolla California. Back then my internet connection was a few hops between satellites and ground stations so latency was really bad: ground/sea based fiber is so much better, the speed of light being limiting after all :-)

For my own use, I am experimenting with a web application that I am slowly developing as I get the occasional few free hours to work on it: I use a public domain JavaScript rich text editing library for editing short papers and a web app that keeps everything on one of my servers. For writing books (both published and the free "web books" that I enjoy working on), nothing beats (I own licenses for Microsoft Office for Mac and Windows, but prefer OOo for serious writing). However, so much of my writing only deals short documents, that I think that a web app will do the job just fine. I have not done it yet, but I am planning OOo import/export.

I may be biased because most of my work is in designing and building interactive web applications but I expect to see almost all use of computers to migrate to web applications. The few exceptions to this trend are: software development, writing, graphics, video editing, etc.

Sunday, October 09, 2005

Writing my own message board for second time in 2 years

No one likes to reuse code more than I do. The first thing that I look for is good quality GPL (when applicable) or BSD/Apache type licensed code to reuse rather than "reinventing the wheel".

My wife asked me to add a message board to our recipe web portal so that people can comment on recipes and suggest improvements for the system. There are several good open source Java JSP/servlet based message board projects but I decided to write my own from scratch. Why? Good question! In our case, the message board needs to be tightly integrated with existing user accounts, knowledge about recipes (so that comments can optionally be linked to specific recipes), etc. I did spend time looking at the source code to existing systems, but decided I wanted a custom solution that is tightly integrated with both the relational database for the web portal and the software that I have already written for it.

The same thing happened a few years ago. I was writing a "SharePoint(tm) replacement" for a customer to manage the workflow for word processing documents. The web portal was highly tailored to their work flow and used a custom Prevayler based persistent data store. Near the end of that project, I received a feature request for a message board and it made more sense to spend an extra day and write something that was tightly integrated with the rest of the system.

Saturday, October 08, 2005

Oracle buying InnoDB owner

I have always preferred PostgreSQL over MySQL - but, I usually end up using MySQL because it is installed and configured on servers and virtual servers that I rent on a monthly basis for my customers' and my web portals.

MySQL Corporation's contract with InnoDB's owners is up next year, but Oracle plans to renew some agreement - it will be interesting to see how the use of MySQL in non-commercial licensed environments holds up.

I am thinking, but have not made a firm decision, of switching over to PostgreSQL for all deployments. This will add to the (now small) overhead for renting a server and deploying a web application. I might also start to favor hosting companies who provide pre-installed PostgreSQL.

Friday, October 07, 2005

I am back from vacation

Carol and I went on a Mexican Riviera cruise with my parents, brother, and sister in law. Carol and I went white water rapids rafting east of Acapulco, rode zip lines 100-150 feet above the ground in a rain forest canopy east of Puerto Vallarta, and went kayaking and hiking on Deer Island (off shore from Mazatlan). Otherwise, I just enjoyed some Mexican beer, tried to avoid over-eating (not easy to do on a cruise ship), enjoyed walking around Mexican port cities, and kicked back. I have been working really long hours for about 8 months straight, so the time off was great.

Friday, September 23, 2005

Predicting the Future: Linux vs. Windows

I wrote Predicting the future: Google vs. Microsoft a few days ago and I enjoyed that exercise enough to try again with Linux vs. Windows.

I have been using Linux since downloading the core of Slackware using a 2400 baud modem and I wrote a commercial application for Windows 1.0. Unlike a lot of Linux enthusiasts, I am fairly happy with Windows dominance in the non-technical world: this makes Windows more of a target for virus attacks, trojans, etc. and makes using Linux and OS X safer. I once spent the weekend at a friend's house and Microsoft's .Net project manager was also there with his family. He told me that I was not Microsoft's target customer - and that suits me fine. I do wish that Microsoft would do the world a favor and support open file formats, but that is their bad.

Windows: Although I believe that Microsoft's desktop dominance and financial position will take a long slow slide down hill, they will be around for a long time. I think that Microsoft's future lies in the idea of charging customers a yearly fee per PC for software use in order to provide a continual revenue source without having to spoil the quality of their products with gratuitous changes in order to get people to upgrade.

This is the scenario that I would like to see: I have two PCs that I dual boot between Linux and Windows. I would prefer to pay Microsoft a (hopefully) small fee per PC per year to get security updates and hopefully rare functionality modifications. This subscription model would allow Microsoft to concentrate on quality and not on new features that a lot of people don't really care about. I would consider a $25/year fee for each of my PCs reasonable. If you buy a new PC, the PC manufacturer would pay the first year fee for you (actually you are paying via a higher purchase price). For individuals who need Microsoft office, I would think that a $50/year fee would be reasonable. I would expect Microsoft to give corporate customers a discount. I would also expect Microsoft to put more effort into making company wide deployment and maintenance of their software products less expensive in IT staff resources.

Linux: I think that the use of Linux will continue to increase in business with a much slower growth in the home desktop market. I think that professional maintenance and work on Linux will continue: large corporations will donate people out of self interest and home/small business Linux users like myself will continue making small financial donations to Debian, Ubuntu, etc. to support continued high quality distributions and improvements. I expect to see Linux Desktop use to continue to cluster more around both Gnome and KDE; alternative desktops will mostly be used for hobbiest or emotional reasons. While the competition between Gnnome and KDE is healthy, I expect to see more distributions smoothly supporting running KDE applications under Gnome, and vice versa.

I think that it will be a long time before Linux scales up to massively SMP servers like Solaris, AIX, etc. I think that for most web based applications running Linux on inexpensive yet fairly reliable hardware will continue dominate the low and middle parts of the server market, slowly but surely increasing market share.

Wednesday, September 21, 2005

Predicting the future: Google vs. Microsoft

One of my customers just asked me what my take is on the future competition between Google and Microsoft. I will start with where I think that each company will be in 5 years:

Microsoft: I think that Microsoft will have limited success in newer markets like music players, gaming consoles, etc. In 5 years, I think that Office and Windows will still be their cash cows. I think that Microsoft's stock price will take a modest hit in the next 5 years, but they will still have huge cash resources.

Google: I think that Google will still be doing well in 5 years, but their stock price will probably come down a bit. Google's advantage is their infrastructure: a scalable platform that lets them relatively easily experiment with new very large scale web applications. Google File System and their map reduce libraries are great technologies. Their architecture scales much better than, for example, Oracle on Unix super servers.

Future of economy: this is the potential spoiler for any predictions that I am making. While the U.S. economy has a lot going for it, it is also subject to some serious threats from the housing and consumer credit bubbles. The "sea change" in our political system (i.e., the total domination of Congress, courts, etc. in the last 10 years or so by our new Corporate Overlords) will probably be OK for the economy as a whole, but say goodbye to much of the middle class: if you want to be OK financially, make sure that whatever you do that you are very, very good at it. The question is not if the U.S. economy is going to tank. Rather the questions are when it will happen, how bad it will be, and will we fully recover in a few years, 20 years, or never. My bet is that the economy will sink slowly rather than an absolute crash (yes, call me an optimist). In any case, the middle class will largely go away, transitioning into an "upper lower class" that still has some qualiy of life, but will struggle under huge debt loads, etc.

Google vs. Microsoft: Microsoft was in the right place at the right time in technology and economic history. They got lucky. Google on the other hand gained market dominance against existing market leaders. This difference, along with their flexible and scalable architectue for large scale web applications, is why I expect Google to improve their position in time while Microsoft is probably looking at a long (very) slow slide downhill.

Sunday, September 18, 2005

Reading code and tech papers

People get very good at games like chess and Go by studying master games. Software design and coding is similar in that we learn patterns and new ways of approaching problems.

I really enjoyed reading the paper on Google's map reduce library this morning. A great way to increase programmer productivity by abstracting away details of handling huge data tuple sets. The paper is detailed enough to make me feel like I could implement their ideas (given the Google file system infrastructure).

While good technical papers make us better at strategy (to keep up the chess and Go metaphor), reading other people's code helps develop good tactics.

Better power management under Linux than Windows XP

I have a Averatec laptop that I dual boot. I get better power management under Ubuntu Linux than I do under Windows XP - with similar 'power saving' settings. I am almost sure that this is because there are fewer memory and CPU hungry processes running under Linux. So, when I am reading online, under Linux it is easier to shut down the disk and fan because everything runs in memory and the CPU is largely idle.

Speaking of Windows XP, although in some instances it is a good OS for coding, it always makes me a little uncomfortable having so much stuff installed by default. Although I wrote a commercial Windows product (SAIC ANSim) using Windows 1.0 beta and delivered on Windows 1.0 and 1.03, Windows NT was the first version of Windows that I liked. I had a "secret" for using Windows NT effectively: I had one boot partition that was totally stripped of everything but my software development tools and another that I would load a bunch of junk on - the trimmed down version with no spurious applications installed was incredibly stable. I have thought of doing the same thing for Windows XP but I don't think that XP's copy protection would let me do this.

I used to use a shared FAT32 partition to share design artifacts and source code between dual booted operating systems, but now it is just so much easier to keep everything in a CVS repository on a server. As I mentioned the other day, this makes it easy to use Windows Linux, or OS X for a task, picking the best platform per job.

Saturday, September 17, 2005

Automating development tasks ...

... is so important! I am going through a tedius process today of getting some of my compiled Java code deployed on a customer's little hardware device. (No, Java is not always portable, but that is OK).

I thought that deploying and testing would only be a few hours work (ha!) but I took the time to automate a compile, package, transfer and burn to device script. This was ugly because I have to work on Windows because of required utilites, so I had to use .bat files.

Anyway, in addition to saving time in the long run, a really good automated system is making this process more fun. I have now gone through about 4 iterations of automating this process, and it is time well spent.

Tuesday, September 13, 2005

Matter of attitude: switching between Windows, Linux, and OS X

Java may be portable, but I still feel the need (and enjoy) switching between Windows, Linux, and OS X as per the requirements of work and research. For Java, I like to customize applications (if they are JFC based) to use tray icons under Windows and support the Mac top of the screen menus under OS X.

There are other good reasons for keeping an open mind to which computer to boot to best get the job done. If I want to use Common Lisp, I find it easiest to run Linux + Emacs + SLIME + SBCL (although I own LispWorks licenses for all three platforms, so I sometimes use LispWorks). For reasons that are not totally clear to me, when I code Ruby and/or Ruby on Rails, I like OS X.

When I am working on a huge Java web framework for one of my customers, I like Windows because of the total "niceness" of using ant build files in TextPad shells.

Anyway, years ago, it used to bug me to have to reboot between Windows NT (and later 2000) and Linux. Now, if I want/need to run under a particular operating system for a few hours, the overhead of switching seems really light and I do not mind doing it.

A few things make it easier to switch (rapidly) between operating systems:
  • Use source code control (either CVS or subversion) for everything - then a fast CVS update gets a development machine current in a few seconds.
  • I mostly use GMail so no more hassles keeping email straight on multiple development systems (my local ISP still does not support IMAP).

Wednesday, September 07, 2005

Where is Apple's iPod/cellphone?

Rumors were of an announcement yesterday. I am just about the only person I know who does not own a cellphone: when I am in my home office working I am (and I should be) instantly available to my customers, family, and friends. However when I am out of the house, I would find it an annoyance to even receive messages - that is my time off. If I am wilderness hiking I will occasionally borrow my wife's cellphone in case of an emergency but I keep it switched off.

I think that I will get Apple's combo iPod/cellphone if they do bring it to market. My MP3 player is over 5 years old and only stores about 80 minutes of music. Also, I can leave the cellphone turned off.

Tuesday, September 06, 2005

That does it - I am going to sell my Microsoft stock

I believe that putting the customer first is the way to run a business. I do not like their position on supporting the international OpenDoc standard.

From a short term business point of view I can understand why Microsoft might feel that it is necessary attempt to lock in customers with proprietary file formats. I believe that such a policy that is so very much against the interests of their customers is a bad business strategy.

As a stockholder, I wrote Microsoft a letter on this subject early this year. I received no response from them, which I find rude corporate behavior.

Sunday, September 04, 2005

Extending the Windows shell to explore JAR files

I started using TortoiseCVS on the advice of a customer. TortoiseCVS extends the Windows UI shell to add CVS functionality as right mouse click menu options - really useful!

I have been looking for similar Windows shell extentions for adding explore Java JAR file options. The best that I have found is Raphaƫl Mounier's ICEOWS package. I used to change a JAR file's extension to .zip, explore, then change it back - a waste of time.

Peter Norvig and Joseph Campbell

I had lunch with Peter Norvig about 18 months ago - yes, as you would imagine, he is a very cool guy - lots of fun to talk with. I just saw a link to his article "Teach Yourself to Program in 10 Years" and re-read it.

I think that his comments go way beyond programming. They remind me a lot of Joseph Campbell's advice "follow your bliss": do those things in life that you most love to do.

While I make my living designing and writing Java software, I invest a huge amount of time studying other programming languages; just in the last few years I have:
  • Studied Paul Graham's great Common Lisp books - I wrote 2 Lisp books for Springer-Verlag many years ago, but I was rusty...
  • Studied Mark Pilgrim's excellent "Dive into Python" to brush up on my Python skills
  • Learned Ruby (see my open source Ruby NLP projects on my main web site)
  • Brushed up on my Prolog skills by working through Sterling and Shapiro's "The Art of Prolog" - I used to be a Prolog guru, and I wanted to get some of my edge back
I also spend a minimum of 5 hours non-work time a week learning new Java frameworks, reading other people's Java code, articles, etc.

How do I get time for this and still have a great life? I suggest not watching TV :-)

When I talk to teenagers, I like to encourage them to follow career paths that involve doing things that they love doing - don't just try to make a lot of money.

Saturday, September 03, 2005

new web portal, some interesting implementation details

I thought about writing a free community web portal ( just for my own community but decided to scale it up to support the U.S. The most interesting technical problem was getting free ZIP code data from the US government, and writing a little code for approximating the distances between different zip codes - I needed this because I wanted people to be able to enter their own zip code and search for all events within a specified distance. This simple web portal took me two evenings to write so I am hopeful that it catches on in my own community. It would be very cool if a few other communities also started using it also. It is currently on a very low cost VPS server and should scale fairly well for use by several hundred communities; more than that and I will move it to a non VPS server.

Friday, September 02, 2005

Best movie of the year: "The Constant Gardener"

For sure Ralph Fiennes and Rachel Weisz will be up for Academy Awards for their roles. Definitely not a movie for kids, but for adults it is a must see. I have spent some time in Kenya and the movie captures the feeling of the place: warm people, but lots of sad situations. If I may, I suggest seeing this movie in the theaters, and not waiting for it to come out on DVD. I don't want to spoil the plot, but as Rolling Stones movie revues said: "[Director Fernando Meirelles] and screenwriter Jeffrey Caine put a human face on John le Carre's novel of sex, lies and dirty politics in modern Africa. Prepare for a thrilling ride."

Wednesday, August 31, 2005

I moved to a much faster server

A few people noted that our recipe web portal occasionally ran slowly; it was running on a very low end virtual linux server during development. I just switched the DNS records so now points to a much, much faster server.


Tuesday, August 30, 2005

MySQL optimizations; free Opera browser license

OK, I know that there is a large overhead for doing single table inserts, even maintaining a pool of connections. This little fact was very obvious this morning: I had a one-shot little program that generates data for about 5 million rows in a database table. The calculations themselves take a while (about 5 minutes on my PC), ignoring database access. Last night, right before I went to bed, I added a few lines of ad-hoc JDBC calls to write out my 5 million rows after everything was calculated in memory. I woke up this morning and it was still running.

So much for being a little sloppy, even for a one-shot (throw it away after it runs) program. I re-wrote it batching up just 2000 inserts at a time (insert into... values (..), (..), (..)...) and the database access code ran in just 20 minutes. After the data was added, it took another two minutes to index the table (obviously you want to do this *after* adding the data). I made a note to myself to review my production code - I think that I almost always batch up inserts when appropriate, but I want to make doubly sure!

re: free Opera browser license: as seen on Slashdot: a short term free license giveaway. I grabbed a free license so I have one more popular browser to test my web applications against.

My IntelliJ review was not well received

Oh well. An editor at DevX asked me to write a review of IntelliJ 5.0. I had never written a product review before but I do *like* IntelliJ, so I thought 'why not?'. Anyway, the reader ranking for the review is low - I wish that people could leave comments because it would be good to get feedback.

Monday, August 29, 2005

Great book on DHTML/Javascript

One of my customers asked me to "beef up" on using Javascript, AJAX, etc. I have been reading a great book: DHTML Utopia.

Usually, I dislike working in Javascript but this book has really turned me on to some good techniques. If your web applications are running on a local area network (low latency), doing every thing on the server is fairly OK but it is better to do some things locally on the client browser when running over the internet. Human usability studies have shown how productivity-destroying it is to make users wait for more than a second or so when they are working.

I especially like the way Stuart sticks to 'the standards'. Personally, I refuse to spend too much time worrying about Internet Explorer strangeness (unless someone is paying me to - even then I don't like it).

We are rebranding our new web portal to

CJ are my wife's first and middle initials, and a nick name. For about 7 years, I used my site for technology demos - a low level of effort. I recently decided to invest in the creation of three public service (i.e., free, but I will probably add a few Adsense ads eventually) web portals:
  • - our knowledge based cooking and recipe portal. Until this morning, this was our site. My wife likes the name change :-)
  • - a web portal (under construction) for announcing local events, searching for events by ZIP code and distance from where you live, etc. This is also an experiment in architectures for scaling for large numbers of users: I have a budget for how much server costs I am willing to donate for this.
  • - a news clipping service that generates customized RSS feeds. (under construction)
I love working on interactive web portals; good thing since this is also how I spend most of my time when consulting for customers.

I mostly enjoy the rapid feedback: sometimes I can almost have a change to a web application done before I get off of the telephone with someone requesting an enhancement or bug fix. While I sometimes think that I would rather be working in a more dynamic environment (like Ruby on Rails, or UnCommon Web for Common Lisp), my development environment rocks: IntelliJ working with JavaBeans, JSPs, custom tag libraries, and some Javascript and CSS. My edit/build/redeploy cycle is so quick that very little of my working time is spent waiting.

The other great thing about web portals is the low cost of hosting. Running Tomcat or JBoss is fairly light weight and efficient. Eventually, all dynamic code gets compiled and run through HotSpot, so the performance is good also (even on the really low end servers that I usually deploy to).

Sunday, August 28, 2005

The "Mom software modification"

My Dad (interesting guy: in his mid 80's and still a member of the US National Academy of Sciences) called with the bad news: my Mom did not like the way food ingredients were showing up on the knowledge-intensive recipe web portal that my wife and I are working on.

Apparently my Mom did *not* like seeing "0.5 cups" instead of "1/2 cup".

Fortunately, I did not have to change anything in the model or controller layers - just one small bit of code in my presentation layer had to be changed.

BTW, for people interested in artificial intelligence: check out the "AI agent" page at

Great George Orwell quotes

Thanks to reddit I discovered the site Here are some great quotes from George Orwell (about 5% of those on the linked site):

Every war when it comes, or before it comes, is represented not as a war but as an act of self-defense against a homicidal maniac.

In a time of universal deceit - telling the truth is a revolutionary act.

All political thinking for years past has been vitiated in the same way. People can foresee the future only when it coincides with their own wishes, and the most grossly obvious facts can be ignored when they are unwelcome.

All the war-propaganda, all the screaming and lies and hatred, comes invariably from people who are not fighting.

The nationalist not only does not disapprove of atrocities committed by his own side, but he has a remarkable capacity for not even hearing about them.

He was an embittered atheist, the sort of atheist who does not so much disbelieve in God as personally dislike Him.

War against a foreign country only happens when the moneyed classes think they are going to profit from it.

When it comes to the pinch, human beings are heroic.

Saturday, August 27, 2005

Fast progress on

I am jazzed: except for the "AI agent recipe creator", the software for the KBrecipe knowledge intensive recipe portal is just about done. Mix a little Java, little AI...

My wife and I have a strong interest in both cooking and healthy food; this portal is our gesture at doing a public service :-)

If you have a few minutes, please try it out - I am interested in feedback. The implementation just uses some standard stuff: JSPs, struts, and a custom database layer (no object relational mapping). My wife and I will hopefully get more recipes entered into the system in the next few days - right now there are only 8 recipes in the system for software testing.

Friday, August 26, 2005

Great (and free) Lisp programming environment

It is good to know more than one programming language: Common Lisp is one of my favorite languages and I often encourage other programmers to learn it. Peter Seibel (author of " Practical Common Lisp" and well known in the Lisp community) has put together a complete programming environment LispBox that integrates a free Lisp development system (CLisp, OpenMcl, or AllegroLisp) for Windows, OS X, and Linux. He included Emacs, Slime, and ASDF (a package system for automatically loading systems from the web).

Neat stuff and a huge time saver if you want to set up your own Lisp development system.

Efficient capital markets

With only a few caveats, I believe in globalization and free market economics. Production and all work should be done in the most efficient way possible, factoring in long term environmental damage to production costs. I just read an interesting guest editorial in the Asia Times (taken from the author's informative and useful blog) that makes a good point about efficiency in international markets: that a global economy driven by tough efficiency will naturally evolve away from using the US dollar for oil and other commodity trading when using the dollar adds cost to all involved (except for the US).

Unfortunately, I think that my country (the US) will continue to take strident and largely ineffective steps to try to prop up the dollar as the primary currency for buying and selling commodities. I believe that it is better for us to concentrate on efficiency through an educated work force and investments in infrastructure than to try to sustain a very good deal that we have had since the early 1970s (i.e., getting off the gold standard, float the dollar, and deals to make the dollar the currency for buying and selling oil). This was a great deal, we should appreciate our good fortune, but realize that this good deal will not last forever.

Even with the current tragedy in Iraq, I am as hopeful for the future of the world as I have been in a long time. I believe that efficient world markets with tightly intertwined economies will bind people together and hopefully lead to more peace and world wide prosperity.

Monday, August 22, 2005

beta version of is available

My wife and I have been working on a public service 'healthy recipes' web portal. For the very brave, a beta version is available at:

I just updated my database schema and data to incorporate some of the USDA nutritional database. After the next software update in a few days you will see nutritional data associated with each recipe.

You can use the portal without logging in, but you will be missing a lot of features: you can maintain a private database of food ingredients that you have on hand and optionally only request recipes for which you can make with what you have in your kitchen.

There are few recipes available right now - I am mostly working on the infrastructure before my wife and I start entering more recipe data into the system.

Google: Desktop 2 beta, $4B for implementing my idea

I tried out Google Desktop version 2 beta after getting home from a friend's birthday dinner late Sunday night: it has a high 'cool factor', but I might tire of the sidebar (that said, toggling between sidebar and taskbar placement is just a mouse click - so, you can get it out of sight quickly while working). Anyway, I think that having their own desktop fits in really well with their information management (and probably in the future knowledge management) business model.

With their new stock sale, Google has plenty of money for internal development and for purchasing tiny technology companies (mostly to get the people).

Like many people, I have an idea for how Google should spend their money :-)

I think that it would be very interesting (but a huge risk) to try writing a Windows/Linux replacement desktop that would act the same on both systems.

For Windows, they would provide a new shell (other people have done this) that would integrate file browsing, search, program launch, command shell (of course!!), FireFox based browsing, etc.

For Linux, the implementation would be a new Window Manager.

In addition to an expensive development effort, there are a few other large problems to be solved:
  • Microsoft would try to torpedo the Windows shell replacement - it would be very bad for their business model.
  • It would be difficult to support the multitude of Linux distributions
One approach for might be to have a Google Linux distribution, but that is really moving away from their information management business model. To make this work, their complete desktop replacement would have to work really well in corporate environments.

Sunday, August 21, 2005

Windows vs. Linux on Slashdot; plain text

I have a little free time before a friend's (who has a picture of himself on stage at Woodstock with Hendrix :-) birthday party tonight - so, I treated myself to some low productivity time on Slashdot.

Another re-hashing of problems with Linux and Windows, but there was something interesting to me: the standard complaints about Microsoft's technically weak, but money making shifting file formats.

Time to step back: why do we write down information? Mostly to record thoughts and information we or other people will need later. I think that formatted text is way overrated.

Notes, memos, short users' guides, email, source code, design notes, requirements, use cases, etc. can all be expressed quite well as plain text files.

Plain text files are easier to store in CVS (even with subversion's binary diff'ing, plain text works best in subversion also).

I have never had problems reading a ten year old plain text file.

Sometimes a little formatting is good - XHTML (or HTML) with limited markup is easy to edit and can be presented nicely with CSS.

I am not just bashing Microsoft Office here: how many people or organizations really need things like DocBook?

If you must write formatted text artifacts, at least use for the transparent file formats.

If someone sends you memos as Word documents, try asking for plain text - convert someone to a better way of working.

Saturday, August 20, 2005

I upgraded to version 5 of IntelliJ

As I just wrote in a review for DevX: JetBrains is very smart to give a short term price promotion for the update because they are their own best competitor: I found IntelliJ 4.5 just about perfect, so without the deep discount (until the end of August) I probably would not have bought the upgrade. For me, the nicest improvements are in Javascript editor support.

There are better programming languages than Java (pick a dynamic language: Common Lisp, Ruby, Smalltalk, Python Scheme, etc.) but there are three things that keep me focused on Java:
  • The quality of IDEs like IntelliJ, Eclipse, and NetBeans
  • The great (and free!) infrastructure software for web portals (Tomcat, JBoss, etc.)
  • Most of my customers insist that I use Java

Friday, August 19, 2005

OK, HSQLDB is cool after all

I complained a week ago about stability problems when using HSQLDB as an embedded database engine. By using a JVM shutdown hook, I totally solved my file locking problems.

Using JVM shutdown hooks is easy; first define a class that extends the Thread class:
class MyShutdown extends Thread {
public MyShutdown() { super(); }
public void run() {
System.out.println("\n\n*** MyShutDown thread started ***\n");
try {
} catch (Exception ee) { ee.printStackTrace(); }
Here I am assuming that my JDBC access utilities are in a class DbUtil that has a static method close_connections() for sending HSQLDB a "SHUDOWN COMPACT" command.

Registering a JVM shutdown hook is also easy:
   MyShutdown sh = new MyShutdown();
Make sure you only register the shutdown hook exactly one time.
When the JVM that is running my application terminates, my shutdown hook is called. This works well for both running unit tests and when used inside a Tomcat container.

I use JVM shutdown hooks fairly often - it is useful to be able to perform cleanup work as a JVM terminates. On Linux servers, be sure to shutdown Tomcat (or JBoss, etc.) cleanly. Shutdown hooks are obviously not called if there is a server crash.