Wednesday, December 29, 2004

Combining Java with other languages

How often is language choice dictated by the availability of existing libraries? For me, quite often! I have been looking at two approaches to leveraging Java code for alternative languages/platforms:
  • IKVM - a JVM implementation (uses GNU CLASSPATH) that runs inside Mono and Microsoft's .Net CLRs
  • PyLucene - the large Java Lucene search application is compiled using GNU gcj into native code and then wrapped using SWIG as a Python library
I have used PyLucene with Python: great to be able to effectively use a large Java application in Python. I have not experimented with IKVM yet, but for a practical Java programmer who might need to reuse some code in the .Net framework: it looks promising.

Monday, December 27, 2004

Lisp vs. Java products, ant build process

A few weeks ago, I was talking with one of me consulting customers and we started discussing my commercial KBtextmaster product. I wrote the original 0.x version in Common Lisp and I am almost done with version 2, also written in Common Lisp. My customer stated the opinion that the marketability of compiled Lisp products is small compared to products written in Java. I decided in the last week that he was correct so I dusted off some old Java code (a port of the Lisp code to Java) written a few years go and decided to 'bite the bullet' and starting with the old Java port proceeded with porting over the new Lisp code to Java also.

Anyway, I spent about 5 hours today "coding like crazy" and I was surprised at both how easy it was to port changes over from Lisp to the old Java port code-base and to also update some of the old Java code with newer version 2 algorithms. If my consulting workload is not too much, I expect to get everything done in Java in a few weeks. I also decided to try a different marketing approach: I am going to sell the product for a very low price with a non-commercial (or educational) license and price it quite a bit higher with a commercial use license.

One of the things that I did today was to spend 20 minutes setting up a good ant build process to manage building and deploying the Java code. Ant is so useful to me that I have not bothered looking at alternatives like Maven (which is layered on top of ant).

While I had the flu last week, I made a first cut at a color brochure for the new KBtextmaster product - when it is done, I will post a PDF file :-) I got the idea for doing a color brochure from the people with the semantic web Sesame project: they produced a brochure for Sesame that does a very good job at promoting Sesame (in my opinion).

Saturday, December 25, 2004

Cut down the trees: payoff to political cronies

This is a sad thing to read Christmas morning. The first effect will literally be in my own back yard (I live in the mountains of Northern Arizona).

My criticism is not just of the corrupt(*) Bush administration: the Democratic leadership in this country are also firmly "in the pockets" of large corporations that are no longer even American companies (since Clinton and Bush both worked to make it easy for large corporations to register off shore and avoid tax burdens).

Our government, both democrats and republicans are now the "best that money can buy". A few people in Congress still have some moral fiber and do more or less what is right - but they are a small minority.

Welcome our new corporate overlords :-(

Oh well.

(*) My opinion because Bush put ex-industry lobbyists in charge of oversight of their industries.

PS. I do study history and understand that it is the nature of people in power to become increasingly more corrupt: this has been going on for thousands of years. I believe that the important thing is for people to continually push back by calling elected representatives to task for their transgressions.

Wednesday, December 22, 2004

I may be Java developer, but...

I have sold two C# products in the last 2 days :-)

It definitely seems worth while porting at least some of my text-mining utility libraries from Java to C#. Most of my sales are for Java products with Python and C# about tied for second.

Oh, the crap they try to feed us

I am taking it easy, recovering from flu, so I have the TV on to the news. A headline news item was something like "US economy doing well: GDP is up 4.5%".

I hope that everyone knows what a load of crap Gross Domestic Product (GDP) numbers are - GDP is a poor measure of economic health. If the US has to borrow even more money from foreign banks to pay for government operating expenses, that increases the GDP. Poor quality products give the GDP a double kick upwards: cheap prices promote sales and products have a short life time and must be replaced. Anyway, there are many factors that are clearly (at least in my opinion) bad for our country yet increase the GDP.

The GDP goes up every year no matter what is happening. I wish our government and the news media would stop using meaningless statistics like GDP and get real. BTW, Dave Pollard has a good writeup on this subject.

Monday, December 20, 2004

Open file formats and AbiWord 2.2.2 for OS X

As an author, I own licenses for Microsoft Office for Windows and the Mac. Still, I try to avoid the use of Word as much as I can because of the horrible Microsoft closed file formats. Frankly, I can not understand how large companies and governments can justify the use of closed source proprietary file formats.

When I primarily used SuSE Linux for my desktop, as long as I had a truetype font server set up correctly, I was very happy using (Strangely, OOo runs really well on Windows - go figure.)

Now that I have almost completely converted my business over to using Macs with OS X, I have found myself grudgingly using Word for the Mac (which is quite good). OOo under X11 on the Mac is usable, but far from perfect. I think that the people working on the NeoOfficeJ project (OOo running natively on OS X using Aqua) are doing a great job (I donated money at their site today - in my own interest to encourage people whose work I use!) and I find NeoOfficeJ to be very usable and getting better. In the future, NeoOfficeJ is likely to be my standard writing tool.

I gave AbiWord another look tonight - they have a minor version release since my last look. I liked what I saw:
  • AbiWord file format is clear and easy to parse XML
  • Light weight - loads in a few seconds even on my old iBook
  • Handles imported graphics well - I just save OmniGraffle drawings as TIFF first
  • Seems to generate tables of contents well (but no index building features - needed for very long documents)
I intend to give AbiWord a fair trial. If I do have to occasionally keep using Word for a while on my Macs, at least OOo has a great batch feature: searches nested directories for Word documents and generates OOo XML document files. In a few minutes in batch mode I can generate OOo XML format files for all of the Word documents on my system.

Great! a web site for home/amateur movies is a very cool site. I am down with the flu today so I have been doing more fun-oriented web surfing than I usually do. This site contains excellent short movies - lots of fun.
Creativity space is infinite...

Lisp world: the old and the new

The new: the "so good I can't believe it is free software" DrScheme Scheme development environment has a new 2.09 version available now.

The old: over on the Lisp blogs recently, I saw a reference to the availability Xerox Parc Medley Lisp for non-commercial use. It is bundled with the LFG grammar toolkit. It is available here.

This is the Lisp that I used in the 1980s on my Xerox 1108 Lisp machine. I downloaded the Medley Lisp and LFG grammar bundle (runtime converted to run on Linux) and took a trip down 'memory lane'. (I am in bed with the flu - not well enough to work, but well enough for messing around with old software).

While I use Common Lisp for my Lisp development now, I am so happy to have a non-commercial version of Medley. Wowza!

Sunday, December 19, 2004

Supporting infrastructure for nurturing creativity

I love coincidences in life. I was just thinking how much I enjoy independent and/or non-formula movies (e.g., both Indy films at Sundance and the fabulous Chinese movie "Hero" that combined an amazing plot twist with eye-candy cinematography). A few minutes later, I saw this article about the British plan to install about 250 digital projectors in theaters throughout the United Kingdom. This will open up the possibility to produce and distribute very low budget movies made with digital video technologies - a very good thing! I would like to see the same thing in the USA.

I admit to being too optimistic, but I look forward to a future where instead of lusting after excess material goods (the production of which places too high of a burden on our delicate ecology), people will become more interested in low-resource cost creative activities like getting together to produce low budget movies and plays, writing, playing games, cooking gourmet food, crafts, writing software <grin>, etc., etc.

Creativity in play and work needs a little infrastructure, but not much compared to expensive social support programs and waging wars. Although the world is suffering from some bad leadership (even a little worse than usual, I think), I am somewhat optimistic about the future.

Why should the USA have to be a net importer of food? And, starving children.

For the first time in about 50 years the US is importing more food than we are exporting.

Another legacy of the Bush administration economics. As Dave Pollard said (unfortunately this is accurate) "Rich Get Richer, Poor Children Dying" is the likely legacy of this era. 400,000 Iraqi children with long term health threatening chronic diarrhea and 1 billion children in the world living in poverty. As US citizens we are responsible for the Iraqi children and we share the shame of 1 billion ill-fed children with the rest of the world.

I like what Colin Powell told Bush about Iraq: "You break it, you own it". Unfortunately Bush seems to be trying to (metaphorically) sneak out of the store after breaking something. Bush's concept of being a Christian and my concept of being a Christian differ greatly, BTW.

Speaking of religion, I received an interesting letter from an official with the Quaker American Friends Service Committee (one of my favorite charities - most of the money donated to them actually goes to helping people build sustainable clean water supplies, build small scale food producing infrastructure, etc.). In this letter is the claim that I believe that many Christian religious leaders in the US personally tried to contact Bush during the lead up to the invasion of Iraq and Bush refused to have any contact with religious leaders who did not agree with his radical foreign policy goals.

Friday, December 17, 2004

NetBeans 4. Free vs. commercial development tools.

I spent some time 'kicking the tires'. Version 4 looks to be a major improvement both in edit/build/test mode and in web app development functionality. It is great that a good IDE like NetBeans is available for free. However, since I earn most of my living designing and writing Java code, I feel that it is worth while buying a 'best of breed' commercial product (which, in my opinion, is IntelliJ).

Same thing with Common Lisp: there are awesomely good free Common Lisp compilers available (e.g., OpenMCL, CLisp, SBCL, CMU). Still, the free tools are just not at the same level as the Xanalys LispWorks and the Franz's Allegro CL. The commercial products provide a richer development environment and library support. Allegro CL is out of my budget price range, so I happily use LispWorks.

Python is the other programming language that I use a lot. I am still trying to find a 'best of breed' Python IDE. I mostly develop on Mac OS X so the ActiveState IDE is not a possibility. I have tried a trial version of the WingSoft IDE (runs under X11 on Mac OS X) and it is good but just not at the same level as IntelliJ or LispWorks. So, I just use Emacs (for about 15 years, I did all of my coding in Emacs, but productivity-wise, Emacs is simply not competitive with commercial IDEs like IntelliJ).

Anyway: great work on the new NetBeans release!

Wednesday, December 15, 2004

OS X 10.3.7 upgrade: wowza!

What a great upgrade! 3D graphics in Croquet is noticeably faster and scrolling long web pages in Safari definitely is smoother and faster.

I used to be a big Linux fan (a Linux user since downloading Slackware over a 2400 baud modem over 10 years ago), but Mac OS X is such a better user experience (for coding, writing, web access) that I just use Linux on my servers now. Oh well.

It is even better that I hardly ever have to boot a Windows machine anymore (but I do keep one handy - work for some customers does occasionally require it).

I seems a little funny that I even care about operating systems: my favorite software (IntelliJ, LispWorks, Emacs,, FireFox, etc.) all runs well on OS X, Windows, and Linux. Still, using a Mac just seems so much nicer to me - really, no comparison.

Sunday, December 12, 2004

I think that I will buy a PlayStation Portable (when available in the US)

I was fortunate enough to work in the game industry for a few years (hired to do 'game AI', but also did multiplayer networking, and lots of other fun stuff), working on two Nintendo games, a PC networked hovercraft racing game, and a virtual reality system for Disney. Anyway, I think that Sony has hit it right with the PlayStation Portable: a balance of great style, features, and portability.

I like Sony products because for me they have 'lasting power'. I bought my Sony DSC-F55 digital camera (with audio & video also) 5 years ago, and I still like it. Until it died a tragic death, my Sony Vaio mini-laptop was a prized possession (and my wife also enjoyed it a lot).

I tend to not buy many toys - mostly because I get more pleasure from time with family and friends, and free activities like hiking. However, when I do buy things, I like to buy things with style and quality. Another good example is my Mac iBook: bought it two years ago, and I still get a lot of pleasure from its style and functionality.

Quality over quantity...

Monday, December 06, 2004

Too cool: a 'Comic Book' to teach you Lisp (online web site)

Conrad Barski (M.D.) has written a great comic book style web site to teach you (yes you!) how to program in Common Lisp. Check it out.

I also have a more conventional Common Lisp tutorial that I wrote(*) 2 years ago available: Loving Lisp, or the Savvy Programmer's Secret Weapon.

(*) actually, so many people in the Lisp community helped so much that I feel that it was a group effort.

Saturday, December 04, 2004

Most productive programming languages and platforms

For me this is a tough call. I am an "old Lisper" (I wrote two Springer Verlag Lisp books a long time ago) and while I have been coding for a new product today using LispWorks Common Lisp, I have to say that when it comes to algorithm development and new coding, nothing comes close to Lisp for programming productivity. I have also been working through Mark Pilgrim's fine 'Dive Into Python' book (*), and I certainly consider Python to be another high productivity language.

That said, for so many of my work tasks, the breadth of Java platfom tools "nukes" the productivity advantages of Python and Lisp. As I have mentioned recently, I am finding XML-RPC to be an excellent bridge for tying together Lisp, Python, and Java code. But, there is a definite performance overhead for XML-RPC calls. Anyway, I keep finding myself making the following decisions over, and over again:
  • If my job task requires any J2EE functionality, then code in Java
  • Else: is it a small job and runtime performance is not much of an issue: then use Python
  • Otherwise: use Commn Lisp for programmer productivity and superb runtime performance
Sorry if this seems like a stupid blog post, but: choosing language/platform is an issue that I face all the time. In the best of worlds, I would have the entire J2EE stack (and other Java goodies) available as Common Lisp libraries and tools.

I can imagine a year or two from now when the Python tools and available software stacks "catch up" with Java - then I might just switch over to Python as my primary development language (and ignore performance problems).

(*) I have started once again seriously studying Common Lisp (mostly with Paul Graham's books)

Wednesday, December 01, 2004

How much productivity is lost due to Microsoft software?

We have just 2 Windows PCs (2000 and XP), but we have 3 Mac OS X systems and one linux box. I spend so much more time keeping the Windows PCs running properly than all of the other computers together, and by a factor of 2 or 3.

Can you feel my frustration :-(

Really, the best computers are the OS X machines: everything works with minimal administration activities.

I think that the problem with Microsoft Windows is simply that marketing and adding new features is more important than producing stable, relatively secure software.

Saturday, November 27, 2004

I really appreciate Google Adsense ads

I just got another check from Google for having Adsense ads on my web sites. Sweet - pays for all of my server and internet costs, and then some... I use up a fair amount of bandwidth because sometimes I often have several hundred copies of my free web books downloaded a day. I also get a surprising number of contributions from people via PayPal for my free web books. I am not sure, but if I stopped consulting and wrote full time, I could probably meet my living expenses by writing free web books(*) and accepting donations. (But, I enjoy consulting too much: I love solving problems). Anyway, I appreciate Google Adsense ads - give them a try yourself.

(*) speaking of which, I have been working such long hours consulting the last 7 or 8 months, that I have not had too much time to write. I am now taking a 4 or 5 month break and only consulting about half time (for a pleasant change), so I will hopefully get some more open content material written. (But I am also trying to wrap up work on a commercial product.) Paul Graham gave me permission earlier this year to use his algorithm in his plan for SPAM paper for extra chapters in my Java AI and Common Lisp Programming web books - that is interesting stuff, and will not take too long to write up.

Thursday, November 25, 2004

Authoring plain web pages vs. Semantic web pages

Very few people add RDF (etc.) to their web sites, preferring to spend effort creating "stylish" looking web sites to creating web sites useful for both software agents and human readers. I am a little guilty of this myself: I have some RDF data on my main site, but it does not reflect the real semantics of my site.

The problem is that when adding new information to a web site, ideally there would be some "standard" OWL ontology that perfectly reflected the subject matter for the new material and an authoring tool would both help "fill in the blanks" to create an instance and provide choices for XSLT to render this instance to HTML. Perhaps an extension to the Protégé system could do this. The problem is still that few people would want to think in terms of what is really knowledge engineering just to add material to web sites.

What is needed is a set of tools that cover all of the bases here: authoring tools that allow fast selection of an appropriate ontology and then instance creation. Then, there would have to be a light weight framework (perhaps server side PHP) to render HTML from an OWL instance and the author's preference for XSLT (assuming that each OWL class also had several available XSLT styles available in the authoring tool -- I am really wishing here!).

Anyway, none of this will happen soon, but I wish that it would. Authors will only want to enter information one time so I am skeptical of systems that have a content creator write content for human readers, then separately try to play nice by supplying RDF, RDFS, OWL, etc.

A "holy grail" of AI research would be to solve the hugely difficult problem of knowledge acquisition directly from text - let's not hold our collective breaths waiting for that to happen: I have been doing some personal research trying to write highly tailored semantic extraction code for a very simple OWL ontology. What a feeling of deja vu: reminds me of my efforts in the 1980s to write conceptual dependency (CD) parsers.

Saturday, November 20, 2004

Weird: so now Alan Greenspan and I are in agreement?

I have been very critical of Greenspan, the Bush administration, etc. for not giving the straight poop to the American public on the danger of widening trade and budget deficits. Now that the election is over, we are finally seeing some honesty.

A good quote from this article:
"What he's [i.e., Greenspan] saying is that the nation's balance sheet is out of whack, and the family balance sheet is out of whack,'' Meese said.
What I believe the real tragedy is: with a little more saving, some real reduction in spending (on a personal level, buying less crap; at the federal level, fewer expensive wars and social programs) everything would be OK for (probably) a long time.

Friday, November 19, 2004

Apache Daisy CMS sytem

I installed the latest release of Daisy this evening. It was a tough install (about 25 minutes to get the MySQL tables created, configure OpenJMS which is required, edit lots of config files, etc.). Anyway, the effort seems to be worth it! Like the Magnolia CMS system, when using a Firefox web browser, you can create/edit rich text HTML documents. However, Daisy also supports "attachment document" that are simply containers for, Word, etc. documents.

Daisy uses the Lucene search engine and both Word documents and plain HTML documents that I created were almost instantly searchable.

I like it!

PS. I have been planning on integrating my next release of KBtextmaster with Plone. I think that I will also supply a plugin for Daisy. KBtextmaster is a standalone service with an XML-RPC interface (and written in Common Lisp, BTW). I am tentatively planning on making any patches to Plone and Daisy (for accessing KBtextmaster vi XMLRPC) public domain. I have not really made up my mind yet - I still have about 150 hours left getting KBtextmaster ready so I will make final decesions on how to market it later. My consulting work load is fairly high right now, so I will be lucky to have a new version ready for sale by next spring.

Magnolia v2 Java-based Content Management System

I had a short while to download and install Magnolia Content Management System. It is designed to allow non-technical people easily build and maintain web sites. So, this is a web site content management system, not a full blown content management system that handles many document types, etc.

You can upload documents (e.g., documents) to Magnolia, but they appear as a named link on a web page.

The install was very easy - by default, two Tomcat instances are run: one for the authoring system and one for the "public" system.

Like the Apache Daisy content management, Magnolia seems like a Plone "want to be". For Java developers, the advantage of Magnolia and Daisy is that they are Java web applications (Plone is written in Python). Daisy has the advantage of supporting many file types (, Word, PDF, etc.) for indexing, search, etc.

Wednesday, November 17, 2004

Distributed knowledge workers

Knowledge workers are software engineers, writers, business development people, architects, etc. - anyone who deals with organizing ideas and creates intellectual property (hopefully licensed under a Creative Commons copyright, open source software, etc. - but also proprietary intellectual property (*)).

Stephen Covey, author of The 7 Habits of Highly Effective People says (in Business 2.0, December 2004 issue):
The industrial age was about control, and the information age, or knowledge-worker age, is about release.
Covey gets this right: in the new economy and new way of doing things, trust and flexibility is key.

Think of the revolution in manufacturing processes through "just in time" delivery of material and sub-systems. The new knowledge economy, even acknowledging the dot-com meltdown, is also about flexibility in getting work done on schedule by pulling together human resources just as they are needed. There is no doubt in my mind that being able to tap into talent on a global scale facilitates this type of flexibility: make time zone differences work as an advantage, form a trust network of people from many countries who can work together, etc.

Networks of trust and cheap communications infrastructure are what makes this happen.

A similar thought: also in the Business 2.0, December 2004 issue: a writup of technology innovators who chose to save lots of money by setting up their new companies in rural areas (that had great broadband support) in order to drastically cut the cost of doing business. This resonated with me: I live in a fairly remote area in the mountains of Northern Arizona - but, I spend a lot of my time designing and writing code for a company in India. I also take advantage of the relatively low rural cost of living to be able to work on commercial products that are definitely "niche market" - a (slightly) lower return on investment while living in an inexpensive rural area that works for me - if I was still living in Solana Beach California, the lower return on investment would make it impossible to work on the types of products that most interest me on a technical level.

In a sense, work networks are scale free networks like the internet: the "hubs" are large cities where closeness to customers and business partners is worth the higher cost of doing business. However, just as smaller "non hub" internet sites add tremendous value to the internet, distributed smaller businesses (especial high technology businesses) in much less expensive rural areas adds to the value of the economy.

(*) My take is that the most efficient way to do business and build industry is a mix of open source/open content and proprietary intellectual property - I am not arguing for one over the other.

Monday, November 15, 2004

Dumping on our kids and grandchildren: raising the national debt limit

Remember: Tax cuts are really tax deferrals to future generations. Too many people in our society lack the moral backbone to live within their earnings.

Some people say "but, running a deficit is a good thing". Well, of course running a moderate deficit is probably a good thing. Our current deficit limits are way out of whack however for what is in the best interests of our country.

Now, when I say best interests of our country, I am talking about people - I no longer count most corporations because most large corporations have moved offshore to avoid their fair share of taxes. They are no longer American.

I believe that in a free society that people should for the most part be free to make their own decisions and live their lives the way that they want to. However, this deeply immoral passing on economic desperation to future generations is sick and selfish behavior - not the legacy that I want to leave to my children and grandchildren. Anyone who is not part of the solution (e.g., repeatedly write to your elected representatives demanding fiscal responsibility, fair taxes for all people and corporations, etc.) is part of the problem. For the average person: do you really need that brand new car bought on credit? For owners of large corporations: do you really need that extra and excess $$$ so much that you are willing to contribute to, and cause severe economic problems for future generations.

Time for everyone to take a deep breath, and decide that dumping on future generations is a sick and immoral act - and that we should do everything in our power to say no to greed and selfishness. What ever happened to basic old fashion American values that I would summarize as "take what you need and leave something for other people". While it is in our nature to constantly strive for better lives for ourselves and our families, I question this new addiction to materialism.

Saturday, November 13, 2004

Semantic Web: replace RDF storage with relational database?

OK, this might not be a good idea for several reasons, but I am going to toss out this idea anyway: for Semantic Web applications instead of modeling and defining application classes (e.g., with OWL, RDFS, etc.) design appropriate database tables and store meta-data, etc. directly in a relational database instead of in RDF.

There are good Java tools for handling RDF, RDFS, OWL, etc. (e.g., Jena, Sesame).

Still, the idea of storing meta-data for online documents, relationships between documents, etc. directly in a database solves a few problems that seem to be holding back progress with Semantic Web technologies: learning curve for RDF/RDFS/OWL, acquisition and learning curve for tools for storing RDF, learning curve for one of the several RDF query languages, etc.

The next release of my KBtextmaster product will include an option for automatic RDF generation from documents and web pages. I think that also supporting extraction of meta-data to a relational database would be useful. In the first case, I will only attempt to extract data for limited schemas (e.g., part of dublin core, my own OWL ontology for news stories, etc.) - every schema needs hand-crafted extraction code so I need to limit what types of information I try to extract. If I also support use of a relational database, I would need to design equivalent tables (sort of an OO to relational mapping problem). Anyway, the semantic extraction problem is most of the hard work and I thought that some potential product users might have an easier time if extracted data was placed directly into a database so that they would not have to learn any new technologies if they did not want to.

Friday, November 12, 2004

My free advice to Steve Jobs: movie equivalent for iTunes music store

I am pleased with Apple for many reasons (*), so, Steve, this advice is free! Current online movie services are lame for a few reasons:
  • They don't run on Macs!
  • DRM limits movie watching to 24 hours
So, following up deals with the music industry to allow Apple's music store (which I really enjoy, BTW), negotiate with the movie industry and support a movie store site that:
  • Works with Macs!
  • With DRM that allows watching a downloaded movie for a whole week
  • Support two pricing structures: 99 cents for a low resolution movie (perhaps 450x250 pixels) and $2.50 for a higher resolution movie (perhaps 800x500 pixels). Support full screen mode with interpolation.

(*) I wrote the free chess program distributed with very early Apple IIs and received fun notoriety for that. I am a slightly wealthier man today because I wrote a commercial Mac application in 1984. The last batch of Apple stock that I bought has gone from $15 to $55 per share. Thanks Apple!

Simplified EJB 3.0 specification: still not sure how much I will use it

When I must use EJBs, I use XDoclet comments, ant build targets, etc. to take most of the hassle out of using EJBs. The new EJB 3.0 specification that uses new features of JDK 1.5 (or 5) looks like a definite improvement.

Still, like a lot of server side Java developers I usually do not need EJBs for my projects. If I can avoid even using a relational database, the use of Prevalyer makes server side back-end development fast and easy - if not, Hibernate is my tool of choice.

I use IntelliJ for about 90% of my Java development. Future versions of IntelliJ will probably simplify using the new EJB 3.0 specification just as the current version greatly facilitates using JSPs, EJBs, etc. So, I might start using EJBs again - wait and see.

Tuesday, November 09, 2004

Using Firefox client, server side Java for a standard platform

I do get tired of supporting multiple browsers when writing interactive web applications. Life would be simpler, especially when trying to write "sort of fat client applications" (e.g., Javascript+HTML like GMail) to have a single platform for client and server: only support Mozilla Firefox on the client side and tailor a development environment with appropriate tag libraries, etc. for dealing with Firefox (e.g., dynamic tree displays, standard Javascript utilities for verifying form input, etc.)

Now, while I consider myself to be an excellent Java, Common Lisp, Python, and Prolog programmer, I am admitedly a little weak using Javascript. For Javascript gurus, my desire for a single standard web browser client probably seems a little lazy :-)

There is a lot of push-back in industry for not supporting the Microsoft Internet Explorer browser, but I would argue that installing Mozilla Firefox is a fairly light weight requirement for using a specific web application.

I often look for commercially available software components to reduce development costs. Most of the systems that I write for customers run on a single server and have a modest number of users so reasonable licensing costs can be a lot cheaper than extra labor costs for devlopment. I have a wish-list for Firefox extensions for form input, inline rich text editors, spreadsheet components, etc. If developers of useful Firefox extensions had a larger market (i.e., lots of web sites and web applications standardized on Firefox) then the cost per component would go down, and everyone but Microsoft wins.

Excellent summary of Bush economic policy

Now that the election is over we can all get back to other important things like work, studying the economy (whoever does not keep a sharp eye on the economy is a fool!), etc.

The New Republic magazine has a very useful summary of the Bush economic policy:
There is a simple way to understand economic policy-making under George W. Bush: Whichever pressure group has the strongest and most direct stake in an issue gets its way. Wealthy individuals and business owners have received large tax cuts; farmers have gotten lavish assistance; and insurance and drug companies won enormous subsidies in the Medicare prescription-drug bill. When steel firms lobbied for tariffs, Bush granted them. When automakers and other manufacturers later lobbied Bush to reverse course, complaining that those tariffs had raised the cost of the steel they buy, he began to back down. If there's a single prominent case where Bush offended a powerful corporate interest--except to benefit an even more powerful corporate interest--we have not come across it.
I think that understanding our president's economic policy is very important for planning investments, trying to figure out which markets may be hotter than others, etc. Trying to figure out how the world really works (in addition to being really interesting) helps one's personal finances.

Monday, November 08, 2004

A great software demo; tools for creating demo movies

I enjoyed watching the Ruby Rails demo that is a 10 minute demo of building a web application using Ruby Rails. This demo was made by capturing a video as the demo creator smoothly went through the steps to build a web app.

I have looked at a few Mac OS X demo-making to Quicktime movie products and I think that I will make my selection soon. I am motivated to do this for a few reasons. One of my customers is interested in ways to demo the web application that I developed for his company and creating a demo like the Ruby Rails demo seems like a good idea. As bandwidth gets cheaper, I have also thought of augmenting the free web books (on my main web site) with free instructional videos.

Sunday, November 07, 2004

Hypertext and Knowledge Management: personal history

I was thinking this morning that this is about my 20th aniversary of experimenting with hypertext technologies (which I view as any chunks of information with typed links): using the Grapher (DAGs) on my Xerox Lisp Machine in the early 1980s to visualize typed links between data; a commercial Smalltalk class library to do the same thing a few years later; Owl Guide hypertext authoring tool on the Mac; ordering and reading the Ted Nelson's Xanadu manefesto document; using a text-based browser to explore physics documents at CERN in the very early 1990s (my first exposure to HTTP protocol and HTML - thanks to Karl Weibe for that!), etc.

In earlier history, some wealthy people had their own personal libraries to store and organize knowledge. Now the tools are better and usually support close to free access to information.

As good as graphical user interfaces are for browsing the web, examining links between data sources, etc., I still think that the "killer application" for knowledge management will have a simple (mostly) text interface that uses natural language understanding and (in a semantic web sort of way) be able to search on content based on meaning.

As an aside: my top priority research project that I am scheduling about half my time for in December and early next year is to add "RDF generation from plain text" to my KBtextmaster system. I will post a link when I have a white paper ready on how I plan on doing this.

Saturday, November 06, 2004

Positive Media Workshop

My friend Tom Munnecke (*) organized a Positive Media Workshop to take place in NYC next week. Tom believes (and puts his effort) in transforming the world in positive ways by sharing good ideas that work.

Please do look at the linked web page - interesting and worth a look. While I agree that the news media business wallows in bad news to increase profits (and I would argue the obvious point of pushing political agendas highly beneficial to corporate owners) I am personally not so optimistic that the mega-size corporation owned news media can be transformed in a positive way. However I also hold out some hope that "person to person" and group related communication technologies like web blogs, wikis, communities built on interlinked web sites, etc. will continue to allow (**) a minority of people to connect in positive ways. My pessimism however is based in a belief that most people are happy enough with 15 second 'sound bite' news that is highly filtered: like putting blinders on a horse -- but, to each their own: some people are satisfied with the world as it is and others want something better.

(*) who took the great picture of my wife and I in front of the Taj Mahal on my main web site.

(**) although I believe that here in the US we may eventually see censorship on the Internet as we see today in China. There are too many powerful business and political interests that might see alternative more personalized communication as a threat to their interests.

Sunday, October 31, 2004

Ongoing education: something a little different

I believe that education is a life long process. I usually spend at least 10 hours a week studying most aspects of computer science: sometimes reviewing previously read or new text books, reading ACM, AAAI, etc. journals, etc. Frankly, I usually enjoy this more than watching TV in the evenings.

I am trying something different: through the ACM Portal, I am working my way through project management courses. Part of this is purely practical: I plan on concentrating on combining open source knowledge management tools with my own commercial products for automaticly mining sematic information from texts, documents, web sites, etc. Rounding out my world view to include the business side as well as technical issues can only help :-)

I have another motivation: a friend of mine likes to point out that the job of management is to get the most work out of people for the least amount of money. While I think that this is a little too simplistic, there is some truth in this; for example, occasionally I run into people like this in my consulting business. The 'bottom liners' who resist spending resources on documentation, formal testing, etc. and need to be convinced that system robustness is as important as 'feature creep'.

Anyway, I am sometimes very formal when it comes to requirements and design (depending on what customers want to pay for), but I look forward to better understand formal management processes.

Saturday, October 30, 2004

Easy cross platform RPC

A few years go, both Microsoft's manager of the .Net framework and I were house guests in the same house for a weekend. He mentioned that he considered SOAP to be a possible universal RPC mechanism for all levels: method calls inside a CLR (a JVM in Java-speak :-), between CLRs running on the same server, and across a network. At the time, this seemed a little heavy weight to me, but...

This afternoon, I was experimenting with Java to Common Lisp communications. I was using the old version 1 of my KBtextmaster product (written in Common Lisp) that converts plain input text to a highly structured data object containing useful semantic information. KBtextmaster produces an XML document with this data.

To get to the point: On the Lisp side I used Sven Van Caekenberghe's nifty XML-RPC package and exposed a single lisp function that accepts a string argument and returns an XML payload as a string. On the Java (client) side, I used about 6 lines of code and the Apache XML-RPC library to test the remote service.

I expected this to be easy (and it was :-) but I was also pleasantly surprised at how low the communications overhead is. I am planning on also supporting REST, which should have even less overhead. I thought about also supporting a SOAP interface, but I am concerned that the overhead for SOAP is too large.

I am hoping that my schedule clears up a little in December and January so that I can get version 2 of KBtextmaster "shipped" - development has been on hold for 5 months, but I am really excited about getting back to work on this project.

PS. I just did a simply timing test; averaging over 100 calls, the RPC and Lisp side semantic processing was taking about 400 milliseconds per call - most of that would be in the semantic processing.

Friday, October 29, 2004

Congratulations Julie!!

I just had a telephone call that made my day: my step-daughter Julie graduated last spring with an accounting degree and is now working on her CPA. She had applied, along with about 70 other applicants, for a job slot with a good accounting firm in San Diego. She got the job!

When Julie was only 15, she got a part time job after school at a local doctor's office. Within a few months, she was doing their billing and some book keeping for them. She has always been very hard working and responsible, so her new employers made a very, very good decision. She is also a Mom of two great kids.

Java Wiki tools

A customer has been interested in adding Wiki support to their web site. There are many Java packages available, but I found two that seemed especialy easy to set up and use:


I have been looking at web based open source document management, authoring tools, knowledge management, etc. that might work well with version 2 of my KBtextmaster product that I hope to release first or second quarter of 2005. The KBtextmaster is a service that provides REST and XML-RPC interfaces for categorizing and summarizing text (*). I am planning on providing free "glue" code for Python based Plone and several Java open source projects.

(*) I m considering support for RDF tuple extraction from text (based on a few ontologies), but that is likely to be a separate product in the future - that code is still 'research quality' and probably will be for some time.

Thursday, October 28, 2004

100,000 Excess Civilian Iraqi Deaths Since War

"The use of air power in areas with lots of civilians appears to be killing a lot of women and children"

read more

Cool: free music CD in November WiRED Magazine

I always enjoy my subscription to WiRED Magazine; this month it contains a free music CD containing Creative Commons Licensed music. I was the "featured commoner" on the Creative Commons web site a couple of years ago because my free web books (see my main web site) use one of their licenses.

Anyway, I liked 6 of the songs well enough to permanently import them into my iTunes music library.

Wednesday, October 27, 2004

OpenCroquet improvements: coming fast

When I got home from San Diego last night I spent some time playing with the recent updates to OpenCroquet. Since the latest version was released a couple of weeks ago, there have been many updates: the system now seems to run much faster on my Mac and many of the virtual worlds either now work or work better. Give it a try!

I can not imagine a better environment than Squeak and OpenCroquet for teaching children all aspects of using computers. Sure, OpenCroquet will not help students write papers, etc., but none the less it is a fascinating experiment in what you can do with 3D graphics, in-the-simulated-world "world creation", real time programming, networking, etc., etc. I think that stepping back from immediate practicality and encouraging experiments is a good thing. My young grandson is in the Lego-blocks-experimenting mode; in a year or two I think that he will be ready for OpenCroquet: he is already creative and as he gets older we will upgrade the learning tools that he has.

Saturday, October 23, 2004

Visiting San Diego - I just saw a large screen iMac

I am visiting my parents in San Diego. My Dad bought my Mom one of the new large screen iMacs: wowza! It is huge! My Mom is 80 - this computer should last her a few years! I will use my trusty old iBook to work while I am here (too much trouble to set up my working environment on the new computer). Still, I am installing OpenCroquet on my Mom's iMac (my Dad enjoys making animated movies - he is getting to be very good at it, so he might be interested in OpenCroquet also).

Tuesday, October 19, 2004

Why doesn't Alan Greenspan level with us about the economy?

This is just my opinion, but I truly believe that Alan Greenspan says things that he knows to be misleading. I think that what he says in public is motivated by two things:
  • The Federal Reserve is not part of the US government: it is privately owned by a few very rich families. Greenspan works for these people and does what is in their best interests.
  • In order to save money on future taxes, the ultra-rich will do anything to re-elect George Bush.
In my opinion, the US economy is heading for a bad fall yet Greenspan fails to give average Americans a heads up to possible problems with the economy.

I believe that Greenspan is trying to postpone any bad economic news until after the reelection of George Bush. If you care about your and your family's economic future, try reading something that is true.

PS. one of my astute readers recommended doing a Google search for owns federal reserve.

Write once, compile and run everywhere

I was thinking last night of how choice of development platform matters less and less. I do most of my development work using three languages/platforms (Java, Common Lisp, and Python, (*)) that are very portable between Linux, Mac OS X, and Windows.

Now, I must admit that I do not do nearly as much application level GUI development as I used to - so, GUI portability is just not an issue with me anymore. Even GUI portability is manageable however: LispWorks Common Lisp has portable UI libraires; Python GUI applications can be built in a portable way with wxwindows, etc.; and, if you must, for Java there is JFC :-)

Freedom to work in a development environment that I enjoy is very important to me. I truely like working in OS X, with SuSE Linux a close runner-up. Did I leave out any other major operating systems?

(*) Notice how I left out C++. I have written 5 C++ books and spent almost 10 years as a C++ developer and mentor. Now, I think that unless performance issues are unusually important that choosing C++ for a software project is fairly much a stupid decision. (And, if performance is that important, just use a high quality Common Lisp compiler: I sell a commercial product written in Common Lisp: the binary executables are small, the runtime performance is superb, and the memory footprint is reasonably small.)

Monday, October 18, 2004

Great New York Times article

This article was fascinating: lots of quotes and material from General Tommy Franks and other military and civilian leaders in Iraq make a fascinating news article. The article is a long 7 pages, but really worthy reading - very interesting stuff.

Tuesday, October 12, 2004

Good book: The Definitive Guide to Plone; my AI products

I just ordered this book recently and it is excellent. I have a keen interest in open source knowledge and content management systems because early next year I plan on releasing version 2 of my commercial KBtextmaster product and I would like to provide wrappers to integrate it with most of the open source content management systems (connected as a remote service using HTTP POST requests and REST responses).

I have not updated my KBtextmaster product in a while. Starting in December, I am scheduling my time to finish off the version 2 functional improvements (better automtic categorization, better identification of proper names like people, places, products, and automatic generation of RDF for a few selected ontologies).

Very high quality knowledge and document management systems are now available for free as open source projects. My plan (interupted by a fun full time 6 month consulting project) is to position the KBtextmaster as an optional add-in tool for people building knowledge management systems.

KBtextmaster is written in Common Lisp and version 2 will only be available as a compiled executable built with LispWorks.

"It's just the bats, Madam"

My wife has been nicely bugging me about what she thought was an air conditioning leak: above our front door (exterior wall), we get a white residue build up. I crawled around in our attic and saw no leaks.

An air conditioning guy was at our neighbor's house and my wife asked him to come over to look at our "leak". He did, took one look and said that we are lucky - we have bats hanging out (literally) above our front door. He said that we should "Google for how to build a bat house" to encourage them to stick around (bats eat a lot of insects and are considered to be useful wild animals up here in the mountains were I live).

This also explains the mysterious tiny brown nobules to the left of our front door: bat droppings! They were so tiny that they just got cleaned up when we periodically clean up our entry way.

Croquet is released

I was up very early this morning checking on a new deployment for a customer, and decided to check the Croquet site before going back to sleep: I was pleased indeed to grab the Mac OS X download distribution and try it out.

I am very busy now (work wise) so it might take a while to get in 20 or 30 hours to learn the Smalltalk Croquet specific classes and write some code - but, I am looking forward to experimenting with this new technology. Croquet combines two interests of mine: large scale network applications and 3D graphics.

Sunday, October 10, 2004

I visited the Arcosanti archology last night

Carol and I, with 4 friends, drove south about an hour to get to the arcology started by internationally renowned architect Paolo Soleri about 30 years ago. I have been to Arcosanti several times and I always enjoy myself there. Arcosanti is only a 90 minute drive north of Phoenix Arizona, so if you are ever in the Phoenix area I recommend a side trip to Arcosanti. The food is excellent (mostly grown on the premises), the people who live there are interesting, and I usually visit when they have concerts in the acoustically perfect amphitheater.

Arcosanti is way ahead of its time. Right now, few people are willing to take a slight hit in standard of living in order to use a very small percentage of our national average use of natural resources. Fast forward 10 or 20 years when gas and heating fuel cost many times what they are today and the elegant low environmental-impact living style of Arcosanti looks good indeed. The beautiful buildings are designed to stay cool in the hot Arizona mountains summers and warm in the brisk winters. Whenever I go to Arcosanti I feel like I am looking at the future of how the human race can comfortably and happily adapt to resource shortages.

Saturday, October 09, 2004

the intellectual exercise of figuring out the world

Note: this weblog was an email that I sent to my family and some friends - then I decided to also publish it here

Even though we can't do much about politics, world peace, etc., I have
a somewhat morbid curiousity that encourages me to try to figure out
how the world really works.

A really good start is to google for "dollar euro oil" and skim
through the first several links.

This exercise can take as little as 10 or 15 minutes if you read quickly!

Here are the first few google links, BTW:

Note on google ranking: these are the first 4 links because these web
pages have the most web links pointing to them. Google web ranking
based on number of incoming links is a good but not perfect indicator
of the world's interest in a web page.

Wednesday, October 06, 2004

OpenCroquet announcement and paradigm shifts

Well, there is a new announcement for OpenCroquet: soon! Even though OpenCroquet is based on Squeak Smalltalk that has some very old roots indeed, I still think that OpenCroquet may be a new paradigm shift - we will see.

Lisa Rein has set up a web site with Alan Kay's Video demos of OpenCroquet.

From the OpenCroquet web site: "a combination of open source computer software and network architecture that supports deep collaboration and resource sharing among large numbers of users".

BTW, the last 2 video files (numbers 5 and 6) contain the actual OpenCroquet demos - but the earlier files are also interesting.

Tuesday, October 05, 2004

Waiting for free software releases

Some of the best software that I use is FREE, so please don't think for a minute that I am complaining. Still, there are two (hopefully!) soon to be released free packages that I can't wait for:
  • Open Croquet - this may change the way we think of online collaboration and generally large scale network applications
  • Apple Computer's early development release of JDK 15. (also known as JDK 5)
Everything else, I can be wait for :-)

Monday, October 04, 2004

GMail is getting better and I am really enjoying Clusty

One of my favorite improvements to GMail is the ability to automatically forward anything in your inbox to another email account. I had been concerned about backing up business email sent to my GMail account. Now, I just forward everything to my POP account on my local ISP - from there it is trivial to do a local permanent archive.

Yesterday I configured my Firefox browser to use the clustering "Clusty" search engine for the defualt browser (although it is very fast to do searches also using Google,, etc.) I left Google as my default search engine for Safari (the browser I use the most).

One thing that makes Clusty so interesting to me is that I have been working (for the last year and a half, although lately because of a full time consulting job I have had to put this development on hold) on version 2 of my KBtextmaster product that uses similar technology: I cluster documents based on what categories I automatically assign to them (for this, I use the Reuter's categories and trained with their excellent 2+ gigabyte tagged news corpus). I also identify people's names, places, etc. and I am planning on using this information to improve clustering also.

Saturday, October 02, 2004

Second brilliant thing I have seen today: Rich Hickey's Lisplets

I saw a reference to Rich Hickley's Lisplets package on Bill Clementson's blog this morning. On several occasions I have hacked around mixing Java and Common Lisp code using both REST (sending a HTML GET request and returning an XML payload) and Sven Van Caekenberghe's very nice XML-RPC implementation for Common Lisp.

However, Lisplets fit in better for my needs: I just about always write web applications in Java but I have a lot of code written in Common Lisp that I would like to use occasionally. (I hate to admit it, but often, since I code quickly, I will just recode in Java once I have figured out how to solve a problem.)

Brilliant: article about database wrappers in Python

This article by Scott Scriven is brilliant. One of the software engineers that I talked with when I spent a day at Google earlier this year was telling me about a cool technique for wrapping database tables to access data in a more "pythonic" way. This article provides the details. HIGHLY RECOMMENDED.

I must say that although much of my programming is in Java (mostly because of the awesome server side frameworks and tools) I find myself spending more and more time developing in Python. Actually, maybe I should re-phrase that as spending less time because Python is such an awesomely productive language that I am sometimes surprised at how quickly programming tasks get done.

Last night I spent a short while doing something I had never bothered doing before: organizing some of the more useful Python code that I have written into modules and set up a PYTHONPATH to the root of my new library setup. Before, I tended to use Python for many small and quick tasks.

It is a little sad, but my increase in use of Python comes at the expense of using Common Lisp and Scheme less (I have written Springer-Verlag books on both Common Lisp and Scheme and really love those languages).

Wednesday, September 29, 2004

My email to Microsoft requesting OOo file format support as a standard feature for Word

This is probably not of too much interest to many people, but the use of open file formats is a very big issue to me. I submitted this email by going, hitting the 'contact us' link, then selecting the 'suggest new feature for product' choice. If you happen to be one of the small number of people who care about file formats, please send them a message of your own! Here is my email:

Hello, I am an author of 14 published books - many of them written using Word for Windows and for the Mac.

While I basically like the functionality of Word, it would make my life very much easier if Word would also support the (soon to be an international standard) XML file format (gzipped XML files in a very clear format).

The problem is that I consider what I write to be **my data** and I would sometimes like to reuse **my data** in other ways. I find the file format to be very easy to process using simple programs written in Python, C#, Java, etc.

To repeat myself a little: the value of my word processing documents increases dramatically when I can access **my data** using simple programs.

In recent years, even though I own licenses for Word for Windows 2000 and Microsoft Office for the Mac (excellent!, by the way) I find myself using more and more because of the file format issue.

Money for software and writing tools is not an issue with me - I would rather spend money for upgrades for future versions of Word because it is such a polished writing tool. However, without simple and native support for the (soon to be international standard) file formats I am unlikely to buy future upgrades.

Thanks for your attention.

-Mark Watson

A great article by Senator John McCain on courage and social responsibility

I believe that the Senator gets this right. It is a fairly long article, but in my opinion well worth reading. I often do not agree with McCain's politics but he is one of the few politions at the federal level that I trust, and he is definitely on my "good guys" list.

Tuesday, September 28, 2004

Small and nimble: small but valuable companies in the future

Paul graham has a new essay that covers good things that came out of the Internet Bubble:
...doing good work will matter more than dressing up-- or advertising, which is the same thing for companies. That people will be rewarded a bit more in proportion to the value of what they create.

To me the interesting thing is the tension between two trends: on one hand huge corporations evolving into efficient world powers and on the other hand the naturally higher productivity of small loose working groups.

I believe that there is a unifying force that will undo the tensions between these two trends: to approach peak efficiency mega size corporations will federate into smaller business units (*) that may become their own cost centers and also take maximum advantage of consultants and other suppliers. Not to sound like a technical-marketing guy, but future technologies like semantics aware web services and SOAs really will allow work processes to become finer grained and distributed for maximum cost efficiency.

Larger organizations do not often become more efficient as they grow. One excellent example is the rise and fall of the nation state (that is happening now, only we are too close to the phenomenon to see it clearly). The British and then the U.S. empires did not have to be efficient - they only had to be able to exert more controlled violence than competitors.

I believe that world dominating mega corporations will not fall into the trap that nation states did: that of slacking off concerning efficiency. While I see the nation state going the way of the dinosaur, I believe that more efficient corporations will do well and I hope that competition will to some extent provide sufficient checks and balances (for at least quasi fair treatment of non-peak performing labor and also at least some protection of the environment) that the largely corrupt governments can not now control because of payoffs and influence peddling.

I may be very wrong about this, but I do not think so: long term, I believe that at least in potential the future looks bright indeed. Starting at the level of individuals, a tightly connected world will provide incentives for trying to achieve peak performance. With Internet technologies, clusters of peak performers can form small highly efficient work groups that might be either independent cost centers in mega corporations, or act as consultants and suppliers. I also expect to see more loosely coupled virtual corporations created by skilled workers coalescing together for tasks as needed.

The competition for quality will likely offer other incentives for decentralization. For example, the food industry has been going through a long period of consolidation that yields cheaper prices but much lower quality. I live in Sedona Arizona and spend lots of time in San Diego; in both locations the locally grown organic health food industries seem to be thriving. Some people really will spend a little more money for better tasting food. Mega scale corporations may end up owning this market but they will probably work through smaller and localized producers and distributers.

So issues of scale will reach equilibrium with desire for quality being a strong force.

(*) I worked for a long while at SAIC: at the time they organized in small cost centers that surprisingly often competed with each other for new work! The small work groups (I never worked in any of the larger divisions) tended to be efficient.

Monday, September 27, 2004

Automatically sorting out "Java blog entries" from "non Java blog entries"

Because of the intense interest in politics (at least for me :-) I have been writing more political blogs than usual. I think that the Java blog agregators should simply automatically filter out non-Java blog entries. For example, if a blog entry does not contain at least on of the words:

"Java", "EJB", or "J2EE"

then do not include it in an agregated feed.

I have recieved some polite requests to not submit political blogs. So far, I have been telling people who complain that after the 2004 elections, I will naturally stop writing about politics to a large degree.

However, like a lot of bloggers, I like to dump all of my blog ideas into one blog (covering a variety of topics). In addition to technical topics like Java, AI, Semantic Web, etc., I also like to sometimes mention my family, hobbies, and yes, even politics and the economy.

To me, the solution is for Java agregators to filter out posts based on lack of a few key words. The same could be done for Lisp agregators, etc.

Major update to one of the best AI programming languages and platforms: Swi-Prolog

New verion 5.4 pre-built binaries and source code are available at Swi-Prolog Home Page.

Windows, Mac OS X, and Linux are suported. If you are into the Semantic Web, the good news is that the semweb library is built in and works great to read and logically query RDF and OWL files. Very useful stuff.

Swi-Prolog also has built in libraries to parse SGML and XML, built in socket servers, socket client libraries, CGI, HTTP support, ODBC support, and supports SSL.

Prolog is not always the best language for a task, but when it is, Prolog programs tend to be very consise - get the job done with relatively few lines of code. In a sense, Prolog is also a late binding language like Python, Lisp, and Smalltalk.

Swi-Prolog is licensed under the LGPL.


Sunday, September 26, 2004

Good thing: European OSCE commision to monitor U.S. elections in November

Whether you voted for Gore or Bush, I think that almost all of us can agree that the disenfranchising of over 20,000 black voters in Florida right before the 2000 presidential election was a national disgrace that we never want to see repeated. A BBC News article has some good information and if you want to see about 600 politically charged responses to this article, head on over to Slashdot.

Seriously: transparency and honesty are all important for a democracy. Inspection of the U.S. election process by respected international organizations can only be a good thing.

I just Google-searched for more information about this; the following two sites were the first two hits and contain material that seems reasonable to me:

Disenfranchising Blacks as Political Strategy
Florida's 'Disappeared Voters': Disfranchised by the GOP

PS. I just noticed that the OSCE was asked by Secretary of State Colin Powell to monitor the election. Also, Congress has recently passed legislation to clean up problems like we had in Florida - so, we are trying and international monitoring of U.S. elections is the next logical step.

Use the right tools for a job: platform choice vs. available open source projects

I have to admit a bias towards using Java on the server side because I have so very much time invested in mastering the J2EE stack. That said, I will be using PHP for a major task for my customer in India because an existing open source project written in PHP gets us about 90% of our requirements 'out of the box'.

I suppose that this is a common enough occurence for consultants: to get a job done efficiently often platform choice is not as important as the availability of open source projects that match project requirements.

For my own research and product development (which unfortunately has been put on hold - I am working on a full time job until near to the end of this year) I have a common pattern: some back end process that I would like to make available through a web interface and/or a SOAP/REST/XML-RPC interface. Much of my research programming is done in Common Lisp because it is a very productive language (I also like to use Python and Smalltalk for research programming). I am a fast coder, so in the past I would prototype in a language like Lisp and then quickly re-code in Java (or whatever other language I wanted to deliver with). With faster servers, I now favor just leaving working code in whatever languages are best for specific development and to live with the overhead of SOAP (etc.) interfaces.

Saturday, September 25, 2004

Why don't we call 'tax cuts' 'tax deferments'?

Really, taxes are not being cut - they are being defered to future generations. It works like this: our government is spending lots of money while reducing revenue with tax deferments. This works by borrowing money (from foreign central banks, investers, etc.) that will either have to be paid back with lots of interest, or our government will have to declare bankrupcy (something that only the most extremist "cheap labor conservatives" want).

There is a lot of public talk by the most extreme neoconservatives in the Bush administration about "starving the government until it is small enough to drown in a bathtub" but I believe that these views are way out of touch with what average Americans want. Really, who wants to see the end of public education, etc.? (Well, the extremist "cheap labor conservatives" do but they hardly represent normal human or American values).

Wednesday, September 22, 2004

One of my favorite things about Java: packaging both code and data in JAR files

Sure, other programming languages like Python, Lisp, and Smalltalk as faster to develop in (for the usual reasons: late binding, more interactive development, etc.). However, one feature of the Java platform that I really like is being able to easily package both compiled code and required data in a JAR file.

Typically, I serialize required runtime data to a binary file and when I create a JAR file I add the binary serialized data file as a top level entry. To read the data into memory, I use something like this:

InputStream ins =
ObjectInputStream p = new ObjectInputStream(ins);
Vector my_vector = (Vector) p.readObject();
// etc.

Then, I can just use the JAR file in other applications and I have both code and required data.

Tuesday, September 21, 2004

Getting people to think

After seeing the (good!) new movie Sky Captain and the World of Tomorrow this weekend, I was reading an interview with the stars Gwyneth Paltrow and Jude Law. The interviewer asked them what they thought about movie stars going public with their politics. Gwyneth Paltrow said something that I thought was right on: she thinks that it is OK if stars want to politic but she said that she would simply like everyone to think for themselves.

That really is what I want also: regular people should be able to respect other people's opinions about politics. I find the polarization going on now in the U.S. more than a little disturbing. I would hope that a liberal and a conservative could sit down and discuss mutually interesting issues like:
  • Benefits and drawbacks of tax cuts
  • Pros and cons of strong environmental laws
  • How to fight crime - effects of longer prison terms vs. community outreach, etc. (what makes our families the safest?)
  • Advantages and drawbacks of military operations in other countries
  • etc.

The political bosses might want people to feel polarized, but after talking with a (more) conservative friend a week ago, I started thinking about how much common ground there is between so-called liberals and conservatives. I also think that is it simply good manners to respect other people's opinions even if they differ from our own. Fear can literally be a killer. Try not to fear people who may think differently than you on a few issues. People who are afraid are generally not thinking straight.

Saturday, September 18, 2004

Amazon's search portal is cool

I know, this is old news, but I have been giving another look. Privacy advocates probably do not like because the first thing the site does is to read your cookie so knows who you are. maintains a search history and is especially nice when search results are from inside a book - you can often read relevant pages out of books. uses Google, the internet movie database, the database, and other information sources.

I have been using the Google SOAP web services API for about 18 months - very cool to be able to use Google easily in my own programs. I would like to see Amazon offer something similar. Amazon does have both a SOAP and a REST API for accessing its main site, but I don't think that this is tied in with their search site yet.

Friday, September 17, 2004

Being an OS X user I forgot about the hassles of Windows and Linux

Really! I use a few Mac OS X computers for my business, but I keep a Chinese PC clone (*) around for the odd job requiring Linux or Windows.

My wife was sharing the PC, but decided that she was just going to use her HP laptop in the future. So, since all my stuff on the PC was old and crufty, I decided to wipe the disks last night after dinner and clean install SuSE Linux and Windows 2000. What a hassle - it took me an hour, and I don't have everything I need re-installed except for the firewalls and security updates. I decided to just install software as I need it since I use Windows and Linux so seldom now.

Yes, I am a happy Mac user :-)

(*) a nice machine: a little slow, but uses very little power, runs cool, etc. cost me $199 at Frys Electronix

Thursday, September 09, 2004

Demand for U.S. Treasury Notes by foreign banks at a historic low

Former President Clinton shocked the hell out of me a month ago when in a speach he refered to the U.S.'s inability to apply any pressure on China because they are our banker. I can not remember the last time I heard a politition from either party be so honest and forthright about an issue that no one wants to talk about: for the last few years foreign banks have been propping up the dollar by buying U.S. Treasury Notes to try to maintain the value of their existing investments. If not for foreign central banks in countries like Japan and China, our economy would have probably have been screwed several years ago.

While our current adminstration has been telling the lie that our economy is just fine, the reality is unfortunatey otherwise.

It is impossible to predict the economy: it is just too chaotic. Still, we can use common sense to figure out what "ought to happen" (*). Low demand for U.S. Treasury Notes is very troubling indeed.

(*) If you can handle reality and not political "feel good" bullshit, I recommend the book Financial Reckoning Day. Hopefully the authors will not mind my summarizing some of their excellent advice:
  1. Prepare for the future by reducing or eliminating your consumer debt
  2. Once you do the first step, try to relax and watch the "show" (e.g., enjoy some Java programming and don't worry about the economy because you have done what you can to protect the future of your family)

Wednesday, September 08, 2004

AI research at Microsoft: question answering system

I just saw this interesting Economist article linked on the latest AAAI alert email.

Eric Brill is one of the best known people working in statistical natural language processing, and Microsoft's ambitious question answering system looks interesting!

BTW, I have a simple question answering demo that I created on my web site.

Budget deficit is $422 billion - or $571 billion counting borrowing from social security

Why do we keep hearing the number $422 billion? I am not picking on just the Bush administration here: the democrats tend to not be transparent on budget issues either. It is a common belief among law professors and economists that if corporations divulged their finances like the federal government did, that there would be a lot more people in jail.

What is wrong with more transparency? Obviously the politicians would not like it, but think about how much better our government would be if:
  • Outlaw voice votes in Congress - this is a way to hide voting records from constituants.
  • By law, force Congress to publish clearly on a web site all voting records and have appropriate selection filters so that people could quickly see how their representatives voted. Correlate voting wih data on campaign contributions.
  • Require people in government to sign off on finance data and make it a crime to lie about it.
Polititions are very well paid and have great benefits - I don't think that I am asking for too much with my suggestions. We pay enough - we deserve transparency - I may be too optimistic here, but I think that if enough people complain to their elected representatives that they might go along with my suggestions if they think that they will not get re-elected if the do not suport transparency in our government.

And by-the-way: arguments about "deficits do not matter" are so much bullshit. Sure, if you are old, have no kids or grandchildren, and basically do not care about the society you leave behind: then deficits do not matter. I find it kind of scary that people will even promote the idea that "deficits do not matter": really out of touch with reality and common sense. We will see a crash in our economy in the next few years no matter who wins the next election; it might not be a "fun" or popular idea, but I think that we should try to make recovering from the next economic crash as soft as possible: for example; try to keep the deficit down!

Tuesday, September 07, 2004

Java article, SOAP, XML-RPC

I saw this article on Patterns and Strategies for Building Document-Based Web Services linked on this morning.

I had a little thrill when I first saw the title: most of my recent work has been designing and writing a web based document repository (a 'Knowledge Management' application). I am extremely interested in managing different document types, searching, automatic categorization (not done in my current project), etc.

The linked article is on a different subject, but still very interesting: on document based interactions instead of RPC type interactions. In other words, using SOAP for RPC style communication is (in my opinion) an overkill. I think that SOAP is best used for requesting a complex structured document. Something like XML-RPC is much better for simply RPC-type communication. This point is discussed both in the linked article and Michael Daconta's (et. al.) useful book The Semantic Web.

Kerry and Bush: keeping fairly quiet about urgent economic problems

This will piss off supporters of both Kerry and Bush, but, I am a little pissed off myself.

There are largely ignored but still overriding issues today that affect the security of Americans: the economy and scarce resources. Sure, both candidates are talking about unemployment, but there are potentially more dangerous issues affecting our security:
  • Huge trade imbalance
  • Reliance on Chinese and Japanese Foreign Banks to prop up the dollar
  • Lack of conservation of limited resources: water (in the South West at least), oil for food production and transportation, and natural gas supplies

Both candidates avoid the hard issues, with the lower grade going to Bush (his reelection team will do absolutely anything to keep the public mind off of economic issues). Still, Kerry is little better, and personally I give him a poor grade also for failing to address threats to our security. (If you think that the security of the U.S. is more threatened by terrorists than the issues that I just mentioned, then we disagree!)

I think that we deserve a lot more than "politics as usual".

Monday, September 06, 2004

Why do I have to read foreign news to get accurate information on my country's economy?

From the BBC

I blame the Democrats(*) and Republicans(*), and our feeble news media.

It is true, that some news sources like the Christian Science Monitor and the New York Times are pretty good at giving us straight talk, but I admit that I am talking in generalities here.

(*) Bill Mahr says that the Republicans are the "feel good" and "party" people: they bullshit about the current economy being OK, bullshit about how we are totally screwing over the Iraq people right now, etc. But, they make people feel good! WooooHoooo!

Mahr describes the Democrats as being downers: they point out and fret about real problems, and make people feel bad.

I think that Mahr has put his finger on how the Republicans hold political power! Really, think about it:

Most polls show that the vast majority of Americans support a woman's right to choose. Most Americans favor universal health care (I don't, but that is another issue...). Most Americans are grossed out by corporate malfeasance. Most people support stem cell research.

Still the Republicans win - even though their platform is largely out of touch with what Americans want. Bill Mahr correctly points out that they largely do this by using fear mongering. The Republicans want people to be scared. I am not too much of a history buf, but still the parallels with the Nazis seems apparent to me: Goebel, their propaganda minister, was quoted as saying that if you lie to the people enough, eventually they will believe you. The Nazis also consolidated power by fear mongering their own people.

It seems all too evident to me: real leaders who care about their people do not go out of their way to strike fear into the hearts of their people. Fear is a crippling influence that while can be used short term for political expediency, is just plain and obviously not in a country's best interests.

Saturday, September 04, 2004

Tailoring web apps to use FireFox; comments on Open Source

I spend a fair amount of time working on interactive web applications and I would love to be able to standardize on using a single browser. FireFox would be the obvious choice because it is fairly standards compliant and available for most platforms.

I saw an article on Slashdot this morning on Mozilla SunBird: a standalone calendar application that can use any WebDAV server to support a shared event space for multiple users. I just spent a couple of days creating a web based multi-user calendar from scratch for a custom project - it would have been good to use something already written, but it was also nice having total integration with our business data models, etc.

In any case, there are many great open source projects that enable consultants like me to put custom systems together quickly and inexpensively. I just started offering a 1/3 discount for any work supporting open source projects - I am booked solid work-wise for several months, but when I do have time available, I hope that customers take me up on this discount offer - I really do believe that individual consultants and small (perhaps even medium) sized companies win BIG with open source.

Thursday, September 02, 2004

JDK 1.5

Or Java 5.0 if you prefer... I am fairly much enthusiastic about version 1.5, mainly because of the auto boxing and unboxing between native types and objects and generics. I guess, the big thing is the ability to make Java programs shorter. That is one reason I love to program in Common Lisp so much: programs can be very terse. I look forward to more terse Java programs.

I basically 'live' in Mac OS X, so I need to wait until Apple has a robust JDK 1.5 distribution. That is OK - by that time, most of my customers will probably be fine with deploying JDK 1.5 with Tomcat 5.5 (which is compatible with JDK 1.5).

I like stable programing languages, and I hope that Sun freezes the core language now.

Wednesday, September 01, 2004

Russell gets it right

Usually I don't simply post link to other people's stuff, but this is just too good not to pass on.

Why Open Source will win in the Knowledge Management and Document Management space

I have friends who believe Microsoft's hold on knowledge and document management via proprietary document formats and the mass acceptance of Microsoft Office is unbreakable. I wish to throw out a contrary point of view:

First, I believe that the open XML document formats of, AbiWord, etc. will make it so much easier for ancillary applications to index documents for search, extract structured information (e.g., Phoenix), etc. that there will be more pressure from IT professionals to move their companies towards open document formats. For example, in Java applications, it is trivial to extract and manipulate documents: open a gzip input stream, feed this through a SAX or DOM parser, use either custom code or XPath to pull required information, etc. Try getting useful information out of Office documents! (If Bill Gates really wanted to support Microsoft customers, he would move the company towards standard documented file formats - not good for Microsoft's business, but good for society at large.)

Secondly, consider free open source knowledge/document management systems like OPEN-XCHANGE that will work natively with like the expensive Microsoft SharePoint works with Office. With companies like Novell behind OPEN-EXCHANGE, development funding is likely to be adequate for good progress.

Third, we are moving towards commodity software: preferably open source. I believe that this is a good thing for developers who can concentrate on augmenting and tailoring open source systems to specific customer requirements.

Tuesday, August 31, 2004

Sometimes I change my mind: Dave Pollard on free trade

Even though I bought, read, and appreciated Arundhati Roy's great little book "Power Politics", I have still been pro "free trade" - but I always qualify my opinion by saying "as long as human rights, environmental laws, etc. are upheld".

Dave Pollard's excellent paper on why free trade does not work has made me change my mind. Actually, unless you are a right wing conservative, changing one's mind is allowed and shows some evidence of a working brain. Radical conservatives (like the neocons in the Bush administration) will never admit that they are wrong - just as people on the far left lack this basic feature of intelligence. I never quite trust the political judgement of people who always vote democratic or republican for this reason - but I digress from what I really wanted to say:

Dave points out "why in a world of massive, hidden government subsidies it creates a hugely unfair playing field". This is the key thing here: the average American would be very surprised indeed if there was an accurate and transparent release of information for Congress giving away subsidies to corporations. This information is indeed hidden well from the public.

So, the problem is not with free trade itself, but the well hidden subsidies given to corporate sponsors of Congress. Other countries (especially in Europe) also do not play fair with subsidies.

Automatic add-on module installs with PLT Scheme, Squeak, etc.

John Wiseman mentioned today that PLT Scheme has an automatic software package install system. Squeak Smalltalk has the same thing in Squeak Map.

PLT Scheme is a very good (and free!) programming environment: a fairly nice UI for writing and debugging Scheme code with fast execution. Small runtime applications are possible with a little work using the underlying MzScheme native compiler and runtime. Cool stuff. PLT Scheme has modules for web servers, web services, XML processing, etc., etc. Usually I use LispWorks Common Lisp (a fairly expensive, but very well supported and very high quality product) for Lisp work, but the free PLT Scheme is fairly competitive. LispWorks allows the easy generation of small and extremely high performance standalone applications but lacks web services support. PLT Scheme has good web services support but is more of a hassle to make standalone applications.

I used to have to use Perl occasionally for work (yuck!). The only thing that I miss about Perl is the CPAN library and auto-installation facility. Now more reasonable programming languages have this facility also.

Sunday, August 29, 2004

Google file system, large scale architectures, Smalltalk Open Croquet

One core piece of infrastructure behind the Google search engine is the Google file system. If you have not already read this paper (PDF) then I suggest that you do so.

Starting with the requirement to be able to efficiently merge work queue data (e.g., spidered links) lead to the design of a file system that supported very inexpensive file append operations.

To me, Google represents at this time the best technology for mega-scale centralized processing. At the other end of the spectrum of large scale design is the Open Croquet project: a design for supporting very large scale peer to peer collaborative environments. The Open Croquet white papers give a great overview to this system. Open Croquet (due to be re-released within a month) combines a scalable immersive working environment with 3D virtual reality (VR) technologies. Open Croquet is vaguely similar to Sun's 3D Looking Glass desktop in appearance, but is really something very different.

Open Croquet covers two of my interests: large scale architecture and VR. (I was a principle in the SAIC VR Systems Division, and was the lead on a VR project for Disney when I worked at Angel Studios). I think that Alan Kay is correct: in computer science, we are just beginning. The real fun is yet to be experienced!

Saturday, August 28, 2004

Outsourcing (Slashdot), and why we will hopefully not have food shortages in the U.S.

Lot's of interesting talk over on Slashdot today about the ACM Queue article on outsourcing. I read this article about a week ago and although I am a defender, in general of outsourcing and globalization (with reasonable care paid to laws, the environment, etc.), I thought that the article was largely bullshit - read it for yourself to form your own opinions.

During a hike and breakfast the other day a new friend Bob (a pharmacist) was talking/worrying about the effects of both outsourcing and, separately, the insane economic agenda of the republicans (just for your information, Reagan got lucky: the economy grew during his watch because of drastically cheaper oil: I believe that the supply side people are idiots). He was more than a little concerned in the U.S. that we are heading for large scale social unrest and possible food shortages. I am a little bit more optimistic - sure, the economy is going to crash sometime (right after the elections, in a year, who knows?...), but I think that Americans are resilient and will adapt to a lower standard of living.

I was a little surprised that another friend of mine Mark (a real estate guy) during a hike this morning agreed about the possibility of social unrest, and food shortages. He pointed out that during just a few short months of gas shortages in the early 1980s, that people were going a little crazy, and that period was thankfully short - the situation is different now: there is global increase in demand (Asia, etc.) and future oil squeezes are not a probability but a certainty, and they will make the early 1980s look mild in comparison.

Still, I am optimistic: here in the U.S., we have so many material blessings, and what do people really need for happy lives? I would suggest that housing, food, family and friends does it. Assholes who drive around in SUVs with no passengers, etc. might have to switch to economy cars when oil supply needs to be diverted to food production, but who cares about these selfish people? I think that people with solid values will be fine.

Tuesday, August 24, 2004

What is up with people setting CLASSPATH?

I have seen a zillion complaints on Slashdot about hassles setting the CLASSPATH environment variable. Setting CLASSPATH is a bad thing to do, period. I have many Java utilities that I run (wsdltojava, zeus, etc.) and I always simply use startup scripts that set the classpath on the command line.

Just out of curiosity: if anyone can think of a good use of setting CLASSPATH, email me.

Slashdot: Why is Java un-cool?

I would have responded directly on Slashdot to this article, but I was already moderating there. Is Java a little un-cool? Sure. So what.

I love doing server side Java. Is Java my favorite language? No. That would probably be Common Lisp. The thing is, for most of the work that I do, Java has several advantages:
  • Great free platform support for web presentation, SOAP, XML-RPC, asynchronous messaging (JMS), etc.
  • There are many skilled Java programmers: when I build a system for a customer, I can feel confident that maintenance will not be a problem.
Common Lisp has advantages also (extremely fast compiled code, small runtime overhead, rich standard library, very terse language) for some types of development (I use Lisp for working on algorithms, natural language processing, and other problems where I am breaking new ground - Lisp is great for experimental programming).

Saturday, August 21, 2004

Tomcat as an application platform: enjoy the simplicity

As a Java developer, I mostly do what people directly pay me to do. Because of this, my comments here might be a bit biased: almost all of my work now involves writing either dynamic web applications or some type of infrastructure for web applications.

I find myself now really disliking writing thick client applications in Java (or Windows, X Windows, or Mac OS X for that matter). I started a long period of writing GUI client apps in 1982 when SAIC bought me a Xerox Lisp Machine - it was so easy to rapidly prototype GUI applications, that I found myself coding lots of demos. I also had to write Microsoft Windows, X Windows and Macintosh native GUI apps for many years.

Maybe I just enjoy doing whatever I am currently doing, but more than that, I just find writing web applications to simply be fun. Keeping code and data on the server just gets rid of all sorts of problems that take the fun out of development: handling updates, diagnosing bugs that are often related to a funny state on a PC, etc.

While there is a definite place for J2EE components like EJBs, JMS, etc., fortunately, for most of my work, I just deal with application specific POJO ("plain old Java objects") classes, simple persistence (e.g., Prevayler, Hibernate, etc.), JSP/struts, etc. In other words: the simple stuff that allows me to simply think about my current application, spending very little time worrying about infrastructure. Even SOA ("service oriented architectures") applications are straightforward with axis SOAP support, WSDL support, XML-RPC, etc.

And yes, Tomcat combined with a good web app development environment (I prefer IntelliJ 4, but Eclipse with MyEclipse is handy also) is good enough, and fairly much worry free.

PS. something obvious, but sometimes overlooked: when building applications on top of Tomcat, it is also easy to handle background processing: write a servlet that is loaded automatically by Tomcat (use a load-on-startup tag in your servlet definition in your web.xml), define your own init(ServletConfig config) method that after calling the super class init, starts up a work thread. I use this whenever I want to do background processing (periodic admin, web scraping, etc.) in a web application. Don't forget to increase Tomcat's heap storage using something like JAVA_OPTS=-Xmx300m in your catalina startup file.

Friday, August 20, 2004

Calling 'return' in a JSP, avoiding sendRedirect(...) problems

A not too uncommon runtime error occurs by trying to call response.sendRedirect(...) twice on a single JSP or calling response.sendRedirect(...) while writing any output to a servlet or JSP output stream.

A trick that I use to avoid this problem is to simply 'return' immediately after calling response.sendRedirect(...). This is required because the page is not redirected instantly - it occurs after all the output to the page is written.

Simple trick, but I never see this in other people's JSP code, so I thought that it was worth mentioning it.

Thursday, August 19, 2004

Using GMail to keep track of competition

I have been using GMail since last Spring when Peter Norvig invited me to try it as a beta tester. Since I use several computers in my consulting business, having a web based email system is convenient - no hassles syncing my email (my local ISP does not support IMAP).

Anyway, one surprising advantage of GMail is the advertisements that are targeted to the text in my emails. I frequently see advertisements for either competitors (of my customers mostly) or interesting information. I don't spend a lot of time checking out competition, but when their advertisements appear along side my email, it is worth a few minutes a day checking them out.