I built my first simple web page at SAIC in 1992 when my good friend Gregg Hanna set up a publicly accessible web server for my working group. Since then I have had a lot of people suggest that my web sites could look better but frankly I have always been more interested in content and developing cool web application functionality. Recently I have been putting some effort into improving my design skills and the best resource that I have found is "The Non-Designer's Design Book" by Robin Williams The author Robin Williams does a fantastic job at explaining four basic concepts of design: contrast, repetition, alignment, and proximity. She then provides good examples that show the reader how to recognize bad design and how to correct design errors. I spent some time redesigning my main web site and really enjoyed the process. I started by determining the worst aspects of the old design based on Robin's advice and then tried to correct the design flaws using her

As I mentioned in my last blog article I am (mostly) shutting down my consulting business in order to have time for work on writing projects and to try to develop three business ideas. All three involve web apps/services and I want to use Clojure for two of them and Rails for the third. Rails 4 should be released early in 2013 but I thought I would get a leg up and start experimenting with Rails 4 now. Fairly easy to set up and try: git clone https://github.com/rails/rails.git cd rails gem install sprockets rake build ; rake install And then version 4 beta is installed: ✗ rails -v Rails 4.0.0.beta ✗ rails new testapp -d postgresql I had to comment out the assets group in the generated Gemfile before bundle install but otherwise everything worked fine.

I would like to wish you Happy Holidays ! I hope you are with family and friends enjoying yourself over the holidays. I wanted to share with you my plans for the future. Starting in January I am planning on mostly shutting down my consulting business. I have been consulting for about 14 years and consulting has provided me with a great life style but it is time for a change. Before consulting I worked at SAIC, Physical Dynamics, and Angel Studios. I will still provide some consulting services but will limit the time directly helping customers on very small projects. I plan to spend most of my "work" time writing and developing a few software as a service business ideas. I have written published books for some great publishers (Springer-Verlag, McGraw-Hill, Morgan Kaufman, APress, Sybex, M&T Press, and J. Wiley) but because I prefer writing on niche subjects (things that are of special interest to me!) I will probably only write free books published as PDFs in the f

I took a ton of hi-def video and pictures. Here is a Google+ photo album of a few of the pictures (small representative sample) I have a Canon T2i camera with a nice 24-105mm L-lens. I mostly take hi-def video hand held. If we were in the same place enjoying a glass of wine together I would show you the hi-def video, but the pictures in the linked photo album are OK as a representation of the experiences Carol and I had.

Note: update: Holland America gave us a fair refund - I withdraw my complaints listed on this blog article My wife and I have been on 15 cruises and the service provided by Holland America on the 23 day cruise we are on right now is so much worse than the other 14 cruises we have been on that I feel motivated to write up our experiences. Note: our cruise was up the Amazon River and we did have some memorable experiences which I will blog about in the next week or so and provide links to some of our pictures and videos. In order of "worse things first": We booked a tour to Santarem (large industrial city) and Alter do Chao (a pretty little town on the water with amazing beaches). The tour guide was from Santarem and spent all 4 3/4 hours of the tour in her home town and blew off taking us to the scheduled stop in Alter do Chao. She did give us lots of unwanted shopping experiences, and wasted time stopping the bus by a new highway project and going into lots of detai

I moderate comments because occasionally someone leaves some SPAM as a comment - so, there is usually just a short delay between the time readers post comments and when I moderate/publish them. I will be on vacation, and for most of the time I will not have an Internet connection, so any comments left on my blog may not get moderated until the end of December when I get back home. Best regards, Mark

I worked in the field of artificial intelligence during the "AI winter" (a backlash against too optimistic predictions of achieving "real AI") and to this day I avoid getting too optimistic of huge short term gains in our field. That said, in the last several months a few things are stirring up my old optimism! I have been enjoying Geoffrey Hinton's Coursera course Neural Networks for Machine Learning and I was pleased to see a front page New York Times article this morning on deep learning . Another really nice reference for deep learning is a very large PDF/viewgraph presentation by Richard Socher, Yoshua Bengio and Chris Manning. Another very good resource is the Deep Learning Tutorial that provides the theory, math, and working Python example code. Deep neural networks have many hidden layers and have traditionally been difficult to train. In addition to very fast processors (graphics chipsets) a very neat engineering trick is pre-training weights

I bought the Kindle edition of the book "ClojureScript: Up and Running" by Stuart Sierra and Luke VanderHart a few days ago. It is well written and a good way to ramp up on using ClojureScript for web client programming in Clojure instead of Javascript. I have experimented with ClojureScript before, and now that two of the three Coursera classes I have been taking this fall are done, getting up to speed on ClojureScript has moved to the top of my side-project to-do list. (BTW, as I have written before, one of the classes I have taken this fall is the Functional Programming Principles in Scala, and that class is also of value to Clojure programmers who might not have a strong interest in Scala specifically, but have an interest into better understanding functional programming - the lectures for that class were especially enjoyable and useful.) I have converted all three of my main web sites to Clojure + Noir in the last few months and this is a great combination (especiall

I am a technology "junky" and I suspect that most people who read my blog regularly or read an occasional blog post from a web search are the same. It is not easy to predict which currently used technologies will end up having a huge impact on human society, but it is fun to make educated guesses. As much as I have been enjoying programming in Scala and Clojure (and some Ruby and Java) I doubt that improvements in programming languages and development tools will profoundly impact society, the economy, and quality of life in general. I also don't think that new "gadget technology" will profoundly effect society, with a possible exception being very low cost smart phones in developing countries. As you can tell by the title of this article, my bet is that the semantic features of HTML5 will have a profound effect on society. I have several reasons for this bet: Semantic tags in HTML5 are minimal but sufficient for web analysis software to detect different typ

Just kidding with the title of this post :-) I believe in using the best tools for any given task, but this is not always possible when working with teams where most developers already know one programming language and/or framework. Also, as a consultant I usually favor using which ever tools are already used in the customer's organization. All that said, I find that alternative JVM languages like Clojure, Scala, and JRuby are so much more effective for the projects that I work on that I have a strong preference to not use Java. I find the decision when to use JRuby to generally be easy, using it on projects requiring fast development, web services, and as glue code for existing Java software. Increasingly though, I am viewing Scala and Clojure to be almost as agile as JRuby, with much better runtime performance. For me, the tough decision is between Scala and Clojure. Taking Martin Odersky's Functional Programming with Scala class definitely affects my decision because I

I am just finishing up today my course work for Andrew Ng's excellent Coursera course in Machine Learning. I am also taking two other classes that I will complete in about a month: Martin Odersky's Functional Programming with Scala class and Geoffrey Hinton's Neural Networks for Machine Learning. Previously this year I also took Natural Language Processing and Software Engineering for SaaS. I have taken the first two or three weeks of several other classes just to get a general feel for their subjects. This morning in one of the last lectures in Andrew Ng's class he showed a precise algorithm for a problem that a customer (a media company in China) and I tried to solve about 7 years ago. We were successful enough to meet my customer's requirements but the next time I see a problem like that (involving collaborative filtering) I will nail the implementation. Every class that I have taken this year has provided many new insights, often on subjects that I thought tha

I rewrote my markwatson.com website recently, tossing the 15 year old PHP implementation and replaced it with a little Clojure + Noir web app. This morning I spent a short while refactoring it to use the noir-bootstrap project on github. Along the way, I read through the Bootstrap scaffolding documentation. Really cool stuff. One thing that I had to add was <meta name="viewport" content="width=device-width, initial-scale=1.0"> to the head section of my Noir page layout template. Before I did this my web site did not identify the media type when I used an iPad and a Samsung Galaxy III S phone for testing. I wrote earlier this year about using Noir and the Dojo Javascript library for my cooking/recipes web app . Dojo makes it easy to write mobile web apps but Bootstrap helps a non-expert Javascript developer like myself write something once and have it work across devices.

I am enjoying Martin Odersky's course "Functional Programming Principles in Scala." I use JRuby and Clojure a lot in both consulting projects and my own projects but my main use of Scala in the past was writing some programming examples for my book "Practical Semantic Web and Linked Data Applications, Java, Scala, Clojure, and JRuby Edition" that is available here for free. I had purchased three Scala books and occasionally played with Scala but it never really "clicked" with me (same situatuon with Haskell: bought three books on Haskell, lots of experiments, never had the "clicked" experience). Taking Martin's class is definitely helping me become more comfortable with Scala. Having an interactive repl for Clojure and JRuby has always been a big win for me over Java development, even with great Java tooling (e.g., when I have to do GWT or SmartGWT development, having both the server and client side code in Java, all debuggable in I

I have done a short sprint the last three days on my NLP SaaS kbsportal.com mostly working on classification/tagging. When I have time in the next month I also make some improvements in sentiment analysis and entity recognition. The text summarization code also needs work but it may be a while before I have more time to work on that. I rewrote KBSportal in Clojure early this year. It used to be a pile of Common Lisp, Scheme, and Java code experiments written over a 10+ year period and it was great to pull just the parts I wanted into a new system. It might be heresy to Lisp programmers but I will (probably) eventually rewrite it again in plain old Java to save a bit on memory footprint and CPU time. Clojure is a very efficient language but not as efficient as Java.

I am going to write up a few of the things that make Ubuntu Linux a more comfortable software development, writing, research, and having fun environment. I hope that readers of this blog add their own suggestions in comments (remember: I moderate comments to avoid publishing SPAM so it might take a short while before I see your comments and approve them). I use Evernote and the Kindle reader a lot and there are no officially supported Linux clients. Evernote has an open source client client NixNote that is OK but I prefer to simply use the web interface on the Chrome web browser. This is a little slower than a native client with local copies of everything but it is OK. I also use the Evernote Chrome plugin. For reading books I buy for the Kindle the Chrome Kindle plugin works fine, especially since I own a Kindle device and my Samsung Galaxy 3 III phone (with 1280x720 screen resolution!!) is also good to read with. One serious problem is watching Netflix movies on Linux. I get by usi

Apple has gone too far in starting legal proceedings to stop sales of the Samsung Galaxy S III phones. Have you seen these? Carol and I both have them. It looks to me like Samsung bent over backwards on this product to not infringe. I have a long happy history with Apple: I wrote the Chess playing program they gave away on the free demo tape for the early Apple II, I did quite well with a commercial AI product for the Mac in 1984, and I respect their technology. That said, I think that the iPad 3 that I bought for my stepson a few months ago might be the last Apple product that I buy unless they have a turnaround in their business strategy and stop trying to be the world's largest patent troll. I have already decided to not buy one of the retina MBPs and to not renew my iTunes Match service subscription when it expires. Apple has the legal right to pursue any legal proceedings they want but as consumers we can vote with our wallets and stop buying Apple products! Obviously I w

I have had the domain markwatson.com for almost 20 years and for about 15 years my site was a simple PHP application hosted on the awesome provider Hurricane Electric . There are a few things that I want to improve on my site so I decided to rewrite using Clojure and Noir, a combination that I have been mostly using for writing web apps in the last year or so. I usually use AWS (and sometimes Heroku) for hosting my customer's web apps and the web apps for my side projects but for my primary domain (important for my business!) I decided to go back to using RimuHosting , a hosting provider I used to use for my own projects and I always liked their service. Some of the new site is still HTML fragments from my old site implementation but I am in the process of converting everything to use hiccup and cleaning up the CSS.

I wrote two days ago and yesterday about my experiments for getting up to speed on Datomic. In the Datomic news group, Rich Hickey suggested that I keep any helper code for data-building separate from helper code for managing database access/transactions. I have reworked my code and added code to experiment with text search. Now that I have a few days of experimenting with Datomic I think I understand how I will structure an application for a side project: write a small helper library that is application independent, much as the code snippets in this article. Write another application specific helper library that layers on top of the application independent library that encapsulates all data store functionality that I will need, and unit test this separately. Then I can write the application layer that will probably not have any Datomic application specific code. After I finish the first version of my side project this morning then I will help my customer on his Datomic project.

I wrote yesterday about getting started with Datomic in a lein based project. Probably because I am not up to speed with Datomic idioms, a lot of the data boilerplate bugs me so I wrote a little wrapper to hide all of this from my view. Starting with a some code by Michael Nygard I saw on the Datomic newsgroup I wrapped creating database attributes and adding data to the data store. I formated the following code in a funky way to make it fit on this web page: (ns datomic-test.core (:use [datomic.api :as api])) (defn attribute [id t c doc] ; by Michael Nygard {:db/id (api/tempid :db.part/db) :db/ident id :db/valueType t :db/cardinality c :db/doc doc :db.install/_attribute :db.part/db}) (defn string-singleton-attribute [conn id doc] @(api/transact conn [(attribute id :db.type/string :db.cardinality/one doc)])) (defn string-multiple-attribute [conn id doc] @(api/transact conn [(attribute id :db.type/string :db.cardinality/many doc)])

Hopefully I can save a few people some time: I flailed a bit trying to use the first released version yesterday but after updating a new version (datomic-free-0.8.3343) life is good. Download a recent release, and do a local maven install: mvn install:install-file -DgroupId=com.datomic -DartifactId=datomic-free -Dfile=datomic-free-0.8.3343.jar -DpomFile=pom.xml Setting a lein project.clj file for this version: (defproject datomic-test "1.0.0-SNAPSHOT" :description "Datomic test" :dependencies [[org.clojure/clojure "1.4.0"] [com.datomic/datomic-free "0.8.3343"]]) Do a lein deps and then in the datomic-free-0.8.3343 directory start up a transactor using the H2db embedded database: cp config/samples/free-transactor-template.properties mw.propreties bin/transactor mw.propreties When you run the transactor it prints out the pattern for a connection URI and I used this in a modified version of the Datomic developer's Clojure

I wrote a simple wrapper that is on github for calling the new API . The old API will not work starting in August 2012 so it was time to update. The README file on github has a simple example for using the JAR created by this project in a Clojure project (a pre-built JAR is also included in the github repository). The wrapper is simple, but will save you a few minutes writing one yourself if you need to use Bing Web Search in Java, Clojure, JRuby, etc. You get 100 free searches/day with the new API and there is a charge if you need more API calls per day.

My almost three year old Droid phone still works great and it was not so easy getting a new phone. What convinced me to upgrade was the Samsung Galaxy S III's screen resolution of 1280 x 720 pixels. Sure upgrading from Android 2.x to 4.x is nice, but having a high density screen makes reading email, browsing the web, reading using the Kindle app, etc. all feel very natural, even on such a small device as a cellphone. Also: the Netflix app is fantastic: unbelievably easy to watch video at this screen resolution, even on a tiny device. I have been an enthusiastic iPad owner, but I think that the Samsung Galaxy S III will replace using the iPad about half the time. I had thought about getting a Nexus 7 tablet for occasions then the iPad was bigger (and heavier!!) than what I needed it for. At least for now, a larger cell phone with a high density screen seems to fill that niche better. One thing that surprised me after using my old phone for almost three years: I didn't manua

Maybe I shouldn't share bad habits with people but occasionally I see articles for setting up local maven repositories, etc. for using local JAR files in Clojure lein based projects. I have a kludge for doing this simply. Now, I have to admit that I tend to use a lot of Java code and 3rd party JARs in my Clojure projects - nice if libraries are in Clojars, but if not I create a directory local_jars in a project directory, put any local JARs there, and instead of using lein deps and lein clean I use a Makefile like: deps: lein deps cp local_jars/*.jar lib/ clean: rm -f -r lib/* Really simple. A side benefit is that I use lein1 in some projects and lein2 in others. Using Makefile targets insulates me from mistakes using the incorrect version of lein. Anyone have a better way of doing this? Please let me know.

I often read opinions about using the best tool for the job in reference to choosing programming languages, frameworks, and libraries. I am a polygot programmer but I am one mostly because I use languages and other software tools that my customers request. Seriously as a consultant I serve my customers' best interests and that process usually does not include trying to get their teams to pivot to use my favorite tools. For my own side projects I am for the most part happy enough to use any of the languages that I feel most comfortable with, my favorites being Ruby and Clojure, but I also really like to use Java and Common Lisp. I usually choose languages for my own projects based on available frameworks and libraires that I can build my code on, and more rarely because of specific language features. Yes, I feel a little heretical saying that! My secret for being comfortable with several languages is that I try to make the development experience similar across programming lang

12.04 was released a few months ago but I just got around to upgrading my Toshiba U505 yesterday. I must say that I don't understand some of the negative comments about Unity that I have read. Unity feels intuitive and so far has just worked for everything I have tried. A minute of google'ing found directions for seting up desktop files in .local/share/applications/ . For an example, here is my .local/share/applications/rubymine.desktop file: [Desktop Entry] Version=1.0 Type=Application Terminal=false Exec=/home/rubymine/bin/rubymine.sh Name=RubyMine Icon=/home/rubymine/bin/RMlogo.svg Obviously change file names and paths as required for your apps and your system. Then anytime when I need to run RubyMine I tap the ALT key and the Unity search bar appears; I start typing the first few letters of RubyMine and when the icon pops up I just click it. I also set up IntelliJ this way for Clojure and Java development. I prefer this to creating task bar icons. Very similar to using Co

After writing about Pharo Smalltalk the other day I started looking at some of my old projects. I created a new github repo for my Pharo code and I'll add code as I have time to re-test it and clean it up. There is not much there right now, just a part of speech (POS) tagger. Edit: 2017/05/20: much improved with summarization, entity detection, and classification. Enjoy!

The last release 2.0 was made available almost three years ago, so I was very happy to see that version 4 is available . OpenCyc is a knowledge base and ontology containing about 270K terms and 2 million triples. There are now many more links to outside (or OpenCyc) knowledge resources like DBPedia, UMBEL, Wordnet, etc. Check out the new features. I downloaded the Linux version (requires Java 6, 3 GB memory, and is only for 64 bit OSs and Java). I had no problems at all running it on 64bit Ubuntu and OS X Mountain Lion (developer preview 4) - it only took a few minutes to install and run. Just follow the instructions in the README.txt file and try the web interface at http://localhost:3602/cgi-bin/cg?cb-start Nice to see this the free version of Cyc getting support and development efforts. I would be very interested in hearing from people who have done projects with either OpenCyc or the OpenCyc OWL/RDF data. 9-9-2012 update: Alyona Medelyan provided a link for downloading

I made a first cut at wrapping Dojo Mobile for use in Clojure Noir web app projects. If you want to try what I wrote this morning here is the github repo . The code is really crude. For example, I just embedded Javascript to test AJAX right in a Noir Clojure view file and I don't make full use hiccup. If I have time in the future, I would like to support a wide range of Dojo control elements and perhaps even use ClojureScript. Pull requests welcomed :-) Here is what it looks like on an iOS device: Controls work and look different on different devices. This is part of the magic that Dojo Mobile does for us. On the Android platform a selection list behaves like an Android selection list:

Pharo is a fork from the Squeak open-source Smalltalk and provides an incredibly rich development environment. As a consultant people pay me to design and write code in Ruby, Clojure, Common Lisp, and Java. That said, for non-work related experiments, Pharo is a lot of fun to use: a modern and free Smalltalk environment. I just wanted to say thanks to the Pharo team: great work! I recently downloaded the 2.0 development build - exciting to see new features. One thing in particular that strikes me as awesome about Pharo is that it is very light weight, using little memory and CPU resources. I wrote a blog 5 years ago about deploying Squeak to Linux servers. I am a little surprised that Pharo is not more widely used for rich web applications but with so many great languages and frameworks (Rails, Sinatra, Clojure Noir, Java Play Framework, GWT, etc., etc.) there is a lot of competition for developer mindshare. My personal interest in Smalltalk started when I got a Xerox 1108 Lisp Mach

I wrote a few days ago that I am excited about how easy it is to make simple one page web apps using Dojo Mobile that look good and work fine on portable devices (Android and iOS) and regular web browsers. I don't do a lot of UI development for my work but when I do write web apps it is great to also be able to support mobile devices with a small amount of additional simple code. In this post I will show you hopefully useful code snippets for cookingspace.com that may save you some time if you want to write the same type of apps. Last weekend I decided that I wanted a new mobile web interface to my old Cooking Space web site. I wanted to be able to quickly look up a recipe on my cellphone, see the ingredient list, and be able to specify how many people need to be served. I also want to see the nutrition data for the recipe. A top level requirement is that once the web page renders then everything is updated with AJAX. There are only two user interactions: Enter a few search term

In my work I specialize in Natural Language Processing (NLP), text mining, and general AI development. That said I find myself writing a lot of web apps and what I really want is an easy to use web client stack with rich controls and that facilitates writing one web app that looks and works OK in web browsers, Android phones and tablets, iPhones, and iPads. Five years ago I wrote a very simple web app cookingspace.com that let me look up the nutrients (using the USDA nutrition database) for recipes we frequently make. I have just spent a few hours rewriting the front end using Dojo Mobile, removing a lot of features that I don't need anymore that is hosted at mobile.cookingspace.com . The new app lets me quickly check nutrients and also on portable devices I can use it while grocery shopping to make sure I get the ingredients I need for making dinner. Both apps are deployed at Heroku (thanks Heroku!). Dojo is really a nice web client toolkit. I suggest you take a look if you

I just finished the excellent Coursera/Berkeley Software as a Service (SaaS) class yesterday. There were two major themes in the class: engaging customers in creating user stories to make sure that you build the right thing, and use BDD and TDD that relies on continuous testing. In particular I enjoyed learning how to use Cucumber that has two parts: an english (or other natural language) style DSL that customers and developers can work on together to create user stories with expectations of what the system should do and an underlying set of steps that map the customer facing DSL descriptions to real testing code. Cucumber is very popular in the Rails world but I had never tried it before. I now like using Cucumber and as I get more used to it for Ruby and also for Rails I look forward to integrating it into my development process for other programming languages. A few nights ago I spent some time with the lein-cucumber project for Clojure. It is still a little rough around the edg

I have had a dotCloud account for a while, but so far only to experiment with. dotCloud has announced new pricing models that look very developer friendly with free lower performance sandbox support and "live" support for production. They may have hit the sweet spot for supporting free development while nudging developers to not deploy small demo apps using their free sandbox model. I would love to see Heroku's and dotCloud's customer stats on free versus paid hosted web apps. The basic idea is that as you add services you may by the amount of memory that those services use. I like their handling of horizontal and vertical scaling . I have not tried it yet, but from the documentation it looks like when you horizontally scale a persistent service like PostgreSQL, MongoDB, MySQL, etc. they automatically set up master/slave, replica sets, etc. as appropriate. You can also vertically scale any service by adding memory. Another interesting PaaS that I have experimente

OK, maybe now we should just call is Wave . As I talked about a few years ago in my blog, I really enjoyed using Wave for interacting with family and friends, and also experimenting writing Wave Robots hosted on AppEngine. Those good times have ended :-( Fortunately, the Apache Wave incubator project provides the code and directions for running Wave on your own servers. If you are lucky, installing Wave is as simple as: git clone git://git.apache.org/wave.git wave cd wave ant compile-gwt dist-server ant -f server-config.xml ./run-server.sh The default data store is the file system but you can use MongoDB instead. After you have started the server, you can use two different web browsers (e.g., Firefox and Chrome) to create two different accounts using the Register a new account link on the right side of the welcome page. After logging in under two accounts with two web browsers then create a new note under one of your Wave accounts and invite the other user account to that note.

My friend Alex Ott left a comment on my blog yesterday asking me if I had looked at OpenShift. I had created an account a while ago but never did anything with it. This morning I briefly tried getting Clojure (not yet supported) running but promptly gave up and switched to Ruby, which along with Java, Python, Node.js, and Perl is supported. It only seemed fair to test Redhat's PaaS with a supported stack. By default Ruby 1.8.7 is supported with a lot of gems pre-installed. I preferred to use RVM and Ruby 1.9.3 so I: I created a new empty "Do-it-yourself" Cartridge using the web console. I was prompted to add a public key, etc. Easy setup. A new empty cartridge runs a trivial Ruby web app. I followed Mark's instructions to install RVM and Ruby 1.9.3. After that, a git commit and a git push redeploys your app. You should take a careful look at .openshift/action_hooks/* that are places where you can customize builds and deployments. Using Mark's instructions, I

Over the course of a year I spend about 2/3 of my working time consulting for customers and the other 1/3 doing my own projects. I view doing personal projects as a form of continuing education. My last major personal project was my natural language processing (NLP) web service KBSportal.com and at the same time I spent a day writing SleepyBird.us for my wife. The project before those projects was an experiment in a live note taking and scheduling system for consultants ConsultingWith.me which I still use but I have turned off the ability for people to create accounts. I never implemented the sister project ConsultingWith.us . Six years ago my side project was CookingSpace.com which I do still use. For me side projects are a great way to try out new ideas and it is well worth it to me to pass on less interesting consulting gigs to free up enough time for my own stuff. I am in general interested in the combination of knowledge management and NLP and my new side project combines t

Full disclosure: for years JetBrains comped me with free products, but for a few years now I have been buying my own licenses. I have been working since 5am this morning on three customer projects in three languages: Python - using PyCharm Ruby - using RubyMine Java - using IntelliJ Switching languages and supporting platforms and frameworks sometimes causes a little cognitive dissonance, taking a few minutes to get my head into a different toolset. With the same (almost) user interface, similar code completion, etc., switching between three languages and projects today was easy with no stress. I will also thank other companies that make it easier for me to do my work: Amazon AWS - Amazon is my favorite technology company. AWS has changed my working life. Since I live in a remote mountain area, I also appreciate getting stuff from Amazon. The Kindle platform with synched reading accross all of my devices is also great. Apple - my MacBook Air and iPad are great devices that I don&

No I am not talking about a super computer, at least not in the conventional sense. I saw a blurb in this month's Technology Review magazine about the Padfone: a cellphone that slips inside a Pad device when you want a larger screen. Good idea as far as it goes but why not add instant connectivity to any compatible keyboard, monitor, and trackpad in the area. I find a slight hassle switching between my MacBook Air Air, iPad, and Droid during the day. Having one tiny device with a copy of everything is a possible alternative to Apple's iCloud, Dropbox, and other distributed "store all of my stuff" ideas.

The first time I used a computer was in 1962 when I was 11 years old. It was not convenient: my Dad had to take me to his office to use one of the early timesharing systems using a local teletype machine with punched tape for saving and reloading stuff. A few years later I took an extension course at a local UC campus, FORTRAN with punched cards with nice keypunch facilities: definitely a large improvement! In the 1980s life really got good: my own Xerox 1108 Lisp Machine and an Internet connection. In the decades starting in 1990 and 2000 there were continual improvements but progress was gradual: better Internet connections, the development of the Web, improvements in software tools like code repositories, IDEs, etc. Skipping ahead to the present time, I am trying to adjust my digital life, including my work and writing flows, to a more mobile lifestyle. Probably the biggest change for me is that I used to keep just about everything that I worked with and touched in a reposit

I signed up for Google's APIs around 2002 and used it until they stopped the service. I later switched to Microsoft's Bing APIs: the cost was great, free! Bing's new entry level cost of $40/month for 20,000 queries does not match my use case. I wish they had a $5/month for 2000 queries. I do a low volume of search for personal research projects and prototypes. Yahoo BOSS's service has an attractive price but I worry about the service being terminated with the deal with Microsoft. Google offers custom site search, but I didn't see any options for a general paid-for web search API. So, I signed up with Yahoo BOSS for search and if the service is ever terminated, I will look elsewhere. The price is good, $0.80 per 1000 queries. 2012-05-19 edit: Bing search now has a free tier for up to 5000 search requests per month. Cool! For research purposes, this is good enough.

Carol and I got home from visiting family in Rhode Island late last night and I have been working catching up on customer work since 4am this morning. Here is a picture of my birthday dinner. The kids got two pound lobsters for everyone - I am holding mine and a plate of grilled veggies. Java : I used to do research programming in Common Lisp or Scheme. For ideas that worked I often re-coded them in Java for better deployment options, for better alignment with customer preferences, etc. I fell out of that habit six or seven years ago because I had a long term Common Lisp development job and also really got into Ruby and Clojure development. Both Ruby and Clojure are great for research programming and for some types of applications the deployment options are good. That said, even Clojure which is efficient (about 1/3 the speed of Java and uses about 3 times the amount of memory) wastes computing resources (hits the environment and the pocketbook). Ruby is much less efficient than Cl

I have been blown away in the last year by Wolfram Alpha but I haven't done much with the developer's APIs . To make it easier to experiment with Wolfram Alpha, I wrote a simple Clojure wrapper for the Java APIs. You can get a copy at github . In case you don't want to grab the github repo, here is most of the code: (ns wolfram) (def appid (System/getenv "WOLFRAM_APP_ID")) (def engine (com.wolfram.alpha.WAEngine.)) (.setAppID engine appid) (.addFormat engine "plaintext") (defn query [input] (let [query (.createQuery engine)] (.setInput query input) (let [result (.performQuery engine query)] {:pods (for [pod (.getPods result)] {:title (.getTitle pod) :sub-pods (for [sub-pod (.getSubpods pod)] (for [contents (.getContents sub-pod)] (.getText contents)))})}))) Notice that you need to set the API key for your application in an environment variable. You get 2000 free API calls a

Even though the news media portrays a dire world situation, I disagree. In the last few decades the world has become a safer place and fundamental shifts in technology keep driving down the cost of computing resources, networks, and storage that enable greatly increased global productivity. For much of the world globalization is a rising tide that floats most people's boats. The problem is that not everyone benefits from new paradigms for constant lifelong learning, diminishing advantages of organizations who hold to old mega-scale production and business models, and a free flow of information. The book The Power of Pull is a good reference for ideas how to take advantage of the transitions that the world is going through, whether you like them or not! The losers in this new world are people and organizations who cannot (or don't want to) adapt and learn and who expect material rewards that are out of touch with their productivity. The biggest potential problem that concer

I don't like doing a lot of client side Javascript (or Coffeescript) development. pjax is a way to minimize client side Javascript while maintaining fast page load times. I became interested in pjax after reading an article on the development of Basecamp Next. DHH indicated that they looked at pjax but then rolled their own similar system. Here is a github repo with a Clojure and Noir example web app using pjax that I wrote this morning. There were a few non-obvious aspects to using pjax with Noir so hopefully this will save you some time. If you don't want to grab the github repo, here are a few interesting code snippets. First, we need to run a little Javascript to process links to set up for AJAX calls setting a "X-PJAX" header: $(function(){ // Activate PJAX test links // Response will be loaded into #wrapper element $('a').pjax('#wrapper') }) I put this code in resoures/public/js/application.js which is loaded in the common Cloj

Well this has been a discovery for me :-) I finished my work early today and started my afternoon working through a simple iOS 5 tutorial. As recreation, I went over to Hacker News and read a linked article by David Heinemeier Hansson on making the response time fast on Basecamp Next while still doing mostly server side processing. The article and the comments were great. This is of some interest to me because I have recently spent a lot of time writing a lot of client side Javascript for a Dojo + Rails app: straightforward but time consuming. DHH in the article and HN comments was making the point that for his company, it was a better developer experience doing more with server side Rails and less custom rich client code in Coffeescript or Javascript. I agree. He, and other people in the comments mentioned pjax as a library for sending back requests to the server that are marked with a HTTP header 'X-PJAX' if the page layout is not to be returned. This makes it relativel

I am a Linux enthusiast (downloaded my first distro over a 2400 baud modem, a long time ago!) and I really like the Android platform. That said, I have really been enjoying the integration between my iPad and my Apple iTV (that my stepson gave me for Christmas) and the Mountain Lion OS X information released today makes me feel fairly certain that the "Apple experience" is what I want when I am not earning money doing server side Java, AI and textmining consulting gigs, etc. For the work I do for making money (i.e., consulting) it doesn't matter what computer and operating system that I use. I am even planning on trading in my Droid cellphone for an iPhone this year. I also have a long history with Apple. I prepaid for an Apple II and received serial number 71. I wrote the simple little chess program that Apple gave away on a cassette tape for a while. When the Mac shipped in 1984 I bought one right away and wrote a commercial app that generated a lot of revenue. So,

As of right now, this new github repo mostly contains the code from the 3rd edition of my book but as I re-write the book, I'll also be updating my code. Some of this Java code really needs a rewrite: many of the examples from the first edition were written in 1998 - a long time ago! I have reworked the code with each new edition. The code examples are licensed under the LGPL but I am considering dual licensing them under Apache 2 also. Any suggestions for code improvements, pull requests, etc. will be appreciated.

I have been using Citrusleaf for a customer ( SiteScout ) task. Interesting technology. Maybe because I am excessively frugal, but I almost always favor open source tools (Ruby, Clojure, Java, PostgreSQL, MongoDB, Emacs, Rails, GWT, etc., etc. that I base my businesses on). That said, I also rely on paid for software and services (IntelliJ, Rubymine, Heroku, AWS services, etc.) and it looks like Citrusleaf is a worthy tool because of its speed and scalability (which it gets from Paxos, using lots of memory, efficient multicast when possible for communication between nodes in a cluster, etc.)

Just announced today: DynamoDB solves several problems for developers: No administration except for creating database tables (including some decisions like using simple lookup keys or keys with range indices and whether reads should be consistent or not) Fast and predictable performance at any scale (but see comment below on the requirement for provisioning) Fault tolerance Efficient atomic counters The probable hassle for developers that I see is in knowing how to provision tables for reasonable numbers of allowed reads and writes per second. When you create tables one option is to get warning emails when you hit 80% of provisioning capacity; I interpret this to mean that you really had better not go over the capacity that you have provisioned. Amazon needs to know how much capacity you need in order to allocate enough computing nodes for your tables. The capacity that you pay for can be raised and lowered to avoid getting runtime exceptions when you go over your provisioned number o

After talking with a friend and a friend of his about the Semantic Web and healthcare yesterday, I re-watched a great video on Web 3.0 by Kate Ray that I bookmarked and blogged about a couple of years ago. I like this video because it frames the problems that the Semantic Web is trying to solve. My last published book (for APress) had Web 3.0 in the title, a term that did not really catch on :-) At least a little bit of my enthusiasm for Semantic Web technologies has diminished over the last ten years because of problems that I have had on customer projects trying to collect linked data from disparite sources and merge it into something useful. There are (apparently) no silver bullets and any data collection and exploitation activities involve a lot of difficult work. I would not be surprised if this problem of merging different data sources is not solved by using Ontologies and webs of linked data sites, but rather, by vendors curating data in narrow domains and selling interfa

Yesterday I wrote about two web portals I have been working on in Clojure. One of them is online: our stock photo web site . This is a simple web app written with Clojure and Noir. I use the excellent stripe.com system for accepting orders for JPEGs (and soon hi-def video clips). In my tests it seems easy enough to buy JPEG files: you just check the ones you want, go to the purchase page, and in a few seconds you are downloading a ZIP file with the JPEGs you purchased. A simple little web app but I think that my wife and I will have fun with it: we are avid photographers.

I have three web portal projects that I have wanted to develop for quite some time. I am close to releasing two of them ( a text analytics web service and a stock photos and video clip store . My wife and I are avid photographers and we have been wanting to travel more and do more photography; I started putting together the photo site yesterday morning and hope to have it fully on line in the next day of two - simple to implement. The text analytics web service will be publicly available within a month or so (right now, just the demo page is active - I short circuited the new account login for now). My third project is a web portal for a single consultant to manage multiple customers. Last year I prototyped this for my own use using Java + GWT + AppEngine and then ported it off of AppEngine, using MongoDB for the data store. I have had such a fun and productive time using Clojure and Noir for my two recent projects that I am considering porting this third project to Clojure. I might