Friday, December 31, 2010

Nice: Neo4j version 1.2 final released

Neo4j is a high performance graph database that I usually use with the JRuby neo4j gem and sometimes with Clojure (documentation here).

Neo4j is open source (AGPL v3) and is alternatively available for a reasonable fee with a commercial license (where you don't need to AGPL your project). I took advantage of the free offer to get your first Commercial Basic Server license, even though I am likely to open source my project anyway.

Wednesday, December 29, 2010

Good SimpleDB performance tips

I don't usually write blogs that just reference other people's material, but this three part article by Siddharth Anand (first installment, then follow links to other articles) is really worth reading if you use SimpleDB.

BTW, while the local SimpleDB simulator simpledb-dev works OK, for development I usually just access SimpleDB remotely from my laptop. One word of warning: while properly constructed SimpleDB queries can run very quickly from EC2 hosted applications, remote access tends to be 5 to 10 times slower.

WebServius wrapper for web services and data services

Webservius is a wrapper that provides for your web service APIs and data sources. Webservius handles billing, tools for your customers to use to monitor their use of your services, logging and use statistics, etc. They charge you 10% of what is billed through their service.

When I have a chance I will try the free version of this service (limited number of API calls per day and access to end users must be free) and write up the experience.

I noticed WebServius on Amazon's blog but Webservius is an independent company that uses the Amazon AWS platform.

Monday, December 27, 2010

Control of news media == ability to set public opinion

In the late 1800s Western Union was able to alter the results of a presidential election (reference: The Master Switch: The Rise and Fall of Information Empires - recommended!)

A similar situation exists today in the USA: all major news sources provide a very slanted pro-corporate agenda that results in a large percentage of the population simply not understanding things like the benefits of a social safety net (a trade of tax money to keep society civil and safer), how we incarcerate a much larger percentage of our population than other countries, that Wikileaks has released a very small percentage of the diplomatic cables and those cables that have been released have been redacted to minimize the chance of putting people in danger, that we spend as much on our military as the rest of the world combined, how military spending enriches only a very small number of powerful people, the level of corruption in Congress (both democrats and republicans), etc.

It is sad that better sources of news come from the Comedy Central channel (for example, The Daily Show), Al Jazeera, foreign services like German Der Spiegel, small companies like, etc.

The USA is in rapid decline. Part of the decline is due to failing infrastructure, large scale corruption, massive military costs, and also a much less informed population. All of these means to decline are fueled by a tightly controlled dishonest news media.

Wednesday, December 22, 2010

Christmas came early: my Google TV arrived at 7pm tonight

Setting it up took 45 minutes because it immediately downloaded a new version of the OS when I set up the wireless Internet connection. I use Directv and it synced up with my DVR and Sony TV with no problems. The keyboard is nice, with a trackpad and mouse button top right corner. It is much nicer using the keyboard rather than the remote for Directv guide and DVR control.

I have experimented with writing Android cellphone apps with the SDK and now I want to try some HTML 5 apps for both the Android cellphones and Google TV. Fun! I had a consulting job two years ago writing some Java blu-ray example apps and the development environment for that was, frankly, painful. Both Android and Google TV seem much more developer-friendly.

You personalize Google TV by logging into your Google/GMail/Apps account. I had to enter my account information multiple times for and Picasa photo albums. There are some wrinklles to iron out but the platform has a lot of promise.

We watched part of a Netflix movie on demand and it looked good. I have a Hulu Plus account but when I went to Hulu I got a pop up dialog warning me that Hulu is building out for Plus access on Google TV. I am looking forward to that because Hulu Plus is a pretty good service. Along with all of the Directv and Netflix material, it seems like there is an infinite amount of good stuff to watch.

A few improvements I look forward to:
  • When looking at my Picasa photo albums it seemed like the Google TV did not cache-ahead very well. I would like a picture to stay on screen until the next one is ready.
  • When watching one of my wife's documentaries for a charity she helps, Youtube playback was pretty good because we uploaded a fairly lowres video to Youtube and it played back smoothly. I then tried one of my videos on Youtube that was fairly hires and there were too many pauses for buffering. Also, if I paused the video it did not seem to buffer ahead like on my laptop.
I am very excited with the Google TV platform and I thank Google for giving me a free Logitech unit. I am not always good at predicting which nacent technologies succeed in the marketplace but to me, at least, the Android platform and the Google TV built on very similar technology show a lot of promise.

Excellent product: RubyMine 3.0

I bought RubyMine when it was first released and recently paid for upgrading to version 3.0. I switch between using TextMate (or GEdit on Linux) and RubyMine for both Ruby and for Rails development but since getting version 3.0 I spend almost all of my time using RubyMine.

Rails 3 support is very good and working with RVM is a nice new feature. I never used to use the Ruby debugger (at all!) but I have used it twice briefly in version 3.0. At least for Rails 3 development I still separately run both rails server and rails console outside of the IDE - a matter of personal preference.

I was using PyCharm (also by JetBrains) a lot for 2 weeks and the autocompletion hints and instant syntax warnings really helped me because my Python and Django skills are light-weight. I don't much need autocompletion hints and instant syntax warnings for Ruby and Rails development but it is unobtrusive and often useful.

The HTML, JavaScript, Erb, and Haml support seems better also. One feature that I don't use in daily development much but that I really like is View->Show Model Dependence Diagram that gives a great overview of your database model classes, their attributes, and their associations. Great for documentation and to serve as a reminder if I have not worked on a project for a while and need to get back into it. Quick navigation support to jump to definitions, etc. also saves me time.

EDIT: Since I wrote this blog I have started using the Model Dependence Diagram fairly often. It would be great to also have an option to show a UML class diagram.

My only real disappointment with version 3.0 is that it does not implement the tear-off editing tabs (i.e., tear-off to create a new and separate window) that the latest version of IntelliJ supports.

That said, using RubyMine is a comfortable experience and the IDE features stay out of my way until I need them.

Getting the Dojo 1.5 rich text editor dijit.Editor working nicely with Rails 3

I could not find much on the web and it took me a little time to get everything working (especially saving edited content) so I'll share some code, hopefully to save you some time. In apps/views/layouts/application.html.erb I added some boilerplate (with non-editor stuff not shown) to be included with each rendered page:
  <link rel="stylesheet" type="text/css"
  <link rel="stylesheet" type="text/css"
    href= "">
  <script type="text/javascript" src=""
          djConfig="parseOnLoad: true">
  <script type="text/javascript">
Then in my view I create an inline rich editor using:
<div dojoType="dijit.Editor" id="editor1"
     extraPlugins="[{name: 'save', url: 'task'}]">
  <%= raw(@task_current.content) %>
It is very important to use the raw method because Rails 3 automatically makes safe HTML encoded strings for any result returned by <%= ... %>. Also notice the url value of 'task': this sends the content data to the index method of the task_controller. This method checks to see if an incoming request is an XHR POST and if so gets the raw content:
if request.xhr?
   if session[:task_id]
     task = Task.find(session[:task_id])
For brevity I did not show any error handling.


Tuesday, December 21, 2010

Reading "The Rails 3 Way"

Obie had his publisher send me a review copy of his book Rails 3 Way that was just printed. It is a very good reference for Rails 3, and it really is a reference book meant to be accessed for specific problems. That said, I am reading it straight through because although I have a lot of Rails development experience, I would like to understand Rails at a deeper level.

While Rails itself is "opinionated" Obie's book is even more so: he bases his business on Rails and the book reflects the way things are done in his company. The book uses Haml and RSpec exclusively. I still mostly use Test::Unit and Erb, and the book will probably not change that (but it might!).

I found Paolo Perrotta's "Metaprogramming Ruby" to be a great resource for getting more into the low level details of Ruby and I expect that "The Rails 3 Way" will serve the same purpose for Rails.

Friday, December 17, 2010

Using cloud services when services like and Wave get cancelled

You can still access my public bookmarks if you want them. I will miss the service, as I miss Google Wave.

I tried several cloud based to-do and getting things done style web sites but ended up writing my own web app but left the door open to other people by releasing the app as open source. I used ran the online word processor for a few years but made sure to provide data export options.

People and companies who provide free services have no real obligation to continue the services forever but they have a responsibility to give users a good exit strategy. I exported my bookmarks and I suggest you do the same.

Wednesday, December 08, 2010

Suggestions for Python SDK and AppEngine

Partly because I don't often code in Python, I have been feeling some pain using the Python AppEngine SDK. I thought that I would pass on a few things that have made the process easier for me:
  • Use an IDE. I have been using PyCharm but other people have mentioned liking Eclipse with the Python and AppEngine plugins. Live hints when I get a local variable name wrong, suggestions to automatically import one of my own modules, and autocompletion have really helped. If I were more familiar with Python and the Python AppEngine SDK then this would not be as beneficial.
  • Unit tests: I have been using to support unit testing and that has been helpful. I get all of the model code and controller helper functions working and tested before doing the UI
  • I find that the default Python SDK template engine (from Django) to be adequate and to be fairly easy to use.
  • I have found Mark C. Chu-Carroll's book Code in the Cloud, Programming Google AppEngine to be a useful tutorial.

Platforms and Infrastructure as service

Good news for the people at Heroku (Salesforce just paid 200+ million for the company). This is definitely a kick up for platforms as a service. I have long considered Heroku to be the best platform as a service offering because it is so developer friendly - very different from AppEngine which I would label as "scalability friendly."

I had to recently make a decision between developing for Heroku or AppEngine for my new business ideas ( and Heroku and Ruby on Rails, along with all of the useful plugins and auxiliary services offered by Heroku, make the most agile web application development and deployment story right now.

In contrast, developing for AppEngine is a pain in many ways. That said, I think that the new AppEngine SDK and services are a solid improvement and since I hope for more than small scale success the relatively inexpensive hosting costs and automatic scalability of AppEngine won me over (at least for these projects).

Infrastructure as a service is also becoming more accepted. My largest consulting customer is considering hosted infrastructure options for very large high performance datastores. As easy as it is to set up a cluster using MongoDB, HBase, or BigCouch, etc. there are so often good business and technical reasons to pay specialists companies to do it for you.

Thursday, December 02, 2010

AppEngine SDK 1.4 release is likely a game changer

I use AppEngine to host my own projects but not for my customers (everyone who has hired me for a few years has wanted Amazon AWS deployments - no exceptions).

I think that the story for using AppEngine is definitely better with the 1.4 release. A charge of $9/month for keeping 3 compute instances always spun up for an application seems like a modest cost to workaround slow loading request times. (I have written twice about this: 1 and 2.)

Ten minute CPU limits for background processes is a nice increase over the old 30 second limits.

I have recently been using the Python AppEngine SDK for my latest side project (something I need for my own use and I am also planning on offering it to other consultants for a small fee).

Platform as a service: I have had the opportunity to help many individuals and companies in 14 years of running my own consulting business and I feel like I have a fairly clear understanding of the opportunity costs of manually managing servers. Small teams especially get hit hard when too much available engineering time is spent on deployments and administration of servers. I expect managed deployment platforms like AppEngine and Heroku will be a strong growth market. Managing servers and deployments may be fun but often does not make business sense.