Posts

Showing posts from 2015

Happy New Year

Hello everyone. I want to wish everyone a happy new year and say a few things about what I expect for the new year. I believe that one of the most important issues facing "first world" countries like the USA and England are the issues of Internet security and privacy. The news this morning of the umbrage of US congress people to the news that NSA is monitoring of their communications with people in the Israeli government is laughable: let us be clear about this: these people don't care about the privacy of US citizens but they do care about their own privacy and the privacy of leaders of another country. This stinks, and badly. While privacy is important I believe that a bigger issue security. I would like to see my government (USA) conduct a multi-year "going to the moon" type project for strengthening our information infrastructures to the benefit of people, companies, and governments. This means that there can be no encryption back doors installed in any

Raspberry Pi and education

I may be late to the Raspberry Pi party - I just bought my first one this week. The Rasberry Pi is everything that I would hope for in an educational computer: cheap enough for all children to own and based on open source software (Debian Linux, LibreOffice, lots of games, and programming languages like Python, Ruby, Java, Scratch, etc. pre installed). The open nature of the Raspberry Pi encourages kids to experiment. RPs might not be as practical as other systems like ChromeBook that have more distributed infrastructure behind them but I think that open systems provide a better better environment for experimenting with computers. I reformatted a 32GB memory card and installed a fresh Debian Linux image provided by the Raspberry Pi project and when hooked up to a large monitor the Raspberry Pi 2 is quite capable. I installed the RubyMine IDE and git cloned a few of my Ruby projects and loaded the manuscript for my current writing project. I find the system is surprising fast with

Digital Life: a modicum of privacy

This post contains my advice for maintaining a reasonable amount of privacy without reducing the utility and entertainment we get from the Internet. It is no news that governments are pushing back against our right of privacy and we should also be concerned by tracking by both corporations and organized crime. Privacy is a basic human right and once rights are lost or reduced in scope they can be very difficult to get back. To start with I believe that everyone should have the privacy enhanced Tor web browser installed. Tor was developed originally by the US Navy in support of journalists and other people living in countries with oppressive regimes. I strongly recommend using Tor for the following reasons: Research any medical conditions that you have. You are interested in buying a product and you don't want advertisers to put ads on web sites you visit because you would rather make independent unbiased purchasing decisions. Visit any sites for any reason that you would not lik

Ruby SVM text classifier

There are several useful Ruby gems/libraries for using Support Vector Machines (SVM) and another to convert text into SVM style feature vectors. I recently packaged up what I needed with a Ruby script to fold the data for testing, etc. Here is the github repository. It took me a short while to get everything working together so hopefully this will save you several minutes of extra effort if you want to use SVM for text classification.

My new book "Power Java" is released today

I recommend that you look through the github repo for my book to see what I cover and if it looks interesting please please consider buying my new book on leanpub . I cover a wide range of topics including machine learning, linked data, network programming techniques for IoT, and some ideas for knowledge management using cloud data.

My Cognition Technology blog and website

I created a new blog yesterday  http://blog.cognition.tech  for news and my personal programming experiments involving machine learning and deep learning. There is a companion website  www.cognition.tech  where I offer consulting, mentoring, and turnkey development. I will continue using this blog for personal posts, programming languages (mostly Haskell, Java, Clojure, and Ruby) and general discusions on technology.

I need some sympathy: spending most of my time coding in Java and Python

As the universe unfolds, I have been spending most of my time recently working with machine learning and for the forseeable future that will not change. Lets face it, many of the really great ML libraries are written in Java and Python. I still love development with Clojure and Ruby, and I am still on my long term quest to become passably proficient with Haskell. That said, it is crazy to not simply use languages that have the best library and framework support for any task.

We are getting closer to the dream of the 1980s and 1990s: software reuse

I worked (mostly) at SAIC in the 1980s and 1990s. In the groups I worked in we developed large software systems (sometimes with hardware components) for customers. Software reuse was a dream back then that was largely unfulfilled. Our procedure for reuse was mostly cut and paste from old projects, with some effort to write reusable libraries. There was also a movement to use commercial off the shelf (COTS) software. It occurred to me recently that we are now much closer to the dream of widespread software reuse. What has changed is a healthy open source (and libre) software ecosystem of trusted and vetted libraries, frameworks, and complete applications. I tend to trust software from FSF and the Apache Foundation, for example. Organizations and individuals are motivated to release software for a variety of reasons: for help in development and bug detection, for good publicity and self promotion, and sometimes for ego. All good reasons! My process when starting a new project is to f

I tried Windows 10 the first day of the rollout (today!)

Installing Windows 10 on my 5 month old HP Stream 11 was easy.I have no comments on that process. Visually two things stand out: windows are all white except for a very thin aqua blue margin and my slow laptop seems to run the UI faster. I don't know how much of the speed bump is making the code more efficient and how much is doing away with some animation effects. The desktop now seems like a mixture between Windows 7 and 8.1. The start menu is back and the bottom icon navigation bar is always visible along the bottom of the screen unless you put an application in full screen mode. Clicking the windows start menu in the lower left corner of the screen brings up a combo: classic looking menu on the left and the metro large icon interface like Windows 8.1 on the right side of the popup - but this popup only covers about 30% of the screen. It all seems a bit odd to me but I really like it - after using it for ten minutes (it took a little while to adjust to the new interface). I

Comparing Clojure + Clojurescript with Scala + Scala.js

Even though I mostly use Java, Ruby, and Haskell, I have also been getting my head back into using Scala in my spare time. I took Martin Odersky's Functional Programming Principles in Scala class three years ago, and although I really enjoyed the class (he is a great lecturer!), I didn't much care for the tooling for Scala at that time. I ended up mostly using Clojure (with a little Haskell) for my day to day most used programming language. I experimented with Scala.js a while back and thought that it compared well with Clojurescript. Sweet to write client code in either Clojurescript or Scala.js but I think that sometimes it is faster to not have the extra complexity and the need to transpile and just use plain old Javascript. I took a class in Typescript this year and really liked it but with ES6 quickly becoming a standard some of the benefits of Typescript go away. This morning I was looking for an interesting template project using Scala for the backend and Scala.js

Ubuntu Linux on my Chromebook without X11

I have been very happy with using Chrome OS on my Toshiba Chromebook 2 but after often reading how easy it is to install Ubuntu I decided to give it a try. I used crouton (which runs Ubuntu along side Chrome OS) and decided to not install X11 to save space - important with just 16GB internal storage. I used: sudo sh ~/Downloads/crouton -t cl-extra After installing Ubuntu I had about 10.5 GB of free local storage (remember, Chrome OS is also installed). After installing the gnu command line development tools, Ruby 2.1, Java 8, lein for Clojure development and several of my Ruby and Clojure projects I still have 7.5 GB of storage. I still want to install Haskell so I will have less space to work with. When developing web apps, you can test using the Chrome web browser since crouton runs Ubuntu right along side Chrome OS. For example, I was running a Sinatra app using port 4567 and could just hit the URL http://localhost:4567 on the Chrome browser in Chrome OS. Easy. I thought that

Experiences with my new Toshiba Chromebook 2

When I worked as a contractor at Google in 2013 I noticed that a lot of people were using Chromebooks. On my first orientation day I received a retina MacBook Pro, that was very nice, and I didn't immediately understand the preference of some people to use a Chromebook. Later I understood that a large amount of work performed at Google could be done in a Chrome web browser. They even had a very nice browser based IDE called Cider that was very great to use because it handled all programming languages and interfaced with Perforce source code control. My curiousity about Chromebooks has persisted. I am going to start teaching free classes at my local library in about two months on Internet secutity and privacy and I used this as an excuse to buy a Tosiba Chromebook 2. I had already bought earlier this year a little HP Stream 11 Windows 8.1 laptop using the same excuse :-) I will start out this "review" with a list of the good and not so good things about the Chromebook

using nitrous.io

I recently signed up for the pro version of nitrous.io since they are phasing out the old version that I used for free. So far I am very pleased. Now you get isolated containers to work in with root access. I use the $15/month version that gives you 1G ram and 20G of disk space for a total of two containers. Since I only use nitrous.io for development, I found it easier just use one container (using the entire 1G ram) and cloning the git projects that I am working on. Some of the things that I particularly like are: The editor built into the web based IDE handles just about all programming languages and file types. There is a separate app than can do two things: sync files between your laptop and your container and also forward ports so you can test run apps in the container and use your local web browser. If you don't want to forward ports, you can use their preview option that opens a new browser tab and lets you test in your browser without any port forwarding. The battery o

More infrastructure changes: Heroku

I am pleased that Heroku has introduced a new low volume pricing tier. I never felt very comfortable freeloading on their free tier and free tier web apps also timed out leading to longer loading request times. Now for $7/month per app Heroku supports a "hobbyist" mode for lower traffic sites that never get timed out or swapped out. So far I have redeployed three of my low traffic sites to Heroku's new plan, moving them from a dedicated server. They reduced their free tier hosting of a site to only being active just 18 hours a day - in other words: great for testing deployments but not good for hosting sites for free 24/7. I think this is a good move on their part although I did hear some complaints on Hacker News about this. Under this new pricing tier my three low volume sites cost about $21/month to host. Under the old paid plan the cost would have been about $105/month. BTW, I would like to thank everyone who took the survey (or emailed suggestions directly to me) a

Some infrastructure changes

In addition to using several programming languages, I also like to experiment with different web infrastructures. A few weeks ago I switched from using gmail as my primary email service to using fastmail.com. I still use gmail as a backup email and for my Google identity but I decided that I liked Fastmail a bit better and the yearly cost is not much. The other change I have made is switching my www.markwatson.com web site from a Ruby + Sinatra web app on my own server to a PHP app running on Google's AppEngine. My absolutely favorite feature of AppEngine is the rolling system logs that can be checked easily from the AppEngine console. When I worked at Google in 2013, I loved the internal development environment (Borg, online system logs, the Cider IDE, and much more). Using AppEngine is, in a small way, reminiscent of Google's internal environment - at least enough so to make me nostalgic :-) I have never been a huge fan of PHP although I have used it over the years for

My two Clojure projects, life in Sedona Arizona, and my new book project

Image
Two Clojure projects? Well, actually, I had just one Clojure project until today. I refer to my project as KB2 (KnowledgeBooks.com 2) and it is basically a kitchen sink for everything that I thought that I wanted in a personal (and perhaps small group) research and content management system: A personal version of Evernote: allows me to collect eBooks, web pages snippets and notes in a personal repository that is searchable. I use a Firefox add-on I wrote to capture multiple selections on web pages and send them to the web app. Uses NLP to identify entities in eBooks, web pages and notes and add an information icon that provides DBPedia (WikiPedia) information on the fly. Uses the Bing search API to find information on what my NLP analysis code considers if the main topic of eBooks, web pages and notes. I enjoy meditation (also practice Yoga since about 1975 and Qigong for about two years) and after my early morning mediation this morning I had one of those ah-ha moments: In us

Net Neutrality (Yeah!), a new distributed wiki, and my current Clojure project

It is, IMHO, a very good thing that net neutrality now has the force of law behind it. The Internet is the most important artifact for sharing information ever and is worth protecting as a neutral platform. Even though there is a lot of value in walled gardens created by Google, Facebook, and Apple I still hope to see many more systems developed that support individual control of our own data and better support for privacy. I think that Ward Cunningham's new Federated Wiki (on github: github.com/fedwiki ) is a very interesting idea for combining local storage with federated sharing of content. Something to keep our eyes on! I have been working on a combined document repository and general research tool that is tailored to my own needs but I will also sell it as a low cost commercial product (or I might make a compiled version free with the source code available with a commercial license for inclusion in other products - I am not sure yet). I want a system where I can store PDF

I bought an HP Stream 11" Windows 8.1 laptop

I have good intentions of starting to give talks at my small town's local library on safe internet browsing, privacy issues, and any topics that I get requests for. I bought a $200 HP Stream 11" laptop to get up to speed on Windows 8.1 since I only use OS X and Linux. I am pleasantly surprised at how much I like this laptop! The display is pretty good, the keyboard is also OK, and the no-fan design is nice. It works well with a second 1080p external monitor also. I ordered the laptop directly from Microsoft: their "signature editions" come with no crap-ware; just Windows 8.1 and nothing else. The shortcomings: Only 32 GB of solid state disk space. After installing Ruby, Java 8, IntelliJ, git command line tools, Pharo Smalltalk, Chrome, and cloning several git projects I have about 11 GB free. Slow CPU. For using Pharo Smalltalk this is no problem. It is a small problem for Clojure development. I use IntelliJ with the Clojure plugin for editing and have a lein r

Happy New Year

I would like to wish everyone a Happy New Year! I am not going to make new year technology predictions, but I do have a few comments on what technology I am using and what I expect to be using this year: First, I am surprised at how much Microsoft technology I am using. This started last spring when for political reasons I wanted to stop using Dropbox (I did not like their appointment of Condoleezza Rice to their board of directors). At the time, I could not find a good Dropbox replacement but after getting a Microsoft BizSpark business development grant last fall, and very much liking the Azure services, I recently decided to take another look at Office 365. It turns out that for the same $100/year I paid Dropbox for a terabyte of storage, with Microsoft I now get for the same cost one terabyte of cloud storage per family member and everyone in my household also gets Office applications and use of the online Office tools (that run really well in Chrome with Chrome apps). The versi