More Clojure Datomic experiments: decoupling data building and transactions, and adding text search

I wrote two days ago and yesterday about my experiments for getting up to speed on Datomic. In the Datomic news group, Rich Hickey suggested that I keep any helper code for data-building separate from helper code for managing database access/transactions. I have reworked my code and added code to experiment with text search.

Now that I have a few days of experimenting with Datomic I think I understand how I will structure an application for a side project: write a small helper library that is application independent, much as the code snippets in this article. Write another application specific helper library that layers on top of the application independent library that encapsulates all data store functionality that I will need, and unit test this separately. Then I can write the application layer that will probably not have any Datomic application specific code. After I finish the first version of my side project this morning then I will help my customer on his Datomic project.

I removed the single core.clj functions and replaced it with two sets of functions. databuilder.clj:

(ns datomic-test.databuilder
  (:use [datomic.api :as api]))

(defn attribute [id t c doc]  ; by Michael Nygard
  {:db/id (api/tempid :db.part/db)
   :db/ident id
   :db/valueType t
   :db/cardinality c
   :db/doc doc
   :db.install/_attribute :db.part/db})

(defn string-singleton-attribute [id doc]
   (attribute id :db.type/string :db.cardinality/one doc))

(defn string-singleton-attribute-searchable [id doc]
  (assoc
    (attribute id :db.type/string :db.cardinality/one doc)
    :db/fulltext true))

(defn string-multiple-attribute [id doc]
  (attribute id :db.type/string :db.cardinality/many doc))

(defn string-multiple-attribute-searchable [id doc]
  (assoc
    (attribute id :db.type/string :db.cardinality/many doc)
    :db/fulltext true))

(defn long-singleton-attribute [id doc]
   (attribute id :db.type/long :db.cardinality/one doc))

(defn long-multiple-attribute [id doc]
  (attribute id :db.type/long :db.cardinality/many doc))

and transactions.clj:

(ns datomic-test.transactions
  (:use [datomic.api :as api]))

(defn- do-tx-helper [conn partition data-seq]
  (let [data
        (for [data data-seq]
          (assoc data :db/id (api/tempid partition)))]
    @(api/transact conn data)))

(defn do-tx-db [conn data-seq]
  (do-tx-helper conn :db.part/db data-seq))

(defn do-tx-user [conn data-seq]
  (do-tx-helper conn :db.part/user data-seq))

Finally, a little bit of test code that also includes a query using text search:

(ns datomic-test.test.core
  (:use [datomic-test.transactions])
  (:use [datomic-test.databuilder])
  (:use [clojure.test]))

(use '[datomic.api :only [q db] :as api])
(use 'clojure.pprint)

;;(def uri "datomic:free://localhost:4334//news")
(def uri "datomic:mem://news")

(api/create-database uri)
(def conn (api/connect uri))

;; create two singleton string attributes and add them
;; to the :db.part/db partition:
(do-tx-db
  conn
  [(string-singleton-attribute-searchable
     :news/title "A news story's title")
   (string-singleton-attribute
     :news/url "A news story's URL")
   (long-singleton-attribute
     :news/reader-count "Number of readers")])

;; add some data to the :db.part/user partition:
(do-tx-user conn
  [{:news/title "Rain Today"
    :news/url "http://test.com/news1"
    :news/reader-count 11}
   {:news/title "Sunshine tomorrow"
    :news/url "http://test.com/news2"
    :news/reader-count 8}])

(def results
  (q '[:find ?n :where [?n :news/title]] (db conn)))

(doseq [result results]
  (let [id (first result)
        entity (-> conn db (api/entity id))]
    (println (:news/title entity)
             (:news/reader-count entity))))

(def search-results
  (q '[:find ?e ?n
         :where [(fulltext $ :news/title "rain")
                 [[?e ?n]]]]
    (db conn)))

(doseq [result search-results]
  (let [id (first result)
        entity (-> conn db (api/entity id))]
    (println (:news/title entity)
             (:news/reader-count entity)
             (:news/url entity))))