I noticed on X this morning that Google dropped a new batch API with a 50% price cut. I use gemini-2.5-flash for speed and low cost and being able to batch large numbers of requests in a JSONL file (JSON where each line is a single legal JSON expression) seems like a big deal to me. Gemini Batch API Docs I have been a little negative on Hacker News and X recently about the energy costs vs. value from LLM use and it seems like Google is striking a good middle ground in cost and environmental impact. Automating NLP and other workflows seems fairly simple: automate writing pipeline requests to a JSONL file, automate sending requests, periodically polling for the results being complete, auto-download and use results in your workflows.

In the late 1970s, I worked long hours on a text based adventure game called land of the dwarf for the Apple II computer. My game was written in Apple Basic and I gave it away for free. I wrote it using a huge sheet of paper, drawing a transition network diagram with bubbles, locations and action codes for things that could be done and arcs between the bubbles being the ability to move from one location to another. Yesterday I was really in the mood to do something fun. We didn’t have anything going on for Fourth of July celebrations until an outdoor symphony in the early afternoon, so I sat down in the morning with Google‘s excellent Gemini CLI coding agent and I described what I wanted, which was to be able to input a story context as a text input file and then use an LLM in a conversation loop and continuously provide you with the text based adventure program experience. I was really surprised how well it turned out. Fun! The generated adventure game code uses a local model running ...

I would like the USA to not lose the ‘AI race’ but I don’t want other countries to lose either. Ensuring continued US leadership in AI without constraining global progress necessitates a dual-pronged strategy that couples strategic national R&D investments with adherence to open international standards. On the technical front, integrating edge computing, specialized AI silicon, and advanced hardware-software co-design with standardized protocols for interoperability enables US institutions to maintain a competitive edge. Simultaneously, promoting collaborative frameworks for data sharing, federated learning, and algorithmic auditing mitigates risks of siloed innovation, ensuring that breakthroughs in efficiency, low-latency inference, and energy-optimized training regimes benefit a global network of research and industry stakeholders. In the USA I would like to see major resources dedicated to both open models such as the ones from Meta, as well as the use of French Mistral open mo...

 I spent a few days experimenting with Cursor last week. Bytedance's Trae IDE is very similar and is currently free to use with Claude Sonnet 3.5 and GPT-4o:  https://www.trae.ai/home  I would like to use Trae with my own API accounts but currently Bytedance is paying for LLM costs. I have been experimenting with the qwen2.5 and qwen2.5-coder models that easily run on my M2Pro 32G Mac. For reasoning I have been going back to using OpenAI O1 and Claude Sonnet, but after my preliminary tests with Deepseek-R1, I feel like I can do most everything now on my personal computer. I am using:  ollama run deepseek-r1:32b I recently published my new book “ Ollama in Action: Building Safe, Private AI with LLMs, Function Calling and Agents ” that can be read free online at  https://leanpub.com/ollama/read

I have been been playing with the Apple Intelligence beta’s in iPadOS and macOS and while I like the direction Apple is heading I am getting more use from Google’s Gemini, both for general analysis of very large input contexts, as well as effective integration my content in Gmail, Google Calendar, and Google Docs. While I find the latest Pixel phone to be compelling, I will stick with Apple hardware since I don’t want to take the time to move my data and general workflow to a Pixel phone. The iPhone is the strongest lock-in that Apple has on me because of the time investment to change. The main reason I am feeling less interested in the Apple ecosystem and platform is that I believe that our present day work flows are intimately wrapped up with the effective use of LLMs, and it is crazy to limit oneself to just one or two vendors. I rely on running local models on Ollama, super fast APIs from Groq (I love Groq for running most of the better open weight models), and other APIs from Mist...

Putting "closer to AGI?" in a blog title might border on being clickbait, but I will argue that it is not! I have mostly earned my living in the field of AI since 1982 and I argue that the existence of better AI driven products and the accelerating rate of progress in research, that we are raising the bar on what we consider AGI to be. I have had my mind blown twice in the last week: Today I took the PDF for my book "Practical Artificial Intelligence Programming With Clojure ( you can read it free online here ) and used it to create a notebook in Google's  NotebookLM  and asked for a generated 8 minute podcast. This experimental app created a podcast with two people discussing my book accurately and showing wonderful knowledge of technology. If you want to listen to the audio track that Google's NotebookLM created,  here is a link to the WAV audio file Last week I signed up for a one year plan on  Replit.com  after trying the web based IDE for Haskell and Python...

 I have only spent a short while experimenting with the ne gtp-o1 models: so far very impressive for science, math, and instruction following. You need a ChatGPT Plus account to try it, or you can perform rate limited queries for half the monthly cost using  Abacus AI The thing I am most impressed with (this week!) is the  Replit.co AI coding agent  that after briefly trying it I pre-paid for a one year subscription. I quickly rewrote a complex Clojure web app I wrote 12 years ago in JavaScript making it much less expensive to host  CookingSpace.com  (new JavaScript version) I gave a live demo of Replit AI in my weekly AI demo and group chat. Please join:  Mark Watson's informal AI presentations and group chats

I wanted to convert my server side web site  CookingSpace.com  to mostly client side JavaScript. I used the Replit.com AI coding agent to do this, and while this is a subject for another blog article, I showed the Replit coding AI a snippet of my JSON recipes and nutrients file, and described in a few paragraphs the functionality of the new web site. I want to share a recipe that I created:  Mark’s African Stew Ingredients Optional: 3/4 to 1 pound of Lamb shoulder, deboned and cut into cubes, leaving some fat on the meat 2 tablespoons of any non-seed oil (Avocado Oil is good). (Avoid seed oils like Canola, Corn, Soybean, Sunflower, etc.) 1 brown onion, coarsely chopped 1 egg plant, partially skinned,  chopped into bite sized pieces, and heavily salt in a strainer. Before cooking, rinse well to remove excess salt! ground coriander (at least 1 teaspoon) Cumin  (at least 2/3 teaspoon) paprika (1 to 2 teaspoons, depending on how you like the taste of paprika) ginger...

Topics: Large context prompts with LLMs vs. RAGs using embeddings vector stores. How to avoid LLM hallucination. 3 code demos. started an informal code demo and group conversation  Meetup group (link)  and today I gave a fifteen minute code demo followed by a conversation with the attendees. Here is a GitHub repo with the code examples:  https://github.com/mark-watson/talk2_LLM_Python_intro Here are my talk notes: Exploring the Future of AI: Introduction to using LLMs using Python Riff on ‘AI grounding’ and how LLMs help:  LLMs, trained on vast amounts of text, excel at recognizing patterns and providing contextually relevant responses. They mimic grounded understanding by referencing large datasets that encompass a variety of real-world scenarios. For example, they can infer meanings from complex contexts by drawing on their training data. When LLMs are integrated with other modalities, such as vision or audio (e.g., vision-language models), the grounding improves. ...

Robert Oppenheimer and Edward Teller facilitated my Dad getting a professorship at UC Berkeley when I was 3 years old. Oppenheimer left Berkeley but Teller was a good friend of my father and I remember him being in our home. Three weeks ago, my wife and I were just leaving to see the new Oppenheimer movie when my Dad called. He mentioned that when I was in grade school he was invited to give a talk at Princeton. After his talk Oppenheimer talked with my Dad and invited him to have dinner at his house. My Dad said Oppenheimer was not well (I think he died of throat cancer soon afterwards) but his wife Kitty carried the conversation. My Dad, Ken Watson, passed away 10 days ago on August 18, 2023.

  As humans we have evolved over a few million years to be both attentive and reactive to danger, live in social communities, and spend much of our time being in the present moment gathering and eating food and socializing. The behavior of rapidly changing short attention to content on social media, too many good short form things to watch on streaming video entertainment platforms, are all rewiring our brains in an unnatural and unhealthy way. I fight back, but in really simple ways that entail little ceremony: Almost every morning I spend 30 minutes scanning Hacker News (about 10 minutes), Apple News (about 5 minutes), and the remaining time on Twitter and Mastodon finding interesting new (mostly tech) things. I make notes in a temporary Apple Note: links of things I may want to research, try, or simply read that day. I like to get this all done at once, and then not feel like I need to interrupt my activities during the day to “catch up” on what is happening in the world. In a w...