At the end of 2024, the AI work I was most excited about involved helping teams find opportunities in government contracting. It gave the technology a concrete purpose: support a difficult business task, rather than add a chat interface simply because one was possible.
I was also working with Shamim Bhuiyan on Generative AI with Local LLM. The book and the project work connected around a practical question: how can we make AI useful while staying deliberate about where data goes and how the system operates?
Start with the work, not the model
Government-contract information is a good reminder that having access to text is not the same as making a useful decision. A team needs to understand whether an opportunity fits, what the requirements mean, and what deserves attention.
A model may help with parts of that process. The quality of the overall workflow still depends on what information it receives, how its output is checked, and where a person makes the final decision.
That is the perspective I wanted to carry into 2025: begin with a recurring problem and a useful output. Only then decide which model or agent workflow belongs in the solution.
Local execution is a design choice
In the original social post, I described local models as a way to protect data and reduce costs. That was too absolute. Local execution changes the available tradeoffs; it does not settle them automatically.
Running inference locally can reduce reliance on an external inference provider. It also gives the team responsibility for the machine, the application, its access controls, and the other services connected to it. A local model does not make a networked application isolated by default.
The cost question is similarly conditional. Hardware, utilisation, maintenance, and the quality needed for the task all matter. A comparison is useful when it covers the actual workflow, rather than only the price of one model call.
Learn by building a small complete workflow
The direction of the book was hands-on: give developers a way to work with local models and understand the surrounding application. A small complete example can teach more than an impressive but disconnected demonstration.
The useful questions are concrete. What data does the application need? What result does it return? Where can it be wrong? How will someone notice? What happens when it cannot complete the task?
These questions give experimentation a purpose. They also make it easier to compare approaches, because the desired outcome stays stable while the implementation changes.
Keep the optimism, make the claims narrower
I ended 2024 enthusiastic about AI and the open-source work around it. I still want to preserve that enthusiasm in this archive. The revision is to be more careful about the promises attached to a deployment choice.
Local language models are worth understanding because they create options. Whether those options improve a particular product depends on the constraints and the evidence from using it.
Explore the book and the projects behind these notes.