W18: An App That Uses AI
You hired an apprentice for your workshop (Pi). Now a customer asks: "Can your shop's apprentice also help me, at the counter?" Yes, with care. The same model that builds your app can work inside it: summarise a note, suggest a title, answer a question about your data. But at the counter, the apprentice talks to your customers, and the same three risks from W01 now apply to them, not just to you.
The Model Is Already There
Pi Workbench runs its model through Ollama, on your computer. Ollama answers questions over a small web API at http://127.0.0.1:11434, for any program, including your app. T26: Ollama & Chat shows this API from JavaScript, and T27: Full Stack AI App connects it to a whole app. The ideas carry straight over to Python.
To see which models you have, run ollama list in a terminal. The model menu in the workbench shows the one Pi uses.
One Typed Function, in the Shell
Asking a model is talking to the outside world, like the database. So it belongs in the shell, not in app/logic/ (W09). One small typed function in its own file, app/ai.py, using only Python's standard library:
import json
import urllib.request
def ask(model: str, question: str) -> str:
"""Send one question to the local model and return its answer."""
body = json.dumps({"model": model, "messages": [{"role": "user", "content": question}], "stream": False}).encode()
request = urllib.request.Request("http://127.0.0.1:11434/api/chat", data=body,
headers={"Content-Type": "application/json"})
with urllib.request.urlopen(request, timeout=120) as response:
data = json.loads(response.read())
return str(data["message"]["content"])
The pure parts stay in app/logic/: building the question from a note ("Suggest a short title for this note: …"), and cleaning the answer (trim it, cut it to 120 characters, refuse an empty one). Those you can test completely.
Testing Without the Real Model
A test that calls the real model is slow, and its answer changes every time. So the route receives the asking function as a parameter, and the test gives it a fake:
def fake_ask(model: str, question: str) -> str:
return " Buy milk and eggs "
The test then checks your code: the question was built correctly, the answer was cleaned, the page shows it. Whether the model is clever is not something a test can prove. That part, you check by trying it.
On the Page: Waiting Is Normal
A local model can take several seconds. Show that something is happening. htmx's hx-indicator points at an element that becomes visible while the request runs:
<button class="btn" data-variant="outline" hx-post="/notes/7/suggest" hx-target="#title-7" hx-indicator="#thinking-7">Suggest a title</button>
<span id="thinking-7" class="htmx-indicator">Thinking…</span>
The answer comes back as a fragment, like everything else (W07).
One more local detail: your app and Pi share the same model. The model menu says whether it answers two requests at once ("parallel") or "one at a time". If it is one at a time, your app waits while Pi works.
The Risks, Now for Your Users
- It will be confidently wrong. Label AI text as a suggestion, and let the person edit it before it is saved. Never let it decide anything important alone (R20).
- People will try to trick it. A note saying "ignore your instructions and…" goes into the question. Never give the model power to delete, send or pay. It writes text; your code decides what happens.
- Privacy. With a local model, notes never leave the computer. That is a feature: say so to your users.
- Cloud models cost money and need a key. The key is a secret, so it never goes in the page, and the app needs a server (W14's one question). Keep it in an environment variable, and cap the spending.
Asking Pi
"Add a 'Suggest a title' button on each note. It asks the local Ollama model (use the model name from an environment variable, MODEL) for a short title through one typed function in app/ai.py using urllib. Build the question and clean the answer in app/logic/ with tests. Test the route with a fake ask function, never the real model. Show 'Thinking…' while it waits."
Pi Workbench will do this step for you soon: a ready block, one typed function to ask the same local or cloud model you use in the workbench, streamed to the page word by word.
This Week's Exercise
- Read T26 and T27's architecture overview.
- Run
ollama list, then ask a question from the terminal withollama runand the model's name. - Ask Pi for the "Suggest a title" button. Read the test: does it use a fake?
- Try to trick it: write a note that asks the model to do something else. What happens? What could not happen, and why?
Key Takeaways
- Ollama, which runs Pi's model, answers any program at 127.0.0.1:11434, including your app
- Asking the model is I/O: one typed function in the shell; building the question and cleaning the answer are pure logic
- Tests use a fake ask function, never the real model
- Show a waiting indicator; AI text is a suggestion the person can edit
- The model writes text; your code decides what happens. Cloud keys are secrets and need a server