Building an AI Patient Chatbot for Urgent Care with n8n, GPT-4, and Langfuse

The first week our chatbot was live, Langfuse — the tracing tool that records every model call — flagged a response that listed outdated holiday hours. Stale schedule data the bot was about to hand to real patients. Nobody had complained yet. We caught it because we could see the full trace: the exact context that went in, the exact answer that came out.

Here’s what that week taught me: in healthcare AI, the model is the easy part. The hard part — the part that earns you the right to deploy at all — is auditability and the context you feed the model. We built our bot to answer the three questions every urgent care patient asks (“What are your hours?” “Do you take my insurance?” “How long is the wait right now?”), and we instrumented it heavily enough that we could catch a bad answer before a patient ever saw it. This post is how.

The problem we were actually solving

Our front desk staff were context-switching all day — checking in the patient at the counter, picking up the phone to repeat our hours, answering the same live-chat question for the fourth time that morning. The questions were predictable. The answers were fixed. Staff time was going to repetition instead of the people standing in the building.

We wanted a system that could answer common patient questions accurately and instantly, escalate anything complex to a human, read live wait times from our queue-management software, and leave a full paper trail for quality monitoring. The paper trail wasn’t an afterthought. In a clinic, “the bot said what?” is a question you have to be able to answer on demand.

Architecture overview

Three pieces do the work: n8n (a visual workflow tool that orchestrates the steps), GPT-4 (OpenAI’s large language model — the model that actually generates the answers), and Langfuse (the observability layer that records everything).

curl https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "max_tokens": 10,
    "messages": [
      {"role": "system", "content": "Classify the patient query into exactly one label: hours | insurance | wait_time | medical_advice | other. Reply with the label only."},
      {"role": "user", "content": "Do you take Blue Cross?"}
    ]
  }'

The obvious alternative — send every message straight to GPT-4 — dies on cost and latency, and it leaves no cheap place to refuse a medical-advice question before it ever reaches the model. The two-stage bucketing above solves both: the cheap GPT-4o-mini classifier runs first, and only the buckets that need real reasoning touch GPT-4.

Why n8n?

We chose n8n as our orchestration layer for several reasons:

1. Visual workflow design — non-technical staff can understand and modify flows
2. Self-hosted — patient data never leaves our infrastructure
3. Extensible — easy to add new tools, APIs, and decision branches
4. Version controlled — workflows are exportable and auditable

The conversation flow

The chatbot workflow handles patient messages through a structured pipeline:

1. Message intake — Patient sends a question via web chat
2. Context enrichment — System pulls current clinic hours, wait times, and service availability from Clockwise.MD (our real-time queue-management system)
3. AI response generation — GPT-4 generates a response using clinic-specific context and conversation history
4. Langfuse trace logging — Every interaction is traced with input, output, latency, and token usage
5. Escalation check — If confidence is low or the query is complex, route to human staff via Teams (Microsoft Teams)
6. Response delivery — Patient receives the answer in under 5 seconds

Langfuse: the observability layer

You don’t deploy an AI system in healthcare and hope for the best. Every response has to be auditable, and you have to catch quality problems before patients do.

We could have used n8n’s built-in logging. It would have given us raw log lines — fine for debugging a single failure, useless for asking “show me every session that burned more than 800 tokens.” Structured traces in Langfuse are what made the difference.

Langfuse gives us:

• Full conversation traces — exactly what context was provided and what the model generated
• Latency monitoring — track response times against the <5 second SLA (our target response time)
• Token usage tracking — cost per interaction (tokens are the chunks of text the model bills by)
• Quality scoring — flag responses that may need human review
• Session replay — review full patient conversations for training and improvement

What we monitor

From our Langfuse dashboard, we track:

• Response accuracy — Are answers factually correct about hours, services, insurance?
• Escalation rate — What percentage of queries require human intervention? (target: <10%)
• Patient satisfaction signals — Follow-up questions that indicate confusion or frustration
• Edge cases — Novel questions that the system hasn’t seen before

Real-world results

After deploying the chatbot across our clinic network:

I want to be honest about those qualifiers. Response time under 5 seconds — verified, we hit it. The 95% accuracy and 10% escalation numbers are targets we’re actively measuring toward, not results we’ve certified. I’d rather say “monitoring” than claim a number we haven’t earned.

The biggest win isn’t in the table. It’s that front desk staff can focus on the patients standing in front of them instead of answering the phone to say “Yes, we’re open until 8 PM.”

Handling edge cases

The system is designed to fail gracefully:

• Unknown questions → Acknowledge the limitation, offer to connect with staff
• Medical advice requests → Firm redirect: “I can’t provide medical advice, but our providers can help when you visit”
• Emotional/urgent situations → Immediate escalation to human staff
• Multi-language queries → Spanish support in development

The medical-advice branch is the one I worried about most. We didn’t try to make the model judge how risky a question was — we hardcoded the refusal. If the cheap classifier buckets a message as medical_advice, it never reaches GPT-4 at all. No model call, no chance of a confident-sounding answer to something it has no business answering.

Lessons learned

1. Context quality matters more than model quality. GPT-4 gives mediocre answers with mediocre context. Feed it accurate, current clinic data and it’s excellent. The model was never the bottleneck; the data we handed it was.

2. Observability is not optional in healthcare AI. Langfuse paid for itself the first week, when we caught the response that listed outdated holiday hours. That single catch was worth the integration effort.

3. Staff trust requires transparency. Showing clinical staff the Langfuse traces — letting them see exactly what the bot says — built confidence faster than any demo. People trust what they can inspect.

4. Start narrow, expand carefully. We launched with just hours/location/insurance queries. Each new domain (wait times, services, booking) was added only after validating the previous one. The temptation is to ship the whole thing at once. Don’t.

What’s next

• Appointment booking integration — Let the chatbot actually schedule visits, not just answer questions about them
• Insurance verification — Pre-check coverage before the patient arrives
• Multi-location support — Scale across our clinic network with location-specific context
• Post-visit surveys — Automated follow-up to capture patient feedback

The takeaway

Building an AI chatbot for healthcare isn’t fundamentally different from building one for any industry — the technology stack is the same. What’s different is the bar for reliability and auditability. In healthcare, a wrong answer about clinic hours is an inconvenience. A wrong answer about services or insurance could send a patient to the wrong place at the wrong time.

The combination of n8n for orchestration, GPT-4 for intelligence, and Langfuse for observability gives us the confidence to deploy AI in a clinical setting while maintaining the transparency that healthcare demands. The model answers the phone. The traces make sure it answers correctly.