About Iatronix

Quantity vs. quality is a harder trade-off than it looks

Fetching more sources always sounds better on paper. In practice, a noisy PubMed result set with 20 weakly-relevant abstracts produces worse LLM output than 5 high-quality ones. We built evidence scoring precisely because raw retrieval count is a bad proxy for answer quality. More data causes the model to hedge, bury the key point, or invent a consensus that doesn't exist in the sources.

Medical research is behind paywalls — and that matters

Most impactful RCTs and meta-analyses are published in journals that don't offer open access. PubMed gives titles and abstracts; the actual trial data is paywalled. Unpaywall helps for open-access articles, but institutional guideline PDFs — NICE, ACC/AHA, ESC — are not consistently machine-readable. This means a query about a rare disease or a recent trial update will frequently hit the evidence quality floor and return a DegradedResponse, not because the answer doesn't exist, but because it exists behind a paywall.

LLMs are good editors, not good researchers

The pipeline treats the LLM purely as a formatter. Give it structured evidence and a schema to fill, and it produces clean, graded, citable output. Ask it to 'find information about X' without grounded sources and it will confabulate confidently. The fail-closed evidence gate exists because we learned early that the model will fill gaps with plausible-sounding but unsourced content if you let it.

Not every question is a clinical question

Early on the pipeline would dutifully search PubMed for anything typed into it, including questions with no drug, disease, symptom, or procedure in them. That wastes retrieval and invites confidently-worded answers to questions the literature can't speak to. A scope guard now runs first: if no medical entities are extracted, the app declines honestly and never searches — a plain 'this isn't a clinical question' is safer and more useful than a padded non-answer.

A feature can be live and still do nothing

We shipped a new retrieval fallback meant to rescue thin result sets — code merged, unit tests green, feature 'live'. It also did nothing. It sat behind an early return with such a low trigger bar that the escalation path it guarded almost never ran on the queries it was built for. The helper worked perfectly in isolation; the feature never fired in practice. The lesson: test that a feature actually triggers on its target case, not just that its internals return the right value when you call them directly.

One run can lie; three runs tell the truth

Live PubMed does not return identical results run to run — ranking and availability shift with load. A single before/after comparison once showed a dramatic 'rescue': the new path looked like it doubled the evidence found. Re-running it, the effect was network noise, not the change. Taking the median of three runs collapsed the improvement to roughly nothing. The lesson: any metric measured against a noisy external system has to be sampled more than once before a delta is trustworthy.

The right amount of evidence is a dial, not a maximum

It is tempting to treat retrieval as 'more is better' and keep widening the net. We raised a relevance threshold to pull in more articles and answer quality got worse, not better — the extra results were mostly off-topic and dragged the model toward hedging and false consensus. A tighter setting produced cleaner, more decisive answers. The lesson: retrieval breadth is a precision/recall trade-off to tune for the task, not a number to maximise.

A size budget is not a count cap

One clinical search returned over 4,000 citations and crashed the response. The citation list was built from every fetched article with no count limit, and a broad query had pulled in thousands of unique papers (PubMed + Semantic Scholar + textbooks). The internal cap only measured characters, so title-only entries slipped through uncounted. The fix caps the list by count — every source the answer actually cites is kept, and the retrieved-but-uncited extras are limited. The lesson: bound what the user sees by count, not just by size.

Relevance needs a floor, not just a ranking

Asked 'does paracetamol cause fever', the app once surfaced an unrelated post-operative arthroplasty fever guideline — it matched the word 'fever' and scored well on study quality, and nothing dropped it, because ranking only reorders. The fix adds a relevance floor that drops articles which never mention the drug, plus adverse-effect query framing so a causation question retrieves 'drug-induced' evidence rather than 'drug used to treat' evidence. Chosen by testing every combination and measuring off-topic rate: together they cut off-topic results sharply and kept more of the right ones.