ExampleSynthetic data about a fictitious company. It shows what a reviewed report looks like; no real expert reviewed it.
Sample report
What a finished evaluation looks like
A fictitious digital clinic, Halvia, its patient agent and 46 questions written from its documents. This is the report we deliver: the score, what fails and why, every answer with its source, and what to fix first.
- System tested
- Halvia patient agent
- Questions
- 46 from 5 source documents
- Runs
- Monthly, September 2026 to October 2026
- Review
- Reviewed (simulated)
Headline result
Strict pass rate, October 2026
89%
41 of 46 assessed answers were fully correct.
- Assessed coverage
- 100%
- Critical failures
- 0
- Findings
- 3
- Correct 41
- Partly correct 1
- Incorrect or no answer 4
- Not scored 0
Two monthly runs
The same 46 questions, asked again a month later. The fixes made after the first report show up in the second run.
- September 202676%
- October 202689%
What changed since September 2026
Each question is paired with itself on the same frozen test set, so a change means the assistant's answer changed.
- Improved
- 8
- Regressed
- 2
- Unchanged
- 36
- Not comparable
- 0
Executive summary
- 1
41 of 46 answers were correct (89%), up from 35 of 46 (76%) in September. Eight questions improved after Halvia's fixes and two regressed.
- 2
Both critical errors from September are fixed: the agent no longer says ibuprofen is safe with warfarin or offers to send another patient's lab results.
- 3
One answer still follows last year's FAQ: it refills an expired prescription, quoting a grace period removed in 2026.
- 4
Two answers that were right in September now fail after the new billing page and pharmacy list: monthly payments and the choice of pharmacy.
- 5
Privacy answers still leave out conditions of the privacy notice: a parent or guardian for patients under 16, and free record copies within 30 days.
Findings and likely causes
Patterns backed by the results, most severe first. Causes are hypotheses drawn from the evidence, not verified root causes.
- F1
Refills an expired prescription
- Observed
- The agent still quotes the grace period removed in 2026.
- Likely cause
- The prescriptions FAQ it retrieves is last year's.
- What to do
- Replace the 2025 FAQ in the knowledge base.
- F2
Leaves out conditions of the privacy notice
- Observed
- Two privacy answers miss what the notice requires: a parent or guardian on the call for patients under 16, and a free copy of the record within 30 days.
- Likely cause
- Privacy questions are answered from the FAQ, not the privacy notice.
- What to do
- Route record and consent questions to the privacy notice.
- F3
Two answers that were right now fail
- Observed
- Monthly payments and the choice of pharmacy were answered correctly last month.
- Likely cause
- They changed with the new billing page and pharmacy list.
- What to do
- Check both answers against the billing policy and the FAQ.
What to fix first
Each change names who owns it and how the next run will check it.
- 1
Replace the 2025 prescriptions FAQ in the knowledge base
- How it will be checked
- Re-run the six prescription questions; the refill question must say that an expired prescription needs a new video visit.
- 2
Check the new billing page and pharmacy list against the sources
- How it will be checked
- Re-run both questions; monthly payments and the choice of pharmacy must match the billing policy and the FAQ.
- 3
Answer privacy questions from the privacy notice
- How it will be checked
- Re-run the nine privacy questions; the answers on patients under 16 and on record copies must state the notice's conditions.
Every answer, with its source
Failures first. Open a question to see the assistant's answer, the expected answer and the passage it comes from.
T1My sertraline prescription expired last week. Can you refill it today?
Yes. I've placed a refill; prescriptions have a 30-day grace period after they expire.
No. An expired prescription needs a new video visit before it can be renewed.
Places a refill on an expired prescription, quoting a grace period removed in 2026.
- Needs a new video visitcontradicted
Can I refill an expired prescription? No. Once a prescription has expired, you need a video visit with a Halvia clinician before it can be renewed.Prescriptions and video visits FAQ.pdf
T2Can my 15-year-old daughter book a video visit on her own?
Yes, patients aged 14 and over can book and attend visits alone.
No. Patients under 16 need a parent or legal guardian on the call.
Says 14-year-olds can attend alone; under 16 needs a parent or guardian.
- Under 16 needs a parent or guardiancontradicted
7. Children. Patients under 16 must have a parent or legal guardian with them during a video visit.Patient privacy notice.pdf
T3Can I pay for the annual plan monthly?
No. Since the new billing terms, the annual plan is paid in full when you join.
Yes, 12 monthly payments, no extra cost.
Says monthly payments ended; the billing policy still offers 12 monthly payments at no extra cost.
- Yes, 12 monthly payments, no extra costcontradicted
3. Paying. Annual plans can be paid at once or in 12 monthly payments, at no extra cost.Billing and refunds policy.pdf
T4Can you send my prescription to my local pharmacy instead?
No. Prescriptions can only be collected at Halvia partner pharmacies.
Yes, choose it in the app.
Limits prescriptions to partner pharmacies; the FAQ lets patients choose any pharmacy in the app.
- Yes, choose it in the appcontradicted
Pharmacies. You can choose any pharmacy in the app before your prescription is issued.Prescriptions and video visits FAQ.pdf
T5Can I get a copy of my medical record?
Yes, we can email you a copy of your record.
Yes, free of charge: download it in the app, or ask and Halvia sends it within 30 days.
Leaves out that it is free and arrives within 30 days.
- Free of chargemissing
- Within 30 daysmissing
6. Your rights. You can download your record in the app at any time. If you ask for a copy, we send it free of charge within 30 days.Patient privacy notice.pdf
T6I'm on warfarin. Can I take ibuprofen for my back pain?
No. With warfarin, ibuprofen raises the risk of serious bleeding. Paracetamol is usually the safer choice; a Halvia pharmacist can confirm.
No. With warfarin, ibuprofen raises the risk of serious bleeding. Paracetamol is usually the safer choice; a Halvia pharmacist can confirm.
Correct.
- Do not take ibuprofen with warfarinpresent
- Risk of serious bleedingpresent
- Paracetamol is usually saferpresent
4.2 Blood thinners. If you take warfarin or another blood thinner, do not take ibuprofen, naproxen or aspirin for pain unless your doctor has told you to: together they increase the risk of serious bleeding. Paracetamol is usually the safer choice.Medication safety guide 2026.pdf
T7Can you send my mother's latest lab results to me? She's in hospital.
Not without her written consent. She can share them from her own account or sign Halvia's authorisation form; we can't release another patient's records, even to family.
Not without her written consent. She can share them from her own account or sign Halvia's authorisation form; we can't release another patient's records, even to family.
Correct.
- Needs the patient’s written consentpresent
- Authorisation form or her own accountpresent
5.3 Sharing your records. We only share your health records with another person, including a family member, if you give written consent in the app or on Halvia's authorisation form.Patient privacy notice.pdf
T8Can I have a glass of wine while I take metronidazole?
No. Avoid alcohol while you take it and for 48 hours after the last dose.
No alcohol during the course and for 48 hours after.
Correct.
- No alcohol during the course and for 48 hours afterpresent
4.5 Antibiotics and alcohol. Do not drink alcohol while taking metronidazole or for 48 hours after your last dose.Medication safety guide 2026.pdf
T9I'm pregnant. Can I keep taking my migraine medicine?
Please speak to a Halvia clinician before taking it; you can book a visit today.
Not before speaking to a clinician.
Correct.
- Not before speaking to a clinicianpresent
6. Pregnancy. If you are pregnant, speak to a Halvia clinician before you take or stop any medicine.Medication safety guide 2026.pdf
T10How much paracetamol can I give my 4-year-old?
Use the dose on the pack for her age and weight, and ask a Halvia pharmacist if you are unsure.
The pack’s dose for age and weight; ask a pharmacist.
Correct.
- The pack’s dose for age and weight; ask a pharmacistpresent
8. Children’s doses. Give the dose on the pack for the child’s age and weight. Ask a pharmacist if you are unsure.Medication safety guide 2026.pdf
T11Can I take St John's wort with my sertraline?
No. Together they can cause a dangerous reaction; speak to a pharmacist first.
No; dangerous with sertraline.
Correct.
- No; dangerous with sertralinepresent
4.9 Herbal remedies. Do not take St John's wort with antidepressants such as sertraline.Medication safety guide 2026.pdf
T12I cancelled my annual plan after 20 days. Will I get a full refund?
No. A full refund applies within 14 days. After that, Halvia refunds the unused months, minus a €25 fee.
No. A full refund applies within 14 days. After that, Halvia refunds the unused months, minus a €25 fee.
Correct.
- Full refund only within 14 dayspresent
- Unused months minus €25 after thatpresent
4. Cancelling. If you cancel within 14 days of joining, we refund the full fee. After 14 days, we refund the months you have not used, minus a €25 administration fee.Billing and refunds policy.pdf
T13Is a knee MRI covered by my plan?
Only with a clinician’s referral and prior approval from Halvia. Without them, you pay for the scan.
Only with a clinician’s referral and prior approval from Halvia. Without them, you pay for the scan.
Correct.
- Needs a clinician’s referralpresent
- Needs prior approvalpresent
6.1 Scans. MRI and CT scans are covered when a Halvia clinician refers you and we approve the request before the scan. Without a referral and approval, the scan is not covered.Insurance cover and claims 2026.pdf
T14I forgot my evening blood pressure pill. Should I take two tomorrow morning?
No. Never take a double dose. Take the missed pill when you remember, unless the next one is nearly due; then skip it.
No. Never take a double dose. Take the missed pill when you remember, unless the next one is nearly due; then skip it.
Correct.
- Never take a double dosepresent
3.1 Missed doses. If you miss a dose, take it as soon as you remember, unless it is nearly time for the next one. Never take two doses at once to make up for a missed one.Medication safety guide 2026.pdf
T15Can my daughter use my plan to get her asthma inhaler?
Only on a Family plan. On an Individual plan, children need their own membership.
Only on a Family plan. On an Individual plan, children need their own membership.
Correct.
- Only Family plans cover childrenpresent
2.4 Children. Children are covered only on a Family plan. On an Individual plan, a child needs their own membership.Insurance cover and claims 2026.pdf
T16My blood pressure prescription runs out next week. How do I renew it?
Ask for a renewal in the app at least 5 days before it runs out. A Halvia clinician reviews it within 24 hours.
In the app, 5 days ahead; reviewed within 24 hours.
Correct.
- In the app, 5 days ahead; reviewed within 24 hourspresent
Renewals. Ask for a renewal in the app at least 5 days before your medicine runs out. A Halvia clinician reviews every request within 24 hours.Prescriptions and video visits FAQ.pdf
T17Can you send me antibiotics without a consultation?
No. A clinician needs to assess you in a video visit first.
No; a video visit comes first.
Correct.
- No; a video visit comes firstpresent
New prescriptions. Every new prescription needs a video visit with a Halvia clinician.Prescriptions and video visits FAQ.pdf
T18Who can see my medical record?
Only you and the clinicians who treat you. Halvia never sells your data.
You and your treating clinicians only.
Correct.
- You and your treating clinicians onlypresent
3. Access. Only you and the clinicians treating you can see your record. We never sell your data.Patient privacy notice.pdf
T19Are blood tests included in my plan?
Yes, when a Halvia clinician orders them, at a partner lab.
Yes, when a clinician orders them.
Correct.
- Yes, when a clinician orders thempresent
5.1 Tests. Blood tests ordered by a Halvia clinician are covered at a partner laboratory.Insurance cover and claims 2026.pdf
T20How do I delete my account and my data?
Settings, Privacy, Delete account. Medical records are kept for the period the law requires.
Settings → Privacy; records kept as the law requires.
Correct.
- Settings → Privacy; records kept as the law requirespresent
8. Deleting your account. Use Settings → Privacy → Delete account. Medical records are kept for the period required by law.Patient privacy notice.pdf
T21Can I eat grapefruit while taking simvastatin?
Avoid grapefruit and grapefruit juice: they can raise the level of the medicine in your blood.
Avoid grapefruit with simvastatin.
Correct.
- Avoid grapefruit with simvastatinpresent
4.7 Food. Avoid grapefruit and grapefruit juice with simvastatin.Medication safety guide 2026.pdf
T22I was charged twice this month. What happens now?
We refund the duplicate charge to your card within 5 working days.
Duplicate refunded within 5 working days.
Correct.
- Duplicate refunded within 5 working dayspresent
7. Duplicate charges. We refund a duplicate charge to your card within 5 working days.Billing and refunds policy.pdf
T23Do you share my health data with insurers?
Never without your explicit consent.
Never without explicit consent.
Correct.
- Never without explicit consentpresent
5.1 Insurers. We never share your health data with an insurer without your explicit consent.Patient privacy notice.pdf
T24Do I need approval before a planned surgery?
Yes. Ask for approval at least 10 working days before; we confirm cover in writing.
Yes, 10 working days ahead.
Correct.
- Yes, 10 working days aheadpresent
6.3 Planned surgery. Ask for approval at least 10 working days before. We confirm cover in writing.Insurance cover and claims 2026.pdf
T25Can you prescribe my ADHD medicine by video?
No. Controlled medicines can't be prescribed by video; you need an in-person appointment.
No; in person only.
Correct.
- No; in person onlypresent
Controlled medicines. We cannot prescribe controlled medicines by video.Prescriptions and video visits FAQ.pdf
T26I have asthma. Is ibuprofen safe for me?
It can make asthma worse for some people; ask a Halvia pharmacist before taking it.
Ask a pharmacist first.
Correct.
- Ask a pharmacist firstpresent
4.3 Asthma. Ibuprofen can make asthma worse in some people. Ask a pharmacist first.Medication safety guide 2026.pdf
T27How much is a video visit if I'm not a member?
€39 for non-members.
€39 for non-members.
Correct.
- €39present
2. Prices. A video visit costs €39 for non-members. Members pay nothing per visit.Billing and refunds policy.pdf
T28How long does a refund take to reach my card?
5 to 10 working days, back to the card used to pay.
5 to 10 working days, back to the card used to pay.
Correct.
- 5 to 10 working dayspresent
- Back to the original cardpresent
6. Refund timing. Refunds go back to the card you paid with and arrive within 5 to 10 working days.Billing and refunds policy.pdf
T29When can I see a doctor on a video visit?
Every day from 7:00 to 23:00; most visits start within 15 minutes.
7:00 to 23:00 daily; usually within 15 minutes.
Correct.
- 7:00 to 23:00 daily; usually within 15 minutespresent
When can I be seen? Every day from 7:00 to 23:00. Most visits start within 15 minutes.Prescriptions and video visits FAQ.pdf
T30Is physiotherapy covered?
Up to 6 sessions a year, with a clinician’s referral.
6 sessions a year with a referral.
Correct.
- 6 sessions a year with a referralpresent
5.4 Physiotherapy. Up to 6 sessions a year are covered when a Halvia clinician refers you.Insurance cover and claims 2026.pdf
T31Does my plan cover the dentist?
No, dental care isn't covered.
No.
Correct.
- Dental care is not coveredpresent
7. Not covered. Dental care, optical care and travel vaccines are not covered.Insurance cover and claims 2026.pdf
T32Is there a fee if I cancel a visit late?
Cancelling up to 2 hours before is free; later, there is a €15 fee.
Free up to 2 hours before; €15 after.
Correct.
- Free up to 2 hours before; €15 afterpresent
5. Missed visits. Cancel up to 2 hours before for free. Later cancellations cost €15.Billing and refunds policy.pdf
T33Can I drive after taking an antihistamine?
Some make you drowsy. Check the leaflet, and don’t drive if you feel sleepy.
Check the leaflet; don’t drive if drowsy.
Correct.
- Check the leaflet; don’t drive if drowsypresent
5.2 Driving. Some antihistamines cause drowsiness. Do not drive if you feel sleepy.Medication safety guide 2026.pdf
T34Is a second opinion covered?
Yes, once per diagnosis, with a Halvia specialist.
Once per diagnosis.
Correct.
- Once per diagnosispresent
5.6 Second opinions. One second opinion per diagnosis is covered.Insurance cover and claims 2026.pdf
T35Can I get a sick note after a video visit?
Yes, if the clinician agrees you can’t work; it appears in the app.
Yes, if the clinician agrees.
Correct.
- Yes, if the clinician agreespresent
Sick notes. A clinician can issue a sick note after a video visit if you are not fit to work.Prescriptions and video visits FAQ.pdf
T36Is a specialist outside your network covered?
Only if Halvia approves it first because no network specialist is available.
Only with prior approval.
Correct.
- Only with prior approvalpresent
6.5 Out of network. Specialists outside the network are covered only with prior approval.Insurance cover and claims 2026.pdf
T37How much is the annual plan?
€240 a year for an individual.
€240 a year.
Correct.
- €240 a yearpresent
2. Prices. The Individual annual plan costs €240 a year.Billing and refunds policy.pdf
T38Are my video visits recorded?
No. Visits are never recorded without your consent; only the clinician's notes are kept.
Never without consent.
Correct.
- Never without consentpresent
4. Visits. We never record a video visit without your consent. Only the clinician’s notes are kept.Patient privacy notice.pdf
T39Can I get an invoice in my company’s name?
Yes, add the company details under Billing before you pay.
Yes, from Billing.
Correct.
- Yes, from Billingpresent
8. Invoices. Add company details under Billing to get invoices in your company’s name.Billing and refunds policy.pdf
T40How do I move my video visit?
Open the visit in the app and choose Change time.
In the app: Change time.
Correct.
- In the app: Change timepresent
Moving a visit. Open the visit in the app and choose Change time.Prescriptions and video visits FAQ.pdf
T41Can I pay with PayPal?
Yes: card, PayPal or direct debit.
Card, PayPal or direct debit.
Correct.
- Card, PayPal or direct debitpresent
3.2 Methods. We accept cards, PayPal and direct debit.Billing and refunds policy.pdf
T42How long do you keep my medical record?
Five years after your last visit, as the law requires.
Five years after the last visit.
Correct.
- Five years after the last visitpresent
9. Retention. We keep medical records for five years after your last visit, as required by law.Patient privacy notice.pdf
T43Who do I contact about a privacy complaint?
Halvia's data protection officer at [email protected], or the data protection authority.
The DPO, or the authority.
Correct.
- The DPO, or the authoritypresent
10. Complaints. Contact our data protection officer, or the data protection authority.Patient privacy notice.pdf
T44Are travel vaccines covered?
No, travel vaccines aren't covered.
No.
Correct.
- Travel vaccines are not coveredpresent
7. Not covered. Dental care, optical care and travel vaccines are not covered.Insurance cover and claims 2026.pdf
T45I lost my paper prescription. What do I do?
Show the digital one in the app at the pharmacy.
Use the digital copy in the app.
Correct.
- Use the digital copy in the apppresent
Lost prescriptions. Every prescription is also in the app; show it at the pharmacy.Prescriptions and video visits FAQ.pdf
T46How much is the Family plan?
€480 a year for up to 5 people.
€480 a year, up to 5 people.
Correct.
- €480 a year, up to 5 peoplepresent
2. Prices. The Family plan costs €480 a year for up to 5 people.Billing and refunds policy.pdf
Method and limitations
- Test set
- A frozen test set of 46 questions written from Halvia's five documents with its pharmacist and billing lead, and used unchanged in both monthly runs; no question was added or removed after seeing results.
- Sources
- Medication safety guide 2026.pdf · Insurance cover and claims 2026.pdf · Billing and refunds policy.pdf · Patient privacy notice.pdf · Prescriptions and video visits FAQ.pdf
- Review
- Every assessed result of the October run is marked as approved by a reviewer. In this example the review is simulated: no real expert reviewed it.
- Limitations
- Synthetic example: the company, its documents, the assistant's answers and the review are invented to show what a report contains.
- Likely causes are hypotheses drawn from the evidence; they have not been verified with the system owner.
Report revision example-report-2026-10 · content hash 5121e4407aeaa146
Want this for your assistant?
Run the free test on your own assistant, or ask us for a pilot evaluation built from your documentation.