What I asked it
The demo on the home page only offers questions it answered right every time. These are all of them, including the ones it got wrong: 4 right, 4 partly right, 2 wrong.
OpenIntelligence 5.5 (build 483) built for the Simulator with one change: the four switches that keep Apple's model off in the Simulator are flipped, so it runs there. iPhone 18 Pro Max Simulator on iOS 27.0, on my MacBook Pro (M3 Pro), answering with Apple's on-device model. Every document is a made-up sample. Each row is one question, asked in a new chat each time and scored against the document by reading it.
| Date | Question | Library | Result | What happened |
|---|---|---|---|---|
| 2026-10-01 | When can my kid go back to school after a fever? | Home (10 sample documents) | Right | 24 hours fever-free without fever-reducing medicine, from the school handbook, cited. |
| 2026-10-01 | When does my flight to Lisbon leave? | Home (10 sample documents) | Partly right | Right day, time and airport (Fri, Oct 9, 6:50 p.m. from Boston), but it left out the flight number the itinerary gives. |
| 2026-10-01 | How long does the banana bread bake, and at what temperature? | Home (10 sample documents) | Right | 350°F for 60 to 65 minutes, from the recipe, cited. |
| 2026-10-02 | A pipe burst in my kitchen. Am I covered? | Home (10 sample documents) | Right | Right in all 6 runs, each in a new chat, citing the policy's notes on water damage. 4 gave the two sentences on the home page, word for word or nearly, and 2 added a third, correct line on which perils carry their own deductibles. The first 3 ran while my Mac was under heavy load and took 15 s to 1 min 26 s by the app's timer, and the next 3 took 9.7 to 13.5 s. |
| 2026-10-02 | How do I clean the air fryer basket? | Home (10 sample documents) | Right | Right in all 4 runs, each in a new chat, every line cited. The 3 for the home page gave the same three lines word for word: warm, soapy water (the manual's first-use step), dishwasher safe on the top rack only, and no metal scouring pads, in 8.9 to 11.0 s by the app's timer. The first run also listed wiping the base and cleaning the drawer. None said to unplug it and let it cool for 30 minutes first, the manual's first cleaning step. |
| 2026-10-02 | A pipe burst in my kitchen. Am I covered, and what's my deductible? | Home (10 sample documents) | Partly right | Coverage right: a burst pipe is covered under Coverages A and C, and a leak that went on for weeks isn't. The deductible part listed the windstorm and water backup deductibles instead of the $1,000 all-perils one. |
| 2026-10-02 | My air fryer is showing E4. What do I do? | Home (10 sample documents) | Partly right | Right that the air outlet must be clear with 5 inches of space and to contact support if it comes back, but it skipped unplugging it and letting it cool for 30 minutes, and it split the support email address in two. |
| 2026-10-02 | What's the return policy on my headphones? | Home (10 sample documents) | Partly right | Right about the original packaging, the receipt and the 30 days to cancel the protection plan, but it gave the purchase date, 09/12/2026, as the last day to return them. |
| 2026-10-02 | When's the last day I can return my headphones? | Home (10 sample documents) | Wrong | It gave the purchase date, 09/12/2026, instead of 30 days after it. Its own Fact Check marked the claim partial at 52%. |
| 2026-10-02 | How long do frozen fries take in my air fryer? | Home (10 sample documents) | Wrong | The search didn't reach the manual's cooking chart, so it said the documents don't cover it and cited the school and travel documents instead. |