What I asked it
The demo on the home page shows a question it answered right each time I asked. Below are that run and every question I asked it in the iPhone Simulator for the demo, including the ones it got wrong: 7 right, 7 partly right, 4 wrong.
OpenIntelligence 5.5 (build 483) built for the Simulator with one change: the four switches that keep Apple's model off in the Simulator are flipped, so it runs there. iPhone 18 Pro Max Simulator on iOS 27.0, on my MacBook Pro (M3 Pro), answering with Apple's on-device model. Every document is a made-up sample. Each row is one question, asked in a new chat each time and scored against the document by reading it. The September 25 row was asked in the Mac app, version 5.4, with the same ten documents. The October 8 rows, the home page's among them, are 5.6, built the same way with one more change: in the Simulator the embedding model runs on the Mac's CPU, so the search by meaning is real there.
| Date | Question | Library | Result | What happened |
|---|---|---|---|---|
| 2026-09-25 | What does Brightwater match on my 401(k)? | Home (10 sample documents), in the Mac app, 5.4 | Right | Right in both runs that morning, each in a new chat: 100% of the first 4% of pay, half of the next 2%, 5% in all, and the match vests after 3 years. Fact Check marked every claim Supported both times. The second run was the one on the home page from October 6 to October 9. |
| 2026-10-01 | When can my kid go back to school after a fever? | Home (10 sample documents) | Right | 24 hours fever-free without fever-reducing medicine, from the school handbook, cited. |
| 2026-10-01 | When does my flight to Lisbon leave? | Home (10 sample documents) | Partly right | Right day, time and airport (Fri, Oct 9, 6:50 p.m. from Boston), but it left out the flight number the itinerary gives. |
| 2026-10-01 | How long does the banana bread bake, and at what temperature? | Home (10 sample documents) | Right | 350°F for 60 to 65 minutes, from the recipe, cited. |
| 2026-10-02 | A pipe burst in my kitchen. Am I covered, and what's my deductible? | Home (10 sample documents) | Partly right | Coverage right: a burst pipe is covered under Coverages A and C, and a leak that went on for weeks isn't. The deductible part listed the windstorm and water backup deductibles instead of the $1,000 all-perils one. |
| 2026-10-02 | My air fryer is showing E4. What do I do? | Home (10 sample documents) | Partly right | Right that the air outlet must be clear with 5 inches of space and to contact support if it comes back, but it skipped unplugging it and letting it cool for 30 minutes, and it split the support email address in two. |
| 2026-10-02 | What's the return policy on my headphones? | Home (10 sample documents) | Partly right | Right about the original packaging, the receipt and the 30 days to cancel the protection plan, but it gave the purchase date, 09/12/2026, as the last day to return them. |
| 2026-10-02 | When's the last day I can return my headphones? | Home (10 sample documents) | Wrong | It gave the purchase date, 09/12/2026, instead of 30 days after it. Its own Fact Check marked the claim partial at 52%. |
| 2026-10-02 | How long do frozen fries take in my air fryer? | Home (10 sample documents) | Wrong | The search didn't reach the manual's cooking chart, so it said the documents don't cover it and cited the school and travel documents instead. |
| 2026-10-02 | A pipe burst in my kitchen. Am I covered? | Home (10 sample documents) | Right | Right in all 6 runs, each in a new chat, citing the policy's notes on water damage. 4 gave the same two sentences, word for word or nearly, and 2 added a third, correct line on which perils carry their own deductibles. The first 3 ran while my Mac was under heavy load and took 15 s to 1 min 26 s by the app's timer, and the next 3 took 9.7 to 13.5 s. |
| 2026-10-02 | How do I clean the air fryer basket? | Home (10 sample documents) | Right | Right in all 4 runs, each in a new chat, every line cited. 3 of them gave the same three lines word for word: warm, soapy water (the manual's first-use step), dishwasher safe on the top rack only, and no metal scouring pads, in 8.9 to 11.0 s by the app's timer. The first run also listed wiping the base and cleaning the drawer. None said to unplug it and let it cool for 30 minutes first, the manual's first cleaning step. |
| 2026-10-08 | When is my rent late, and what's the fee? | Home (10 sample documents), iPhone Simulator, 5.6 | Right | Right in all 3 runs that day, each in a new chat: late after 11:59 p.m. on the 5th, a $75.00 late charge, capped at $150.00 a month, and $35.00 for a returned payment. Fact Check marked every claim Supported each time, and the app's timer read 12.3, 19.1 and 15.2 s. None of the three mentioned the $10.00 a day the lease adds after the 6th. The third run is the one on the home page. |
| 2026-10-08 | Which parking space is mine? | Home (10 sample documents), 5.6 | Partly right | Space #17, right, but two of its sentences stop short: a car in someone else's space "may be towed at the owner's", and guest parking is "from 6:00" where the lease says 6:00 p.m. to 8:00 a.m. Its own Fact Check marked the first claim partial at 48%. |
| 2026-10-08 | When are quiet hours? | Home (10 sample documents), 5.6 | Wrong | The lease says 10:00 p.m. to 7:00 a.m. every day. The answer said quiet hours vary by day, quoted the section heading twice with two different end times, and went on to grills and guest parking. |
| 2026-10-08 | How much notice do I have to give before I move out? | Home (10 sample documents), 5.6 | Partly right | It gave the 30 days for a month-to-month tenancy and the 30 days with a fee for ending early, both in the lease, and left out the first rule: 60 days before the end of the term. Fact Check marked both claims Supported. |
| 2026-10-08 | What's my home insurance deductible? | Home (10 sample documents), 5.6 | Wrong | It described what the all-perils deductible applies to and never gave the amount, $1,000. |
| 2026-10-08 | When can my kid go back to school after a fever? | Home (10 sample documents), 5.6 | Partly right | 24 hours fever-free without fever-reducing medicine, right and cited, then a stray line about the price of school lunch that stops in the middle. |
| 2026-10-08 | When does my flight to Lisbon leave? | Home (10 sample documents), 5.6 | Right | Fri, Oct 9 at 6:50 p.m. from Boston on Coastline Airways CW 412, arriving Oct 10, cited. Fact Check marked all three claims Supported, and the app's own badge still read Unverified. |
How it found this
When is my rent late, and what's the fee?
- Searched by meaning and keywordHybrid search, picked by the app, over the library's 117 passages
- Wrote it with Apple's on-device model58 tokens at 9 a second, 15.2 s by the app's timer
- Checked each claim against the sources3 of 3 supported, 73% to 79%
Fact check3 verified
- Rent is considered late if it is received after…
- Monthly rent is $1,675.00, due on the 1st da…
- A returned payment costs $35.00 .
From the source
Rent received after 11:59 p.m. on the 5th day of the month is late. A late charge of $75.00 applies on the 6th, plus $10.00 per day after that, with total late charges capped at $150.00 for any one month.
- Library
- Home: 10 made-up sample documents, split into 117 passages and indexed in the app.
- Embeddings
- all-MiniLM-L6-v2, 384 dimensions with mean pooling. In the Simulator it runs on my Mac's CPU.
- Search
- Hybrid: vector search by meaning and BM25 by keyword, merged with reciprocal rank fusion. The app chose it for this question by itself.
- Answer
- Apple's on-device model, through the Foundation Models framework. 58 tokens at 9 tokens a second, and 15.2 s on the app's own timer.
- Fact Check
- The app split its answer into three claims and checked each against the passages it retrieved. All three came back Supported, at 73%, 79% and 79%, with three sources each. The rows above are as the app printed them, cut short where it cut them.
- The source
- The recording doesn't open a claim. The quote above is from section 5 of the lease as the document has it, the same two lines the recording shows beside the answer. The rent is section 4, and the returned-payment fee is the next line of section 5. The answer leaves out the $10.00 a day after the 6th.
- Recorded
- OpenIntelligence 5.6, built for the iPhone 18 Pro Max Simulator on my MacBook Pro (M3 Pro), October 8, 2026. The App Store build keeps Apple's model off in the Simulator, so this build turns it on there and runs the embedding model on the CPU. The 16 s wait plays at 8.1x, the typing and the end of the answer at about 1.3x, and the test script's pauses are cut.