RETRIEVAL_PIPELINE.mdnarrative source-verified at v4.6; claims re-checked 2026-09-01View source

Retrieval Pipeline — narrative source-verified at v4.6; claims re-checked 2026-09-01

Documentation status: Sections 1–5 source-verified 2026-07-15 against v4.6 and not re-narrated since; 4.8–4.9 changed retrieval behaviour in ways they do not describe. The numbered items 12–22 and the Suggested questions section are individually dated and evidence-tagged and are the current record.

Corrected 2026-10-20 against the working tree (b868fc0 plus the 5.7 work in progress): the flow diagram in section 2 (Standard’s query analysis, where the similarity floor sits, the λ its MMR step uses, the two packing paths and the order of step-by-step passages, what a failed gate check does and which gates run, with a note on how Deep Think and Maximum differ), stage 3’s chunk sizes and cutting rule, stage 4’s two keyword-index writes, its page rows and its flat vector file, stage 7’s “compressed” and its overlap skip, stage 8’s “per query”, its per-call routing in Deep Think and Maximum and the question to an empty library that has no route plan, item 13’s list of what goes through sentence selection, item 14’s account of Needle Rescue and item 15’s note on it, item 18’s note on which synthesis it describes and which passages it is given, item 22’s places a locked value is used, the known-drift line that named executeReasoningChain and the callers its tag names, the section on an extracted value (the function its five places share, and in its 2026-10-20 note the count of places that follow the model’s answer and what the other three share), the answer mode’s λ said to be read by logging, the re-pack after the route plan (“Retrieval’s best passage reaches the model”), the condition on the measured iPhone budget and the limit on candidates, and the minimum Deep Think and Maximum pass to each search. A claim that rests on a text search of the code (“no caller”, “the only one”) is tagged grep_verified and names the search. Each correction is marked where it is made. Line numbers are from the working tree as read at 21:41 that day and at about 23:00 (the files read at both times had not changed), and they move as the 5.7 work goes on.

Shipped versions: read Docs/SHIPPED_VERSION.json, the per-platform record. This document deliberately states no current version number, because each one it stated went stale: before 2026-09-01 this block said “iOS/macOS 4.9 is the shipped version” while the title said v5.0, and the correction made that day (iOS 5.0, macOS 5.0.2) has since been superseded too. The 2026-09-01 note that PCC device/distribution validation was pending predates PCC shipping; Docs/SHIPPED_CAPABILITIES.json records private_cloud_compute as shipping since 2026-09-10.

Claims re-checked against source 2026-09-01, all holding: the seven RetrievalTraceCollector stages and their order; executeFullRetrievalPipeline taking a defaulted trace parameter (item 19); the collector defaulting to nil everywhere, so production pays a nil check; the Core ML TinyBERT cross-encoder with heuristic fallback in the flow diagram, which is loaded at RAGEngine.swift:82 and invoked at :354; and every referenced document existing.

The benchmark directories cited in items 12 and 19 — BenchmarkRuns/20260811-133233-matrix and BenchmarkRuns/paired-retry — are archived, not deleted. All 90 run directories are in ~/OpenIntelligence-BenchmarkArchive/BenchmarkRuns-2026-09-01.tar.gz, verified byte-identical before removal. Extract to read them; see BenchmarkRuns/LEDGER.md. Known drift as of 2026-08-05 — each of these is in the changelog under 4.9 (moved on 2026-09-24 from CHANGELOG.md to Docs/Archive/CHANGELOG_2.0_to_5.2.md) but not yet reflected below:

  • Gate E measures query coverage as the better of combined-evidence and best-single-chunk, not the per-chunk maximum.
  • DomainIsolationService relaxes only the cross-domain-mixing clause when the retrieved chunks all come from one documentId.
  • DomainIsolationService (5.7, 2026-10-15) matches its keywords as whole words, has no two-letter abbreviations, and engages only when the question scores two or more of them; it had failed right answers about everyday documents by matching “um” inside “number” and “ip” inside “trip”.
  • The Deep Think and Maximum front half (AgenticOrchestrator.executeHoldingAnswerStream) searches the question as the user wrote it when its search phrasings (from the query planner, or from one on-device model call with fixed phrasings as the fallback) fuse zero passages and none of them was that wording, before the relevance gate judges the library. (Corrected 2026-10-20: this line named executeReasoningChain, which starts after that gate and has no such retry.) [evidence_level: code_verified, confidence: exact, evidence_source: AgenticOrchestrator.swift:126, AgenticOrchestrator.swift:180, AgenticOrchestrator.swift:204-213, AgenticOrchestrator.swift:237-295 (the relevance gate), AgenticOrchestrator+Retrieval.swift:785-859 (generateSearchQueries), AgenticOrchestrator+ReasoningChain.swift:180 (executeReasoningChain)] [evidence_level: grep_verified, confidence: high, evidence_source: grep -n "Retrying with" AgenticOrchestrator.swift (one hit, AgenticOrchestrator.swift:206) and grep -rn "executeReasoningChain(" OpenIntelligence OpenIntelligenceTests --include='*.swift', 2026-10-20: four calls, at :480 and :480 in the front half, at RAGService+Agentic.swift:77 in executeStandardReasoningChain, Standard's chain, which runs only with the developer switch forceReasoningChain (RAGService+QueryAnswer.swift:659-667, called at RAGService+QueryPipeline.swift:655), and at RAGService+Import.swift:1135 in executeClusterChain, reached only from executeMultiChainReasoning (:1583), which has no caller] (Removed 2026-11-06: executeClusterChain and executeMultiChainReasoning are deleted, so the fourth call is gone; three remain, at AgenticOrchestrator.swift:481 and :719 in the front half and RAGService+Agentic.swift:77. [evidence_level: grep_verified, confidence: high, evidence_source: 2026-11-06, grep -rn "executeReasoningChain(" OpenIntelligence OpenIntelligenceTests --include='*.swift' (three callers plus the definition at AgenticOrchestrator+ReasoningChain.swift:180), the deleted call was AgenticOrchestrator+MultiChain.swift@b914298b:406])
  • (Added 2026-11-04.) The planner is reused: QueryExecutionPlannerService.buildPlan keeps a plan for the same question, quality mode and tool setting for 120 seconds (recentPlans, planReuseSeconds) and lets a second call wait for one in flight, so the mode choice, decideIfRetrievalNeeded and generateSearchQueries share one draw. decideIfRetrievalNeeded makes no model call of its own, but the plan it asks for is a guided on-device model call (a heuristic plan is the fallback), unless one is reused. [evidence_level: code_verified, confidence: exact, evidence_source: QueryExecutionPlannerService.swift:71-101, :104-136; AgenticOrchestrator+SelfRAG.swift:200-218, AgenticOrchestrator+Retrieval.swift:793-798]
  • makePostRetrievalModelPlan takes the maximum similarity over the chunk set rather than reading chunks.first, which is not the best chunk after MMR, expansion, and Lost-in-the-Middle reordering. Source of truth: the code. The codebase audit this line used to cite in Docs/AUDIT/ is not in the repository (Docs/AUDIT/ is gitignored). For anything retrieval-related in 4.8–4.9, read those sections of Docs/Archive/CHANGELOG_2.0_to_5.2.md, where they moved from CHANGELOG.md on 2026-09-24. Scope: Describes shipped behavior unless explicitly labeled experimental, developer-only, or scaffolded.

Large libraries, changed 2026-10-10 for 5.7. Three steps described in this document used to walk every passage of the active library: the query-expansion vocabulary (CorpusVocabulary.build, a part-of-speech tagger per passage, built for the first question after a launch or an import), the suggested-question sampler (RAGService.getSampleChunks) and the cross-reference graph (a regular expression compiled per pattern per passage). Above 4,000 passages the first two now work from an even sample of the library (the sampler also keeps every document summary), and the graph’s patterns are compiled once. A vocabulary built from a sample can miss a term that appears in few passages; that is the trade. Later the same day: the hybrid search joins keyword hits through a lookup of only the keys a pass needs (HybridSearchService.buildNeededChunkLookup), and the extractive “spec” scan scores an even sample of 20,000 passages above that size (RAGService.passagesToScanForSpecSniper). [evidence_level: test_verified for the sampling rules; code_verified; not timed on a device, confidence: medium, evidence_source: LargeLibraryCostTests, QueryEnhancementService.swift, RAGService.swift, GraphIndexService.swift]

Where passages come from, changed 2026-10-12 for 5.7. The vector store no longer holds every passage’s text after it opens a library. A vector search scores every vector as before and decodes only the passages that come out on top. Two lookups in hybrid search that used to load every passage when none were handed in now ask the store’s records which passages they need and read those by id: the search inside named documents (searchInsideScope) and the lookup that turns keyword hits into passages (HybridSearchService.neededChunkPositions, which picks the same passages buildNeededChunkLookup picked from the full array). The question path itself still loads every passage once, through readableChunks(), because its summary, enumeration, parent-expansion, cross-reference, extractive-scan and multi-hop steps read all of the text, so what a question retrieves is unchanged. A passage the store cannot read is left out of that array and of every search. [evidence_level: test_verified for the two lookups (VectorStorePassageIndexTests), code_verified for the rest, confidence: high, evidence_source: HybridSearchService.swift, BNNSVectorDatabase.swift, RAGService.swift; CHANGELOG.md 5.7 Changed]

A question limited to named documents is ranked by vector as well as by words, corrected 2026-10-13 for 5.7. searchInsideScope took each passage’s similarity from the vector on the passage. The vector store hands passages back without their vectors, so every similarity was 0 and the scoped ranking was by the question’s words alone. It now asks the store how close each passage is (similarities(to:forChunkIDs:), worked out where the vectors are kept); a store that hands passages back with their vectors is ranked from them as before. Up to 6,000 passages in the named documents, every passage is scored both ways. Past that (one document of 144,247 passages exists) every vector is scored and words are compared for the 2,000 closest passages and for the keyword index’s best 400 hits inside the scope, whether the passages were handed in or are read by id; with no vector to score, every passage is read, and a question whose own embedding failed counts as that. The keyword hits come from the index’s chunk search, which is library-wide and matches every word before it matches any, so a passage outside the scope that holds every word leaves the scope without keyword candidates. The log lines [Hybrid] Document scope: ... say which happened and how many passages were compared by vector. [evidence_level: test_verified (HybridSearchServiceTests, nine tests), not run on a device, confidence: high, evidence_source: HybridSearchService.swift searchInsideScope; BNNSVectorDatabase.swift similarities]

Parent expansion, changed 2026-10-13 for 5.7. ParentDocumentService.expandWithSiblings gathers only the passages of the documents its retrieved passages come from and sorts each document once. It used to index and group every passage of the library for each question. The siblings it adds are the same. [evidence_level: test_verified (ParentDocumentServiceTests), confidence: high, evidence_source: ParentDocumentService.swift]

The first question’s vocabulary is built off the main actor and kept, changed 2026-10-13 for 5.7. CorpusVocabulary.build ran for the first question after every launch, as a plain call on the main actor (2.6 to 2.7 s on the 144,247-passage benchmark store, from its 4,000-passage sample). It now runs on the concurrent pool, and the vocabulary is kept on disk under a name that carries the library’s documents and passage count (LibraryIntelligenceCache), read back while the library is what the name says, and removed when it is not. It is kept only when it was built from every passage in the store and each belongs to a listed document. The standard path and the Deep Think and Maximum path, which built its own copy, share one function (RAGService.libraryVocabulary). It holds a vocabulary in memory for later questions only when it is of the whole library as the library is when the build ends: a question limited to named documents on the Deep Think or Maximum path builds from those documents for itself and leaves nothing behind (from 2026-10-09 until this change it left that vocabulary as the library’s; no version shipped with it). [evidence_level: test_verified for the kept file and the rule for keeping it (LibraryAnalysisSampleTests), measured on the Mac for the build's time, code_verified for which actor it runs on, confidence: high, evidence_source: RAGService.swift libraryVocabulary, queryInternal, performFullRetrievalPipeline; AdaptiveEmbeddingOptimizer.swift LibraryIntelligenceCache]

1. Overview

The retrieval pipeline is the core engineering idea in OpenIntelligence: answers are grounded in user-provided material and expose the exact evidence that influenced them. The app is built to bias responses toward groundedness, showing uncertainty when evidence is weak rather than inventing confident prose.


2. RAG Retrieval Flow

flowchart TD
    UserQuery[User Query Input] --> QueryPlan[Query Analysis: heuristic intent, lookups kept literal, no expansion or HyDE in Standard]
    QueryPlan --> EmbeddingGen[Generate Query Vector - Core ML or Core AI]
    GPUProfile[GPU Execution Profile] --> EmbeddingGen
    EmbeddingGen --> HybridSearch[Hybrid Search: FTS5 Lexical + Vector Similarity]
    GPUProfile --> HybridSearch
    HybridSearch --> RRF[Reciprocal Rank Fusion - RRF]
    RRF --> Cutoff{Adaptive Ceiling Cutoff}
    Cutoff --> Rerank[Core ML Granite Reranker on the Neural Engine / Heuristic Fallback]
    Rerank --> Floor[Dynamic Similarity Floor, after a wider search or a keyword retry when the first is weak]
    Floor --> MMR[MMR Diversity Selection, lambda from the library settings]
    GPUProfile --> MMR
    MMR --> ContextExpand[Context Sibling Expansion - ParentDocumentService]
    ContextExpand --> PackChoice{Lookup question, not step-by-step?}
    PackChoice -- Yes --> LeadWhole[Best passage whole as S1, then question-relevant sentences]
    PackChoice -- No --> LostInMiddle[Passages packed in rank order, then placed in Lost-In-Middle order when four or more fit, step-by-step ones in document order, then Needle Rescue]
    LeadWhole --> Evidence[Post-Retrieval Evidence + Exact Token Budgets]
    LostInMiddle --> Evidence
    HybridSearch -.-> Trace[[RetrievalTraceCollector - eval runs only, nil in production]]
    RRF -.-> Trace
    Cutoff -.-> Trace
    Rerank -.-> Trace
    Evidence -.-> Trace
    Trace -.-> StageScore[[Per-stage recall@k / MRR / nDCG - RetrievalStageEvaluator]]
    Policy[Persistent Picker Policy: Hybrid / On-Device / PCC] --> ModelRoute
    Evidence --> ModelRoute{ModelExecutionPlanner v2}
    Capability[Signed Entitlement + Availability + Quota + Consent State] --> ModelRoute
    ModelRoute -- Insufficient Evidence --> AbstainRefusal
    ModelRoute -- Local --> LocalLLM[SystemLanguageModel.default]
    ModelRoute -- PCC Candidate --> Minimize[Minimized Cloud Evidence Envelope]
    Minimize --> Consent{Consent Valid?}
    Remembered[Canonical Remembered Consent - No Launch Prompt] --> Consent
    Consent -- Yes --> PCCLLM[PrivateCloudComputeLanguageModel]
    Consent -- No / UI Unavailable --> LocalLLM
    LocalLLM --> Verification[Verification Gates A-I - Negation & Word-Overlap]
    PCCLLM --> Verification
    Verification --> Decision{Every Gate That Ran Passes? Gate E runs only with embeddings}
    Decision -- Yes --> GroundedAnswer[Render Grounded Answer with Citations]
    Decision -- No --> FailRule{Reliability mode off, or gate average under 0.50, 0.25 for a lookup?}
    FailRule -- Yes --> AbstainRefusal[Abstention Refusal Message]
    FailRule -- No --> Labelled[Answer shown, labelled unverified with the failed gates, a lookup then through the source-only check]
    GroundedAnswer --> RouteBadge[Receipt-backed Route Badge]
    Labelled --> RouteBadge

Corrected 2026-10-20 against the working tree. The diagram is Standard’s path. Deep Think and Maximum leave it once the question is profiled (queryInternal returns the answer of executeAgenticQuery before the rewrite, the embedding and the search) and search through executeFullRetrievalPipeline, where each search runs the reranker, then MMR, then the floor. From 5.7 the answer of a Deep Think or Maximum run is labelled by the verification gates, and never abstains on them, unless the source-only check replaced it (rewrote it from the passages, or declined to answer). A check that does not run, or runs and keeps the answer, leaves the gates to label it. Through 5.6 those answers read Not Checked unless the source-only check replaced them, and a value the precision lookup returns before any run still does (item 22). In Standard a failed gate check abstains only when reliability mode is off (it is on by default) or the gates’ average confidence is under the mode’s bar, 0.50 or 0.25 for a lookup. Otherwise the answer is shown labelled unverified: with the failed gates, and a lookup’s answer then goes through the source-only check, which can rewrite it from the passages or, for a precise value or a state, decline to answer. The critical-gate flag (shouldAbstain, gates A, C and E) decides nothing: its one reader, StructuredAnswer.applyVerification, adds its reason to the answer’s list of what is missing. Gate E runs only when the answer’s and the passages’ embeddings are at hand. When the plan keeps the answer on the device and the context does not fit there (usually because the first pass was sized for Private Cloud Compute), or consent is refused, the context is packed again passage by passage (RAGEngine.assembleContext), a lookup’s too. Standard expands no question, so the part-of-speech tagging that picks a question’s key terms for synonym lookup (QueryEnhancementService.extractKeyTerms, inside expandQuery, which keeps nouns, verbs and adjectives and tags no names) runs only for Deep Think and Maximum, and the router’s entity tagger (QueryRouterService.extractQueryEntities) has no caller. This diagram used to read “Query Analysis: NER & Expansion”, went from the reranker straight to MMR, with no λ named, and from siblings to Lost-in-the-Middle for every question, asked “Critical Gates Pass?”, and sent every failed check to abstention. [evidence_level: code_verified, confidence: exact, evidence_source: RAGService+QueryPipeline.swift:181, RAGService+QueryPreparation.swift:40 (the profile), RAGService+QueryPipeline.swift:212-231 (the branch), RAGService+QueryPreparation.swift:78, RAGService+QueryPreparation.swift:413-419, RAGService+QueryPreparation.swift:436, RAGService+QueryPreparation.swift:456, RAGService+QueryRetrieval.swift:556, RAGService+QueryRetrieval.swift:626, RAGService+QueryRetrieval.swift:724, RAGService+QueryRetrieval.swift:865, RAGService+QueryRetrieval.swift:935-942, RAGService+QueryContext.swift:1207-1221 (step-by-step passages sorted by their place in the document), RAGService+QueryContext.swift:1291, RAGService+QueryContext.swift:447-454, RAGService+QueryContext.swift:479, RAGService+QueryContext.swift:510-532, RAGService+QueryRetrieval.swift:170-229, RAGService+QueryExits.swift:625, RAGService+QueryAnswer.swift:833, RAGService+QueryAnswer.swift:361-403, RAGService+AnswerFinishing.swift:455, RAGService+AgenticQuery.swift:775-799 (the source-only check after a run sets the gating decision only when it returns an outcome), RAGService+AgenticQuery.swift:801-816 (the gates, when it set none), RAGService+AnswerFinishing.swift:422-554 (sourceOnlyOutcomeIfNeeded returns nil when it declines or keeps the answer), RAGService+Generation.swift:979-1026, RAGService+ToolHandler.swift:253, RAGService+ToolHandler.swift:323, RAGService+ToolHandler.swift:341, RAGService+ToolHandler.swift:353, VerificationGateService.swift:200-223 (Gate E), :249-254, StructuredAnswer.swift:974, ConfidencePolicyService.swift:48-54, RAGQualityMode.swift:226-243, SettingsStore.swift:532-535, QueryEnhancementService.swift:730-739, :802-831 (extractKeyTerms enumerates .lexicalClass only), QueryRuntimeCoordinator.swift:197-200, and executeAgenticQuery in v5.6.0's RAGService.swift (lines 8849-9416 of that file), where every verificationResult is nil] [evidence_level: grep_verified, confidence: high, evidence_source: 2026-10-20, grep -rn "extractQueryEntities\|\.shouldAbstain" OpenIntelligence OpenIntelligenceTests --include='*.swift' (extractQueryEntities: its definition only, QueryRouterService.swift:292, and on a RAGVerificationResult .shouldAbstain is read in the app only at StructuredAnswer.swift:974, the other hits being other types' flags or tests), and git show v5.6.0:OpenIntelligence/Services/RAG/Orchestration/RAGService.swift piped to grep -c verifyAgenticAnswer (0)]


3. Pipeline Stages

  1. Import: Files enter through Apple platform document workflows.

  2. Extraction: Text, layout, and metadata are extracted.

  3. Chunking: Chunks are generated with metadata using semantic and structure-aware rules, applied to word counts. A cut goes, in this order of preference, at the start of a section, at the end of a table or before a transition phrase (“However,”, “In conclusion,”), when one lies within the size limits, and otherwise near the target size, at a sentence end within 100 characters of it or else between words. The phrases are the semantic part, and the code preset leaves them out. The target and overlap come from the library’s chunking plan when it has one, and otherwise from the file’s preset: 240 and 45 words for a PDF, an image or a CSV file, 180 and 35 for code, and 260 and 50 for other text. Either way they are never more than 260 and 50, and the ceiling is 310 words. Every library the app makes adapts itself (autoAdaptDimension) until the person turns that off in Library Settings, and a library saved before the setting existed reads it as off. An adapting library writes a plan from each file’s opening text before it cuts the file (a PDF’s first ten pages, or a file that reads as UTF-8 text), and that plan always asks for 280 words or more with an overlap of 50 or more, so in such a library every file is cut at 260 and 50 once a plan exists, and the plan’s kind can turn the phrases on for code too. An image, an Office file or a PDF with no text layer gives no opening text and writes no plan, so it is cut by the library’s last plan, or by its preset when the library has none. A library set by hand uses the two values chosen in Library Settings. A passage of under 15 words is merged into the one before it, with no size check, so a merged passage can pass the ceiling. A table stays one passage only up to 380 words, a larger one is split, and each table also gets a summary passage. A passage over 430 WordPiece tokens is split again. The embedding-based topic boundaries described at the top of SemanticChunker.swift never run: the app cuts with chunkText, from a chunker made without an embedding service, and chunkTextAsync, the only path to detectEmbeddingBoundaries, is called only by a test. (Corrected 2026-10-20: this said only “semantic and structure-aware rules”. The sizes, the cues and the merge and split rules are added, and “semantic” is narrowed to the phrase cues.) [evidence_level: code_verified, confidence: exact, evidence_source: SemanticChunker.swift:7-16, SemanticChunker.swift:168, SemanticChunker+Chunking.swift:80-91, SemanticChunker+Chunking.swift:104-136 (the presets), SemanticChunker+Chunking.swift:141-208 (recommended(for:strategyOverride:), where a dense or elastic plan turns the phrases on), SemanticChunker+Chunking.swift:304-314, SemanticChunker+Chunking.swift:392-398, SemanticChunker+Structure.swift:356-374 (the transition phrases), SemanticChunker+Boundaries.swift:109-228 (findOptimalChunkRange: a section start, then a table end, then a phrase, then a sentence end or a word boundary), SemanticChunker+Chunking.swift:528-541, SemanticChunker+Chunking.swift:624, SemanticChunker+Boundaries.swift:31-36, DocumentProcessor+Import.swift:404-409 (the plan's values, else the preset's), DocumentProcessor+Import.swift:422-426 (clamped to 260 and 50), DocumentProcessor+Import.swift:470, DocumentProcessor+Import.swift:494, DocumentProcessor+StructureChunks.swift:466, DocumentProcessor+StructureChunks.swift:162-168, DocumentProcessor+StructureChunks.swift:213-226, DocumentProcessor+TokenLimits.swift:36, DocumentProcessor+TokenLimits.swift:90, RAGService+Import.swift:503, RAGService+Import.swift:618-648 (the plan written before a file is cut), RAGService+SelfTuning.swift:77-87 (chunkingOverride), RAGService+SelfTuning.swift:89-107 (the opening text), AdaptiveEmbeddingOptimizer.swift:241-337 (every plan asks for 280 words or more and an overlap of 50 or more), KnowledgeContainer.swift:110, :172, ContainerSettingsSheet.swift:670-682] [evidence_level: grep_verified, confidence: high, evidence_source: 2026-10-20, grep -rn "chunkTextAsync(\|detectEmbeddingBoundaries(\|SemanticChunker(" OpenIntelligence OpenIntelligenceTests --include='*.swift' (chunkTextAsync is called only at SemanticChunkerTests.swift:233, detectEmbeddingBoundaries only from chunkTextAsync, and each SemanticChunker() in the app is made without an embedding service) and grep -rn "autoAdaptDimension:" OpenIntelligence --include='*.swift' (every library is made with the default, true, ContainerService.swift:121, :140, :493)]

  4. Indexing: Chunks are written into local search (SQLite FTS5) and vector databases (BNNSVectorDatabase.swift). The keyword index is written twice: the document’s whole text before the text is cut, with one row per page as well for a PDF of more than one page and for each batch of a PDF imported in batches, and one row per passage after its vectors are stored, all stemmed and case-folded by porter unicode61. A large text, Markdown or JSON Lines file is imported in pieces, and its whole text is written once, after every piece is in. The vectors are kept in arrival order in one flat Float32 file per library, which each save writes again whole, and a search compares the question with every vector (on the GPU from 1,000 vectors when the device policy allows it). There is no approximate index and no grouping by topic. (Added 2026-10-20.) [evidence_level: code_verified, confidence: exact, evidence_source: DocumentProcessor+Import.swift:42-53 (which files are imported in pieces), DocumentProcessor+Import.swift:331-337 (pages are numbered for more than one page or a PDF batch), DocumentProcessor+Import.swift:351-353 (a piece's text goes back to its caller), DocumentProcessor+Import.swift:359 (before chunking at :396), RAGService+TextStreaming.swift:317-321 (the whole text, after the last piece), RAGService+Import.swift:339 (vectors), RAGService+Import.swift:1205 (passage rows), SQLiteFullTextService.swift:191-195, SQLiteFullTextService.swift:242-252, SQLiteFullTextService.swift:321-327, BNNSVectorDatabase.swift:29-36 (each file written whole on a save), :1036-1048, :1063-1091 (storeBatch appends in memory, persist saves), :1104-1126, VectorStoreRouter.swift:160-161] [evidence_level: grep_verified, confidence: high, evidence_source: grep -rn "pageBreakSentinel" OpenIntelligence --include='*.swift', 2026-10-20: the page break is written only at DocumentProcessor+PDFText.swift:1005 and DocumentProcessor+StructuredParsing.swift:1339, both in PDF extraction]

  5. Query Analysis & Planning: Incoming questions are classified, scoped, and prepared for retrieval.

  6. Retrieval: Candidate chunks are selected from the active library or workspace container.

  7. Reranking and Packing: Evidence is scored using a local Core ML cross-encoder, since 5.8 IBM’s granite-embedding-reranker-english-r2 on the Neural Engine (TinyBERT through 5.7; see “The reranker, from 5.8” below), with a proximity-based heuristic fallback if the model is absent, deduplicated (MMR, which acts from 5.7, see “MMR and the passages’ vectors” below), expanded with sibling context, and packed into the budget. When the packing goes passage by passage (RAGEngine.assembleContext), a passage whose words overlap 75% or more with one already packed is skipped. A lookup’s first pass packs chosen sentences instead (item 13), and there a sentence whose words overlap more than 70% with one already chosen gives way to the better-scored of the two. Contextual compression, a model call that cuts irrelevant sentences, runs on no answer path: Standard’s mode turns it off, and Deep Think and Maximum leave Standard’s path before its one compression step, although RAGQualityMode.usesContextualCompression reads true for Deep Think. (Corrected 2026-10-20: this said “and compressed”.) [evidence_level: code_verified, confidence: exact, evidence_source: RAGQualityMode.swift:277-284, RAGService+QueryPipeline.swift:212-231, RAGService+QueryContext.swift:1076-1086, RAGService+QueryContext.swift:80, RAGService+AgenticQuery.swift:905-910 (the fallback rerun is Standard), RAGService+ToolHandler.swift:92-526 (executeFullRetrievalPipeline and performFullRetrievalPipeline, no compression step), RAGService+QueryContext.swift:447-457 (a lookup's first pass), RAGService+SentenceExtraction.swift:444-505 (the sentences' own overlap test), QueryRuntimeCoordinator.swift:222-227, RAGEngine.swift:563, :605-624 (the 75% test, in assembleContext)] [evidence_level: grep_verified, confidence: high, evidence_source: grep -rn "ContextualCompressionService()" OpenIntelligence OpenIntelligenceTests --include='*.swift', 2026-10-20: one hit, RAGService+QueryContext.swift:80] Only the first min(count, max(100, min(250, topK × 5))) fused candidates reach the reranker (RAGEngine.rerank); from 5.7 a warning line says how many were left out whenever that ceiling binds, which nothing showed before (roadmap F-03). [evidence_level: code_verified, confidence: high, evidence_source: RAGEngine.swift, the adaptiveCeiling lines]

  8. Post-Retrieval Model Routing & Generation: ModelExecutionPlanner combines evidence sufficiency and synthesis burden with the captured persistent policy, foreground state, network, signed entitlement, live quota/availability, and SDK token/context budgets. Hybrid chooses per question in Standard and per planned call in Deep Think and Maximum, On-Device is local-only, and explicit PCC requests cloud but records a truthful local completion when any cloud gate prevents use. (Corrected 2026-10-20: this said Hybrid chooses “per query”.) Local execution uses SystemLanguageModel.default; eligible iOS/macOS 27 synthesis may use native PCC after the exact minimized envelope is consented. iOS/macOS 26 remains local-only. [evidence_level: code_verified, confidence: high_pending_physical_device_validation, evidence_source: ChatScreen.swift, ModelExecutionPlanner.swift, LLMService.swift, RAGService.swift] In Standard, for a library that has passages, the plan is made once, before the answer is generated. A Standard question typed to a library with no documents gets no plan: the model answers it alone, with the library’s tools switched off. (Corrected 2026-11-04: this said the route policy’s branch for a call with no plan can send it to PCC with no consent sheet, under Hybrid when it is longer than the device’s window, and at any length when PCC is picked, even with the permission on Don’t Allow. That described 5.6 and the 5.7 work before 2026-10-21. FoundationModelRoutePolicy.determineRoute now returns .onDevice for a call with no plan, so this question never reaches PCC; Docs/PRIVACY_AND_ROUTING.md, “The exception: a question to a library with no documents”, has the history. [evidence_level: code_verified, confidence: exact, evidence_source: FoundationModelRoutePolicy.swift:49-65 ("No plan means on device (5.7)"), RAGService+QueryExits.swift:794-799 (disableTools = true, no plan)]) [evidence_level: code_verified, confidence: high, read link by link and not run on a device, evidence_source: RAGService+QueryPipeline.swift:327-329, RAGService+QueryExits.swift:730-801 (the empty library), RAGService+Generation.swift:1480-1487, RAGService+StandardRetrieval.swift:873-881 (no plan, no consent), FoundationModelRoutePolicy.swift:28-66, FoundationModelSessionFactory.swift:86-102] In Deep Think and Maximum each call made through RAGService.generateWithProperConsent is planned on its own, and the calls made through RAGService.generateWithFreshSession are never planned and run on the device. Under Hybrid only the calls that write an answer can be planned to PCC. Such a call goes there when PCC is usable, the device is online, the person can be asked (the app is in front, or a Siri or Shortcuts action can bring it forward) or has already allowed it, and the call’s context fits PCC’s window, and when, besides, its context does not fit the device budget or, outside Standard, its passages come from more than one document. Consent is then asked before anything is sent, unless a choice is remembered. The other planned calls in those modes (the sessions, research’s search-or-answer decisions and Maximum’s running answers) are held on the device unless the person picked PCC in the model picker (Docs/PRIVACY_AND_ROUTING.md, “Fallback and stream integrity”). (Added 2026-10-20.) [evidence_level: code_verified, confidence: exact, evidence_source: RAGService+QueryRetrieval.swift:122 (Standard's one plan), RAGService+Generation.swift:1222-1232, RAGService+Agentic.swift:87-132, RAGService+Agentic.swift:214-220, RAGService+Agentic.swift:248-256, ModelExecutionPlanner.swift:45-56, :77-85]

  9. Fidelity Verification & Receipt: Responses pass through local verification checks. Route metadata is persisted as a ModelExecutionReceipt that separates intended, attempted, actual, fallback, and completed targets. [evidence_level: code_verified, confidence: high, evidence_source: VerificationGateService.swift, ModelExecutionReceipt.swift, RAGQuery.swift] From 5.7 the numeric check compares a number as a value (NumericValueExtractor): “2.5 billion” has to be stated as 2.5 billion, a number that is part of a name or code (“Qwen-2.5-3B”) is matched as that name, and digits inside a larger number do not match. A sentence whose figure is not stated in the passages is Partial, not Supported. The same reading is applied in the source-only check that follows a Deep Think or Maximum answer. From 2026-10-20 a claim’s check (Fact Check, Check a Claim) also counts an ordinal (“the 16th”, against the same number in digits or words) and an amount written in words (“four hundred dollars”, NumericValueExtractor.spelledAmounts); the numeric gate over a whole answer counts neither, and a passage’s amounts in words are values an answer’s digits can match. [evidence_level: test_verified (NumericSanityGateTests), confidence: high, evidence_source: NumericValueExtractor.swift, VerificationGateService.swift] [evidence_level: test_verified for Standard's gate and claim check, build_verified for the source-only check; confidence: high; evidence_source: NumericSanityGateTests, 2026-10-09; SourceOnlyAnswerService.swift]

  10. Presentation: Answers are shown with liquid glass UI indicators, citations, quality gauges, and review affordances.

  11. Continuous Evaluation: RAGEvalRunner runs the cases in Benchmarks/rag_eval_v1.jsonl through the live pipeline and scores answer match, retrieval recall, citation precision and abstention.

    Corrected 2026-08-08. This item previously read “verified against quality targets (e.g. Recall@5 $\ge 0.85$, Citation Precision $\ge 0.90$) using the native Evaluations harness”. Two parts of that were untrue and are withdrawn rather than deleted, so the record shows what was claimed. The \ge 0.85 recall gate was never met and could not be: retrievalRecallAt5 returned exactly 0.0 on every run regardless of retrieval quality, because the runner only scored recall when a case supplied groundTruthChunkIds and every committed case carries null there. And there is no native Evaluations harness: AppleEvaluationsBridge.swift is named for Apple’s Evaluations framework but imports only Foundation and FoundationModels. [evidence_level: code_verified, confidence: exact, evidence_source: AppleEvaluationsBridge.swift imports, Benchmarks/rag_eval_v1.jsonl all 20 records, CHANGELOG.md 4.9]

  12. Per-stage retrieval measurement (added 2026-08-08): A RetrievalTraceCollector may be attached to a query. It records the rank-ordered output of each retrieval stage, and RetrievalStageEvaluator scores recall@1/3/5/10, MRR, nDCG@5/10 and precision@5 per stage rather than once at the end.

    RetrievalTraceCollector.Stage has seven cases, in pipeline order: vector, lexical, fusion, boosted, candidates, rerank, final. The first five are recorded by HybridSearchService in both its search paths; rerank is recorded by RAGService after RAGEngine.rerank, because that is where reranking runs; final is recorded at the queryWithAudit boundary from the chunks on the response.

    The split matters for attribution. Reranking cannot recover a chunk the first stage never returned, so a single end-of-pipeline number cannot tell a first-stage regression from a reranker regression, and a later stage can mask an earlier one. A recall drop between boosted and candidates means top-K truncation is too tight; a drop between candidates and rerank means the cross-encoder demoted the right chunk.

    That last inference holds only when ground truth credits every document the question requires, and on 2026-08-11 it did not. scripts/build_eval_dataset.py emitted a single-element expectedCitations from the manifest’s singular expected_source, and scripts/run_quality_matrix.py passed that one filename to the harness. All five multi_hop_project_m* cases ask a two-part question whose halves live in different fixtures, so ranking the other required document first scored as a demotion. That produced rerank MRR@10 0.972 and final 0.917 against a vector stage at 1.000, and the arithmetic closed exactly at (17 + 0.5)/18 and (15 + 1.5)/18 while all five cases answered correctly. The reranker was not demoting anything. Both scripts now read a plural expected_sources, falling back to the singular key, and the manifest carries both filenames for those five cases. Before attributing a late-stage drop to the reranker, check that every case in play credits all of its required documents. [evidence_level: measured+code_verified, confidence: exact, evidence_source: BenchmarkRuns/20260811-133233-matrix/results.json, build_eval_dataset.py, run_quality_matrix.py, tiny_research_suite/manifest.json]

    The collector is one instance per query, passed explicitly as a defaulted parameter. It is nil in production, where the only cost is a nil check per stage. It is deliberately not a shared sink: retrieval runs concurrent child tasks, and a process-wide collector would interleave stages from overlapping queries with no way to separate them. [evidence_level: code_verified, confidence: exact, evidence_source: RetrievalTraceCollector.swift, HybridSearchService.swift, RAGService.swift, RetrievalStageMetrics.swift]

  13. Sentence selection, which runs after the last measured stage (added 2026-08-13): RAGService.extractRelevantSentences turns the retrieved chunks into the text the model actually sees. It runs downstream of final, so none of the seven stage metrics above observe it. A query whose every stage figure is excellent can still reach the model with nothing usable in its context, and on 2026-08-13 one did. From 5.6 a Deep Think chain session skips this stage when its window’s chunks fit the device’s context window in tokens and reads them whole (SessionEvidencePlan). [evidence_level: code_verified, confidence: high; not run on a device] Standard’s lookups, the Needle Rescue that follows Standard’s passage-by-passage packing (item 14), Maximum’s own loop, Self-RAG’s synthesis (executeComprehensiveSynthesis) and every window that does not fit still go through it. Standard’s other questions are packed passage by passage by RAGEngine.assembleContext, and so is a lookup whose sentence selection comes back empty or whose context is packed again after the route plan. (Corrected 2026-10-20: this said Standard as a whole goes through it.) [evidence_level: code_verified, confidence: exact, evidence_source: RAGService+QueryContext.swift:447-454, RAGService+QueryContext.swift:479, RAGService+QueryContext.swift:510-537, RAGService+QueryRetrieval.swift:186, RAGService+QueryRetrieval.swift:213, AgenticOrchestrator+Retrieval.swift:449, AgenticOrchestrator+ReasoningChain.swift:340, AgenticOrchestrator+ReasoningChain.swift:698, AgenticOrchestrator+Maximum.swift:852]

    Line and sentence cuts, corrected 2026-10-09 for 5.7. The stage split each passage at every line break and then split each line at every ”. ”. A sentence wrapped over two lines became two fragments, scored and kept separately, and “Rent is late after 11:59 p.m. on the 5th.” was cut after “p.m”, so the model could be handed “on the 5th” without the first half of the sentence (roadmap row 3f349a74d54f81748e49d885416a4ced). EvidenceLineSplitter (Services/RAG/Tuning/EvidenceLineSplitter.swift) now does both cuts. lines(in:) joins two lines only when the first is at least 40 characters and ends in a word, a comma or an abbreviation’s period (“p.m.”), the second starts with a lowercase letter and is neither a field (model: on-device, monthly rent: 1200, retries = 3; a name of up to four words in any script) nor a lettered list item (a) ..., b. ...), and neither is a table row. So the fields of an imported record, rows that end in a figure, lines of code and a line that ends a sentence stay as they were; a blank line always ends a line; and a heading under 40 characters stays on its own line. A heading of 40 to 80 characters directly above a lowercase line is joined to it, which the old code did not do. Whether a line-ending period ends a sentence is judged with a capitalised word after it, because the sentence splitter reads a lowercase word after any period as the same sentence going on. units(in:) splits a line with AnswerSentenceSplitter, the splitter the answer cleanup has used since 5.6, which does not cut inside a time, an amount or an abbreviation; a row with a tab, two spaces or a pipe is kept whole, as before. All six callers of the stage share it, so Deep Think and Maximum evidence changes with Standard’s. [evidence_level: test_verified, confidence: high, evidence_source: EvidenceLineSplitterTests on an iOS 27.0 simulator, 2026-10-09; RAGService.swift extractRelevantSentencesOffMain] No answer has been compared before and after on a device.

    What a device log showed. For “How often must reprocessing occur?” retrieval completed in 88.5ms with 177 candidates, 57 of them found by FTS5 and missed by vector search, and the reranker normalised the top chunk from 0.10 to 0.90 with the correct section in first place. Sentence selection then handed the reasoning chain 67 characters against a 3000+ character budget: a section label and a filename. Deep Think ran eight sessions over that, its insights degenerated to “[S1] contains a section titled”, and the final generation failed to parse. The user’s query failed while every retrieval metric was healthy.

    Two independent causes, both fixed. Selection kept a sentence only if it contained a query keyword as a raw substring, with no stemming, while FTS5 indexes with porter unicode61. Retrieval therefore matched “reprocessed” to a query saying “reprocessing” and selection then discarded those same sentences, leaving only lines carrying the literal token. Separately, a line qualified as a heading only if it contained no digits at all, which rejects every heading in a numbered technical manual (“4 Reprocessing”, “4.5.1 Overview”), so no sentence could inherit its section’s keywords. Keyword matching is now additive across raw substring and a shared suffix-stripped stem, the digit test applies only after a leading section number is removed, and the section is seeded from chunk.metadata.sectionTitle rather than re-derived from raw text. [evidence_level: measured+code_verified, confidence: exact, evidence_source: iPhone console capture 2026-08-13, app 4.9 build 150, not retained in the repository (root .txt is gitignored: these captures run to several MB and carry container UUIDs and document names); the figures it produced are quoted inline above and in the 2026-08-13 CHANGELOG entry; RAGService.swift extractRelevantSentencesOffMain; DocumentChunk.swift ChunkMetadata.sectionTitle]

    Confirmed on device the same day. A second capture after the fix, on a different question (“What safety protocols are mandatory?”), recorded session contexts of 855, 1468, 850, 242, 252, 252, 242, 899 characters against the pre-fix 106, 123, 67, 53, 167, 111, 67, 67. The four sessions landing near 242 are the new too-small fallback firing. Insights carried real cited findings rather than “contains a section titled”, and no session echoed its system prompt. The query still failed, which disproved the assumption that context starvation was contributing to the final parse error: that failure is independent, and is a rate limit. See the 2026-08-13 CHANGELOG entry on escalating backoff. [evidence_level: measured, confidence: exact, evidence_source: second iPhone console capture 2026-08-13, same caveat on retention]

    The general lesson, and it is the same one item 12 records one level up. Stage metrics measure retrieval, not what the model receives. AgenticOrchestrator had a fallback to raw chunk content, but it tested isEmpty, and 67 characters of section title is not empty. Guard the layers between retrieval and generation on usefulness, not on presence.


4. Library Isolation

Library and workspace boundaries are critical because retrieval quality depends on scope. The app is designed so that a query is answered strictly against the user-selected document container rather than all files indiscriminately, preventing cross-container leakage.

From 5.7 a question can also be held to named documents inside the library. The three Shortcuts actions that name a document pass a RAGQueryScope, run in Standard, and refuse an answer whose passages are not all from those documents. [evidence_level: code_verified, confidence: high; not run from Shortcuts or Siri on a device, evidence_source: RAGAppIntents.swift AskActionRunner; Docs/OPENINTELLIGENCE_ARCHITECTURE_ATLAS.md, the 5.7 Shortcuts paragraph]


5. Diagnostics & Telemetry

Diagnostic and telemetry surfaces are included for inspecting chunks, retrieval quality, answer details, and pipeline behavior. These are engineering tools for iteration and must not be interpreted as validation for regulated or safety-critical workflows.

  1. Citation labels are one namespace, and the response must be able to resolve them (added 2026-08-14): RAGEngine.assembleContext packs chunks until the character budget runs out and labels them [S1]…[S{used}]. Whatever did not fit is handed to Needle Rescue, which runs extractRelevantSentences over the dropped chunks and appends the result to the same prompt. Rescue previously labelled its sources from [S1] again, so one prompt presented two different chunks under the same label. A device trace on 2026-08-14 shows the consequence: four packed sources, then rescue sources numbered from one, and an answer citing [S5] and [S6], being the fifth and sixth sources the model had been shown. Those resolved to nothing, because includedRetrievedChunks was orderedCandidates.prefix(actualChunksUsed) and carried only the packed four.

    Corrected 2026-10-20: not everything that did not fit reaches the rescue. For a question that is not step-by-step, packing fills the budget in Lost-in-the-Middle order (ranks 1, 3, 5 and onward, then the even ranks reversed), but the rescue takes orderedCandidates.dropFirst(assembled.used), the input list as it was before that reorder. With ten candidates and room for three, the prompt holds ranks 1, 3 and 5, the rescue is offered ranks 4 to 10, rank 5 among them again, and rank 2 is neither packed nor rescued. It is the prefix(used) defect of item 15 seen from the other side, and it is in the working tree of 2026-10-20. [evidence_level: code_verified, confidence: high, read from the code and not run, evidence_source: RAGEngine.swift:500-504, :779-797, RAGService+QueryContext.swift:1291, RAGService+QueryContext.swift:510-532]

    Fixed later the same night, in the 5.7 working tree. assembleContext now chooses the passages that fit in rank order and places only those (best first and second best last, when four or more fit). The graph packer that serves investigate, compare and findings questions no longer reorders its own output (ContextPackingService.pack returns priority order), so the assembler is the one place that places passages. The rescue reads the candidates after where the assembler stopped (RAGService.candidatesPastThePacked). With ten candidates and room for three, the prompt now holds ranks 1, 2 and 3, and the rescue reads ranks 4 to 10. The trace logs the candidates a whole-passage context kept, in label order ([RAG] Whole-passage context keeps candidates [1, 3, 4, 2] of 17, in label order, from RAGService.candidatePlaces). [evidence_level: code_verified, confidence: high, evidence_source: RAGEngine.swift assembleContext; ContextPackingService.swift pack; RAGService.swift candidatesPastThePacked; tests CitationLabelNumberingTests.testTheSecondBestPassageIsKeptAndPlacedLast, ContextPackingServiceTests.testPackKeepsPriorityOrderForTheAssembler]

    Two rules now hold together, and neither works alone. Rescue continues the packed block’s numbering through a labelOffset, and the chunks rescue contributed are attached to the response. SentenceExtractionResult.usedSourceIndices exists to make the second possible; returning only a count is what let the prompt and the response disagree about what a citation meant.

    The general rule: anything that puts text in front of the model under a citation label must also put the corresponding chunk in the response. Three separate layers already drop out-of-range citations (StructuredAnswer.citedEvidence, the empty-evidence confidence floor, normalizeStructuredCitations) and Gate B fails on them. None of that helps, because the answer prose is not validated and it is the only part a reader sees.

  2. The prompt’s source array and the response’s source array must be the same array (added 2026-08-14): RAGEngine.assembleContext reorders chunks for Lost-in-the-Middle mitigation before numbering them, so [S1]…[Sn] are labels over front + back.reversed(), not over the input. Every caller independently rebuilt the citation list as prefix(used) of the array it had passed in, which is a different set for any query with four or more chunks. [A,B,C,D] is presented as [A,C,D,B], so a citation of [S2] meant C and resolved to B.

    This was undetectable by every guard in place. Indices were in range, chunks existed, and the citation resolved to a real source. It was simply the wrong one. Five call sites had the same defect, which is the signature of a label related to its chunk by convention rather than by construction.

    assembleContext now returns sources: the chunks in the exact order their labels were assigned. Callers must use it and must not recompute a prefix. One caller still does the same thing from the other side: Standard’s Needle Rescue takes dropFirst(assembled.used) of its own input (item 14, corrected 2026-10-20). Since later that night (5.7) the assembler chooses the passages that fit before it places them, its labels still number the final order and sources still returns it, and the rescue no longer takes dropFirst (item 14). If you add a stage that puts text in the prompt under a citation label, it must return the chunks it labelled. Item 14 records the related case where needle rescue appended labelled text whose chunks reached no one.

  3. The right fusion weight is corpus-dependent, and the app can already tell which kind it is facing (added 2026-08-14): the offline sweep (Docs/EVALS.md) found MRR@10 maximised at dense weight 0.00, monotonically. That was measured on qasper_external_v1, whose questions were written by readers shown only a title and abstract and therefore reuse the paper’s exact vocabulary, which favours the lexical arm. It is a property of the fixture, not of retrieval.

    Device captures on 2026-08-14 show the arms inverting between real libraries. A manuals corpus returned 120 vector + 60 FTS5 with 57 lexical-only hits. A research corpus asked “What is the role of serotonin signaling?” returned 140 vector + 6 FTS5, and the log explains why: it reports that the keyword serotonin hit 51% of chunks and therefore received zero boost. On a corpus where the domain term is on every page, lexical search discriminates nothing and the dense arm is carrying the entire query. Zeroing the dense weight there would sink 140 candidates beneath six.

    Do not set a single global weight from a single fixture. HybridSearchService already computes per-keyword hit rates and builds discriminativeKeywords before fusion runs, so the signal needed to choose a weight per query already exists at the moment the weight is applied. The completion log now reports unique: N vector-only, M lexical-only with the weights actually used, which is the evidence any future weight change should be argued from.

  4. Retrieval has a second consumer, and it needs the passage judged rather than the output judged (added 2026-08-15): SuggestedQuestionsService runs the same retrieval and turns the chunks into the questions offered before the user types anything. buildGroundedPassages took chunks.prefix(limit) with no quality test, so whatever retrieval returned became question material.

    The service already carried five filters, and every one of them inspects the generated question: isJunkString, sanitizeGeneratedQuestion, isUsableLLMGeneratedQuestion, isUsableGeneratedQuestion, isStructuralOrMetaQuestion. A reference list defeats all five at once. It is grammatical, it is dense with capitalised domain nouns, and the question it produces is well formed, so no test applied to the question can separate “What is the role of Neurosci Yagishita Transient?” from a real one. The defect is only visible in the source text.

    questionSourcePenalty scores the chunk instead, on citation density, dot leaders, access boilerplate, digit-to-letter ratio, vowel-free token ratio, and absence of sentence enders. It demotes rather than filters, because a hard filter returns an empty set for a document that is entirely a reference list or a scanned table, which disables the feature precisely where it is already weakest.

    Hand-built samples were not enough to get this right, and the corpus run is what corrected it. The first version passed all nine hand-built cases and still hard-demoted 51 chunks of ordinary body prose out of a 5,425-chunk sample of the real library. Two signals were wrong in a way no synthetic sample exposed. A parenthetical year counted three or more times is the marker of an author-year citation in running text, so it fires hardest on the related-work paragraph, which is good question material; a numbered reference list writes 2017;6:e20975 instead, so the signal was evidence against a bibliography. And matching author initials as [A-Z][a-z]{2,}\s+[A-Z]{1,3}[,.] without requiring the entries to be consecutive matched “Detection DET.” and “Twitter PHEME,” in any paper whose body is full of acronyms.

    Author detection now requires three consecutive Surname AB, entries with distinct surnames. The surname test is the part that generalises beyond papers: “Phase A, Phase A, Phase B,” is shape-identical to “Correia PA, Lottem E, Banerjee D,” and a repeat count cannot separate them, while “Section A, Section B, Section C” and “Option A, Option B, Option C” are ordinary content in manuals, forms and agreements. [evidence_level: measured, confidence: high, evidence_source: the shipped function extracted verbatim and run over a stratified 5,425-chunk sample of LocalCache/FTS5/fulltext.sqlite; 99.2% score 0 and zero are falsely hard-demoted, while bibliography, numbered references, table of contents, running header and numeric table all still demote hard]

    The general rule this is an instance of: item 12’s stage metrics stop at final, and both known defects since have been downstream of it. Item 13 was sentence selection, this is question generation. A stage figure cannot tell you that a consumer of the stage output is unusable.

  5. A fixed-size cut over a Lost-in-the-Middle ordering removes the best chunk by construction (added 2026-08-17): executeFullRetrievalPipeline finishes with a Lost-in-the-Middle reorder, and Deep Think’s synthesis then truncated what it received. Both stages were defensible alone. Composed, they discarded the single highest-scoring chunk on every query large enough to matter.

    The reorder sorts by score and interleaves: even ranks are appended, odd ranks are inserted at the midpoint. For 85 chunks that yields [c1, c3, c5, ...] with rank 0 at index 42, so the best chunk sits in the exact middle of the array. executeSearchStepWithChunks then rendered chunks.prefix(10), which for 85 chunks is ranks 1, 3, 5 … 19 and excludes rank 0 outright, and executeDirectSynthesis applied String(searchResults.prefix(3000)) to what was left. The comment above that cut justified 3000 characters against a system prompt it no longer matched.

    Nothing about this was visible. The cut logged nothing, the citations that survived resolved correctly, and Self-RAG scored the result relevance=70%, citations=1/1, confidence=88%. The observable symptom was an inversion: on “How does dopamine affect social behavior?” Deep Think answered that the documents provide no evidence, in 202.7s, while Standard assembled 3 chunks and 522 words, fit inside its budget, and answered correctly in 6.7s. [evidence_level: measured, confidence: exact, evidence_source: device capture Standard+DeepThink+Xcodeconsole.txt 2026-08-16, gitignored per the item 13 retention caveat; figures quoted inline]

    Three changes, and the ordering one is the load-bearing one. Synthesis now packs from the chunks rather than the pre-rendered string, sorted by score with array position as a deterministic tiebreak, so truncation removes the least relevant evidence instead of the most relevant. The budget is derived from FoundationModelTokenBudget against the actual system prompt, query and output reserve rather than a hardcoded character count. And every assembly emits one line naming chunks kept, tokens used and chunks dropped, so a future instance of this is readable in a trace instead of inferable from a wrong answer.

    Parent document expansion is now gated on the same budget. It grew an 18-chunk result to 85 immediately before a cut that could hold roughly eight, so it was not neutral: it added nothing that fit and displaced the ranked set that did. Primary matches are never gated, only siblings, because dropping retrieval’s own results here would trade one silent loss for another.

    Rank-ordered packing does not license recomputing the citation list. Item 15’s invariant still binds: the budgeted subset is passed to generateWithProperConsent as sourceChunks in the order it was labelled, because [S1] resolves positionally. Reordering for relevance without carrying the order forward would have swapped one defect for a misattribution. [evidence_level: code_verified, confidence: exact, evidence_source: AgenticOrchestrator.assembleBudgetedEvidence and executeDirectSynthesis; RAGService.swift step 7.5; reorder reproduced directly for n=85]

    Not device-verified. The change is build-verified and passes the 236-test suite, and this path has no test coverage, so the claim that Deep Think stops abstaining on the dopamine query remains unverified until the identical query is re-run on hardware and compared against the saved trace. [evidence_level: code_verified, confidence: high_pending_device_validation]

    Which synthesis this is (note of 2026-10-20). The synthesis here is the single one a Deep Think or Maximum question gets when its first search grades moderate, or low with Speculative RAG falling short (executeDirectSynthesis), and the decomposed pipeline’s (executeSynthesisStep), which a low grade can reach. When the first search grades excellent or good, Deep Think runs the reasoning chain instead, and so does Maximum on good (on excellent it runs its own loop). The chain’s closing synthesis is written from the sessions’ findings, at most 3,000 characters in Deep Think and 5,000 in Maximum, and its context holds no passages when the plan keeps it on the device. (Corrected 2026-11-12, for 5.8: Deep Think’s closing synthesis now also reads the passages its sessions cited, chosen by exact token count, 1,890 tokens with the notes on a 4,096 window and at most 3,200 on 8,192, in its prompt (ChainSynthesisEvidence); the 3,000-character notes-only prompt is the fallback when the prompt cannot be counted or would not fit. The notes are fitted rather than cut from the end. Maximum’s 5,000 characters are unchanged. [evidence_level: code_verified, confidence: exact, evidence_source: AgenticOrchestrator+ReasoningChain.swift sourcePassagesSynthesisPrompt; SessionEvidencePlan.swift ChainSynthesisEvidence]) When the plan picks Private Cloud Compute, generateWithProperConsent puts in place of that empty context a minimized envelope of the pool’s best-ranked passages, up to 12,000 characters, and if consent is then refused or cannot be asked, the call runs on the device with the same envelope. The front half also sorts the fused passages again by a blend of the best reranker score and the fusion score, so the chain’s pool is not in Lost-in-the-Middle order. [evidence_level: code_verified, confidence: exact, evidence_source: AgenticOrchestrator.swift:387 (excellent), AgenticOrchestrator.swift:683 (good), AgenticOrchestrator.swift:789 (moderate), AgenticOrchestrator.swift:885 (low), AgenticOrchestrator.swift:838, AgenticOrchestrator.swift:865, AgenticOrchestrator.swift:920, AgenticOrchestrator.swift:965, AgenticOrchestrator+Retrieval.swift:404, AgenticOrchestrator+Retrieval.swift:642, AgenticOrchestrator+Retrieval.swift:1128-1137, AgenticOrchestrator+ReasoningChain.swift:1106-1184 (Maximum's exhaustive synthesis), AgenticOrchestrator+ReasoningChain.swift:1197-1270 (Deep Think's synthesis), AgenticOrchestrator+ReasoningChain.swift:1175-1183 and :1262-1270 (both pass an empty context and the pool as sourceChunks), RAGService+Agentic.swift:264-290 (the envelope), RAGService+Agentic.swift:306-320 (a refused or unavailable consent), RAGService+Agentic.swift:366-372 (generated from the routed context), ModelExecutionPlanner.swift:139-178 (CloudEvidenceMinimizer, passages by rank), RAGService+Generation.swift:1309-1314 (the envelope as text)]

  6. The agentic path was never measured, which is why its recall gap could not be attributed (added 2026-08-19): item 12 records seven per-stage metrics — vector, lexical, fusion, boosted, candidates, rerank, final. Those come from HybridSearchService and from queryWithAudit. Deep Think and Maximum reach retrieval through AgenticOrchestrator, which calls executeFullRetrievalPipeline, and that function took no trace parameter at all.

    The consequence showed up the first time the two modes were benchmarked against each other. Standard reported all seven stages; Deep Think reported exactly one, final, recorded downstream from the response.

    stagestandard r@10deep-think r@10
    vector0.571—
    lexical0.857—
    fusion0.714—
    rerank0.857—
    final1.0000.625

    Standard’s retrieval is perfect on this set: r@5 and r@10 both 1.000 across seven scored cases. Deep Think finds the gold document 62.5% of the time on the same eight. That gap is the entire quality difference between the modes, because correct == (gold_recall == 1.0) held in 14 of 15 scored runs — synthesis was right whenever it had the evidence and wrong whenever it did not.

    Two hypotheses were formed and refuted by reading source within the hour. That query expansion degrades the lexical arm: false, searchWithFTS5 searches originalQuery and uses expansions only as a fallback when the original returns nothing. That expansion replaces the original query: false for the same reason, and the keyword boost also uses originalQuery. A third guess was not attempted. The path is now instrumented instead.

    executeFullRetrievalPipeline takes a trace, and queryWithAudit holds the collector for the query’s duration so all nine agentic call sites inherit it without a signature change. That property is diagnostic-only and not concurrency-safe: two overlapping traced queries would share a collector. The harness runs one case at a time and production passes nil, so it cannot arise today; a second concurrent traced query would need a task-local instead.

    Also confirmed by the same table, and already an open row: fusion ranks below the lexical arm it is fusing, 0.714 against 0.857. The keyword arm alone would have retrieved more than the hybrid did.

    [evidence_level: measured, confidence: exact_for_the_gap, cause_unattributed_pending_instrumented_run, evidence_source: BenchmarkRuns/paired-retry stage_summaries, 8 cases, pool_limit 10, qasper_external_v1]

  7. Retrieval is nondeterministic, which subordinates every measurement in this document (added 2026-09-01): two runs of one build return different retrieved evidence, and different answers, for the same question against the same library.

    That does not invalidate the per-stage machinery in item 12 — the stages, the metrics and the attribution logic are all sound. It invalidates comparison. Every figure above that contrasts one thing with another assumes the only difference between two runs is the change under test, and right now that assumption does not hold. Item 19’s Standard-versus-Deep-Think table, item 16’s arm inversion, and any future A/B over fusion weights, reranking or contextual retrieval all inherit the problem.

    The practical rule until this is fixed: treat a single run as an observation, never as a measurement, and do not attribute a difference between two runs to a code change without establishing that the same run repeats. Item 12’s warning about attribution was about which stage lost recall; this is a level beneath it, about whether the numbers are stable enough to attribute at all.

    The cause is upstream of tie-breaking and is not yet identified. Tracked as a Future Backlog row at High priority because it gates every other retrieval row, including the embedding migration and contextual retrieval. [evidence_level: observed, confidence: high, evidence_source: repeated runs of a single build returning different retrieved evidence and different answers for one question] (2026-10-17: “upstream of tie-breaking” was written when seven sorts already broke ties by name and place (byScoreThenId in four places and stableTieBreakKey in two more in HybridSearchService, and fusion’s keyword ranking in RAGEngine); eight others ordered by score alone, and at these stages a passage’s name is often empty or “Unknown”, so ties between documents remained even where the key was used. See the next paragraph.)

    2026-10-17, for 5.7: every score sort in retrieval breaks ties the same way. Eight sorts ordered passages by score alone, so passages with equal scores kept whatever order they arrived in: the keyword boost, the structure boost and the scoped search (HybridSearchService, now byScoreThenId), and the heuristic rerank, the brute-force vector search, reciprocal rank fusion and both cross-encoder sorts (RAGEngine, by HybridSearchService.stableTieBreakKey). The key is the document’s name, the passage’s place in it and, from that day, the document’s id: vector hits carry no name and keyword-only hits are named “Unknown” at these stages, and keyword results reach fusion in a dictionary’s order, so passages of two documents at the same place still tied without it. The brute-force search sorts by document id and place. The cross-encoder’s penalties floor scores at 0.01, so ties there are common right at the cut. This removes one known source of the difference between runs; whether two runs of one build now agree has not been measured (a greedy A/A pair on the Mac closes the row). Since 2026-10-18 both helpers are nonisolated: RAGEngine calls stableTieBreakKey off the main actor, and the module’s default isolation made each call a main-actor call from a nonisolated context (ten warnings, two at each of five sorts, in the demo copy’s Release build of b19b9b5), and the same build warned three times at three sorts inside HybridSearchService, “call to main actor-isolated static method ‘byScoreThenId’ in a synchronous nonisolated context”; there are none of either in the test build of the next commit’s tree. [evidence_level: measured (that build log: 10 and 3 warnings), build_verified (0 of either, 2026-10-18), confidence: high, evidence_source: HybridSearchService.swift, RAGEngine.swift] [evidence_level: test_verified (HybridSearchServiceTests: tied passages, named and unnamed, come out in one order whatever order they went in), not yet measured as A/A, confidence: medium, evidence_source: HybridSearchService.swift, RAGEngine.swift]

  8. The cross-encoder reranker is real and lives in Orchestration, not Retrieval (added 2026-09-01): stated here because a search scoped to Services/RAG/Retrieval/ finds only heuristic scoring and reads as though no reranker exists. It does. RAGEngine.swift:82 loads ReRankerModel at launch and rerankWithCrossEncoder runs at :354 whenever the model and its tokenizer are both present, falling back when either is missing to the heuristic scorer in the same function, not to fusion order: each candidate’s incoming score plus keyword, proximity, chunk-position and metadata boosts, minus the TOC, question-bank and bibliography penalties, with a Cross-encoder unavailable ... heuristic fallback warning logged every time. Corrected 2026-09-24: this sentence said “falling through to fusion order”, which contradicted the flow diagram and item 7. [evidence_level: code_verified, confidence: exact, evidence_source: RAGEngine.swift:354, :366-370, :372-400] It is bound to ibm-granite/granite-embedding-reranker-english-r2 (since 5.8; cross-encoder/ms-marco-TinyBERT-L2-v2 through 5.7) by exact path in THIRD_PARTY_NOTICES.md.

    The flow diagram in section 2 has always shown this correctly as “Core ML TinyBERT Reranker / Heuristic Fallback”. An audit on 2026-08-29 nonetheless concluded the reranker was absent, and that claim reached a published document and a roadmap row before Docs/ai/ARCHITECTURE.md contradicted it. [evidence_level: code_verified, confidence: exact, evidence_source: RAGEngine.swift:82 and :354; Resources/MLModels/ReRankerModel.mlpackage]

  9. The extractive lookup may answer only when the question asks for the kind of value it found (added 2026-09-26): SpecificationExtractor.extract runs the specification patterns over the retrieved chunks and scores every value they match. RAGService.highPrecisionLookupOverrideAnswer makes a span scoring at or above EvidenceScoringPolicyService.precisionLockThreshold (0.82, or 0.90 for a forced attempt) the answer. Deep Think and Maximum return that span before any session or model call when the precision lookup locks one, and a span locked after a Deep Think or Maximum run replaces that run’s answer. Standard swaps it in after generation (from 5.7 only when the model’s answer does not already state the value), and the source-only check declines any source-locked answer (RAGService+AnswerFinishing.swift:452). A span returned before the run is checked by no gate and reads Not Checked. (Corrected 2026-10-20: this left out the replacement after a run, and cited lines from before 5.7.) Through 5.6 nothing downstream asked whether the span answered the question, so the extractor’s own scoring was the only defence against a wrong lock. From 5.7 the value also has to share a word with the question’s subject (RAGService.extractedValueAnswersQuestion, see “An extracted value has to be about the question” below). [evidence_level: code_verified, confidence: high, evidence_source: RAGService+AnswerFinishing.swift highPrecisionLookupOverrideAnswer at :1164, which calls highPrecisionLookupOverride at :1175, at the live call sites RAGService+AgenticQuery.swift:251 and :280 (the precision lookup), RAGService+AgenticQuery.swift:757 (after a run) and RAGService+Generation.swift:100 (the reliability fallback), and highPrecisionLookupOverride called directly at RAGService+QueryAnswer.swift:210 (Standard), RAGService+AgenticQuery.swift:124-134 (lookups and precise values, not conceptual questions), RAGService+AgenticQuery.swift:163 (gating decision agentic_precision_extractive_override), RAGService+AgenticQuery.swift:175-181 (its source-only call is source-locked and declines), RAGService+AnswerFinishing.swift:1220-1221 (extractedValueAnswersQuestion), the call at RAGService+QueryPipeline.swift:564 is inside a commented-out block, EvidenceScoringPolicyService.swift:82-84, VerificationOutcome.swift:33-60]

    What a device showed. On 2026-09-25, on the macOS Debug build of 5.4 in Deep Think, “How much notice do I have to give before I move out?” was answered “1 lb.” from an air fryer manual’s cooking chart, with a Verified badge; the lease in the same library says 60 days. Two scoring rules stacked. “Much” and “many” appended five volume words to any question, which made every “how much” question a measurement question and added +0.12 to every measurement in the evidence. The liquid-unit test matched ” l” as a substring, which “1 lb” contains, for another +0.22. Together they put “1 lb” at 0.85.

    Now. “Much” and “many” add the volume words only when the question also names an anchor the extractor already keeps for measurement answers: gas, gasoline, fuel, tank, capacity, volume, coolant or oil. “Capacity” and “hold” add them as before. The unit test compares the measurement’s unit exactly, so a pound is never a liquid unit, and “4.5L” written without a space now counts as one. PrecisionLockAnswerTypeTests pins both rules. [evidence_level: code_verified+measured, confidence: exact_for_these_runs, evidence_source: SpecificationExtractor.parseQueryEntities and measurementUnitMatchesQuery; the extractor built from source with Swift 6.4 on Linux fails 3 of the 4 tests before the change and passes 4 of 4 after, Docs/AuditArtifacts/RAGArchitectureAudit_2026-09-26/swift/]

    What an 18-question probe shows about the rest of the lookup. The change removes three wrong locks and gains one right one. It costs one: “How much water does the reservoir take?” now goes to the model instead of locking the right “1.2 L”, because “water” is not an anchor. It leaves three wrong locks exactly as they were. On a car manual’s capacities list, both fuel-tank questions and the coolant question lock the engine oil’s figure. When several volumes in one passage score the same, the extractor treats them as variants of one specification and keeps the first. No unit or keyword rule can say which value belongs to which label, and that is the roadmap row on regex extraction in the answer seat. [evidence_level: measured, confidence: exact_for_these_runs, evidence_source: swift/probe_before.txt and probe_after.txt in that folder, over fictional passages; not run against real ingested chunks]

    Not device-verified. None of this has been built in Xcode, run on the iOS 27 simulator, or re-asked on a device.

    Since then (note of 2026-09-29). On 2026-09-26 the notice question, re-asked on a macOS Debug build of the fix, no longer locked “1 lb.”, but Deep Think answered “at least 30 days” where the lease says 60 and still marked it Verified, so the roadmap row stays open. On 2026-09-28 the full iOS 27 simulator suite passed with the four regression tests (504 tests, 0 failures), and Xcode Cloud built it as 5.5, builds 480 and 481, for TestFlight on iOS and macOS. No iPhone has re-asked it. [evidence_level: test_verified, confidence: high, evidence_source: CHANGELOG.md ## 5.5 Fixed, Retrieval entry; Docs/ai/STATE.md verification of 2026-09-28; App Store Connect build list read 2026-09-28]

The query cache (corrected 2026-10-14, for 5.7)

A Standard question that is not limited to named documents first looks for its own earlier search in semantic_query_cache (SQLiteFullTextService.getCachedQuery), and a hit skips query rewriting, the embedding and the hybrid search. After a live search the passages are written back (cacheQuery).

From 5.7 the key is the normalized question followed by a digest of every library’s documents with their passage counts, of the embedding model and its dimension, and of the app’s version and build (RAGService.queryCacheKey), so an import, a deletion, a re-import, a change of model or an update gives the question a fresh search, and a hit is used only when every passage in it is from a document still in the libraries (cachedResultsAreCurrent). There is no lookup by similarity any more. Through 5.6 the key was the question and the library’s id alone, nothing cleared an entry, and a question within 0.95 cosine of a cached one also took its passages (getCachedQueryBySimilarity, which decodes every cached question’s embedding for the library each time): a question asked again after an import left the new document out, and after a deletion it brought the deleted document’s text back into the answer. [evidence_level: code_verified+test_verified (QueryCacheKeyTests), confidence: high, evidence_source: RAGService.swift queryInternal; SQLiteFullTextService.swift]

From 5.7 the rows are also deleted once no question can reach them (added 2026-10-15). Every key names every library’s documents and the app’s version, so after any import, deletion, re-import or update no older row can be found again. Through 5.6 nothing deleted a row, and the rows kept the questions and the passages found, a deleted library’s text among them, until the app was removed. After the documents are saved and at launch, RAGService.deleteUnreachableCachedSearches compares a digest of the documents with their passage counts and the app’s version (queryCacheDocumentsToken) with the one recorded in the app’s settings at the last deletion, and when they differ deletes every row (SQLiteFullTextService.deleteAllCachedQueries, added at the owner’s word; the table is unchanged). A change of a library’s embedding model alone leaves that library’s rows unreachable until the documents next change. [evidence_level: test_verified (QueryCacheClearingTests), confidence: high]

An extracted value has to be about the question (added 2026-10-15, for 5.7)

SpecificationExtractor can lock a value as the answer at five places (highPrecisionLookupOverride is the one function they share, four of them through its wrapper highPrecisionLookupOverrideAnswer and the Standard path directly), on an additive score of 0.82 and nothing else. From 5.7 the lock also needs RAGService.extractedValueAnswersQuestion: the value’s label, or the sentence of the line that holds it (RAGService.valueLine), shares a word with the question’s subject, which is its words of three letters or more with genericValueWords (question words, quantity words, units, function words and question verbs) taken out and the extractor’s synonyms added (SpecificationExtractor.subjectSynonyms). Corrected 2026-10-15: the first version read the extractor’s surroundingContext, which for a value with a decimal point was 50 characters each side and reached the neighbouring lines, so the 2026-09-26 probe’s wrong locks still passed; the tests now run the real extractor on the probe’s passages. A plural’s final “s” is dropped only from a word of five letters or more and not after “ss”, “us” or “is” (lookupWords), so “does” and “this” stay whole and come out as question words (corrected 2026-10-15: folding them to “doe” and “thi” left them as the subject, and every value failed). A question with no subject words left is not checked. A value that fails is not used and the model answers. The answer-type and answerability checks the roadmap row also asks for are not built. [evidence_level: test_verified (ExtractedValueRelevanceTests), confidence: medium] (Corrected 2026-10-20: the shared function was named as highPrecisionLookupOverrideAnswer, which is the wrapper.) [evidence_level: code_verified, confidence: exact, evidence_source: RAGService+AnswerFinishing.swift:1164-1171 (the wrapper), RAGService+AnswerFinishing.swift:1175 (highPrecisionLookupOverride), RAGService+AnswerFinishing.swift:1218-1230 (the subject check), RAGService+QueryAnswer.swift:210 (the Standard path's call)] [evidence_level: grep_verified, confidence: high, evidence_source: grep -n "highPrecisionLookupOverrideAnswer(\|highPrecisionLookupOverride(" RAGService.swift, 2026-10-20: the wrapper is called at RAGService+AgenticQuery.swift:251, RAGService+AgenticQuery.swift:280, RAGService+AgenticQuery.swift:757 and RAGService+Generation.swift:100, the function itself at RAGService+QueryAnswer.swift:210, and one more call sits in a commented-out block at :4164]

Updated 2026-10-20. Of the five places, two follow an answer the model has already written: the Standard path, after generation, and the end of a Deep Think or Maximum run, where a locked value still replaces the run’s answer outright. On the Standard path the model’s answer now stands when it already states the extracted value (RAGService.answerStates(value:in:), case and spacing ignored), and it is not marked source-locked, so the source-only pass can still run if the gates fail. When an extracted value does stand in and has no label, the answer is the whole sentence that holds it (valueLine, with a table row read as a sentence by readableValueLine) rather than the extractor’s window of characters around it, unless the value is a measurement or the question asks for a precise value, which are answered with the value itself. On 2026-10-19 that window turned “What is the harbor lantern code, and where is it kept?” into “e harbor the crew measured 3163 units and noted the weather as foggy. the final instruction: the harbor lantern code is VERMILION-4417”, which began mid-word and left out ”, kept in the blue ledger”. The other three places, the precision lookup’s two, which answer before any Deep Think or Maximum model call, and Standard’s reliability fallback, which answers when generation failed or came back empty, have no model answer to compare, so the test for a value already stated is the Standard path’s alone. The whole sentence and the subject check above are in the function all five share, so they hold at every place. (Corrected 2026-10-20: this said one place follows the model’s answer, that the four places that answer before it runs are unchanged, and that every value with no label is answered with its whole sentence.) [evidence_level: measured (iOS 27.0 Simulator trace and screenshot, 2026-10-19), test_verified (ExtractedValueOverrideTests, five tests, in a full run of 1,264 on 2026-10-20), code_verified for the five places (2026-10-20), confidence: high, evidence_source: RAGService+AnswerFinishing.swift highPrecisionLookupOverride, answerStates, readableValueLine, and RAGService+AgenticQuery.swift:251 and :280 (the precision lookup), RAGService+AgenticQuery.swift:757-761 (after a Deep Think or Maximum run), RAGService+QueryAnswer.swift:210-228 (Standard), RAGService+QueryAnswer.swift:222 (answerStates), RAGService+Generation.swift:86-104 (the reliability fallback, buildReliabilityFallbackResponse, reached on an empty or a failed generation), RAGService+AnswerFinishing.swift:1175, :1218-1230 (the subject check), :1234-1249 (a label, then a precise value or a measurement, then the whole sentence)] [evidence_level: grep_verified, confidence: high, evidence_source: grep -rn "answerStates(" OpenIntelligence OpenIntelligenceTests --include='*.swift', 2026-10-20: one call in the app, RAGService+QueryAnswer.swift:222, and the rest in ExtractedValueOverrideTests]

Updated 2026-11-13. highPrecisionLookupOverride declines a value made only of numbers the question itself gives (RAGService.valueRestatesQuestion): asked about a row “under ADHD (n = 3026)”, it had locked onto “3026” on a misread figure caption and returned that line as Deep Think’s whole answer. [evidence_level: code_verified, confidence: high]

Count questions, 2026-11-13. A question with “how many” no longer tells the model to state a count first: it gives a stated number as the excerpt writes it, each group’s number where excerpts differ, never a sum, and counts only items listed without a number. After the responsiveness step a headline number no passage states as a count of something the question names, and that is not the count of the list under it, is removed (RAGService+AnswerHeadline.swift).

Also 2026-11-13. Two switches in RetrievalAblation for paired runs: answerFirst (Deep Think’s and Maximum’s final step open with a one-sentence answer and never call a given value missing) and maximumSourcePassages (Maximum’s final step reads the top retrieved passages beside its claim bank). EvidenceReachBenchmarkTests asks Maximum answers on path m. A third, draftAbstainNeedsFailedClaim, lets the source-only check’s drafting model flag an answer Needs Verification only when one of its claims failed its check (SourceOnlyAnswerService). [evidence_level: code_verified, confidence: high]

MMR and the passages’ vectors (corrected 2026-10-14, for 5.7)

The vector store hands passages back without their vectors (BNNSVectorDatabase keeps the vectors in their own file), and two steps after it compared the vectors on the passages they were handed:

  • MMR (RAGEngine.applyMMR) measured how alike two passages are with those vectors, so every pair measured 0 alike and MMR kept passages in score order whatever λ said, in every answer mode, in 5.6 and before. From 5.7 the two callers fetch the candidates’ vectors by id (RAGService.storedVectors, getEmbeddings(forChunkIDs:)) and pass them in, so a passage that is nearly the same as one already picked gives way to a different one when their scores are close. Every figure in BenchmarkRuns/LEDGER.md was measured with MMR doing nothing; a paired run is the measure of the change. Each answer mode’s own λ (RAGQualityMode.mmrLambda: 0.60, 0.55 and 0.50) is read nowhere that runs. Its one reader is the featureToggles dictionary, which nothing calls. Both MMR call sites use the library’s λ (RetrievalConfig.mmrLambda, 0.6 by default), and Standard raises it to at least 0.85 for a step-by-step question. (Corrected 2026-10-20: this said the mode’s λ is read only by logging.)
  • Lexical recall (HybridSearchService.lexicalRecallCandidates), the path a search takes when no library has keyword rows, skipped every passage without a vector of the question’s length and so kept none. From 5.7 it scores the passages it matches with similarities(to:forChunkIDs:).

Sentence selection (stage 13 above) also dropped every two-character word from the question and from each line, so a code such as “E1” never counted as a keyword. From 5.7 a two-character word that mixes letters and digits counts (RAGService.isEvidenceKeyword). [evidence_level: code_verified+test_verified (MMRStoredVectorsTests), confidence: high for the mechanism; no benchmark run yet, evidence_source: RAGEngine.swift applyMMR; RAGService.swift storedVectors, isEvidenceKeyword; HybridSearchService.swift, and for the modes' λ (code_verified 2026-10-20) RAGQualityMode.swift:210-217, :315-320, RAGService+QueryRetrieval.swift:906, RAGService+QueryRetrieval.swift:935, RAGService+ToolHandler.swift:338-345, KnowledgeContainer.swift:266-272] [evidence_level: grep_verified, confidence: high, evidence_source: grep -rn "mmrLambda\|featureToggles" OpenIntelligence OpenIntelligenceTests --include='*.swift', 2026-10-20: RAGQualityMode's mmrLambda is read only at RAGQualityMode.swift:320, inside featureToggles, whose only hit is its definition, and every other mmrLambda read is of a library's RetrievalConfig, a tuning report or a local]

Retrieval’s best passage reaches the model (added 2026-10-15, for 5.7)

The answer path reranks the candidates and then, for a lookup question, packs the compact context from sentences that repeat the question’s words, ordered by EvidenceScoringPolicyService.extractivePriorityScore. A passage can answer a question without repeating its words, so that step could drop the passage the reranker ranked first: on 2026-10-15 “What happened on September 30” ranked the September 30 section of a monthly log first and answered from September 24 and 25. From 5.7 the reranked first passage stays first in the extractive ordering (RAGService.leadingWithTopRanked) and goes into the context whole as [S1] (extractRelevantSentences(leadingCandidateWhole:)), at most three fifths of the budget and cut at a line or sentence end; chosen sentences from the other passages fill the rest. Not measured on the retrieval benchmark: the lookup cases there, which the extractive ordering was tuned on, may move. [evidence_level: test_verified (LeadingEvidenceTests), confidence: medium] This holds for the first pass. When the plan keeps the answer on the device and the context does not fit there (usually because the first pass was sized for Private Cloud Compute), or consent is refused, Standard packs the context again with RAGEngine.assembleContext, a lookup’s too: the best passage still comes first, cut at a sentence to about a third of the budget when it is long, and the chosen sentences of the other passages are not added (item 13). (Added 2026-10-20.) [evidence_level: code_verified, confidence: exact, evidence_source: RAGService+QueryContext.swift:1282 (best passage first), RAGService+QueryRetrieval.swift:170-229 (the two re-packs), RAGEngine.swift:500-504, :556-560, :612-622]

Passages whose document is in no library (added 2026-10-15, for 5.7)

A passage stays in its library’s vector store after its document is deleted only if something went wrong, and an import that stops before recording its document leaves its passages too. Searches kept such passages: queryInternal named them “Unknown” and searchDocumentsRaw (Deep Think, Maximum, the model’s search tool) named them by getDocumentName. Both leave them out now, before naming and before the no-sources check, so a search left with nothing answers as one that found nothing (RAGService.passagesFromKnownDocuments). A passage the pipeline made itself, such as a camera or barcode capture, carries its own name and is kept. The launch cleanup that removes them from Local Only libraries is in Docs/INGESTION_PIPELINE.md, “Text left by an unfinished import”. [evidence_level: test_verified (PassagesFromKnownDocumentsTests), confidence: high]

Updated 2026-10-19. Two more ways in, both found by tests in the iOS 27 Simulator that day. (1) A source, and a regression of 5.7’s own: since the import’s model work moved after the save (2026-10-15), a document deleted while its overview was written (enrichImportedDocument, up to 12 s) kept that overview as a passage of no document. The overview and the suggested questions are now stored only for a listed document, which includes one in its undo seconds, and taken out again when the deletion began while they were stored (enrichmentTargetState). (2) The steps after the first check. In Standard the retrieval cascade and the keyword retry named such a passage “Unknown” and kept it, and the steps that read the library’s whole passage list (the spec sniper, cross-references, parent expansion, lexical recall with cachedChunks) added it back; on 2026-10-19 the spec sniper put a planted one into an answer. Deep Think and Maximum search through executeFullRetrievalPipeline, not searchDocumentsRaw as the paragraph above had it, and that path had no check. Now the cascade and the retry apply passagesFromKnownDocuments, the passage list both paths read leaves such passages out (chunksOfListedDocuments, which logs how many and keeps the store’s array when there are none), executeFullRetrievalPipeline checks its result before naming the passages, and semanticSearch (the Semantic Search screen’s) checks before it cuts its list to the number asked for. They stay stored in an iCloud library and in one with no documents, as before. [evidence_level: test_verified (DeletedDuringEnrichmentTests; ShortcutsActionTests.testAPassageOfNoDocumentIsNotUsedInAnAnswer, a planted overview asked in Standard twice and in Deep Think), confidence: high for the paths named]

A day written as an ordinal (added 2026-10-15, for 5.7)

The keyword index (FTS5) keeps “30th” as one word, so a question that writes a day as an ordinal shares no word with notes that write it as a number. On 2026-10-15 “What happened on September 30th” found only “september” (in over 40% of the passages of that library, so discounted) and “happened”, and the reranker put September 1’s passage first; asked as “September 30”, the same question put that day’s section first. DayOrdinals.cardinal reads a day number with st, nd, rd or th beside a month name as the number, in queryInternal before anything else uses the question, and in executeFullRetrievalPipeline for the searches Deep Think and Maximum make. Nothing else in the question changes. A document that itself writes “30th” no longer matches that keyword and is left to the vector search and the reranker [inferred]. [evidence_level: test_verified (DayOrdinalsTests), confidence: medium]

Graph packing is sized to the device’s window (added 2026-10-19, for 5.7)

For an investigate, compare or findings question (AnswerIntent.benefitsFromMultiHop), ContextPackingService.pack gathers the retrieved passages plus their parents, neighbours and cross-references up to a token budget, and returns them in document order with the added passages scored 0.3. RAGEngine.assembleContext then fills the model’s window in that order, after a lost-in-the-middle reorder that assumes relevance order. (Corrected 2026-11-04: both halves of that were 5.6 and the 5.7 work before 2026-10-20. ContextPackingService.pack now returns priority order, core passages by rank, then parents, neighbours and references, and assembleContext packs the passages in that order first and applies the Lost-in-the-Middle placement only to the passages that fit, when four or more did (packed.count >= 4). [evidence_level: code_verified, confidence: exact, evidence_source: RAGEngine.swift:495-501, :683-691, :785-786; ContextPackingService.swift:267-271]) The budget was 6,000 tokens on every device. On the owner’s iPhone 16 Pro Max the window is 4,096 tokens and the Standard path had 3,190 left for passages (4,470 characters, its own trace of 2026-10-16, which a comment in the code quotes), so about half of a 6,000-token pack was cut by position, not relevance, and a top passage late in a document could lose to a neighbour early in it. The trace’s base=4096 is the device’s window, which the budget starts from only when the first pass is not sized for the cloud, so in that run the cloud route was not eligible or Private Cloud Compute had been refused. When the cloud route is eligible (Apple’s model answering, online, Private Cloud Compute allowed by the model picker, not On-Device, consent not denied), Standard asks for at least 50 candidates and sizes its first pass for a 32,768-token window instead, up to 45,000 characters (never in the Simulator), and packs again for the device when the plan keeps the answer there and the context does not fit, or consent is refused. The candidates are capped by an adaptive limit: 75 on an A17 Pro, 100 on an A18, 150 on an A19 and 250 on an M-series chip, half that under critical memory pressure and 10 when the device is critically hot, and also 10 for a question of three words or fewer with no question mark, and at most 50 for one the app rates complex (several parts, more than 25 words, or a word such as “compare” or “why”) while the device is under no pressure. The battery level, Low Power Mode and heat short of critical change none of this, and only Standard’s path reads the limit. (Added 2026-10-20.) [evidence_level: code_verified, confidence: exact, evidence_source: RAGService+QueryPreparation.swift:149-156 (the cap), RAGService+QueryPipeline.swift:482-495, RAGService+QueryPipeline.swift:511-516 (the base window), RAGService+QueryContext.swift:1198-1200 (the log line that prints base), RAGService+QueryRetrieval.swift:170-229, QueryRuntimeCoordinator.swift:150-158, :232-233, FoundationModelTokenBudget.swift:28-40, AdaptivePipelineOptimizer.swift:56-70 (the level: critical heat, then critical memory pressure), :115-192 (the caps by chip), :205-262 (half, or 10), :324-370 (by question), :583-587 (a memory warning), :606-631 (how a question is rated)] [evidence_level: grep_verified, confidence: high, evidence_source: 2026-10-20, grep -n "adaptiveConfig" RAGService.swift AgenticOrchestrator.swift (read at RAGService+QueryPreparation.swift:45, RAGService+QueryPreparation.swift:151, RAGService+QueryRetrieval.swift:996 and RAGService+QueryContext.swift:1085, all on Standard's path) and grep -rn "isLowPowerModeEnabled" OpenIntelligence --include='*.swift' (no hit in AdaptivePipelineOptimizer.swift or RAGService.swift)] [evidence_level: artifact_derived, confidence: high, evidence_source: the 2026-10-16 iPhone 16 Pro Max log line quoted at RAGService+SentenceExtraction.swift:86-92] [evidence_level: inferred, confidence: medium, evidence_source: base=4096 in that line, read against RAGService+QueryPipeline.swift:495 and RAGService+QueryPipeline.swift:511-516 as they are now] RAGService.graphPackingTokenBudget(windowTokens:) now gives three quarters of the window the device reports, at most 6,000 (unchanged at 8,192 and above) and at least 1,500; a device that reports no window counts as 4,096. Private Cloud Compute’s 24,000 is unchanged. Not measured on a benchmark or a device: a long question or a long transcript can still leave less than the pack, and the document order itself is unchanged. [evidence_level: code_verified for the order and the fill, measured for the iPhone's window and budget, test_verified for the arithmetic (GraphPackingBudgetTests), confidence: medium]

A table’s passage is read as its rows (added 2026-10-19, for 5.7)

A passage carries two texts: content, which is embedded and indexed, and parentContent. For prose the parent is the window around the passage (SemanticChunker.buildParentContent), which holds it, and the answer path reads the window for more context. For a structured passage (a table, a list, rows of key values) DocumentProcessor stores the section’s title there instead (parentText: currentSectionTitle). Every reader took parentContent ?? content, 57 of them across context assembly, reranking penalties, the verification gates, Check a Claim, Fact Check and Deep Think, so a table’s passage was read as its title. On 2026-10-19, in the iOS 27.0 Simulator, “Where is VERMILION-4417 kept?” ranked the last page of a 60-page PDF first and the model was given “Field log page 60 of 60: the harbor survey.” and not the row under it, and it answered that no document mentioned the code. In that library 238 of 520 passages were table passages whose parent was their title. Readers now use DocumentChunk.contextText: the parent when it is at least as long as the passage and holds the passage’s first line (compared without whitespace, past any “[Part N]” label), the passage otherwise. Nothing stored changes. SpecificationExtractor.extractionContent already read such passages as the text plus the title and is unchanged. [evidence_level: measured (Simulator trace, store metadata), test_verified (TablePassageContextTests, six tests, in a full run of 1,259 on 2026-10-19), confidence: high, evidence_source: DocumentChunk.swift, DocumentProcessor.swift, RAGService.swift extractRelevantSentences]

A library’s search settings (added 2026-10-20, for 5.7)

Library Settings has a Search section: a style (Automatic, Balanced, High Precision, Technical Manual, Exploratory) and, under Advanced Controls, the four values a style sets. Through 5.6 the styles were not shown and the controls sat behind SettingsStore.developerRAGTuningEnabled, which the app forces off, so a library’s values were whatever it was created with or tuning after an import picked (RAGService.autoTuneRetrievalConfigIfNeeded). What each value does, read from the code on 2026-10-20:

  • Minimum score (RetrievalConfig.minSimilarity). Standard’s filter starts from it (RetrievalPolicyService.filteringDecision), and compares it with the re-ranker’s score, which is relative: the cross-encoder’s scores for one question are rescaled to 0.1 to 0.9 and then lowered for tables of contents, question banks and bibliographies (RAGEngine.rerank), so the best passage found scores high whether or not it matches well. Left to the app, the starting point is the lower of the library’s minimum and the mode’s (0.28, 0.25, 0.20), so a minimum between the mode’s and 0.45 did nothing; chosen by the person, it is used as given. The filter then lowers it for a question whose best five passages average below it, for a lookup or keyword question (by 0.04, and by that rule to no lower than 0.18) and when the words don’t match (vocabularyMismatch), raises it to 0.25 for a step-by-step question, and with nothing over it the answer is written from the closest passages (the acceptance override, or reliabilityModeEnabled). Find Passages applies the same minimum to the hybrid search’s score, before any re-ranking, and never lowers it (searchDocumentsRaw), so a high minimum can leave it with nothing. Deep Think and Maximum pass their own minimum to every search they make: 0.05 for the first searches that build the pool, 0.03 for the precision lookup, and RetrievalPolicyService.agenticMinSimilarity for the rest. (Corrected 2026-10-20: this named agenticMinSimilarity for every search, and two pass a fixed minimum.) [evidence_level: code_verified, confidence: exact, evidence_source: AgenticOrchestrator+Retrieval.swift:1075 (in executeMultiQuerySearch), RAGService+AgenticQuery.swift:270, and AgenticOrchestrator.swift:938, AgenticOrchestrator+Retrieval.swift:732, AgenticOrchestrator+Retrieval.swift:1351, AgenticOrchestrator+Retrieval.swift:1469, AgenticOrchestrator+Maximum.swift:767, AgenticOrchestrator+MultiChain.swift:32, AgenticOrchestrator+MultiChain.swift:53] [evidence_level: grep_verified, confidence: high, evidence_source: grep -rn "executeFullRetrievalPipeline(" OpenIntelligence --include='*.swift', 2026-10-20: nine calls, each passing minSimilarity]
  • Passages Needed (minConfidentChunks). With 2 or more, a Standard answer is written only when that many of the passages it would use score at or over the minimum the person set, each at the score the re-ranker gave it (a rescued table passage is raised to just under the best score so it reaches the answer, and is not counted at that); otherwise the reply lists the closest three with their scores and how many made it (strongPassageShortfall, gating decision high_accuracy_blocked). Through 5.6 this acted only when the library’s values equalled the High Accuracy preset exactly, and the acceptance override, set for nearly any library of ten or more passages, skipped it there, as did the lenient switch, which the app keeps off.
  • Exact words and meaning (vectorWeight, lexicalWeight), in every mode. Each question moves them by up to 10 points by its kind, and each then stays between 35% and 65% (QueryProfile.adjustedHybridWeights), so High Precision’s 75% acts as 65% on every question. The slider shows the stored value from 35% to 75%.
  • Variety (mmrLambda), in every mode; Standard keeps at least 0.85 for a step-by-step question.
  • Citations are not a setting: every mode asks the model for them (RAGQualityMode.requiresCitations), though an answer can still come back without one (with streaming or the reliability fallback on, RAGService), so the old “Require Explicit Citations” switch changed nothing and is not shown. Its stored field, requireExplicitCitations, now records that the person chose the values (RetrievalConfig.isChosenByPerson): the two places that read it already treated it so (the filter’s starting point above, and tuning after an import, which leaves such a library alone, RetrievalConfig.autoTuned). The library’s saved form and its iCloud copy are unchanged.

A library reads as Automatic while tuning after an import may still change it; one at a minimum of 0.5 or more that nobody chose (an older library’s strict mode) reads as Custom, since tuning never changes it. A style is shown when the values equal its preset (within 0.001), so a moved slider reads as Custom; High Precision’s 0.2 temperature cap follows the same test (style == .precise). Moving a control marks the library as chosen only when the value changes, so opening Advanced Controls changes nothing. Automatic picks Technical Manual when more than half of a library’s documents are code, or at least two thirds (and two or more) are spreadsheets, and Balanced otherwise (RetrievalConfig.recommended). [evidence_level: code_verified; test_verified (RetrievalStyleTests); measured (a robot run in the iOS 27.0 Simulator, 2026-10-20: a minimum of 70% with Passages Needed at 5 held back "How much is the pet deposit?"); confidence: high for the mechanism, not measured on the retrieval benchmark]

Suggested questions

Services/Query/UX/SuggestedQuestionsService.swift writes the chips on the Chat tab’s empty state. Since the 2026-11-05 split (5.8, a pure move) the actor’s steps sit in files of their own: +Bank (the persisted bank and the public API), +ChunkSelection (step 1), +LLMGeneration (step 2), +ContentFallback (steps 3 and 4) and +Grounding; scripts/split_linemap.py SuggestedQuestionsService.swift:N finds where an older line went. It is retrieval-adjacent rather than retrieval: it reads chunks, never the index. Three paths, in the order they are tried:

  1. The curated set, for a library made only of sample documents. Hand-written questions that skip generation. Since 2026-09-14 a sample is recognised by identity (sampleIdentity(for:)): each canonical sample filename or a numbered copy of it (<stem>-<digits>.md, which managed storage produces when a filename collides). Before that the check was an exact filename match and one numbered copy switched the curated set off. Updated 2026-10-20, for 5.7: curatedSampleQuestions(forFilenames:) returns the questions of the samples the library holds, in the list’s order, where isSampleWorkspace(filenames:) required all three; any other document returns nil and the library goes to path 2. The chat screen shows the list from the first frame (ChatScreen.curatedSampleQuestions), and its own older list of four, which came first whenever the architecture and Private Cloud Compute guides were both there under their own names, and so kept this set off the screen unless one of them was a numbered copy, is gone. Each question names the heading of its guide it was written from, pinned by SampleQuestionHeadingsTests.
  2. Model-written questions, per document, at ingestion for user documents and on demand for each document with none in the bank (until 5.8, only for a library whose bank was empty; see “An interrupted run saves nothing” below). Accepted only when at least two survive passage grounding.
  3. Template questions from specifications, procedures, warnings, definitions and terms. Since 2026-09-14 these fail closed when the model is present: an empty model result shows no question rather than a raw template. They still ship on a device with no model, and a bank that holds no model-written question is rebuilt once per launch when the model becomes available (shouldRebuildBank), because the sample library’s bank is built on first launch, when Apple Intelligence is least likely to be ready, and used to be kept forever. isAcceptableTemplateTopic rejects lone abstract words, article-plus-word phrases and runs of the document’s own title, and the research templates key on documentCategory == .scientificPaper rather than the word “study” in a passage.

Sample duplicates are removed by SampleDocumentManager.removeNumberedSampleCopies, on every refresh pass and only when the canonical file is also present.

[evidence_level: code_verified, confidence: exact, evidence_source: SuggestedQuestionsService.swift sampleIdentity, curatedSampleQuestions(forFilenames:) (isSampleWorkspace(filenames:) until 2026-10-20), shouldRebuildBank, isAcceptableTemplateTopic, generateQuestionsForDocumentOnly; SampleDocumentManager.swift numberedCopiesToRemove; SuggestedQuestionsSampleWorkspaceTests and SampleDocumentCopyCleanupTests, 2026-09-14; SampleQuestionHeadingsTests and the rewritten SuggestedQuestionsSampleWorkspaceTests, 2026-10-20]

Development-build journal (added 2026-11-03)

Debug builds only (VerificationJournal). retrieval.graphPack records each graph pack’s tokens against its budget and the device’s window, whether it fit (withinBudget) and whether it was cut (truncated), with the core and added passage counts. vocabulary.learnedEntities records the Vision-detected entities a library learned, as counts by kind (address, phone number, email, date, other), never the values. [evidence_level: code_verified, confidence: high, evidence_source: RAGService.swift graph-packing block, ContainerVocabularyService.learnFromDetectedEntities]

An interrupted suggested-questions run saves nothing (added 2026-11-07, for 5.8)

On the owner’s iPhone on 2026-11-07 a two-file import showed “No grounded suggestions are ready yet” for a minute, and one file would never have had any: its import-time run was cancelled when iOS ended the import’s background task, six runs Chat started were cancelled by library and tab switches or timed out behind the other file’s deferred enrichment, and each was logged as “Model produced no grounded question” and saved as an empty result. Now a CancellationError, a timeout or a cancelled task marks the run interrupted (SuggestedQuestionsService.isInterruption, the interrupted: argument of generateDirectlyWithLLM and generateWithLLM), and generateQuestionsForDocumentOnly returns nil for an interrupted run that wrote nothing. Nil is never saved: the import path returns, and a rebuild of a template-only bank keeps the saved bank and is tried again. generateQuestions calls fillMissingQuestions, which writes questions for the documents with none in the bank (documentsNeedingQuestions) and merges them in: never one whose import is still adding its own (RAGService.deferredEnrichments, ingestionQuestionsInFlight), one Chat is already writing (questionFillInFlight), one with a finished run this launch (questionFillSettled), or one interrupted three times this launch; at most three per refresh, and only while the bank holds fewer than its 25 questions (maxBankSize). A finished run that wrote none still saves none, so the fail-closed rule above holds. Each file is read from its own passages (RAGService.passagesForSuggestedQuestions, up to 80 spread through it plus its overview, passed as passagesForDocument), not the library’s sample of about 50, which on 2026-11-07 held none usable of a file whose passages are mostly reference list. [evidence_level: device_log_proven for the failure (pipeline_trace and journal, ~/Library/Developer/OpenIntelligence-evidence/2026-11-07_180251), test_verified for the selection (SuggestedQuestionsFillTests), confidence: high; device check pending]

A value from another table row is not grounded (added 2026-11-08, for 5.8)

Gate C counted an answer’s number as grounded when the passages held it anywhere. VerificationGateService.valuesFromOtherRows reads the passages’ table rows (“ROW n:” lines and cell lines joined by ” | ”) and their labels (first cell); when some row’s label shares a word with the question’s subject, an answer value held only by rows about something else, and by no line outside a table, is counted as unverified. Questions no row is about, and values also stated in prose, are judged as before. A table none of whose row labels names the subject is read the other way round: each cell is about its row’s label and its column’s name (“ROW 2: Adults | DHA: 1-2 g” holds the DHA value), decided per table so that a column such as “Daily Dose” does not make every row about a question that says “daily dose” (2026-11-08). [evidence_level: test_verified, confidence: medium: the rate on correct answers is not measured]

A retrieved table summary brings its table (added 2026-11-07, for 5.8)

A table is stored as its full passage (tableStructural, split into parts when long), a summary (tableSemantic) and, for some tables, one passage per row, all sharing a siblingGroupId. Before this, a summary that ranked first went to the model alone, and summaries written before 5.8 list two rows. RAGService.withTablesForSummaries (RAGService+TablePassages.swift) replaces each summary candidate with its table (the tableStructural siblings before it in the document), at the summary’s place and score; a table already among the candidates is not added twice, and a summary whose table is not found stays. Standard runs it on the context candidates before orderContextCandidates; Deep Think and Maximum run it on every search’s result (performFullRetrievalPipeline and the raw search in RAGService+ToolHandler.swift). [evidence_level: test_verified (TablePassageFidelityTests), confidence: high; device answer not yet re-asked]

An answer has to answer its question (added 2026-11-07, for 5.8)

The grounding checks ask whether an answer is in the sources; AnswerResponsivenessJudge (Services/RAG/Safety/AnswerResponsivenessJudge.swift) asks whether it gives what the question asks for. One guided generation on the on-device model (ResponsivenessReview: what the question asks for, whether the answer says the sources do not have it, full/partial/none, what is missing). An answer with no letter or digit is none without the model; a failed or unavailable call returns no judgment and changes nothing.

  • Standard, step 6r (RAGService+AnswerResponsiveness.swift, ensureAnswerResponds, called right after the integrity repair in queryInternal): a partial or none answer is written once more with the judge’s note (ResponsivenessJudgment.regenerationNote), not streamed, and the rewrite is kept only when the judge rates it higher (responsivenessRank). The gates and the source-only check then run on whichever text is kept.
  • Standard’s precision override (step 6g): an extracted value the judge reads as none is set aside and the model’s answer stands, when that answer was not itself none.
  • Deep Think and Maximum: the precision lookup’s extracted value is judged before it skips the run; none falls through to the full run. The final answer is judged and labelled, not rewritten.
  • Every mode appends answers:<verdict> to the gating string; VerificationOutcome.from(gatingDecision:) reads answers:none as not supported, lets a Standard source-only rewrite judged full stand as supported after failed gates, and leaves the badge for partial (a warning names what is missing).

Measured on the Mac, four wordings over 160 cases, in BenchmarkRuns/2026-11-07-responsiveness/README.md. This is the answer-type check that the 2026-10-15 section above says was not built. All five places an extracted value can lock the answer judge it first and set it aside when it is none: Standard’s override (the model’s answer stands), the precision lookup’s two (the full run happens), the end of a Deep Think or Maximum run (the run’s own answer stands, RAGService+AgenticQuery.swift), and the reliability fallback (the passages answer instead, buildReliabilityFallbackResponse, whose gating string now carries the verdict). [evidence_level: measured (Mac probe), test_verified (AnswerResponsivenessJudgeTests, VerificationOutcomeTests), confidence: medium; the iPhone's verdicts not yet read]

The reranker, from 5.8 (added 2026-11-09)

The cross-encoder is IBM’s granite-embedding-reranker-english-r2 (149 million parameters, 22 layers, Apache 2.0), replacing cross-encoder/ms-marco-TinyBERT-L2-v2 (2 layers, 4.4 million). Two reasons, both measured or read:

  • Commercial use. TinyBERT was trained on MS MARCO, whose terms allow non-commercial research only; IBM’s card says the Granite reranker’s training data leaves MS MARCO out “due to its non-commercial license” (read 2026-11-09 at revision d09d3d69). Docs/ai/PURPOSE.md rule 1.
  • Accuracy. On 246 QASPER questions over 80 papers (3,912 passages, the app’s [file] [section path] prefix, hybrid vector and BM25 with reciprocal rank fusion at k=60, the top 50 reranked), the first gold passage reached the top 5 for 37.0% of questions with no reranker, 41.9% with TinyBERT and 48.8% with Granite; Granite against TinyBERT was better on 25 questions and worse on 8 (sign test p=0.005). The bundled Core ML file scores 48.4% (4-bit palettized, 2 questions better and 2 worse than 8-bit). BenchmarkRuns/2026-11-09-reranker-swap/.

How it runs (RAGEngine.setupReRanker, rerankWithCrossEncoder):

  • ReRankerModel.mlpackage holds two fixed-shape functions, len256 and len512, sharing one 76 MB weight file; each pair runs on the shorter function that holds it. Fixed shapes are what the Neural Engine runs (enumerated shapes were planned onto the CPU); 887 of 897 operations are planned there. Compute units are CPU and Neural Engine, which ran it faster than the GPU on an M3 Pro: 10.4 against 20.8 ms a pair at 256 tokens, 31.6 against 38.8 at 512. TinyBERT took 0.8 ms. The first load after an install compiles the model for the Neural Engine (about 50 s on the M3 Pro, 45.5 s on an iPhone 16 Pro Max by the journal’s reranker.load, 2026-11-09; 0.2 to 1.5 s from Core ML’s cache afterwards; RerankerSetupCard shows “Setting up search” on every tab meanwhile). Core ML finds that compiled form by the model’s location, so since 2026-11-12 the model loads from a copy at one path in Application Support (RerankerModelLocation, Models/ReRankerModel-<fingerprint>.mlmodelc, excluded from backup): before it, the phone compiled again on most launches (about 30 loads of 20 to 130 s from 2026-11-09 to 12, most with no install between) and kept 202 compiled copies, 1.47 GB. A copy loads with identical scores (20 of 20 outputs equal, M3 Pro); the journal’s reranker.load says fixedPath. RerankerWarmup starts the load four seconds after a launch into the foreground (until 5.8 RAGEngine.shared, whose initializer starts it, was first created by the first search, so the first question would have paid the compile); a load past one second shows “Preparing search” on the model pill and, for a question that waits on it, as a step (RerankerSetupStatus).
  • Inside the app (DeepThinkRetrievalGoldenTests, iOS 27 Simulator, 71 documents, 7,078 passages, one pass under the Simulator’s window, scored by the question’s expected document with BenchmarkRuns/2026-11-09-reranker-swap/scripts/score_golden.py): Deep Think at 15 passages, 92 questions, the expected document first 28.3% before and 33.7% after, in the top 5 48.9% and 50.0%, MRR 0.372 and 0.413 (23 better, 19 worse, p=0.64); Maximum at 15, 36 questions, top 5 50.0% and 55.6% (9/7); Standard at 20, 19 questions, top 5 52.6% and 47.4% (2 better, 6 worse, p=0.29). None of these differences is significant. Most of these questions are QASPER’s (“What is their evaluation metric?”) and name no paper, so the expected document is often not identifiable from the question in a 71-document library; the passage-level QASPER test above is the sharper measure. The Simulator runs Core ML on the CPU, so its search pass took 3,236 s against 281 s and says nothing about a device.
  • Inputs are the token ids and the mask, without TinyBERT’s token types; a pair longer than 512 tokens loses the passage’s end and keeps its closing [SEP], as the model’s own tokenizer truncates. The output is one logit, min-max normalised over the batch to 0.10 to 0.90 as before, so every downstream threshold reads the same scale.
  • The tokenizer is the model’s own (byte-pair encoding, reranker_tokenizer.bundle), pinned by RerankerModelTests to the token ids and scores the build script printed.

[evidence_level: measured (accuracy, timings, load), code_verified (the loading and scoring), documented (the licence and training data, the model card); confidence: high for the ranking gain on QASPER, unmeasured on the iPhone's Neural Engine]

Side searches go behind retrieval’s best passage (added 2026-11-09, for 5.8)

Three steps add passages found outside the reranker to an extractive question’s candidates: cross-reference resolution (addCrossReferencedPassages, Step 4.6), Standard’s spec sniper (runSpecTableSniper, Step 4.8, keyword and number co-occurrence over every passage) and Deep Think’s spec sniper (RAGService+ToolHandler). Each inserted at position 0. Sentence extraction keeps the first candidate whole (leadingWithTopRanked, 5.7, up to three fifths of the budget), so whatever a side search found decided the answer. On the owner’s iPhone, 2026-11-09, “How much lysine is present?” (Standard, 48 documents, 8,202 passages): both reranks ranked the Elfar spirulina composition table first, the sniper put two passages from an autism-biomarker paper ahead of it on “lysine” and “present”, the prompt held 2 of 162 passages (both from that paper) and the answer said the topic was not in the documents. All three now insert through RAGService.insertingBehindTopRanked: right behind the top passage, ahead of every other candidate, so they still survive the budget. When the reranker’s first pick is wrong and a side search’s is right, the side search’s passage is second instead of first; that is the trade, and it is not yet measured on the golden set or QASPER.

The same run ended blank for a second reason, downstream of retrieval: the model stopped after “1”, the responsiveness judge called it a full answer, and cleanupResponseText deleted a line that was only a number as an orphaned list marker. isDegenerateAnswer now sends an answer with no letters back for one rewrite and, failing that, Standard shows notFoundAnswer; isOrphanListMarker requires the marker’s period or bracket.

Checked on the phone at 17:40 the same day: the spirulina paper was S1 and the answer had words, but it opened with “ONE” and quoted the prose beside the table (“small amounts”) instead of its row (lysine 4.4 to 26.2 mg/g). The count came from the “State the exact count FIRST” instruction, which “how much” questions also got; it is now for “how many” only (getsCountInstruction). Whether the lysine row was in the lead passage (“[Part 1]” of the table) is open: [RAG] Lead passage now logs the lead’s document, page, structure, and how many of its characters the room kept.

That line settled it at 17:44: the lead was the prose paragraph (p.5, 1,579 of 1,579 characters), not the table. The cross-encoder had ranked the table first at 0.90, and RAGEngine.computeTOCPenalty took 0.18 off it because its section-number pattern (\b\d+\.\d+\.?, 8 or more matches) counts every decimal in “15.0-25.0” and “4.4-26.2”; the gate read 0.720 at the top. For a passage stored as a table (isStoredTable) the penalty now counts only dot leaders and the words “contents” or “table of contents”; numbers, short lines and list markers describe every data table. [evidence_level: artifact_derived, test_verified (AnswerFloorTests), confidence: high for the mechanism]

[evidence_level: code_verified, artifact_derived (pipeline_trace.log, 2026-11-09 14:56 to 15:05), test_verified (AnswerFloorTests); confidence: high for the mechanism, unmeasured for accuracy across sets]

Quotes only stops before generation (added 2026-11-10, for 5.8)

With Quotes only selected (RAGService.quotesOnlySelected), Standard skips query rewriting, HyDE and contextual compression, and after the hard relevance gate returns quotesOnlyResponse: up to six excerpts from the passages assembled for the model (excerptBullets, shared with the excerpts fallback), each labelled with its source. Deep Think and Maximum run as Standard. [evidence_level: code_verified, test_verified (ReleaseFinishTests), confidence: medium]

Every rule after the reranker, measured (added 2026-11-10, for 5.8)

Thirty rules change a passage’s score, rank, inclusion or place between the cross-encoder and the prompt; twelve can override its first pick, and several were written for TinyBERT (“the reranker has an inherent prose bias”). Each was switched off on its own (RetrievalAblation, development builds) and measured by whether the evidence reached the model (EvidenceReachBenchmarkTests, scripts/evidence_reach.py): 246 QASPER questions (prose) and 158 table questions over 7 PDFs (a numeric cell read off the page image). Full tables: BenchmarkRuns/2026-11-10-rule-audit/README.md.

  • Corrective retrieval measurably helps: switched off, Standard’s evidence coverage on QASPER fell 0.617 to 0.595 (15 better, 33 worse, p=0.01). The focused neighbour window probably helps (0.608, 6/15, p=0.08).
  • Deep Think’s spec sniper is removed. On tables the cell’s passage came first in 72.2% of searches with it and 75.9% without (7 better, 1 worse, p=0.07); on QASPER 32.5% and 32.1%. Standard keeps its sniper (off: 2 of 158 cells lost, p=0.50).
  • No other rule measurably helps or hurts on either set: the three penalties (the question-bank and bibliography ones never fired; the contents-page one is neutral with tables exempt), definition boost (QASPER 9 better, 4 worse switched off, p=0.27), spec rescue, summary hybrid, cross-references, extractive priority, graph packing. They stay.
  • Not measured: contextual compression (a model call; the harness’s stand-in model skips it) and Deep Think’s multi-query blend (the harness does not call executeMultiQuerySearch).

With everything on, the table cell reached Standard’s prompt for 83.5% of questions, and Apple’s model answered 125 of 158 right, 124 of the 131 whose cell reached it: the loss is retrieval and import, not the model. On the import side a table whose title spans its first row is stored with “Column 2 | Column 3” as its header and the real header as row 1 (tables/fidelity.score.txt).

[evidence_level: measured (iOS 27 Simulator, 2026-11-10), code_verified; confidence: high for corrective retrieval and for the absence of large effects, medium for the sniper (p=0.07); tables limited to 7 PDFs]

Look-alike numbers are not duplicates, and the reranker’s pool was measured (added 2026-11-08, for 5.8)

  • Pool cut before the boosts: tried, not adopted (HybridSearchService, the FTS5 path). The boosts (keyword match, structure type, state-lookup anchor) re-sort by the vector score plus up to 0.20 over every fused result before the top-K cut, so the reranker’s pool is the boosted top K; the retrieval benchmark measured MRR falling from the fused list to the boosted one, 0.708 to 0.635 in Standard (n=24) and 0.719 to 0.583 in Deep Think (n=66, BenchmarkRuns/LEDGER.md). Cutting in fused order first was measured on 2026-11-08 with the model-judged AnswerQualityEvaluationTests on one build: 0.40 correct with it, 0.45 without (20 cases; one case apart, within the run-to-run spread). MRR before the reranker does not decide it, since the reranker reorders its pool; the pool’s recall does, which the retrieval benchmark measures. The default is unchanged; OI_RERANK_POOL=fused selects the fused cut for that measurement.
  • Deduplication keeps numbers (RAGEngine, context assembly). Words under three characters were left out of the Jaccard comparison, so “6” against “61” looked identical. Numbers of any length now count, and a passage that states a number the kept passage does not is never a duplicate.
  • The citation bound follows rescued sentences (RAGService+QueryContext, +SentenceExtraction). Sentences rescued after the context was built came after the “Cite only sources S1 to Sn” line; the line is restated after them (withCitationBound).

[evidence_level: code_verified, measured (the MRR figures, BenchmarkRuns/LEDGER.md; the answer evaluation, logs eval-newpool and eval-oldpool of 2026-11-09 recorded in BenchmarkRuns/2026-11-08-extraction-fidelity/README.md); confidence: low for the pool question: n=20, one run each]

Suggested questions wait for an answer (added 2026-11-04, for 5.8)

SuggestedQuestionsService calls AnswerModelPriority.waitWhileAnswering before each model call, outside its 10 and 12 s timeouts, so suggestions are written after an answer, not during it. On the owner’s iPhone on 2026-11-04 six suggestion requests ran during one Deep Think answer and its calls waited 3.7 to 13.3 s for their first word. Which questions are suggested does not change. [evidence_level: code_verified, confidence: high, evidence_source: SuggestedQuestionsService.swift; his iPhone trace 2026-11-04]

(Added 2026-11-04.) The side jobs that call AnswerModelPriority.waitWhileAnswering are SuggestedQuestionsService (two call sites), DocumentSummaryService (documentSummary), ContentTaggingService (contentTags), ImageUnderstandingService (imageUnderstanding, two call sites), ClusterLabelService (atlasLabels) and LibraryVisualizationEngine (atlasProfile and atlasAnalysis, two call sites). (Corrected 2026-11-07: the Atlas profile’s wait was added 2026-11-05.) The wait is skipped inside an answer’s own task tree (AnswerModelPriority.isAnswerPath), polls every 500 ms and records model.sideJobWaited in a development build when it waited 0.5 s or more. [evidence_level: grep_verified, confidence: high, evidence_source: grep -rn waitWhileAnswering OpenIntelligence (9 call sites in 6 files, re-counted 2026-11-07), AnswerModelPriority.swift:27-39]

An answer also pauses while the app is in the background. RAGService.waitWhileAppIsInBackground (checked once a second, the answer’s Live Activity and task card read “Paused, tap to resume” through answerPausedNotification) is called in the retry ladder after a rate-limited model call: iOS rate-limits the on-device model for an app in the background, so the retry waits for the app to come back instead of spending the 2, 5 and 12 second backoff. It is called in RAGService’s primary retry ladder and its local fallback ladder and in AgenticOrchestrator’s retry; it does not pause a call that is not failing. [evidence_level: code_verified, confidence: high, evidence_source: RAGService+ImportEnrichment.swift:262-286, RAGService+Generation.swift:1671, RAGService+Generation.swift:1785; RAGService+Agentic.swift:121]

A question is planned once (added 2026-11-04, for 5.8)

QueryExecutionPlannerService.buildPlan asks Apple’s model for a guided QueryExecutionPlan. Deep Think asked for it three times per question (mode choice, the retrieval decision, the search phrases), each a separate model call; it now reuses a plan for the same question, quality mode and tool setting for 120 seconds, and a call made while one is being planned waits for that one. The plan’s content is unchanged: the three calls were independent samples of the same prompt. [evidence_level: code_verified, confidence: high, evidence_source: QueryExecutionPlannerService.swift]

Added 2026-11-05: Deep Think’s merge is now deterministic, and parent expansion groups by document. AgenticOrchestrator.executeMultiQuerySearch merges the five searches by 60% the best re-ranker score and 40% the normalized reciprocal-rank sum, keeps twenty, and since 5.8 breaks a tie by where a passage first appeared across the searches; before, a tie fell to dictionary order, which differs between two dictionaries in one process, so the same question could keep different passages (28 of 57 merged searches differed between two passes over an unchanged library). Parent expansion in performFullRetrievalPipeline finds a result’s neighbours in its own document’s passages (Dictionary(grouping:by:), in the library’s order) instead of filtering the whole library for each result. DeepThinkRetrievalGoldenTests records 760 searches under three windows and fails if two passes differ; scripts/golden_retrieval.py compare compares two runs. Before and after: 703 of 703 single searches identical; merged 57 of 57 identical across passes and launches after. [evidence_level: measured, confidence: exact, evidence_source: iOS 27 Simulator, 2026-11-05]

Proving a Standard change changes nothing (added 2026-11-06, for 5.8)

StandardPipelineGoldenTests asks 35 fixed questions in Standard against a fixed library (23 fixtures: the household documents and the research fixtures that are not near copies of one another) and records, per question, the answer as the app returns it, the audit snapshot, every retrieval stage’s passages in order with their score bits, and every model call with a fingerprint of its prompt and context. The model is a stand-in that subclasses AppleFoundationLLMService, so the pipeline takes the branches it takes for Apple’s model, and answers with the first line of its context and the prompt’s fingerprint. The questions reach the cache-hit path, external evidence (OCR text, a barcode), a library with no documents, a model failure and the negative controls; the recorder executes 65% of queryInternal’s lines, and most of the rest cannot run in Standard (query expansion, HyDE, compression and Step 5.9 sit past the agentic return) or needs Private Cloud Compute or Apple’s model. It exists for the 2026-11-06 decomposition of queryInternal into steps (BenchmarkRuns/2026-11-06-decomposition/PLAN.md). Two runs on the same build match on 34 of 35 questions; the 35th ties two passages’ final scores exactly and breaks the tie differently between launches, and is reported, not failed (scripts/golden_compare.py, KNOWN_LAUNCH_SENSITIVE). [evidence_level: measured, confidence: high, evidence_source: runs std-a and std-b, 2026-11-06, iOS 27 Simulator]

The Standard pipeline, broken into steps (2026-11-06, for 5.8)

RAGService.queryInternal in RAGService+QueryPipeline.swift is being cut into methods, one step per commit, each a move with no change in behaviour: every commit builds for the iOS Simulator, an iOS device and macOS, and StandardPipelineGoldenTests records the 35 Standard questions identically to the commit before (BenchmarkRuns/2026-11-06-decomposition/specs/query/README.md has the order, the steps that need a hand-written exit, and what stays in queryInternal).

StepWhat moved
Q019 (response) (builds the response metadata and the RAGResponse and logs the total time) into buildStandardResponse
Q029 (gating) (builds the gating summary, final warnings and the structured answer) into summarizeGatingAndWarnings
Q038 (calibrates confidence and logs and emits the pipeline completion) into calibrateConfidenceAndLogCompletion
Q046g (log) (logs the model, timing, length and token rate of the generated answer) into logGeneratedResponse
Q056 (telemetry) (measures the generation time and emits the ‘Response generated’ events) into emitGenerationTelemetry
Q066 (chain gate) (decides whether the developer-forced reasoning chain runs) into decideReasoningChain
Q076 (history) (adds conversation history or memory to the prompt and decides whether citations are required) into buildGenerationPrompt
Q086 (output budget) (caps maxTokens to the room left after the context and applies the procedural and trivial adjustments) into capGenerationTokens
Q096 (prompt) (writes the intent-specific system prompt and the temperature caps into the generation config) into composeStandardGenerationConfig
Q10SV (computes the pre-generation confidence and decides whether the Evidence-First prompt is used) into assessEvidenceBeforeGeneration
Q115 (citations) (logs the assembled context, lists the passages the prompt labelled and records them for recovery) into finishIncludedPassages
Q12 to Q21Q12, Step 5 (ordering): orders the candidates for procedural, extractive or default questions into orderContextCandidates; Q13, Step 5 (budget): computes the token and character budget for the context into computeContextCharBudget; Q14, Step 5 (tools): starts context assembly and turns tools off when the context is already assembled into disableToolsForAssembledContext; Q15, Step 4.7 (gates): decides whether contextual compression runs and logs each reason it is skipped into gateContextualCompression; Q16, Step 4.8: adds specification-table passages found by keyword and number co-occurrence for extractive questions into runSpecTableSniper; Q17, Step SE: adds a BM25 second pass over every passage for enumeration questions into addEnumerationKeywordPass; Q18, Step SF: narrows strong short single-document questions to a neighbor window around the top passage (on-device Apple model only) into applyFocusedNeighborWindow; Q19, Step SX: blends document summaries with detail passages for questions that need a document summary into addSummaryDetailHybrid; Q20, Step SX (setup): starts the context candidates from the MMR list and detects enumeration questions into prepareContextCandidates; Q21, Step 4.4: adds passages from other documents before diversification and records the retry candidates into ensureMultiDocumentCoverage
Q22 to Q31Q22, Step 4.3 (rescue): reports the dropped passages and rescues specification passages for extractive questions into rescueDroppedSpecChunks; Q23, Step S4d: applies the contextual-definition boost to the re-ranked list into boostContextualDefinitionMatches; Q24, Step 3.5: boosts passages whose section metadata matches the question and logs the top passage into applySectionMetadataBoost; Q25, Step S3x: inserts OCR and barcode evidence as virtual passages at the front of the retrieved list into injectExternalEvidence; Q26, Step 3 (gate): decides whether iterative retrieval runs for this intent and logs when the intent overrides the setting into decideIterativeRetrieval; Q27, Step 2.5: starts the retrieval clock and sets the hybrid weights, RAPTOR-lite search levels and filtered passage pool into planSearchRouting; Q28, Step P4e: logs the pipeline box and feature summary and emits the planning events and the ‘Query received’ telemetry into logStandardPipelinePlan; Q29, Step P3: applies the opt-in adaptive generation profile (UserDefaults adaptiveInferenceProfiles) to the inference config into applyAdaptiveGenerationProfile; Q30, Step 9 (source-only): checks an extractive answer against its sources when the Standard gates flagged it into applySourceOnlyCheck; Q31, Step 7.5 (run): runs verification gates A-G against the answer and the evidence embeddings into verifyAnswerAgainstEvidence
Q32 to Q41Q32, Step 7: scores answer confidence and quality warnings into assessAnswerQuality; Q33, Step 6g (override): replaces the answer with the source-locked precision-lookup value when the model did not state it into applyPrecisionLookupOverride; Q34, Step 6 (integrity): normalizes malformed output and runs one grounded repair pass when needed into repairResponseIntegrity; Q35, Step 6 (citations): retries once for a cited answer when citations are required and missing into retryForMissingCitations; Q36, Step 6 (start): logs the generation step, applies the adaptive cooldown and starts the generation clock into beginGenerationStep; Q37, Step 5 (audit): emits the context telemetry and stores the RAGAuditSnapshot into recordContextAuditSnapshot; Q38, Step 5 (assembly): builds the context by sentence extraction or whole-passage packing with needle rescue into assembleStandardContext; Q39, Step SR: runs corrective lexical retrieval and adopts or rejects its candidates into runCorrectiveRetrieval; Q40, Step 4.9: packs graph neighbors and references around the candidates for multi-hop intents into packContextWithGraph; Q41, Step 4.7: compresses the leading candidates with the contextual compression model into compressContextCandidates
Q42 to Q51Q42, Step 4.6 (cross references): adds the passages that extractive candidates refer to into addCrossReferencedPassages; Q43, Step 4.6: expands the candidates with sibling passages from their parent sections into expandParentDocuments; Q44, Step 4.5: diversifies the filtered passages with MMR (or takes the top K when MMR is off) into diversifyWithMMR; Q45, Step 4.3 (filter): computes the adaptive similarity floor and filters the re-ranked passages into filterLowConfidenceChunks; Q46, Step S4k: retries with a cleaned keyword query when the top similarity is below 0.45 into retryWithCleanKeywordQuery; Q47, Step S4c: runs the lexical-heavy cascade search and re-rank when the policy asks for it into runRetrievalCascade; Q48, Step 4: re-ranks the candidates (cross-encoder or similarity order) and records the rerank trace stage into rerankCandidates; Q49, Step S3n: attaches source names, demotes interrogative FAQ passages and emits the retrieval telemetry into attachSourcesAndDemoteQuestions; Q50, Step S3k: drops passages whose document is in no library into dropPassagesOfUnknownDocuments; Q51, Step S3s: narrows the retrieved and cached pools to the documents a Shortcuts action named into applyDocumentScope
Q52 to Q61Q52, Step SC (write): stores a live search’s passages in the semantic query cache into writeSemanticQueryCache; Q53, Step 3 (multi-vector): adds passages from vector searches on up to two appended query expansions into runSupplementaryVectorSearch; Q54, Step 3 (single pass): runs the hybrid vector and BM25 search into runHybridSearch; Q55, Step 2 (embedding): embeds the query, or reuses the cached vector on an exact cache hit, and checks its dimension into embedQuery; Q56, Step 2 (HyDE): starts the embedding clock, logs the HyDE gates and generates a hypothetical document when HyDE applies into gateAndGenerateHyDE; Q57, Step 1.6: builds the effective query profile and derives the answer intent and its flags into resolveStandardAnswerIntent; Q58, Step 1.5: expands the query from the planner, heuristics, container vocabulary and gazetteer into expandQueryForRetrieval; Q59, Step 1: rewrites the query with the model and conversation context when enabled and not an exact cache hit into rewriteQueryWithModel; Q60, Step 0: loads the cached or newly built corpus vocabulary and every readable passage, translates the query and decides whether to rewrite it into loadCorpusVocabulary; Q61, Step SC: looks the question up in the persistent semantic query cache and takes an exact, still-current hit into lookUpSemanticQueryCache
Q62 to Q66Q62, Step P4c: sizes topK from the live passage count, the quality mode, the query profile and the adaptive candidate cap into sizeStandardTopK; Q63, Step 6 (route plan): applies the post-retrieval model plan (Private Cloud Compute envelope, consent, local fallback, abstain) into applyPostRetrievalModelPlan; Q64, Step 3 (iterative): runs multi-pass iterative retrieval with self-correction into runIterativeRetrieval; Q65, Step P4b: resolves the developer-tuning and relay flags, the selected container, its retrieval config and vector database, and records the query’s container into resolveQueryContainer; Q66, Step P1b: unpacks the coordinator’s QueryRuntimeContext into the pipeline’s locals, records the query on self, logs the header and refreshes cloud consent into unpackRuntimeContext
E01 to E07E01, Step OC: runs the reliability fallback from the recovered passages when it applies, else logs the failure and rethrows (the outer catch body) into recoverFromQueryFailure; E02, Step ND: answers a question to a library with no documents, abstaining for actions and grounded-only mode and otherwise by direct chat into answerWithoutDocuments; E03, Step IC: builds a recovery response from the generated text when processing the answer threw (the inner catch body) into recoverFromResponseProcessingError; E04, Step 7.5 (abstain): logs the verification gates and abstains when they failed in grounded-only mode or below the quality mode’s threshold into abstainOnFailedVerification; E05, Step 6g (empty): answers from the recovered passages when the model returned nothing, else throws modelNotAvailable into fallBackFromEmptyAnswer; E06, Step 6d (abstain): abstains when citations are required, missing after the retry, and ungrounded fallback is off into abstainForMissingCitations; E07, Step 5.9: answers a summarize question from an extractive summary of the included passages, falling through to generation when it fails into answerWithExtractiveSummary
E08 to E14E08, Step SG: returns a not-found answer when the included passages are lexically and semantically irrelevant to the question into applyHardRelevanceGate; E09, Step 5 (empty): answers by direct chat or abstains when context assembly produced no passages into handleEmptyAssembledContext; E10, Step SH: returns the closest sources instead of an answer when fewer passages than Library Settings requires reach its minimum score into enforcePassagesNeeded; E11, Step 4.5 (empty): answers by direct chat or abstains when MMR returned no passages into handleEmptyDiverseSet; E12, Step 4.3 (gate): when no passage clears the similarity floor, takes the top re-ranked passages under override, lenient or fallback rules, else abstains into handleNoConfidentPassages; E13, Step 4 (empty): answers by direct chat or abstains when re-ranking left no candidates into handleEmptyRerank; E14, Step 3 (empty): answers by direct chat or abstains when retrieval found no passages into handleNoRetrievedPassages

Maximum, the compute profile, turns the work up (added 2026-11-10, for 5.8; unmeasured)

With the Compute Profile set to Maximum, the app in front, the device not hot and Low Power Mode off (DeviceCapabilityService.isMaximumFidelityActive; constants in DeviceCapabilityService.MaximumFidelity; OI_MAXIMUM_FIDELITY=on|off overrides it for a benchmark): the reranker scores up to 400 passages and at least 150 (RAGEngine, 250 and 100 otherwise) and loads with Core ML’s fastPrediction specialization, and keyword search takes up to 120 results (HybridSearchService, 60 otherwise). Until this change the profiles only chose which chips may run a job. None of it is measured: benchmark day (2026-11-11) compares Maximum with Balanced, and it ships only if it reads and finds at least as well. [evidence_level: code_verified, confidence: low until measured]

The profiles ramp (added 2026-11-10, for 5.8; unmeasured). Depth now steps up per profile through DeviceCapabilityService.Depth and activeDepth: Efficiency and Balanced keep 5.7’s values, Performance takes the midpoint and Maximum the top. Efficiency saves energy only by halving what runs at once (paced(_:for:): text recognition, pages parsed and drawn, passages embedded), never by reading or searching less, because accuracy comes first. OI_COMPUTE_PROFILE=efficiency|balanced|performance|maximum overrides the profile for depth and pacing on one build (OI_MAXIMUM_FIDELITY still works). For search: rerank ceiling 250 / 250 / 325 / 400 and floor 100 / 100 / 125 / 150 (RAGEngine), keyword results 60 / 60 / 90 / 120 (HybridSearchService). [evidence_level: code_verified, confidence: low until measured]

What the Intelligence Mode page claims per mode (traced 2026-11-10, 5.8). Deep Think and Maximum return to executeAgenticQuery at RAGService+QueryPipeline.swift:249, so any step that exists only later in queryInternal runs only under Standard: HyDE (:375, and Standard has usesHyDE off), contextual compression (:489, Deep Think only by flag), graph context packing, the lost-in-the-middle order that reaches the prompt, conversation memory and sigmoid confidence calibration. IntelligenceModeCard lists only steps with a live call site and the modes that reach them; the Settings card’s step ids are pinned by IntelligenceModeCardTests. Unfixed retrieval findings from the same trace are in Docs/ai/STATE.md. [evidence_level: code_verified, confidence: high]

Fusion reads score order (2026-11-10, 5.8; unmeasured). Deep Think’s and Maximum’s multi-query reciprocal rank fusion ranks each search by score (AgenticOrchestrator.rankedForFusion) before fusing, because executeFullRetrievalPipeline returns lost-in-the-middle order. OI_RETRIEVAL_ABLATE=fusionScoreOrder restores the old order. [evidence_level: code_verified+test_verified, confidence: medium until measured]

Retrieval settings are the same on every device (added 2026-11-16, for 5.8)

Until 5.8 the device tier set the retrieval candidates (75 on an A17 Pro to 250 on M-series), iterative retrieval, the similarity floor and the MMR balance. Every answer benchmark ran at the top tier, so AdaptivePipelineConfig.base(for:) now applies those measured settings on every supported device and keeps only pacing from the tier; live thermal, battery and memory adjustments still apply. The reranker’s default compiled form serves search first; on Maximum the .fastPrediction form replaces it once prepared in the background, with the same weights (RerankerFormTests). How much longer an answer takes on an A17 Pro or A18 with 250 candidates is not yet measured. [evidence_level: code_verified, test_verified on the Simulator, confidence: medium]