OpenResponses

SwiftUI developer client for the OpenAI Responses API and the successor to OpenAssistant.

Version 2.9

Version 2.9 is about seeing what the assistant is doing while it uses its tools. Each tool step shows a spinner while it works and says Completed or Failed when it’s done, steps have their real names (Code Interpreter instead of code_interpreter_call), and a Code Interpreter step shows the Python it ran. A chart from Code Interpreter shows once, each file it makes is listed once, and reasoning summaries show their formatting instead of raw asterisks. See the 2.9 release notes, its What’s New and the changelog.

App Store, checked October 2: version 2.9 is live. App Store Connect reports 2.9 READY_FOR_SALE with build 54, the Xcode Cloud run 54 archive of commit baff2fc, and the public store lookup carries a release timestamp of 2026-10-02T16:24:23Z (09:24 Pacific).

Version 2.8

Version 2.8 adds GPT-6.1 Sol, lists new OpenAI models on its own with settings read from OpenAI’s docs pages, runs Python on the device after you approve each run, searches conversations by meaning on the device, signs in to MCP providers that refuse the app’s own callback, stops GPT-Live when you talk over it, and holds 4.5:1 text contrast and 44-point tap targets on the chat, Settings and Conversations screens. See the 2.8 release notes and the changelog. It went live September 30, 2026 as build 51, the Xcode Cloud run 51 archive of commit ab87186.

Version 2.7

Version 2.7 adds GPT-6 Sol and GPT-6 Luna, GPT Image 2.5 Flare and Sunburst with X-High and Max quality, and version-aware recognition of later general-purpose GPT releases. See the 2.7 release notes.

Version 2.6 documentation

The complete v2.5 → v2.6 release dossier covers the full source comparison, What’s New, changelog, technical behavior, upgrade steps, validation, and store/reviewer copy. Its 129-file inventory includes committed work and the September implementation changes.

Overview

OpenResponses is a native SwiftUI Playground for OpenAI Responses API. It functions as a mobile developer playground and testing workspace, exposing low-level model parameters, tool execution, token-level streaming data, and raw request visibility without hiding the API behind a custom proxy layer.

• Target Audience: AI engineers, prompt designers, and developers needing direct client-to-API control.
• Core Problem Solved: Lack of visibility in standard AI interfaces. OpenResponses exposes raw token counters, network statuses, expandable reasoning summaries for supported reasoning models, and outbound/inbound JSON payloads.
• Technical Characteristics: Direct client-to-endpoint connections, local document parsing (with Vision OCR), and sandboxed browser automation loops.
• Feature Tiers:
  • Core Playground: Responses API (Chat, Tool Calling, Vision, Models)
  • Developer Lab: Batch API
• Product Lineage: OpenResponses is the active evolution of Gunnar Hostetler’s API-tooling work and supersedes the older OpenAssistant Assistants API client.

---

Product Snapshot

Dimension	Detail
Platform	iOS 17.0+ and iPadOS 17.0+ (iPhone and iPad)
Language	Swift
UI	SwiftUI
Architecture	MVVM-S
Primary APIs	OpenAI Responses API, Notion API, EventKit, Contacts
Storage	Keychain, sandboxed JSON files
App Store	Download
Status	Active
License	MIT

---

Key Capabilities

• Direct API Connections: Outbound HTTPS traffic routes directly from the iOS client to OpenAI and Notion endpoints without intermediate proxy servers.
• Asynchronous SSE Streaming: Uses Swift Concurrency (AsyncThrowingStream) to parse Server-Sent Events line-by-line, dispatching UI updates to the @MainActor to avoid layout race conditions.
• Realtime Voice WebSockets: Includes Voice Mode using wss:// for bi-directional 24kHz PCM16 audio streaming (Direct BYOK WebSocket mode).
• Developer Labs: Batch job management with complete output/error exports, defaulting to the Responses endpoint. Assistants, fine-tuning and published prompts were removed in 2.7 after OpenAI shut them down or deprecated them.
• Secure Keychain Storage: API keys, Notion tokens, and custom Model Context Protocol (MCP) headers are stored inside the secure iOS Keychain. Request inspection/logging includes targeted credential redaction; keys are transmitted to the relevant service when needed for authentication.
• On-device Browser Automation: Persistent WKWebView with serialized DOM and screenshot actions, precise element references, cancellation, deadlines, and per-turn limits. Pending computer safety checks pause both tool paths. See browser execution.
• Local Ingestion & OCR: Extracts text from PDFs using PDFKit and recognizes text in image attachments using the native Vision OCR framework locally on-device.
• Observability Tools: Includes inline collapsible reasoning panels, live connection monitors, and a Request Inspector rendering raw JSON payloads.

---

How It Works

The following flowchart outlines the request lifecycle, tool branches, and approval gates:

flowchart TD
    A[Compose request] --> B[Send to Responses API]
    B --> C[Read response events]
    C --> D{Execution owner}
    D -->|Hosted tools| E[OpenAI executes configured tools] --> C
    D -->|Client tool| F[Check enabled handler and execution rules]
    F --> G[Execute and return actual result] --> B
    D -->|Computer safety check| S[Pause for turn-scoped user decision]
    S -->|Allow| G
    S -->|Deny| X[Cancel pending work]
    D -->|Completed| H[Render answer, summaries and artifacts]

---

Architecture

The codebase separates views from network and system frameworks using the MVVM-S pattern:

flowchart LR
    View[SwiftUI Views] <--->|Observe state| VM[ChatViewModel]
    VM <--->|Request completions| Services[Response runners / OpenAIService / ComputerService]
    Services -.->|Authenticate| Keychain[iOS Keychain]
    Services <--->|API Payload| OpenAI[OpenAI Responses API]

For a detailed layer-by-layer system map and data flow boundaries, see ARCHITECTURE.md.

---

Core Workflows

The local file conversion and ingestion workflow converts attachments prior to payload transmission:

flowchart TD
    A[Select file attachment] --> B[FileConverterService evaluates extension]
    B --> C{Format?}
    C -->|PDF| D[PDFKit extracts text] --> G[Pack into prompt payload]
    C -->|Image| E[Vision OCR recognizes text] --> G
    C -->|Text| F[Read raw content] --> G
    G --> H[Transmit request payload to OpenAI]

---

Data Flow

Data boundaries separate on-device storage, Keychain secrets, and third-party APIs:

flowchart TD
    Keychain[(Keychain)] -.->|inject headers| API
    Disk[(Local Disk JSON)] <--->|load/save history| UI[User Interface]
    UI -->|direct HTTPS request| API[OpenAI / Notion APIs]
    API -->|SSE Stream| UI

---

File Entry Points

Concern	Files	Responsibility
App Entry	OpenResponsesApp.swift	Initial bootstrapping and startup migrations.
DI Container	AppContainer.swift	Service locator for dependency injection.
Main UI	ContentView.swift	Navigation shell and tab container.
Chat View	ChatView.swift	Chat rendering and text/attachment inputs.
Chat ViewModel	ChatViewModel.swift	Session state management, settings, and tool approvals.
OpenAI Client	OpenAIService.swift	Payload assembly and SSE stream parsing.
Keychain Storage	KeychainService.swift	Secure credentials management.
Browser Automation	ComputerService.swift	Sandboxed browser automation and capture loops.
File Extraction	FileConverterService.swift	On-device file conversions and OCR text recognition.
Notion Client	NotionService.swift	Direct Notion workspace database integrations.

---

Configuration

The configurations map to UserDefaults (for preferences) or the secure Keychain (for keys).

Setting	Storage	Default	Required	Purpose
OpenAI API Key	Keychain (openAIKey)	None	Yes	Authenticates all OpenAI network requests.
Notion Token	Keychain (notionApiKey)	None	No	Authenticates Notion integration requests.
Model Selection	UserDefaults	gpt-6-sol	Yes	Responses model; GPT-6.1 Sol, GPT-6 Sol, Astra and Luna and the GPT-5.6 family are built in, and the model menus also list later general-purpose releases the account can use, with settings read from OpenAI’s docs pages.
Reasoning Effort	UserDefaults	medium	No	Configures model-aware effort choices; current models also expose higher efforts where supported.
Web Search	UserDefaults	true	No	Toggles OpenAI web search capabilities.
Code Interpreter	UserDefaults	true	No	Toggles OpenAI sandboxed Python containers.
Computer Use	UserDefaults	false	No	Toggles local browser automation tool.
Notion Integration	UserDefaults	true	No	Toggles Notion tool access.
Apple Integrations	UserDefaults	true	No	Toggles Calendar, Reminders, and Contacts access.

---

September 2026 API refresh

The playground now includes a shared current-model catalog, Astra-compatible reasoning controls, GPT Image 2, current Realtime transcription and voices, opt-in automatic compaction, persisted reasoning, pro reasoning, hosted shell, and deferred function/MCP loading through tool search. Saved presets keep working; a preset on a retired model moves to OpenAI’s documented replacement (2.7).

Native chat now executes configured function/custom tools, Astra async lookups, programmatic tool calls, and multi-agent responses. Multi-agent uses WebSocket result injection so waiting agents can resume immediately. Read-only calls can overlap; writes run sequentially. Root answers, subagent activity, tool results, and image previews appear in the chat. Interrupted turns preserve known results and mark uncertain outcomes without automatically retrying writes. MCP configuration is available again. These orchestration features apply to current-model foreground requests with Computer Use disabled.

Settings → Model → API Workbench exposes editable requests over HTTP, SSE, and persistent Responses WebSockets. It includes steering, tool-result batches and live injection, full response export, input token counting, standalone compaction, response retrieval/cancellation/input items, and conversation retrieval/items. Examples cover asynchronous functions, custom text tools, programmatic tool calling, hosted shell, image generation, apply patch, and the multi-agent beta. Workbench client tools use actual results supplied by the user; native chat executes its configured tools.

Manual chat compaction preserves the complete returned output window, including opaque items, then replays it on the next turn. It never treats a compaction ID as a response ID. The workbench displays unknown events and exports the complete final response; its on-screen event preview is bounded.

Current API contracts and verification details are recorded in API refresh notes. These are source/build capabilities, not an App Store deployment claim.

Settings coverage: The shared ResponseSettingsRegistry and modern response controls expose saved prompt settings. Request mapping is model-aware: verbosity/reasoning use nested API fields, unsupported sampling is omitted, and preset names remain local metadata. See the technical settings table for current defaults and execution constraints.

---

Build & Run

Local Setup

1. Clone the Repository:

   git clone https://github.com/Gunnarguy/OpenResponses.git
   cd OpenResponses

2. Open in Xcode:

   open OpenResponses.xcodeproj

3. Requirements:

   • Xcode 16.1 or newer.
   • iOS 17.0+ deployment target.
   • Active OpenAI API key.

4. API keys: Enter your OpenAI API key in Settings. To use Notion, a new user connects it by signing in under Settings → MCP; an existing integration token can be managed under Settings → Tools. Keys and tokens are stored in the Keychain.

---

Testing

Test Type	Command / Procedure	Expected Result
Build Target	Build project in Xcode (Cmd+B)	Compilation completes with no errors.
Unit/integration tests	Use an available simulator and the command in the validation ledger.	Latest implementation run (October 1, 2026): 343 tests, zero failures, 3 skipped.
Secret Scan	python3 scripts/secret_scan.py	CLI tool returns success with no keys detected.
Preflight check	bash scripts/preflight_check.sh	Confirms Info.plist privacy descriptions are present.

---

Privacy & Security

OpenResponses operates under a local-first threat model:

• Network boundaries: The device contacts configured API providers over HTTPS or secure WebSockets. Hosted tools, including MCP, may contact their configured services from OpenAI infrastructure.
• Keychain Storage: Storing API keys securely in the iOS Keychain.
• Opt-In Safety Notice: Requires explicit user confirmation prior to sending the first completions payload.

For details, refer to SECURITY.md and PRIVACY.md.

---

Documentation

Document	Purpose
2.6 release dossier	Complete release documentation and evidence index
Changelog	Detailed categorized 2.5 → 2.9 changes
Architecture	System design, data flow, and service boundaries
Security	Secret handling, local storage, and release checks
Privacy	Data storage, API transmission, and user controls
Roadmap	Current status, planned work, and known gaps
App Store Notes	App Store metadata, review notes, and release checklist
Case Study	Engineering retrospective and implementation notes
Contributing	Local development setup and contribution guidelines
Release Notes (v2.8.0)	Summary of changes, fixes, and updates in version 2.8.0
Release Notes (v2.9.0)	Version 2.9.0, the version on the App Store, with its What’s New

---

Roadmap

The completed 2.6 source includes current-model native orchestration, Workbench, voice recovery, hosted MCP discovery, browser/search hardening, full job exports and persistence fixes. Versions 2.6, 2.7, 2.8 and 2.9 have shipped. Remaining work includes broader physical voice/accessibility/device checks, private MCP OAuth coverage and account-dependent service validation. See the current roadmap and release plan.

---

License

OpenResponses is released under the MIT License.