Enox AI

Model-Adaptive Composer

Problem: Users were confused when selecting a TTS model but seeing a standard chat input. Solution: The composer dynamically changes based on model type — emerald-themed with voice pills for TTS, purple-themed with Sparkles icon for image/video, and standard chat with attachments for text. This immediately communicates what the model expects without any instructions.

Inline Media Generation (Not Separate Pages)

Problem: Switching between "chat" and "image generator" tabs destroys conversational context. Solution: Media generation happens inline via tool calling. You ask "draw a sunset" in a normal chat, and the image appears in the conversation. The AI decides when to generate, keeping the experience conversational rather than transactional.

Thinking Mode as a Toggle, Not Default

Problem: Extended thinking is useful for complex reasoning but annoying for quick questions — it adds latency and visual noise. Solution: A one-click toggle in the composer lets users opt into thinking mode per-message. When active, thinking streams into a collapsible panel so it doesn't overwhelm the response. Casual users never see it; power users love it.

Model Pinning with Quick-Switch Dropdown

Problem: With 6+ providers and dozens of models, the model selector becomes overwhelming. Solution: Users pin up to 3 favorite models. The dropdown organizes by Pinned → Recent → All, so your daily-drivers are always one click away. Type badges (IMG/VID/TTS) provide instant visual cues about model capability.

Privacy-First Attachment Handling

Problem: Users hesitate to upload sensitive documents (contracts, medical images) to AI platforms that might store them. Solution: Attachments are explicitly never stored in the database. Only a placeholder like [Image sent] is persisted. The user can verify this — chat history shows only placeholders, not their actual files. This was a deliberate trust-building UX decision, not just a technical one.

Skeleton Shimmer for Media Generation

Problem: Image/video generation takes 5-30 seconds. Without feedback, users think the app is broken and re-submit. Solution: A generating SSE event immediately shows a type-appropriate shimmer skeleton (image aspect ratio, audio waveform, or video player shape). This communicates "I'm working on it" within 200ms, even though the media won't arrive for seconds.

Server-Sent Events over WebSockets

I chose SSE because the data flow is strictly unidirectional: the server streams tokens to the client. WebSockets are bidirectional, which adds complexity (heartbeat pings, reconnection state, socket lifecycle management) for zero benefit here. SSE also works natively with HTTP/2 multiplexing, auto-reconnects on disconnect via the EventSource API, and passes cleanly through every reverse proxy and CDN without special upgrade headers. WebSockets require an HTTP Upgrade handshake that some corporate firewalls and load balancers block.

When I'd pick WebSockets instead: If Enox had real-time collaborative editing or live presence indicators (multiple users typing), I'd need true bidirectional communication. For a chat streaming use case, SSE is strictly simpler and more reliable.

Supabase over Raw PostgreSQL

I chose Supabase because it bundles three things I'd otherwise have to build myself: authentication (Google OAuth with JWT out of the box), Row-Level Security policies with a clean SDK, and a hosted Postgres instance with connection pooling. Setting up raw Postgres + a custom auth server + session management + OAuth provider integration would have taken weeks and introduced security surface area I'd have to maintain forever.

The tradeoff: Supabase's RLS is powerful but opaque — policy bugs are hard to debug because queries silently return empty results instead of throwing errors. I also can't use advanced Postgres features like logical replication or custom extensions without their approval. For this project, the speed-to-ship and built-in auth far outweighed those limitations.

Express.js over Fastify

I chose Express because the AI SDK ecosystem is built around it. The OpenAI SDK examples, Anthropic streaming guides, and Google GenAI tutorials all use Express. Fastify is ~2× faster in raw benchmarks, but the bottleneck in an AI chat app is never the HTTP framework — it's the AI provider response time (800-3000ms). Saving 0.5ms on routing is meaningless when you're waiting 2 seconds for GPT-4o to think. Express also has a vastly larger middleware ecosystem (Helmet, CORS, express-rate-limit) that Just Works.

When I'd pick Fastify instead: For a high-throughput API that handles 10,000+ req/s where framework overhead matters (e.g., a REST API serving cached data). Enox's backend handles maybe 50 concurrent streams — Express is not the bottleneck.

Zustand over Redux / Context API

I chose Zustand because streaming chat creates extreme state update pressure — every token triggers a store update. Redux's reducer dispatch overhead and middleware chain add measurable latency at 30+ updates/second. React Context causes full subtree re-renders on every update (catastrophic for a chat UI). Zustand's useShallow selectors and direct state mutation give me surgical re-render control with zero boilerplate.

The tradeoff: Zustand has no built-in dev tools as mature as Redux DevTools, and the "single store" pattern can get unwieldy. I mitigated this by keeping the store interface clean and using localStorage persistence selectively (7 keys) rather than syncing everything.

Monorepo over Separate Repositories

I chose a monorepo because the three projects (app, backend, admin-app) share types, constants, and deployment context. With separate repos, a database schema change would require coordinated PRs across 3 repositories. In a monorepo, one commit can update the schema, backend route, and frontend type simultaneously. The mental overhead of "which repo has the bug?" disappears.

The tradeoff: No shared package extraction (like a /packages/types workspace) — types are duplicated between frontend and backend. I accepted this because the duplication is small (a few interfaces) and a Turborepo/Nx setup would add tooling complexity disproportionate to the project's size.

Native Gemini SDK + OpenAI SDK (Dual Path) over Single SDK

I chose to maintain two code paths because Google's OpenAI-compatible endpoint lies about streaming. It buffers the entire Gemini response (~5-8 seconds), then "streams" it as a burst of chunks. The native @google/genai SDK delivers true token-by-token streaming with ~200ms time-to-first-token. For the most popular free model (Gemini 2.5 Flash), this made the difference between "feels instant" and "feels broken."

The tradeoff: Two code paths means double the maintenance surface for streaming logic, error handling, and tool calling. Every new feature has to work on both paths. I mitigated this with the yield* delegation pattern — both paths yield the same typed events to the same consumer, so the downstream code is unified.

Token Extraction

Extracts Bearer token from the Authorization header. Returns 401 immediately if missing.

In-Memory Cache Check

Checks a Map<token, {profile, ts}> cache with a 5-minute TTL. If valid, skips DB calls entirely.

Supabase Verification

On cache miss: calls supabase.auth.getUser(token), then fetches the full user profile from public.users. Caches the result.

Instant SSE Open

The SSE stream opens immediately — before any database work. This cuts ~1s off perceived latency. A 2KB comment padding flushes proxy buffers (Cloudflare, nginx, Traefik).

Mega Batch Pre-flight

All pre-flight checks run in a single Promise.all(): model lookup, agent data, rate limit check, and user API keys fetch. This parallelizes 4 database queries into one round-trip.

API Key Resolution

If BYOK is enabled, uses the user's key (bypasses rate limiting). Otherwise, uses the platform's key. For tool calling (image/TTS generation), merges both key pools to maximize available providers.

Typed SSE Event Streaming

The stream yields typed events: thinking_start, thinking_content, thinking_done, chunk (text), generating (skeleton), media (base64), clear_content, done, and error.

Fire-and-Forget Persistence

After [DONE] is sent, the response ends immediately. Message saving and usage increment happen asynchronously in background promises — the user never waits for DB writes.

Browser → Express  POST /api/chat/send

Client sends the message payload (modelId, message, attachments, settings). Express validates with Zod.parse(body).

Express → Browser  T+0ms

SSE stream opens instantly — before any DB work. Response headers flushed with a 2KB padding comment to force proxy buffer flush.

Express → Supabase  Promise.all()

Four parallel queries execute simultaneously: model lookup, agent data, rate limit check, user API keys. All 4 results return in a single round-trip.

Express (internal)  T+80ms

resolveApiKey() — O(1) lookup from pre-fetched keys. buildGenApiKeys() — merges user + platform keys for tool calling. insertChat() — creates new chat session (new chats only).

Express → Browser  meta:{chatId}  T+120ms

Chat ID is sent to the client. The frontend updates activeChatId in the Zustand store and prepends the new chat to the sidebar list.

Express → AI Provider  T+200ms

buildMessages() constructs the messages array, then stream.open() initiates an HTTP/2 stream to the AI provider via cached client connection.

AI Provider → Express → Browser  Thinking Phase

The AI model's internal reasoning streams back in real-time:

AI Provider → Express → Browser  Token Streaming  T+800ms first token

Text tokens stream through the async generator pipeline. Each token is forwarded as a chunk SSE event with <1ms relay overhead. May also yield generating → media events for tool-called content.

Express → Browser  done + [DONE]  T+3200ms

Stream completes. done:{chatId} event signals the frontend to finalize the message and enable regeneration. [DONE] terminates the SSE connection.

Express → Supabase  Fire-and-Forget  T+end (async)

After the client connection is closed, two background promises persist data without blocking the user: message INSERT and usage increment via atomic upsert. Zero impact on perceived latency.

What happened

Early on, all 6 providers used the same OpenAI-compatible SDK code path. Everything seemed fine — until I tested Gemini 2.5 Flash side-by-side with GPT-4o. Gemini had a consistent 5-8 second delay before the first token appeared, while GPT-4o started streaming in ~800ms. Users reported "Gemini is broken" even though it was technically working.

Root cause: Google's OpenAI-compatible endpoint (generativelanguage.googleapis.com/v1beta/openai/) doesn't actually stream. It buffers the entire response server-side, then sends all chunks in a rapid burst. The "streaming" is fake — you get 0 tokens for 6 seconds, then all 500 tokens in 200ms.

The fix: I built an entirely separate native Gemini code path using the @google/genai SDK, which supports real token-by-token streaming. This required duplicating all streaming logic, tool calling handling, and error management for Gemini specifically. The result cut Gemini's time-to-first-token from ~6s to ~200ms — but the cost was maintaining two parallel streaming implementations (§5g).

Lesson: Never trust "OpenAI-compatible" claims without benchmarking the actual streaming behavior. Compatibility layers optimize for correctness, not latency.

What happened

Tool calling (for image/TTS generation) worked perfectly on OpenAI and Gemini. Then I tested it on Claude via OpenRouter, and the chat just... printed raw JSON. The model understood the tool definition but instead of making a function call, it wrote {"tool": "generate_image", "prompt": "a sunset over mountains"} as plain text into the chat.

It got worse: Every model that "faked" tool calls did it differently. Some wrapped JSON in markdown code blocks. Some used action/action_input DALL-E format. Some just inferred a type: "image" field. I found at least 5 distinct JSON patterns across providers, and new ones kept appearing as I tested more models.

The fix: The detectFakeToolCall() function (§5c) — a multi-pattern matching pipeline that extracts JSON from markdown blocks or raw text, then tries to match it against 5 known tool-call formats. When detected, it clears the JSON from the UI via a clear_content SSE event and executes the generation transparently. This was the messiest code I wrote, and it still occasionally fails on novel model outputs.

Lesson: The AI ecosystem's "function calling" standard is a lie. Every provider implements it differently, and many models will simply ignore the schema and do their own thing. You need both a clean path and a dirty fallback.

What happened

Streaming worked perfectly in local development. The moment I deployed behind Cloudflare, SSE events were delayed by 15-30 seconds. The entire AI response would accumulate silently, then dump to the client all at once. Users saw a blank screen for half a minute, then the full response appeared instantly. It looked completely broken.

Root cause: Reverse proxies (Cloudflare, nginx, Traefik) buffer response data until they accumulate enough bytes to justify a network flush. SSE events are tiny (~50-200 bytes each), so they sit in the proxy buffer waiting for more data that never comes. The proxy's "optimization" was destroying the entire streaming UX.

The fix: Three-layer workaround: (1) A 2KB SSE comment (: + 2048 spaces) sent immediately to overflow the proxy buffer threshold. (2) X-Accel-Buffering: no header to explicitly disable nginx buffering. (3) socket.setNoDelay(true) to disable Nagle's TCP algorithm, which was batching small SSE frames. All three were necessary — removing any one brought the delay back in certain deployment configs.

Lesson: Local development is a lie for streaming applications. Always test SSE through at least one reverse proxy layer before calling it "done."

What happened

Gemini 2.5 Flash supports thinking mode + tool calling. Gemini 2.0 Flash supports tool calling but not thinking. Gemini 1.5 Pro supports neither. There is no API endpoint to query which features a model supports. I only discovered this through trial and error — sending a request with thinking enabled, getting a 400 error, and then realizing I had to maintain a compatibility matrix in my head.

It got worse: Google sometimes updates model capabilities silently. A model that didn't support tools on Monday might support them on Wednesday. Hardcoding a feature matrix would become stale instantly.

The fix: The three-stage progressive fallback state machine (§5g). Instead of trying to know in advance what each model supports, I attempt the most capable configuration first (thinking + tools), catch the error, check if it's an "unsupported feature" error, and retry with a simpler configuration. This makes the system self-healing — it adapts to model capabilities at runtime without any hardcoded knowledge.

Lesson: When working with third-party AI APIs, design for capability discovery at runtime rather than static configuration. APIs change under you.

What happened

Gemini TTS returned audio data with MIME type audio/L16;rate=24000. I Base64-encoded it, sent it to the browser, and... silence. The <audio> element refused to play it. No error, no console warning — just nothing. Chrome, Firefox, Safari all silently failed.

Root cause: Browsers cannot play raw PCM audio. They need a container format (WAV, MP3, OGG). Gemini returns headerless 16-bit linear PCM samples — just raw bytes with no metadata about sample rate, channels, or bit depth. The browser has no way to interpret the data.

The fix: I wrote a manual pcmToWav() function (§5e) that constructs a 44-byte RIFF/WAVE header byte-by-byte using DataView, then prepends it to the raw PCM data. The sample rate is extracted from the MIME type string via regex. I also had to add magic byte detection (checking for RIFF and 0xFF 0xE0 MP3 sync headers) because OpenAI TTS returns MP3 directly, and wrapping an MP3 in a WAV header produces garbage.

Lesson: "Returns audio" in an API doc doesn't mean "returns playable audio." Always check the actual byte-level format, not just the MIME type.

What happened

I built a public agents feature where users can share their custom agents. Another user can chat with your agent for free — great for discoverability. Then I realized the exploit: a user creates 10 public agents, shares them, and 50 people use them. All 500 daily requests consume the platform's API key budget, but nobody's individual rate limit is hit because usage was only tracked against each consumer individually.

The fix: Dual usage attribution (§5h). When someone uses a public agent on the platform key, usage is counted against both the consumer (who sent the message) and the agent creator (who published the agent). This prevents creators from bypassing their rate limits by laundering usage through public agents. The creator's count is only incremented when the platform key is used — if the consumer brings their own key, the creator isn't penalized.

Lesson: Any "sharing" feature in a rate-limited system is a potential bypass vector. Always ask: "Who pays for the compute when content goes viral?"

What happened

Voice recording worked perfectly on Chrome desktop and Android. On iOS Safari, the MediaRecorder API silently produced empty blobs. The recording UI appeared to work — the timer ticked, the animation played — but the resulting audio file was 0 bytes.

Root cause: iOS Safari doesn't support audio/webm (the default format on Chrome). It supports audio/mp4 and audio/aac, but doesn't throw an error when you request an unsupported format — it just produces garbage output.

The fix: The VoiceRecorder component now probes format support at initialization with a priority list: audio/mp4 → audio/aac → audio/webm. It uses MediaRecorder.isTypeSupported() to find the first working format. This cascading approach handles iOS, Android, and desktop without user-agent sniffing.

Lesson: Never trust MediaRecorder to fail loudly. Always probe format support before recording, and always test audio features on actual iOS hardware (simulators lie about codec support).