System Design: OpenAI/Gemini Platform

Name: How ChatGPT Works Technically | ChatGPT Architecture
Uploaded: 2023-04-24T08:30:15-07:00
Duration: 7 min 53 s
Channel: ByteByteGo
Description: The eight-minute version of what sits behind a model endpoint, before you design your own.

You run an LLM platform like OpenAI/Gemini. Provide API for completions/chat, fine-tuning, moderation, usage tracking, billing, and safety with global latency under 300 ms.

System Design: OpenAI/Gemini Platform

What clarifying questions would you ask the interviewer?