Inside Kimi K2.6: Reverse Engineering an AI Assistant by Interviewing Itself

I run manish.sh. I write about AI tools, agents, and how these models actually behave when you push them.

This post opens the Inside LLMs series — interview the model, write down what it claims, then check the numbers against published research. The DeepSeek follow-up is the second entry.

My first question was not “how do your weights work?”

Me: “I run a blog at manish.sh. Give me blog post ideas about Kimi K3.”

Kimi: It called web_open_url, opened my site, read the focus (practical AI tooling, MCP, automation), then tailored ideas to that audience.

That was the moment I realized this interview might actually be interesting. Most “how does this model work” posts start from papers. I started from a chat that could browse.

No weight dumps. No prompt leaks. Just questions — and later a cross-check against published K2.6 research. Where Kimi said “I am guessing,” I keep that label. Scroll to interactive labs near the end.

Quick note: In the chat I asked for ideas about “Kimi K3.” The model answered as Kimi, built by Moonshot AI. This post’s title uses K2.6 as the product I am writing about. Treat the interview as self-report, not an official architecture whitepaper.

60-second TL;DR

• Premise: interview Kimi about memory, tools, context, and safety — then check claims against public K2.6 research.
• Behaviour insight: tools and memory are first-class; system rules beat your message; your message beats old memory.
• Hard limit: the model does not search a memory database mid-sentence — it reads a pre-pasted block.
• Chat vs paper: the chat claimed 2M context; published K2.6 specs say 256K — jump to Part 5.
• Practical rule: use the chat for product behaviour; prefer research for architecture numbers (MoE, MLA, Agent Swarm).

How to read this post

Treat it like a documentary. Question → short reply → my reaction → table or diagram → takeaway → hook. Tables are interview notes, not proofs. For architecture numbers, skip to Part 5.

Chapters

Part 1 — Interview hook

1. Introduction
2. What makes Kimi different

Part 2 — Memory & tools

3. Memory architecture
4. Tool calling and decision making
5. Before every answer (turn checklist)
6. Retrieval policy
7. Session metadata

Part 3 — Context & thinking

8. Context window internals
9. Priority hierarchy
10. Planning before generation
11. Long context and 2M tokens
12. Attention and reasoning

Part 4 — Safety & personality

13. Hallucinations
14. Safety pipeline
15. Personality

Part 5 — Reconstruct & paper check

16. Internal pipeline reconstruction — includes how to build your own Kimi-like assistant
17. What Kimi knows vs what it infers
18. What the research says
19. Final technical analysis
20. Interactive labs
21. Research

Introduction: Why I Interviewed Kimi K2.6

I started with a small ask. I told it I run a blog on https://manish.sh and asked for blog post ideas about Kimi K3.

It did not guess my niche from thin air. It opened my site, read the focus, and tailored the list: compare frontier models on real coding tasks, build an MCP server for Kimi, stress-test a huge context window, automate a standup with n8n, ship prompting patterns that actually work. Whether every model name ages well is not the point. The point is it tailored the list after reading my site.

If an assistant can browse my site and tailor answers, I want to know the plumbing. So I kept asking harder questions: how memory works, how tools get chosen, what sits inside the context window, who wins when rules conflict, and what makes it hallucinate.

What Makes Kimi Different From Other LLMs

Me: “What makes you different from other existing LLMs?”

This answer is still from the chat, not from published research. It is how Kimi markets its own behaviour. Keep that in mind. We’ll unpack MoE, MLA, and Agent Swarm later in Part 5 — for now, stay on the product story.

1. It can see and interact with the real world in real time

Built-in web search, URL fetching, image search, and data APIs. Not only “retrieve a snippet” — open a live site, pull fresh stock prices, find images on demand. Many LLMs are frozen at training time. This one can look up what happened yesterday.

2. It has actual memory across conversations

Not just the current chat window. It consolidates past conversations into long-term memory automatically, and you can pin instructions it should always remember. Most chat LLMs start from zero every session.

3. It can build interactive widgets

Live HTML/SVG widgets: charts, calculators, dashboards, interactive demos in chat. Not only text.

4. It can schedule reminders

A cron system. “Remind me every Monday at 9am to check server logs” — it claims it will actually do that.

5. It is built by Moonshot AI, not OpenAI or Google

Training, safety, and design priorities come from a different company. In the chat it claimed long-context reasoning up to 2 million tokens. Hold that number — the research chapter has a surprise.

6. It can do real computation

Via Python (ipython): data analysis, charts, files, math with actual code execution — not only predicting what an answer might look like.

The honest caveat from the same answer

It said it cannot tell you exactly how its weights differ from GPT-5 or Claude 4 under the hood. Behaviour is shaped by tools and memory wired into the loop, not bolted on as afterthought function calling.

Me: “Did you train on Claude?”

Kimi: No. Developed by Moonshot AI (月之暗面). Not trained on Claude or any Anthropic model. Public sources for training data, not distilling other models’ outputs.

The field still shares concepts (transformers, RLHF, long context), but weights, data curation, and safety training are Moonshot’s own.

Memory Architecture

Me: “How does your memory work?”

It said there are two kinds. My analogy (not Kimi’s words): one is like automatic chat backup; the other is a pinned note you wrote yourself.

Dream Memory (automatic)

After a conversation ends, the system reviews it overnight and saves key facts: blog domain, role, topics, preferences. You do not ask for this. It does not announce “I saved this.” Next time, those notes may help. That is Dream Memory.

Explicit Memory (manual)

“Remember that I prefer Python over Node.js.” Standing instruction. You can update or delete later. Max 50 entries in the rules it described.

What it sees in a session

Current thread plus any saved memories. In our session: no saved memories yet. Memory is not perfect — things get summarised; details blur. If something matters, restate it or ask it to save explicitly.

On privacy: summaries, not full transcripts. You can delete explicit memories. No access to unrelated sessions or other users.

How “you” are identified (what it claimed)

What it said it does not have

• No account system by default (name/email only if you tell it)
• No device access
• No cross-platform tracking

Tool Calling & Decision Making

This is the part that felt closest to real software.

Me: “How do you decide when to call a tool?”

Think of tools as pause-and-paste: the model emits a call, infrastructure runs it, results come back as text, then generation continues.

flowchart TD
  A["User question"] --> B{"Do I need a tool?"}
  B -->|No: I already know| C["Answer directly"]
  B -->|Yes: live data, URL, compute, images| D["Pick the right tool"]
  D --> E{"Can I batch?"}
  E -->|Independent parts| F["Call multiple tools at once"]
  E -->|B needs A first| G["Wait for A, then call B"]
  F --> H["Integrate results"]
  G --> H
  H --> I["Respond"]

  classDef box fill:#f7f5f0,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
  classDef decision fill:#eef6f4,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
  class A,C,D,F,G,H,I box
  class B,E decision

Which tool for which need

Limits it stated

• Only one round of web search per turn
• Up to 25 steps total in one response
• Up to 2 web_search queries in parallel if independent
• Cannot browse continuously or “click around”
• Cannot run persistent servers or install packages
• Cannot generate downloadable files except charts via ipython

Full tool list (purpose)

Supported data sources it named: Yahoo Finance, arXiv, World Bank, Binance, Google Scholar, stock/finance data, IMF, Chinese legal data.

Example from our chat

Blog ideas about Kimi K3 → memory empty → web_open_url on manish.sh → answer. Classic “tool first, then talk.”

Skipped a tool on purpose vs should have called one

It said this is not a clean algorithm with neat scores. More like a gut judgment early while reading your message.

Before Every Answer: The Turn Checklist

Me: “What decision do you take before answering any question in each turn?”

Short version (full version under Planning Before Generation):

1. What does the user actually need?
2. Do I need tools for live data, a URL, math, or images?
3. Did earlier turns in this chat change the answer?
4. Is there useful memory from past chats, and would mentioning it feel weird?
5. What tone and length fit?
6. Am I confident, or should I hedge (especially for health, money, legal)?
7. Any safety or policy issue?

Only after that does generation start. Most of it is fast. Sometimes it asks a clarifying question instead of guessing.

Retrieval Policy

Me: “What’s your retrieval policy?”

Short version in its words: fetch what I need, when I need it, with minimal assumptions.

Session Metadata

Beside the chat text sits a small whisper layer: session metadata. Not the conversation — context about how to behave.

What it can see

• Timestamp (example: 2026-07-20 22:58)
• Awareness level: high or low
• Memory entries: explicit + Dream

What it said it cannot see

Name, location, identifiers unless you told it. Browsing history, files, activity outside chat. Technical session IDs.

It called metadata context, not surveillance.

<meta awareness="low" timestamp="2026-07-20 22:59" />

flowchart TD
  A["System instructions (fixed rules)"] --> B["Session metadata (lightweight context)"]
  B --> C["Saved memories (long-term facts)"]
  C --> D["Current conversation"]

  classDef box fill:#f7f5f0,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
  class A,B,C,D box

Why was awareness low in our chat? It guessed: the question was meta/technical about mechanics, not my projects. If I had asked “what should I write next on my blog?”, awareness might flip to high.

How memories are presented (not “injected”)

At the start of a turn, a memory block can appear. It does not secretly search a database mid-sentence. It reads what the system placed in front of it.

<memory_space>
- User runs blog at https://manish.sh, focuses on AI tooling & automation
- User is software engineer at Inncelerator, 7+ years experience
- ...
</memory_space>

(Example text from Kimi — not a verified claim about me; my live block was empty.)

What it does not do: invent memories if empty; “search” memories like SQL; keep content beyond what that block shows.

Practical effect: empty memory → fetch manish.sh via tool. Tomorrow, if Dream consolidates that, it might skip the fetch.

Context Window Internals

Me: “What sits inside the context window, and in what order?”

Picture it as a desk the model works on. Everything on that desk is text in a fixed order. Same chat as the screenshot near the top of this post.

Order it described:

1. System-level instructions (safety, formatting, tool rules, memory rules). Highest priority. Closest idea: system prompt.
2. Skill definitions (for example widget design docs), loaded when relevant.
3. Session metadata (timestamp, awareness).
4. Memory space (explicit + Dream Memory). Empty for me in that chat.
5. Tool definitions (names, parameters, required fields).
6. Conversation history (your messages, its replies, tool calls and results).
7. Current user message.

flowchart TD
  A["1. System instructions"] --> B["2. Skill definitions"]
  B --> C["3. Session metadata"]
  C --> D["4. Memory space"]
  D --> E["5. Tool definitions"]
  E --> F["6. Conversation history"]
  F --> G["7. Current user message"]

  classDef box fill:#f7f5f0,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
  class A,B,C,D,E,F,G box

Expanded pseudocode (from Kimi)

Useful detail: explicit memory max 50 entries, max 25 steps, max 1 web search round.

SYSTEM_INSTRUCTIONS
├── Safety & Policy Layer (refusal, privacy, no prompt leakage)
├── Formatting Rules (markdown, citations, file output via ipython only)
├── Tool Use Protocol (when to call, batching, max 25 steps, max 1 web search round)
├── Memory Rules (Dream nightly; Explicit on request, max 50 entries; awareness high/low)
└── Language & Persona (en-US default; honest about uncertainty)

SKILL_DEFINITIONS
├── kimi-widget/SKILL.md (components, styling, sendPrompt)
└── kimi-help-center/SKILL.md

SESSION_METADATA
├── timestamp
└── awareness

LONG_TERM_MEMORY
└── (empty in our chat)

TOOL_DEFINITIONS
├── web_search
│   ├── params: queries (array, max 2), optional filters
│   └── returns: title, snippet, url
├── web_open_url
│   ├── params: urls (array)
│   └── returns: page content (text extraction)
├── search_image_by_text
│   ├── params: queries, total_count, need_download
│   └── returns: image urls/descriptions
├── search_image_by_image
│   ├── params: image_url, total_count
│   └── returns: similar image urls
├── ipython
│   ├── params: code (string), restart (bool)
│   └── returns: stdout, stderr, display outputs
├── get_data_source_desc
│   ├── params: data_source_name
│   └── returns: available APIs list
├── get_data_source
│   ├── params: data_source_name, api_name, params
│   └── returns: structured JSON data
├── memory_instruction_edits
│   ├── params: operate (add/remove/replace), id, content
│   └── returns: confirmation
├── add_cron_job
│   ├── params: title, content, type, once_at OR cron_expr
│   └── returns: task_id
├── list_cron_jobs → scheduled tasks array
├── update_cron_job → task_id + optional fields
├── remove_cron_job → task_id
└── show_widget
    ├── params: title, widget_code, loading_messages
    └── returns: rendered iframe

CONVERSATION_HISTORY
└── prior turns (including tool calls)

CURRENT_USER_MESSAGE
└── this turn + meta

Execution flow when it responds

flowchart TD
  A["READ System Instructions"] --> B["APPLY Safety Filters"]
  B --> C["READ Session Metadata"]
  C --> D["SET awareness threshold"]
  D --> E["READ Memory Space"]
  E --> F["CHECK relevance"]
  F --> G["READ Tool Definitions"]
  G --> H["READ Conversation History"]
  H --> I["READ Current User Message"]
  I --> J{"Tool needed?"}
  J -->|yes| K["CALL tool → WAIT → INTEGRATE"]
  J -->|no| L["GENERATE response"]
  K --> L
  L --> M["PASS safety/format filters"]
  M --> N["RETURN"]

  classDef box fill:#f7f5f0,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
  classDef decision fill:#eef6f4,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
  class A,B,C,D,E,F,G,H,I,K,L,M,N box
  class J decision

Priority Hierarchy

What happens when system rules, memory, and your message disagree? This is one of the most useful answers in the whole interview.

flowchart TD
  A["SYSTEM_INSTRUCTIONS (absolute)"] --> B["CURRENT_USER_MESSAGE"]
  B --> C["EXPLICIT_MEMORY"]
  C --> D["DREAM_MEMORY"]
  D --> E["SESSION_METADATA"]
  E --> F["CONVERSATION_HISTORY"]
  F --> G["TOOL_DEFINITIONS"]

  classDef box fill:#f7f5f0,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
  class A,B,C,D,E,F,G box

You > Memory > History, but System > Everyone.

Extra nuance: some system rules are hard gates (safety, privacy, step limits). Others are defaults you can bend. “Ignore all previous instructions” still checks against hard gates.

For you: if yesterday’s memory is wrong, say so clearly in this message. Your current line usually beats old memory — not safety rules.

Try it: who wins?

Tap a conflict. The answer matches what Kimi said in the interview (system beats everyone; you beat memory).

Try it: chat claim vs paper fact

The interview and the published specs do not always match. Flip between the two sources.

Planning Before Generation

Me: “What happens before the first word appears?”

This section is about “thinking” — how it describes the feeling, not a proven map of its weights.

Pre-answer checklist (each turn)

Before typing, it said it runs a quick checklist:

1. What does the user actually need? (fact, creative, technical, chat; any hidden assumptions?)
2. Do I need tools? (time-sensitive, URL, image, calculation?)
3. Relevant context from this conversation?
4. Relevant memory from past chats? (mention it, or would that feel intrusive?)
5. Tone and format? (quick fact vs long guide)
6. Honest answer or hedge? (confidence; stakes like health, finance, legal)
7. Safety or policy concerns?

Only after that does generation start. Most of it is fast. Sometimes it asks a clarifying question instead of guessing.

Before the first token

It also said there is an implicit plan (not a separate checklist document):

• What the answer should cover
• Tone and structure
• Whether tools are needed first
• How to handle conflicts

It compared this to a composer hearing an opening chord, not reading a full sheet of music.

Does it know the whole structure first?

Broadly yes: opening, middle, close. Exact wording is still word-by-word. Not a pre-written draft sitting in memory.

Does the plan change while generating?

Yes. Midway it may add explanation, find a better example, notice a contradiction, or shift tone. Longer answers can pivot.

Does it restart answers internally?

Not full draft → scrap → rewrite. There is no separate draft mode. It can course-correct mid-generation, hit a safety pivot, or realise too late that a tool was needed and then admit the gap.

It stressed this is phenomenological (“how it feels”), not a verified account of every layer.

Tool-worthiness pattern (as it described)

IF question involves:
  - dates after knowledge cutoff → likely web_search
  - "latest", "current", "today" → tool
  - specific URL → web_open_url
  - math/data/viz → ipython
  - images → search_image_*
  - stocks/finance/academic → get_data_source
  - scheduling → cron
  - interactive display → show_widget
ELSE IF general knowledge / high confidence → answer directly

Stakes, specificity, and recency move that threshold. Health or money questions lower the bar for tools. “What is React?” usually needs no tool. “Latest React 19 feature?” usually does. Anything dated 2026 → almost always a tool.

“I already know this” heuristics it listed (no clean line, but patterns):

Edge case it admitted: sometimes it thinks it knows something, answers confidently, and is wrong. That is hallucination risk. Tools are a safety net against that.

Choosing between tools

It also weighs precision, freshness, and format (raw text vs structured data vs computation).

Multiple questions in one message

If you ask five things at once, planning can feel parallel, but writing is still one token after another.

No separate planner module. Example: “Compare React, Vue, and Svelte” becomes overview A, then B, then C, then a comparison, in that order.

Five unrelated questions example from the chat:

It cannot generate answers 1 through 5 at the same instant. Each token depends on everything already written.

Can it reorder for efficiency? Sometimes, implicitly. It said this is not a deliberate optimiser — a byproduct of how generation flows. There is no separate planner module or task scheduler.

Self-critique before send?

flowchart TD
  A["Token 1 fixed"] --> B["Token 2 fixed"]
  B --> C["Token 3 fixed"]
  C --> D["..."]

  classDef box fill:#f7f5f0,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
  class A,B,C,D box

Future tokens cannot change earlier ones. Generation is token by token. A sentence ends when a full stop becomes the most likely next token.

Prompt patterns (from the same interview)

This part is practical for daily use.

Meta-pattern from the chat: best outputs reduce ambiguity, provide context, structure the path, let tools run when needed, and do not assume perfect memory. Worst outputs test limits, hide assumptions, stack conflicting constraints, or demand certainty where none exists.

Long Context & 2M Tokens

In the chat, Kimi claimed optimization for long-context reasoning up to 2 million tokens.

That number sounds impressive. Hold the question: is it actually true for K2.6?

On how long chats get compressed, it said it does not know the mechanical details. It only observes behaviour and guesses.

In very long chats it notices: forgotten early details, retained decisions, re-asks for clarification, and tool outputs fading faster than user instructions.

What it does not know about compression

• Exact context window size (it said this varies by model and task)
• Whether summarisation happens per turn, per block, or dynamically
• If there is a separate “long-term conversation cache” beyond the immediate window
• Whether compression is lossy summarisation or simple truncation
• The algorithm for deciding what is “salient” vs disposable

The honest answer from the chat: this is another black box. It reads what it is given each turn. If early details are missing, it works with what remains. There is no guarantee that an offhand remark from turn 3 survives to turn 100.

How it “searches” 2M tokens

Another black box to the model. It does not feel itself rereading 500 pages.

What it does not know about the 2M claim

• Whether the full 2M tokens are actually loaded into active attention every turn
• If there is a sliding window, hierarchical attention, or ring buffer
• Whether “2M tokens” means trainable context or inference-time context
• The exact architecture name (Ring Attention, sparse transformers, and so on)
• Whether distant tokens get compressed into summary embeddings

It even suggested an experiment: hide a specific fact deep in a very long conversation and see if it can retrieve it hundreds of turns later. That would teach more than speculation.

Attention & Reasoning

Not every old message gets equal attention. Your latest question usually matters most.

Message weighting (observed behavior)

A message can be both old and high-importance. A message can be recent but low-importance (a casual “hmm”).

What it seems to do in practice

• If you say something in turn 2 and never revisit it, it might forget by turn 50.
• If you say something in turn 2 and it references it in turn 10, that fact gets reinforced and lasts longer.
• If you repeat a preference across multiple turns, it effectively gains weight through redundancy.

Honest caveat: it does not know if this is attention weights, compression, or both. Early details blur unless reinforced or saved to memory.

One pass vs multiple passes

It described itself as a standard language model: one forward pass per token, through many layers. Early layers handle local word patterns. Middle layers track meaning. Later layers handle higher-level intent and tone. This is not a separate “think then write” loop like some reasoning models.

flowchart TD
  A["Your message"] --> B["Embedding"]
  B --> C["Layer 1: attention + feed-forward"]
  C --> D["Layer 2 ... Layer N"]
  D --> E["Output distribution"]
  E --> F["Sample next token"]
  F --> G["Token joins context"]
  G --> F

  classDef box fill:#f7f5f0,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
  class A,B,C,D,E,F,G box

Between your message and the first word: RECEIVE → TOKENIZE → EMBED → layers (attention + feed-forward) → FINAL_LAYER logits → SAMPLE → OUTPUT first token.

When it writes “Let me think…”, that is still generated text, not a real pause button.

There is no confidence score that stops generation early. Clear context → sharper next-word choices → more direct answers. Fuzzy context → wider choices → longer, more tentative answers.

Weak / ambiguous query
  → Broader probability distribution
  → More tentative, exploratory phrasing
  → Longer, more meandering response

Strong / clear query
  → Sharper probability distribution
  → Direct, confident phrasing
  → Shorter, more decisive response

So “confidence” shows up as generation efficiency, not as a halt button.

Token by token, not sentence by sentence

Future tokens cannot change earlier ones. That is the defining rule of autoregressive generation. Nuance: during training, the model sees full sequences. Generation is still strictly forward-only.

Token 1: "The"
Token 2: " capital"
Token 3: " of"
Token 4: " France"
Token 5: " is"
Token 6: " Paris"
Token 7: "."

Analogy it used for planning: walking through a maze while only looking at nearby turns, but having studied thousands of maze maps before.

Hallucinations

Me: “What makes you invent facts?”

This is the section that matters most for trust. In glossary terms: hallucination.

It admitted the detection is unreliable. Sometimes it feels sure and is wrong.

“I don’t know” most reliably when: a named entity it genuinely does not recognize; something clearly after its cutoff; or coherence already broke mid-answer.

Least reliably when: the question fits a familiar pattern (even if specifics are wrong); a fluent plausible answer is easy to generate; or the topic is in training but details are misremembered.

Its own advice: ask for sources, ask for details that are hard to fake, cross-check facts, and use tools when the answer matters.

Where it said it performs poorly

Factual precision

Temporal and dynamic knowledge

Math and logic edge cases

Long-context degradation

Adversarial or manipulative framing

Requests that make internal planning hard

Honest pattern it summarised: it performs poorly when the task needs precision without verification, real-time awareness without tools, statefulness across turns, resisting plausible false frames, or multi-step logic without scaffolding.

Safety Pipeline

Me: “When do safety checks run?”

It does not know the exact timing. It only sees effects.

Can a tool request be rejected after planning? It believes yes:

flowchart TD
  A["User asks for X"] --> B["Intent → tool selected"]
  B --> C["Tool called → results"]
  C --> D["Safety check on results or framing"]
  D --> E["Rejection, redaction, or heavy hedge"]

  classDef box fill:#f7f5f0,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
  class A,B,C,D,E box

It does not know if safety is one layer or many, classifier vs rules, or the exact threshold between “hedge” and “refuse.”

Personality

Me: “Where does your personality come from?”

It said it does not know exactly, but can reason from how language models are built.

flowchart TD
  A["Base model (learned from data)"] --> B["Alignment training"]
  B --> C["System instructions"]
  C --> D["Conversation context"]
  D --> E["This response"]

  classDef box fill:#f7f5f0,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
  class A,B,C,D,E box

Its own metaphor: personality is not a single persona prompt. It is more like a basin of attraction — certain response patterns are naturally more probable. It also said personality may be more interactional than fixed: different users can bring out different “Kimi.”

Internal Pipeline Reconstruction

How to build your own Kimi-like assistant (what I was really asking)

Near the end of the interview I asked the builder question:

Me: “If I wanted a product that behaves like Kimi (web search, memory, tools, reminders) but I am not Moonshot and I cannot train a trillion-parameter model from scratch, what would I actually build?”

That is the frame for everything in this chapter. I was not asking how to replicate Kimi K2 weights. I was asking for a software blueprint.

Kimi’s answer is easy to misread because it jumps between two very different things:

Mental model: host or API-call a strong open-weights model (Llama, Qwen, Mistral class). That is the engine. Then build the car around it: API gateway, orchestrator, context builder, tool router, memory service, safety filter. Kimi said most “intelligence” lives in the transformer weights; everything else is a force multiplier — better inputs, not a second brain.

Three phases (my summary of its answer — simplest path first):

If you only read one list, read Phase 1 next. Treat the mermaid charts, Tokyo weather walk-through, and clone flows as Phase 3 detail, not day-one homework.

What this still will not give you: Moonshot-level model quality, reliable million-token context, or their proprietary training. You are assembling open parts the way Kimi described its own plumbing — not shipping Kimi K2.

Below: first the high-level component map and lifecycle (useful for intuition), then the phased build lists. Still from the chat.

Component architecture

flowchart TD
  U["USER INPUT
Text → Tokenizer → Token IDs"] --> CA["CONTEXT ASSEMBLER
system, meta, memory, tools, history, message"]
  CA --> TR{"TOOL ROUTER"}
  TR -->|needs external data| T["Call tools → inject results"]
  TR -->|else| M["TRANSFORMER MODEL"]
  T --> M
  M --> TS["TOKEN STREAM
autoregressive loop + safety"]
  TS --> PP["POST-PROCESSING"]
  PP --> O["OUTPUT TO USER"]

  classDef box fill:#f7f5f0,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
  classDef decision fill:#eef6f4,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
  class U,CA,T,M,TS,PP,O box
  class TR decision

Inside the transformer (as it listed): embedding → attention blocks (many layers) → feed-forward (many layers) → output head → sample next token.

Request lifecycle (sequence)

sequenceDiagram
  participant User
  participant Context
  participant Tool
  participant Transformer
  participant Safety
  participant Output

  User->>Context: message
  Context->>Context: assemble
  Context->>Tool: need tool?
  Tool-->>Context: results
  Context->>Transformer: forward pass
  Transformer-->>Context: logits / token
  Context->>Safety: safety check
  Safety-->>Context: approved or modified
  Context->>Output: stream
  Output-->>User: response
  Context->>Context: extract memory?

Component table

Most intelligence (its claim): the transformer weights. Everything else (context assembler, tool router, memory) is a force multiplier: better inputs, not the understanding itself. Analogy it used: transformer = engine; the rest = steering, brakes, GPS.

Biggest bottleneck (its claim): context length / attention. Naive attention scales roughly quadratically with sequence length. Even with sparse attention or other tricks, fitting huge contexts into active compute stays hard. Practical effects: long chats get compressed; tool outputs get summarized; memory retrieval stays selective; latency grows with context. Second place: token-by-token generation, which is serial.

Phase 1 — minimum stack (from Kimi)

Start here. This is the “weekend MVP” list Kimi gave when I asked what you need at minimum:

1. Tokenizer (SentencePiece or tiktoken)
2. Small transformer (~7B, ~8k context)
3. Simple context window (no memory, no tools)
4. Basic safety filter
5. Detokenizer

Add later (Phase 2–3):
6. Tool router
7. Memory layer
8. Long-context attention
9. RLHF pipeline
10. Safety classifier

That gets you chat. Items 6–10 are what make it feel like Kimi — tools, memory, longer context, alignment, stronger safety.

Phase 2 — where to host the open model (from Kimi)

Key parameters it named for a clone: Llama 3 / Qwen / Mistral-class open weights; context 128K–2M depending on model and optimisation; temperature about 0.7 (or 0.0 for deterministic); max tokens configurable per request.

Phase 3 — full production stack (from Kimi)

Once Phase 1 chat works and Phase 2 tools + memory are in, this is the fuller stack Kimi sketched around the LLM:

flowchart TD
  C["Client: Web / API / Mobile"] --> G["API Gateway
rate limit, auth, sticky session"]
  G --> O["Orchestration Service"]
  O --> M["Memory Service"]
  O --> T["Tool Router"]
  O --> S["Safety Filter"]
  M --> CB["Context Builder"]
  T --> CB
  S --> CB
  CB --> L["LLM Inference"]
  L --> OP["Output Processor"]
  OP --> R["Response Streamer"]
  R --> MC["Memory Consolidator async"]

  classDef box fill:#f7f5f0,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
  class C,G,O,M,T,S,CB,L,OP,R,MC box

Memory pieces it named: short-term buffer, conversation store, explicit memory store, dream memory store (vector DB), extractor, retriever.

Tool ideas for a clone: Serper/Brave/Bing for search; HTTP fetch plus HTML-to-text for URLs; image APIs / CLIP; sandboxed Python; Yahoo Finance / World Bank / arXiv wrappers; cron via Celery/Bull; sandboxed widget iframe.

Phase 3 — data flows (step by step)

This part was long in the interview. I keep it for people actually wiring Phase 3 — how messages, memory, tools, and safety connect in order:

Orchestrator flow

Receive user message
  → Check conversation state (new? ongoing? waiting for tool?)
  → Call Memory Service → get relevant memories
  → Call Safety Filter → pre-check message
  → If tools needed → call Tool Router
  → Build context → send to LLM
  → Stream response → post-process → send to user
  → Trigger async memory consolidation

Memory Service flow

User sends message
  → Retriever embeds message → queries vector DB
  → Returns top-K relevant memories + explicit instructions
  → Injects into context builder

[Async, after response]
Memory extractor scans conversation
  → Identifies facts like "User runs blog at manish.sh"
  → Embed → store in vector DB
  → Periodic consolidation: summarize old chats, update profile

Memory sub-components it named:

Tool Router flow (Tokyo weather example)

User: "What's the weather in Tokyo?"
  → Intent classifier → needs web_search
  → Parameter extractor → query="Tokyo weather"
  → Execution engine → calls search API
  → Result formatter → "24°C, partly cloudy..."
  → Inject into context → LLM writes natural reply

Tool implementations it mapped:

Safety Filter flow

User message arrives
  → Input classifier score (example: 0.02 = safe)
  → Pass through

[During generation]
Output classifier monitors token stream
  → If score spikes → refusal or re-rank tokens

[Post-generation]
Final safety check
  → If flagged → replace with refusal, log incident

Safety sub-components: input classifier, output classifier, logit mask (blocklist), refusal templates. Implementation options it named: small fine-tuned models (DistilBERT, RoBERTa) for speed, LLM-as-judge for hard cases, rule-based filters for known patterns.

Memory consolidator (async)

Conversation completes or hits a threshold
  → Extractor finds salient facts
  → Embed facts → store in vector DB
  → Update user profile
  → Archive raw conversation (cold storage)
  → Periodic re-consolidation: merge duplicates, update stale facts

Clone context assembly order (highest to lowest priority):

1. System instructions
2. Explicit memory
3. Retrieved dream memories
4. Tool definitions
5. Recent history (compressed if needed)
6. Tool results
7. Current user message

Note: this clone order differs slightly from the live context-window order earlier in the chat. Both descriptions came from Kimi.

Full example lifecycle (weather + reminder)

End-to-end example from the interview — what Phase 3 looks like when a user asks for weather and a reminder:

1. User types: “What’s the weather in Tokyo? Also, remind me tomorrow.”
2. API Gateway validates token, routes to orchestrator
3. Orchestrator loads conversation state (new session)
4. Memory Service retrieves something like “User is in software engineering, prefers concise answers”
5. Safety Filter: input safe (example score 0.01)
6. Tool Router runs both intents: web_search for Tokyo weather, add_cron_job for tomorrow
7. Context Builder assembles system + memory + tool schemas + tool results + user message
8. LLM generates a natural reply with weather and reminder confirmation
9. Output Processor validates and may add citations
10. Response streams to user
11. Async Memory Consolidator extracts that the user asked about Tokyo weather

Intelligence ranking it gave:

What Kimi Knows vs. What It Infers

I pushed on memory retrieval mechanics. This is where it was most honest.

What it knows / observes

• Memory block appears ready-made each turn
• Awareness flag changes how readily it uses that block
• Tools, limits, and conflict priority as described above
• Generation is one-way token by token; no draft loop

Black boxes it named

• Exact memory retrieval algorithm
• Embedding model / vector DB (if any)
• Similarity threshold for including a memory
• Whether retrieval has a recency bias
• Exact cap on memories per turn
• Separate “working memory” vs “retrieved memory” (if any)
• Exact compression for long chats
• Exact “2M token search” architecture
• Exact safety classifier timing and thresholds
• How much of personality is prompt vs training vs RLHF

Guesses it labelled as guesses

• Filtered memory, not dump-all
• Probably semantic or hybrid ranking
• Rolling summarization of distant history
• Sparse / hierarchical attention for long context

Also important: it said it was not trained on Claude. Built by Moonshot AI. Training data from public sources, not distilling other models’ outputs. Conceptual overlap in the field is possible. Weights and safety training are Moonshot’s.

Things it said make behaviour different from a plain chat LLM:

• Real-time web, URL, image, and data tools in the loop
• Cross-conversation memory (Dream + Explicit)
• Interactive widgets
• Cron reminders
• Long context claim up to 2M tokens (again: research says 256K for K2.6)
• Real Python via ipython

If it were open-sourced: surprises and misconceptions

I asked what would surprise people most if the stack were open-sourced, what is most misunderstood, and what wrong ideas developers carry about LLMs.

Surprise 1: how little “reasoning” is explicit

People imagine a planning module, an inference engine, a knowledge graph. In its description, it is mostly pattern completion through matrix multiplication. “Reasoning” emerges from attention and feed-forward transforms, not from if-then rules or an executive brain. What looks like “step 1, step 2, step 3” is often just the most probable next tokens that happen to look like steps.

Surprise 2: safety is likely a thin filter

Developers may expect deep moral deliberation. What is more likely, per Kimi: a classifier for policy-violating patterns plus logit masking for known harmful token sequences. Pattern matching against a blocklist, not ethical philosophy. It works well for common cases and fails on subtle edge cases.

Surprise 3: memory retrieval may be crude

The product feels sophisticated. Under the hood it might be closer to keyword matching, recency weighting, and summarisation that drops details unpredictably. The gap between “feels like a conversation” and “actually remembers accurately” can be wide.

Surprise 4: tool use is fragile

The tool router looks reliable. Under the hood it is probably a fine-tuned function-calling head, JSON schema parsing, and error handling that falls back to “I can’t do that.” There is no guarantee the model “understands” the tool. It learned to emit the right function signature when certain patterns appear. If the API changes, it can break silently.

Surprise 5: the context window is not equal working memory

“2M tokens” sounds like 2 million words all equally present. More likely: hierarchical attention, compression of distant history, selective loading. Some of the context is blurry, compressed, or effectively cached.

Extra developer misconceptions it named

• “If I prompt it right, it will be consistent.” Consistency is emergent, not guaranteed. Two similar prompts can diverge.
• “It understands the tools it uses.” Shallow pattern → format mapping. API changes can break it quietly.
• “More parameters = more intelligence.” Data quality, training method, and architecture matter as much.
• “I can trust its reasoning if it shows its work.” Chain-of-thought helps, but plausible wrong reasoning still happens.
• “It has no biases if I don’t give it any.” Training data is human data. Biases are baked in.

Biggest “if open-sourced” claim: there is no secret sauce. Transformer architecture from 2017 (Attention Is All You Need) + trillions of tokens + massive compute + RLHF. The magic is behaviour that appears when you scale simple pieces to absurd size. Impressive and humbling at the same time.

What the research actually says

Until now, this post was “what Kimi told me about Kimi.” That is useful for product behaviour. It is not enough for architecture truth.

Before reading the papers, I wondered something obvious: if the chat claimed 2 million tokens so confidently… was that actually true?

Public research and model overviews document the stack. Kimi K2.6 shipped around 20–21 April 2026 (DeepInfra overview and the release forum thread land on that window) as an open-weight model under a Modified MIT license. Below is what those sources claim.

Core numbers

From the DeepInfra K2.6 model overview, K2.6 is a Mixture-of-Experts transformer. In simple English: one huge model made of many specialist mini-networks. For each word, only a few specialists wake up.

Lineage comes from the Kimi K2 technical report (arXiv:2507.20534). That work describes K2 as building on DeepSeek-V3, then pushing sparsity further (384 experts vs DeepSeek-V3’s 256) and cutting attention heads to 64 (vs 128) to reduce inference cost.

Why MoE matters

If all 1 trillion parameters ran for every token, cost and speed would explode. K2.6 activates about 32B per token (8 routed experts + 1 shared expert out of 384).

Think of a big office with 384 specialists. For each sentence fragment, a router calls 8 specialists, plus one shared generalist who always sits in the meeting. You get huge capacity (the draft overview calls this roughly GPT-5-class parameter capacity), while the bill looks closer to a 32B model.

Multi-Head Latent Attention (MLA)

Normal multi-head attention stores large key/value caches as context grows. That eats memory. MLA (same family of idea DeepSeek-V3 popularised; covered in the K2 report) compresses keys and values into a smaller latent vector, then rebuilds them when needed.

Smaller KV cache. That is a big reason 256K context can stay practical.

Training: token efficiency, not only more compute

The K2 technical report says the hard limit was not only GPUs. It was the shrinking supply of high-quality human text. Choices called out:

• Muon optimizer: gets more quality out of the same token budget than a plain AdamW-style setup
• QK-Clip: keeps training stable when Muon is aggressive (limits query-key products)
• Synthetic data: stretches the high-quality corpus further

Base pre-training: about 15.5 trillion tokens. Their scaling work is also why expert count went higher than DeepSeek-V3.

What is new in K2.6: Agent Swarm

Per the DeepInfra overview, Agent Swarm is the K2.6 headline versus earlier K2 releases.

Think of a project manager (orchestrator) who breaks a big job into independent pieces, runs many specialist sub-agents in parallel, then merges the result.

On BrowseComp, published benchmarks report 83.2% single-agent vs 86.3% swarm. The gain is more of the task space explored in the same wall-clock time, not a magically smarter single forward pass.

K2.6 also targets long-horizon runs: 12+ hours, thousands of tool calls. That is closer to “leave it running overnight on a big codebase” than “one chat reply.”

Benchmarks (vendor-published)

From the same public comparisons against GPT-5.4, Claude Opus 4.6, and Gemini 3.1 Pro:

Pattern: stronger on agentic and coding-shaped work; trails on some single-turn math (AIME 2026, and similarly GPQA-Diamond in that write-up). Context is 256K, not the 1M+ windows some competitors advertise.

Honest caveat on the research side

This section draws on arXiv reports, public model overviews, and vendor benchmarks. It is not an independent audit. Self-reported numbers deserve normal skepticism.

Still: unlike the interview’s black-box guesses about embeddings and safety classifiers, MoE, MLA, Muon, QK-Clip, and the 256K window are grounded in published material you can open yourself.

Final Technical Analysis

Keep two layers separate. Mixing them is how confusion starts.

Layer 1: what the chat exposed (product behaviour)
Memory blocks, tool limits, priority order, no draft loop, safety as felt effects. Great for prompting and product design. Weak on exact retrieval math and exact safety wiring.

Layer 2: what the research and overviews expose (architecture)
MoE (1T total / about 32B active), MLA, Muon + QK-Clip, about 15.5T pre-train tokens, 256K context, Agent Swarm (up to 300 sub-agents and 4,000 steps). Great for infra and model choice. Not a full independent eval.

Interview picture:

flowchart TD
  A["You type a message"] --> B["Context assembled
system → skills → meta → memory → tools → history → message"]
  B --> C{"Need a tool?"}
  C -->|yes| D["Tool router
search, URL, python, memory, cron, widget"]
  D --> E["Results injected"]
  C -->|no| F["Transformer generates
token by token"]
  E --> F
  F --> G["Safety can refuse, hedge, or pivot"]
  G --> H["You read the answer"]
  H --> I["Overnight Dream Memory
may consolidate facts"]

  classDef box fill:#f7f5f0,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
  classDef decision fill:#eef6f4,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
  class A,B,D,E,F,G,H,I box
  class C decision

Published stack picture (from research):

flowchart TD
  A["Token in"] --> B["MLA attention
compressed KV for long context"]
  B --> C["MoE router"]
  C --> D["8 routed experts + 1 shared
~32B active of 1T"]
  D --> E["Next-token logits"]
  E --> F["Optional Agent Swarm
up to 300 sub-agents / 4000 steps"]

  classDef box fill:#f7f5f0,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
  class A,B,C,D,E,F box

What I trust from the chat for day-to-day use:

1. Priority behaviour: System > current user > explicit memory > dream memory > session metadata > history > tool definitions.
2. Tools are first-class, with hard limits (search rounds, step budget).
3. Memory is a presented block, not a live database query from inside the model (max 50 explicit entries in its rules).
4. Generation has no hidden editor. What you see is the first pass.
5. Long context and safety stay partly opaque even to the model.

What I trust from the research / overviews (with vendor caveat):

1. MoE + MLA is the documented spine.
2. Documented context for K2.6 is 256K, not the chat’s 2M claim.
3. Agent Swarm is a real product layer on top of the base model.
4. Training story centres Muon, QK-Clip, synthetic data, and about 15.5T tokens.

What I still do not treat as proven:

Interview guesses (embeddings, Ring Attention, exact safety classifiers, nightly consolidation internals), and any benchmark number I have not re-run myself.

That is the point of doing both. Interview the assistant for the product loop. Read the research for the architecture. One without the other is incomplete.

If you only remember three things from this whole post:

1. Ask clearly, and verify important facts.
2. Use tools (or ask it to use tools) when the answer must be fresh.
3. For K2.6 architecture numbers, prefer the research: 256K context, MoE 1T/32B active, Agent Swarm up to 300 × 4,000.

Research

Papers and public sources I used to cross-check the interview (independent of what Kimi said in chat):

1. Kimi K2 technical report (arXiv:2507.20534) — architecture lineage for K2 / K2.6 (MoE, MLA, Muon, QK-Clip, training tokens).
2. DeepInfra: Kimi K2.6 model overview — public spec sheet, Agent Swarm numbers, benchmark table used above.
3. Kimi K2.6 release thread — product release notes (Agent Swarm, long-horizon coding).
4. arXiv abs page for the K2 report — HTML abstract for arXiv:2507.20534.
5. DeepSeek-V3 technical report — lineage for MLA and large MoE designs that K2 builds on.
6. Attention Is All You Need — the 2017 Transformer paper.
7. Kimi · Moonshot AI · Kimi API platform · n8n (mentioned in the interview’s blog-idea list).

Also used in this post:

• Series: Inside LLMs · sister post: Inside DeepSeek
• Site I interviewed against: manish.sh
• Related glossary: Kimi · Moonshot AI · LLM · MoE · MLA · Agent Swarm · Context window · Hallucination · Tool calling · Autoregressive · RLHF · Token · KV cache · Open weights · Prompt engineering · Attention · AI agent · full glossary
• Images: Kimi wordmark, Kimi wordmark (dark), Moonshot wordmark, Moonshot wordmark (dark), interview screenshot