"is that cpu enough?"
· 1 min read
rewriting follow-up questions before retrieval, and why it fixed more than a bigger model did.
the pc builder chatbot answers build requests over a real parts catalogue. the first version was great at the first question and useless at the second.
follow-ups have no nouns
people don't ask "is the ryzen 5 7600 enough for 1440p gaming with an rtx 4070?". they ask "is that cpu enough?". embed that and you retrieve every cpu in the catalogue, or none.
rewrite, then retrieve
before retrieval, each turn is rewritten into a standalone question using the chat history. the retriever never sees the follow-up; it sees what the follow-up meant.
rewrite = PromptTemplate.from_template(
"given the conversation so far:\n{history}\n\n"
"rewrite the last message as a standalone question "
"that names every part it refers to.\n\nlast message: {question}"
)
standalone = llm.invoke(rewrite.format(history=history, question=question))
parts = retriever.invoke(standalone)the retriever can only find what the question names.
the change cost one extra model call per turn. it fixed more bad answers than swapping in a bigger model did, because the bigger model was still answering over the wrong parts.