RAG vs fine-tuning: the decision in one page

A pharmaceutical company's assistant answers questions about internal standard operating procedures that are updated weekly. The company wants answers to reflect the latest version without retraining any model. Which approach BEST meets this requirement?

• ALower the temperature so the model recalls the stored procedures more precisely
  Temperature cannot give the model knowledge of documents it has never seen.
• BRetrieval-augmented generation over an index refreshed when procedures change
  Correct. RAG grounds each answer in the latest documents, so weekly updates flow through without retraining.
• CFine-tune the model every week on the full set of procedures
  Weekly fine-tuning adds cost and delay, and it is what the company wants to avoid.
• DPre-train a new foundation model whenever a procedure is revised
  Pre-training is far too expensive and slow for weekly document changes.

A consulting firm wants its GenAI assistant to answer questions using its private project reports and to show which report each answer came from. Which approach BEST meets both needs?

• AFine-tune a model on the reports and ask it to recall the source file names
  Fine-tuned models do not reliably reproduce exact sources, so citations may be invented.
• BUse RAG so answers are grounded in retrieved report passages that can be cited
  Correct. Retrieved chunks carry source metadata, which the application can show as citations.
• CUse a larger model whose pre-training covered more consulting reports
  Bigger models know more, so this is tempting. The firm's reports are private and were never in any pre-training data, and pre-trained knowledge cannot point to which report an answer came from.
• DRely on the base model's pre-training knowledge of general consulting best practices
  The base model has never seen the firm's private reports and cannot cite them.

A two-person startup wants a foundation model to return customer feedback summaries in a fixed three-part format. It has almost no budget and only a handful of example summaries. Which approach should the startup try FIRST?

• AFine-tuning the model on the handful of example summaries the startup already has
  A few examples are too little data for effective fine-tuning, and training adds cost.
• BContinued pre-training on the startup's archive of raw customer feedback
  It teaches domain vocabulary from unlabeled text but not a three-part output format, and it costs far more than a startup with almost no budget can spend.
• CA RAG knowledge base that stores the example summaries for retrieval
  RAG adds facts from a data source; it needs infrastructure and is not the cheapest way to control format. Placing the examples directly in the prompt is simpler.
• DIn-context learning with a few example summaries in the prompt
  Correct. Including a few examples in the prompt (few-shot, in-context learning) often teaches format with no training cost and can be tested immediately.

A company needs a model to answer accurately in a narrow domain and is weighing fine-tuning against RAG. From a cost perspective, which statement is MOST accurate?

• ARAG requires retraining the model whenever documents change, so it costs more
  Changing knowledge sounds like it needs retraining, so this is tempting. RAG only re-indexes documents; it is fine-tuning that must be repeated to update the model's knowledge.
• BFine-tuning removes the need to pay per token for inference
  Owning a custom model sounds cheaper to run, so this is tempting. Custom models still incur inference charges, and fine-tuning adds training and hosting or storage costs.
• CRAG has no ongoing cost once the documents are embedded
  Embedding feels like a one-time step, so this is tempting. RAG keeps costing money for the vector store, query embeddings, and the extra retrieved tokens in every prompt.
• DFine-tuning adds training and hosting costs; RAG adds retrieval and token costs
  Correct. Each approach shifts cost to a different place, so the choice depends on the need.