I don’t think you’d be fine-tuning a whole model in such cases. That seems over the top, no? I assume you’d get sufficiently far with big context windows, vector search, RAG. Etc.
It's an interesting question. I'm not sure we really know yet what the right mix of RAG and fine tuning is. IMO small-scale fine tuning might be under-appreciated.
