When a model of your own pays off
A dedicated language model is rarely the first step. In most cases citation-grounded RAG, a clean knowledge graph and PageIndex already solve the task without touching any weights. So we check first whether retrieval, prompt structure and reranking are fully used. Training starts only when a recurring behaviour must live permanently inside the model, when a narrow technical vocabulary has to be hit reliably, or when latency and cost at high volume become the limiting factor.
For domain-specific behaviour we use parameter-efficient fine-tuning with LoRA and QLoRA in 4-bit. This keeps GPU demand low and lets several variants share one base of open weights such as Llama, Mistral, Qwen or Teuken. For search and RAG we train your own embeddings: a bi-encoder for the first pass, a cross-encoder reranker for the final order. Distillation compresses a large teacher into a small, fast SLM. With DPO we align tone and answer shape to your rules.
- Adjust RAG instead of training when knowledge changes often
- LoRA becomes viable from a few hundred cleanly labelled examples
- Train embeddings when generic search misses your terminology
- SLM and distillation when latency and unit cost matter
- DPO when style and format must stay reproducible
Your training data stays with you, in Germany. The model runs on our own GPU systems with vLLM, operated for you. Full transfer of the model IP is available as an enterprise option.