TODO: 23.12.29 Updated

Training Method

• LoRA: Low-Rank Adaptation of Large Language Models
• QLoRA: Efficient Finetuning of Quantized LLMs

Dataset

• Orca: Progressive Learning from Complex Explanation Traces of GPT-4

Model

• Textbooks Are All You Need: Phi-1
• SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling

Extra

MoE(Mixture of Experts)

• MoE 관련 논문들 Velog