这次来谈谈LoRA微调。

一、基础概念#

先看一下LLM训练的全过程：

在预训练阶段，模型在大规模标注文本上学习对下一个token的分布表示；在下游应用中，通过微调可使模型适应特定任务领域或风格（如问答、代码生成、对话系统等）。微调本质是教会模型遵循用户指令，把知识按照要求表达出来。

微调分为全参微调和低参微调两大类，低参微调又有Prompt-Tuning、 P-Tuning、 Prefix-Tuning、Adapter Tuning、 LoRA等多种方法，本篇就重点讲解目前工业界应用最广泛的 LoRA 微调。

LoRA 微调：通过矩阵分解将大矩阵近似表示为两个小矩阵的乘积，只训练分解后的两个小矩阵，可大幅降低显存需求。例如，将 (4096×4096) 的矩阵分解为 (4096×4) 和 (4×4096) 相乘，参数数量从 160 万下降到 3 万左右，降低了 98%。

假设这是一个4×5的矩阵：

W= \begin{bmatrix} w_{11} & w_{12} & \dots & w_{1n} \\ w_{21} & w_{22} & \dots & w_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ w_{m1} & w_{m2} & \dots & w_{mn} \end{bmatrix}

LoRA 不改动原模型权重，用两个低秩小矩阵相乘模拟原权重的微调增量：

\Delta W = A \times B

A= \begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \\ a_{31} & a_{32} \\ a_{41} & a_{42} \end{bmatrix},\quad B= \begin{bmatrix} b_{11} & b_{12} & b_{13} & b_{14} & b_{15} \\ b_{21} & b_{22} & b_{23} & b_{24} & b_{25} \end{bmatrix}

矩阵乘积A×B是一个4×5的矩阵：

A\times B= \begin{bmatrix} a_{11}b_{11}+a_{12}b_{21} & a_{11}b_{12}+a_{12}b_{22} & \cdots & a_{11}b_{15}+a_{12}b_{25} \\ a_{21}b_{11}+a_{22}b_{21} & a_{21}b_{12}+a_{22}b_{22} & \cdots & a_{21}b_{15}+a_{22}b_{25} \\ \vdots & \vdots & \ddots & \vdots \\ a_{41}b_{11}+a_{42}b_{21} & a_{41}b_{12}+a_{42}b_{22} & \cdots & a_{41}b_{15}+a_{42}b_{25} \end{bmatrix}

仅训练A、B实现参数高效微调，训练结束后W+微调增量可以合并回原权重，推理无额外计算开销：

h = (W + BA)x = (W + \Delta W)x

**参数对比：**原W参数4×5=20个；LoRA (A+B) 总参数4×2+2×5=18，r越小参数量压缩越夸张。

虽然Lora微调能极大地节省计算开销，但是能做全量微调还是尽量不要做 LoRA 微调，LoRA 微调精度有损失，只适用于部分简单场景，涉及重要能力的微调需用全量微调。

二、手撕实战#

这里以对DeepSeek-R1-Distill-Qwen-7B这个模型做医学问答方面的微调为例。

需要用到的包如下：

torch：深度学习底层框架PyTorch

transformers：LLM 加载和 Tokenizer 核心库

datasets：数据集处理工具

peft：参数高效微调库

trl：LLM 训练流水线库

accelerate：简易分布式 / 混合精度工具

wandb：训练日志可视化平台

deepspeed：微软超大模型显存优化框架

整体链路如下：

1
torch(底层) → transformers(模型+分词) + datasets(数据) → peft(LoRA) + trl(SFT训练器) → accelerate/deepspeed(分布式显存优化) → wandb(日志监控)

1. 配置模型和数据路径#

1
model_id_or_path="DeepSeek-R1-Distill-Qwen-7B"
2
dataset_path="dataset/medical_o1_sft_Chinese.json"
3
ouput_dir="sft_lora_model_int_output_new"
4

5
tokenizer=AutoTokenizer.from_pretrained(model_id_or_path,trust_remote_code=True)
6
model=AutoModelForCasualLM.from_pretrained(model_id_or_path,trust_remote_code=True)

2. 加载数据集#

1
print("Loadding Dataset...")
2
dataset=load_dataset("json",data_files=dataset_path,split="train")
3
print(f"数据集一共有{len(dataset)}组数据")
4
print(f"数据集:{dataset}")
5
print(f"数据集列{dataset.column_names}")

3. 处理数据集#

1
train_prompt_style_zh = """
2
请根据下方指令和提供的具体问题，撰写一个恰当的回复。
3
在回答前，请仔细思考问题，并展现一步一步的思考过程，以确保回复的逻辑性和准确性。
4

5
### 指令:
6
你是一位医学专家，精通临床推理、诊断和治疗计划。
7
请回答下面的医学问题。
8

9
### 问题:
10
{}
11

12
### 回答:
13
<think>
14
{}
15
</think>
16
{}
17
"""

固定对话 Prompt 模板，3 个{}依次占位：

第 1 个{}：数据集字段Question → 用户医学问题

第 2 个{}：数据集字段Complex_CoT → CoT 思考推理过程（分步解题思路）

第 3 个{}：数据集字段Response → 最终标准答案

1
# eos_token：模型结束标识符（如<|endoftext|>），每条样本末尾强制拼接结束符，告诉模型一句话到此结束，微调时学习终止生成。
2
EOS_TOKEN=tokenizer.eos_token
3
def formatting_prompts_func(examples):
4
    inputs=examples["Question"]
5
    cots=examples["Complex_CoT"]
6
    outputs=examples["Response"]
7
    texts=[]
8
    for input,cot,output in zip(inputs,cots,outputs):
9
        text=train_prompt_style_zh.format(input,cot,output)+EOS_TOKEN
10
        texts.append(text)
11
    return {
12
        "text":texts,
13
    }

最终生成新字段text，是后续 Trainer 训练唯一输入字段。因为我们后面是用 trl 来训练，所以把输入输出拼接成一个 text 就可以了。如果想正常用 peft 来训练的话，需要把数据集处理成text, input_ids, attention_mask这三个标准的列然后传入模型去训练。trl 库是对整个流程进行简化，只需要数据集里有 text 这一列就可以了。

1
print("Formating dataset...")
2
# 数据集映射处理
3
dataset=dataset.map(formatting_prompts_func,batched=True)
4
print("Dataset after formatting",dataset)
5
print("Dataset[0]",dataset[0]["text"])
6

7
if tokenizer.pad_token is None:
8
    tokenizer.pad_token=tokenizer.eos_token

pad是padding 填充符，batch 训练时补齐短句长度必需。很多大模型（Llama、Qwen、Mistral）原生没有 pad_token，只有 eos_token，所以赋值pad_token = eos_token，避免 Trainer 训练时报错。

4. 模型量化#

1
# 配置量化
2
bnb_config=BitsAndBytesConfig(
3
    #这里我采用的是4bit量化，想改成8bit量化直接把下面四个参数替换成load_in_8bit=True就行
4
    load_in_4bit=True,   # 开启 4bit 量化
5
    bnb_4bit_use_double_quant=True, # 双重量化，更省显存
6
    bnb_4bit_quant_type="nf4",  # 用更精准的 4bit 格式（QLoRA 专用）
7
    bnb_4bit_compute_dtype=torch.bfloat16 # 计算时用 bf16
8
)
9
# 加载模型
10
model=AutoModelForCausalLM.from_pretrained(
11
    model_id_or_path,
12
    quantization_config=bnb_config, # 传入量化配置
13
    device_map="auto",   # 自动把模型分配到 GPU/CPU
14
    trust_remote_code=True # 运行模型自定义代码
15
)
16
print("model loaded")
17
print(model)

5. LoRA配置#

1
# 把量化模型设置成可以训练的模式。
2
model = prepare_model_for_kbit_training(model)

主要有下面三个作用：

关闭模型的冻结，让量化层支持梯度更新
关闭不需要的缓存，节省显存
让量化权重可以参与反向传播

1
lora_config = LoraConfig(
2
    r=16,   # LoRA 的秩
3
    lora_alpha=16,   # 缩放系数，一般设置成和r一样就行
4
    # 把 LoRA 加在哪些层上。下面是给模型的所有线性层都加上了LoRA，q/k/v/o是注意力层，gate/up/down是前馈层
5
    target_modules=["q_proj","k_proj","v_proj","o_proj","gate_proj","up_proj","down_proj"],
6
    lora_dropout=0, # 随机丢弃一部分训练权重，防止过拟合
7
    bias="none",   # 是否训练偏置项
8
    task_type="CAUSAL_LM"  # 模型类型：自回归大模型
9
)

1
# 给模型绑定上LoRA适配器，冻结原模型的全部权重，新增少量可训练的LoRA权重
2
model = get_peft_model(model, lora_config)
3
print("Lora applied to the model")
4
model.print_trainable_parameters()

6. 训练配置#

1
# wandb 实验可视化日志
2
import wandb
3
wandb.init(project="test")
4

5
print("Config Train Arguements")
6

7
# 训练核心超参
8
training_arguments=TrainingArguments(
9
    output_dir=ouput_dir,     # LoRA权重保存目录
10
    per_device_train_batch_size=1,   # 每个设备的训练批次大小。这里一张显卡一次只喂 1 条数据进模型训练。
11
    gradient_accumulation_steps=8,   # 梯度累积步数。这里是连续累积 8 步的梯度，再更新一次模型权重。
12
    optim="adamw_8bit",   # 优化器。这里使用最常用的8bit 精度的 AdamW 优化器
13

14
    save_steps=500,   # 每训练多少步自动保存一次 LoRA 权重
15
    logging_steps=2,  # 每几步打印一次训练日志
16

17
    learning_rate=1e-4,  # 学习率。LoRA 微调常用范围：1e-4 ~ 3e-4。太大模型不收敛，太小训练太慢
18
    num_train_epochs=1,  # 训练几个轮次
19
    max_grad_norm=0.3,   # 梯度裁剪，防止梯度过大导致模型训练崩溃。大模型微调常用：0.3~1.0
20
    lr_scheduler_type="cosine",  # 学习率调度策略。这里使用的是余弦退火，学习率会按照余弦曲线慢慢下降。
21
    weight_decay=0.01,   # 权重衰减，防止模型过拟合。常用值：0.01 ~ 0.1
22
    warmup_ratio=0.03,   # 学习率预热比例。这里训练前 3% 的步数，学习率从 0 慢慢升到设定值，让训练初期更稳定。
23
    fp16=False,     # 混合精度训练。半精度（关闭）
24
    bf16=True,      # 混合精度训练。脑浮点精度（开启）
25

26
    report_to="wandb",  # 日志上传到wandb
27
)
28

29
print("SFT TRAINER START...")
30

31
# TRL库专用的有监督微调训练器
32
trainer=SFTTrainer(
33
    model=model,
34
    tokenizer=tokenizer,
35
    args=training_arguments,
36
    train_dataset=dataset,
37

38
    peft_config=lora_config,
39
    dataset_text_field="text",
40

41
    dataset_num_proc=2,  # 2进程并行做数据集tokenizer预处理，提速
42
    max_seq_length=2048, # 单条文本token上限2048，超长截断、不足补padding。
43
    packing=False,       # 关闭序列打包
44
)
45

46
print("Start training...")
47
trainer.train()
48
print("End training...")

nohup.out如下：

1
The following values were not passed to `accelerate launch` and had defaults used instead:
2
  `--num_processes` was set to a value of `1`
3
  `--num_machines` was set to a value of `1`
4
  `--mixed_precision` was set to a value of `'no'`
5
  `--dynamo_backend` was set to a value of `'no'`
6
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
7

8
Loading weights:   0%|          | 0/339 [00:00<?, ?it/s]
9
Loading weights:  31%|███       | 104/339 [00:00<00:00, 1032.65it/s]
10
Loading weights:  61%|██████▏   | 208/339 [00:00<00:00, 949.35it/s]
11
Loading weights: 100%|██████████| 339/339 [00:00<00:00, 1122.88it/s]
12
Loadding Dataset...
13
数据集一共有20171组数据
14
数据集:Dataset({
15
    features: ['Question', 'Complex_CoT', 'Response'],
16
    num_rows: 20171
17
})
18
数据集列['Question', 'Complex_CoT', 'Response']
19
Formating dataset...
20
Dataset after formatting Dataset({
21
    features: ['Question', 'Complex_CoT', 'Response', 'text'],
22
    num_rows: 20171
23
})
24
Dataset[0]
25
请根据下方指令和提供的具体问题，撰写一个恰当的回复。
26
在回答前，请仔细思考问题，并展现一步一步的思考过程，以确保回复的逻辑性和准确性。
27

28
### 指令:
29
你是一位医学专家，精通临床推理、诊断和治疗计划。
30
请回答下面的医学问题。
31

32
### 问题:
33
根据描述，一个1岁的孩子在夏季头皮出现多处小结节，长期不愈合，且现在疮大如梅，溃破流脓，口不收敛，头皮下有空洞，患处皮肤增厚。这种病症在中医中诊断为什么病？
34

35
### 回答:
36
<think>
37
这个小孩子在夏天头皮上长了些小结节，一直都没好，后来变成了脓包，流了好多脓。想想夏天那么热，可能和湿热有关。才一岁的小孩，免疫力本来就不强，夏天的湿热没准就侵袭了身体。
38

39
用中医的角度来看，出现小结节、再加上长期不愈合，这些症状让我想到了头疮。小孩子最容易得这些皮肤病，主要因为湿热在体表郁结。
40

41
但再看看，头皮下还有空洞，这可能不止是简单的头疮。看起来病情挺严重的，也许是脓肿没治好。这样的情况中医中有时候叫做禿疮或者湿疮，也可能是另一种情况。
42

43
等一下，头皮上的空洞和皮肤增厚更像是疾病已经深入到头皮下，这是不是说明有可能是流注或瘰疬？这些名字常描述头部或颈部的严重感染，特别是有化脓不愈合，又形成通道或空洞的情况。
44

45
仔细想想，我怎么感觉这些症状更贴近瘰疬的表现？尤其考虑到孩子的年纪和夏天发生的季节性因素，湿热可能是主因，但可能也有火毒或者痰湿造成的滞留。
46

47
回到基本的症状描述上看，这种长期不愈合又复杂的状况，如果结合中医更偏重的病名，是不是有可能是涉及更深层次的感染？
48

49
再考虑一下，这应该不是单纯的瘰疬，得仔细分析头皮增厚并出现空洞这样的严重症状。中医里头，这样的表现可能更符合‘蚀疮’或‘头疽’。这些病名通常描述头部严重感染后的溃烂和组织坏死。
50

51
看看季节和孩子的体质，夏天又湿又热，外邪很容易侵入头部，对孩子这么弱的免疫系统简直就是挑战。头疽这个病名听起来真是切合，因为它描述的感染严重，溃烂到出现空洞。
52

53
不过，仔细琢磨后发现，还有个病名似乎更为合适，叫做‘蝼蛄疖’，这病在中医里专指像这种严重感染并伴有深部空洞的情况。它也涵盖了化脓和皮肤增厚这些症状。
54

55
哦，该不会是夏季湿热，导致湿毒入侵，孩子的体质不能御，其病情发展成这样的感染？综合分析后我觉得‘蝼蛄疖’这个病名真是相当符合。
56
</think>
57
从中医的角度来看，你所描述的症状符合“蝼蛄疖”的病症。这种病症通常发生在头皮，表现为多处结节，溃破流脓，形成空洞，患处皮肤增厚且长期不愈合。湿热较重的夏季更容易导致这种病症的发展，特别是在免疫力较弱的儿童身上。建议结合中医的清热解毒、祛湿消肿的治疗方法进行处理，并配合专业的医疗建议进行详细诊断和治疗。
58
<｜end▁of▁sentence｜>
59
Config BitsAndBytes for int4 quantization...
60

61
Loading weights:   0%|          | 0/339 [00:00<?, ?it/s]
62
Loading weights:   0%|          | 1/339 [00:01<10:39,  1.89s/it]
63
Loading weights:   1%|          | 2/339 [00:09<29:31,  5.26s/it]/home/stw/miniconda3/envs/peft/lib/python3.10/site-packages/bitsandbytes/backends/cuda/ops.py:213: FutureWarning: _check_is_size will be removed in a future PyTorch release along with guard_size_oblivious.     Use _check(i >= 0) instead.
64
  torch._check_is_size(blocksize)
65

66
Loading weights:   1%|          | 4/339 [00:11<13:50,  2.48s/it]
67
Loading weights:   1%|▏         | 5/339 [00:12<11:35,  2.08s/it]
68
Loading weights:   2%|▏         | 6/339 [00:13<09:50,  1.77s/it]
69
Loading weights:   3%|▎         | 10/339 [00:13<03:43,  1.47it/s]
70
Loading weights:   4%|▎         | 12/339 [00:13<02:44,  1.98it/s]
71
Loading weights:   5%|▍         | 16/339 [00:15<02:27,  2.19it/s]
72
Loading weights:   5%|▌         | 17/339 [00:17<03:28,  1.54it/s]
73
Loading weights:   5%|▌         | 18/339 [00:18<04:20,  1.23it/s]
74
Loading weights:   6%|▋         | 22/339 [00:19<02:23,  2.20it/s]
75
Loading weights:   7%|▋         | 24/339 [00:19<01:58,  2.65it/s]
76
Loading weights:   8%|▊         | 28/339 [00:21<02:08,  2.42it/s]
77
Loading weights:   9%|▊         | 29/339 [00:24<03:46,  1.37it/s]
78
Loading weights:   9%|▉         | 30/339 [00:25<04:31,  1.14it/s]
79
Loading weights:  10%|█         | 34/339 [00:26<02:27,  2.06it/s]
80
Loading weights:  11%|█         | 36/339 [00:26<01:56,  2.59it/s]
81
Loading weights:  12%|█▏        | 40/339 [00:27<01:46,  2.82it/s]
82
Loading weights:  12%|█▏        | 41/339 [00:28<02:06,  2.36it/s]
83
Loading weights:  12%|█▏        | 42/339 [00:28<02:09,  2.29it/s]
84
Loading weights:  14%|█▎        | 46/339 [00:29<01:11,  4.08it/s]
85
Loading weights:  14%|█▍        | 48/339 [00:29<00:57,  5.06it/s]
86
Loading weights:  15%|█▌        | 52/339 [00:29<00:44,  6.49it/s]
87
Loading weights:  16%|█▌        | 54/339 [00:30<00:53,  5.29it/s]
88
Loading weights:  18%|█▊        | 62/339 [00:30<00:24, 11.20it/s]
89
Loading weights:  19%|█▉        | 65/339 [00:30<00:25, 10.92it/s]
90
Loading weights:  20%|██        | 68/339 [00:30<00:22, 12.19it/s]
91
Loading weights:  22%|██▏       | 76/339 [00:30<00:15, 17.27it/s]
92
Loading weights:  23%|██▎       | 79/339 [00:31<00:16, 15.71it/s]
93
Loading weights:  26%|██▌       | 88/339 [00:31<00:11, 22.53it/s]
94
Loading weights:  27%|██▋       | 91/339 [00:31<00:12, 19.34it/s]
95
Loading weights:  29%|██▉       | 100/339 [00:31<00:09, 25.90it/s]
96
Loading weights:  30%|███       | 103/339 [00:32<00:10, 21.94it/s]
97
Loading weights:  33%|███▎      | 112/339 [00:32<00:07, 28.55it/s]
98
Loading weights:  34%|███▍      | 116/339 [00:32<00:08, 25.46it/s]
99
Loading weights:  37%|███▋      | 124/339 [00:32<00:06, 30.91it/s]
100
Loading weights:  38%|███▊      | 128/339 [00:32<00:07, 27.57it/s]
101
Loading weights:  40%|████      | 136/339 [00:33<00:06, 33.13it/s]
102
Loading weights:  41%|████▏     | 140/339 [00:33<00:06, 28.88it/s]
103
Loading weights:  44%|████▎     | 148/339 [00:33<00:05, 34.79it/s]
104
Loading weights:  45%|████▍     | 152/339 [00:33<00:06, 30.77it/s]
105
Loading weights:  47%|████▋     | 160/339 [00:33<00:04, 37.01it/s]
106
Loading weights:  48%|████▊     | 164/339 [00:34<00:07, 23.89it/s]
107
Loading weights:  51%|█████     | 172/339 [00:34<00:05, 30.52it/s]
108
Loading weights:  52%|█████▏    | 176/339 [00:34<00:05, 28.29it/s]
109
Loading weights:  54%|█████▍    | 184/339 [00:34<00:04, 33.45it/s]
110
Loading weights:  55%|█████▌    | 188/339 [00:34<00:05, 29.65it/s]
111
Loading weights:  58%|█████▊    | 196/339 [00:34<00:04, 34.61it/s]
112
Loading weights:  59%|█████▉    | 200/339 [00:35<00:05, 27.66it/s]
113
Loading weights:  61%|██████▏   | 208/339 [00:35<00:03, 32.96it/s]
114
Loading weights:  63%|██████▎   | 212/339 [00:35<00:04, 28.06it/s]
115
Loading weights:  65%|██████▍   | 220/339 [00:35<00:03, 32.82it/s]
116
Loading weights:  66%|██████▌   | 224/339 [00:35<00:04, 28.38it/s]
117
Loading weights:  68%|██████▊   | 232/339 [00:36<00:03, 34.03it/s]
118
Loading weights:  70%|██████▉   | 236/339 [00:36<00:03, 29.36it/s]
119
Loading weights:  72%|███████▏  | 244/339 [00:36<00:02, 35.75it/s]
120
Loading weights:  73%|███████▎  | 248/339 [00:36<00:02, 31.35it/s]
121
Loading weights:  76%|███████▌  | 256/339 [00:36<00:02, 37.12it/s]
122
Loading weights:  77%|███████▋  | 260/339 [00:37<00:02, 32.22it/s]
123
Loading weights:  79%|███████▉  | 268/339 [00:37<00:01, 38.57it/s]
124
Loading weights:  81%|████████  | 273/339 [00:37<00:02, 24.02it/s]
125
Loading weights:  83%|████████▎ | 280/339 [00:37<00:02, 28.51it/s]
126
Loading weights:  84%|████████▍ | 284/339 [00:38<00:02, 24.47it/s]
127
Loading weights:  86%|████████▌ | 292/339 [00:38<00:01, 30.53it/s]
128
Loading weights:  87%|████████▋ | 296/339 [00:38<00:01, 27.77it/s]
129
Loading weights:  90%|████████▉ | 304/339 [00:38<00:01, 34.74it/s]
130
Loading weights:  91%|█████████ | 308/339 [00:38<00:01, 26.98it/s]
131
Loading weights:  93%|█████████▎| 316/339 [00:38<00:00, 33.28it/s]
132
Loading weights:  94%|█████████▍| 320/339 [00:39<00:00, 28.95it/s]
133
Loading weights:  97%|█████████▋| 328/339 [00:40<00:00, 13.57it/s]
134
Loading weights:  98%|█████████▊| 331/339 [00:40<00:00, 13.98it/s]
135
Loading weights: 100%|██████████| 339/339 [00:40<00:00,  8.37it/s]
136
wandb: [wandb.login()] Loaded credentials for https://api.wandb.ai from WANDB_API_KEY.
137
wandb: Currently logged in as: 3262818908 (3262818908-personal) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
138
wandb: setting up run awwiyurj
139
wandb: Tracking run with wandb version 0.27.0
140
wandb: Run data is saved locally in /home/stw/PEFT/wandb/run-20260605_115430-awwiyurj
141
wandb: Run `wandb offline` to turn off syncing.
142
wandb: Syncing run trim-thunder-2
143
wandb: ⭐️ View project at https://wandb.ai/3262818908-personal/test
144
wandb: 🚀 View run at https://wandb.ai/3262818908-personal/test/runs/awwiyurj
145
[transformers] warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
146
/home/stw/miniconda3/envs/peft/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py:100: FutureWarning: Deprecated argument(s) used in '__init__': dataset_text_field, dataset_num_proc, max_seq_length. Will not be supported from version '0.13.0'.
147

148
Deprecated positional argument(s) used in SFTTrainer, please use the SFTConfig to set these arguments instead.
149
  warnings.warn(message, FutureWarning)
150
[transformers] warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
151
/home/stw/miniconda3/envs/peft/lib/python3.10/site-packages/trl/trainer/sft_trainer.py:300: UserWarning: You passed a `max_seq_length` argument to the SFTTrainer, the value you passed will override the one in the `SFTConfig`.
152
  warnings.warn(
153
/home/stw/miniconda3/envs/peft/lib/python3.10/site-packages/trl/trainer/sft_trainer.py:314: UserWarning: You passed a `dataset_num_proc` argument to the SFTTrainer, the value you passed will override the one in the `SFTConfig`.
154
  warnings.warn(
155
/home/stw/miniconda3/envs/peft/lib/python3.10/site-packages/trl/trainer/sft_trainer.py:328: UserWarning: You passed a `dataset_text_field` argument to the SFTTrainer, the value you passed will override the one in the `SFTConfig`.
156
  warnings.warn(
157
[transformers] The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': 151646, 'pad_token_id': 151643}.
158
model loaded
159
Qwen2ForCausalLM(
160
  (model): Qwen2Model(
161
    (embed_tokens): Embedding(152064, 3584)
162
    (layers): ModuleList(
163
      (0-27): 28 x Qwen2DecoderLayer(
164
        (self_attn): Qwen2Attention(
165
          (q_proj): Linear4bit(in_features=3584, out_features=3584, bias=True)
166
          (k_proj): Linear4bit(in_features=3584, out_features=512, bias=True)
167
          (v_proj): Linear4bit(in_features=3584, out_features=512, bias=True)
168
          (o_proj): Linear4bit(in_features=3584, out_features=3584, bias=False)
169
        )
170
        (mlp): Qwen2MLP(
171
          (gate_proj): Linear4bit(in_features=3584, out_features=18944, bias=False)
172
          (up_proj): Linear4bit(in_features=3584, out_features=18944, bias=False)
173
          (down_proj): Linear4bit(in_features=18944, out_features=3584, bias=False)
174
          (act_fn): SiLUActivation()
175
        )
176
        (input_layernorm): Qwen2RMSNorm((3584,), eps=1e-06)
177
        (post_attention_layernorm): Qwen2RMSNorm((3584,), eps=1e-06)
178
      )
179
    )
180
    (norm): Qwen2RMSNorm((3584,), eps=1e-06)
181
    (rotary_emb): Qwen2RotaryEmbedding()
182
  )
183
  (lm_head): Linear(in_features=3584, out_features=152064, bias=False)
184
)
185
Config Lora
186
Lora applied to the model
187
trainable params: 40,370,176 || all params: 7,655,986,688 || trainable%: 0.5273
188
start train
189
Config Train Arguements
190
SFT TRAINER START...
191
Start training...
192

193
  0%|          | 0/2522 [00:00<?, ?it/s]/home/stw/miniconda3/envs/peft/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py:1298: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. Starting in PyTorch 2.9, calling checkpoint without use_reentrant will raise an exception. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.
194
  return fn(*args, **kwargs)
195

196
  0%|          | 1/2522 [09:55<416:57:49, 595.43s/it]
197
  0%|          | 2/2522 [15:46<316:06:25, 451.58s/it]
198

199

200
  0%|          | 2/2522 [15:46<316:06:25, 451.58s/it]
201
  0%|          | 3/2522 [21:25<279:57:43, 400.10s/it]
202
  0%|          | 4/2522 [27:20<267:21:31, 382.24s/it]
203

204

205
  0%|          | 4/2522 [27:20<267:21:31, 382.24s/it]
206
  0%|          | 5/2522 [33:09<259:03:45, 370.53s/it]
207
  0%|          | 6/2522 [39:10<256:40:06, 367.25s/it]
208

209

210
  0%|          | 6/2522 [39:10<256:40:06, 367.25s/it]
211
  0%|          | 7/2522 [44:30<245:39:45, 351.64s/it]

这里我的电脑配置太拉了，就不等它训练完了，太慢了

三、分布式训练#

上面我们说的都是单GPU的情况，如果想去做多卡训练，也很简单：

1.在项目根目录新建ds_zero2.json，怎么写可以参考：https://huggingface.co/docs/accelerate/usage_guides/deepspeed。

也可以参考我这个：

1
{
2
    "bf16": {
3
        "enabled": true
4
    },
5
    "zero_optimization": {
6
        "stage": 2,
7
        "offload_optimizer": {
8
            "device": "none"
9
        },
10
        "allgather_partitions": true,
11
        "allgather_bucket_size": 5e8,
12
        "overlap_comm": true,
13
        "reduce_scatter": true,
14
        "reduce_bucket_size": 5e8,
15
        "contiguous_gradients": true
16
    },
17
    "gradient_accumulation_steps": "auto",
18
    "gradient_clipping": "auto",
19
    "train_batch_size": "auto",
20
    "train_micro_batch_size_per_gpu": "auto"
21
}

2.TrainingArguments 中加一行 deepspeed="./ds_zero2.json"

3.运行（这里以2卡为例）：CUDA_VISIBLE_DEVICES=0,1 accelerate launch --num_processes=2 test.py

四、LlamaFactory框架#

最后推荐一下LlamaFactory这个框架吧，这个框架可以实现模型训练的可视化操作，我觉得无论是新手还是老手使用这个框架去进行工作都是更方便的：

前面的学习可以说是为了夯实基本功，但真正干活的时候还是怎么方便怎么来。