模型支持情况说明
更新时间:2024-06-04
本文介绍了模型支持情况,在调用模型精调V2版本部分API时,需查看此文档各参数支持情况。
对话续写类
SFT
ERNIE系列
model | trainMode | parameterScale | hyperParameterConfig |
---|---|---|---|
ERNIE-Lite-8K-0308 | SFT | FullFineTuning、LoRA | (1)epoch:[1,50],默认值1 (2)learningRate: FullFineTuning:[0.000001,0.00004],默认值0.00003 LoRA:[0.000001,0.001],默认值0.0003 (3)maxSeqLen: 单选,512 或 1024 或 2048 或 4096 或 8192,默认值4096 (4)loggingSteps:1 (5)warmupRatio:[0.01,0.5],默认值0.1 (6)weightDecay:[0.0001,0.1],默认值0.01 (7)gradientAccumulationSteps:[0,32],默认值0 (8)pseudoSamplingProb:[0,1],默认值0 (9)seed:[1,2147483647],默认值42 (10)lrSchedulerType: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值linear (11)numCycles:[0.1,0.5],默认值0.5 (12)lrEnd:[0.00000001,0.000001],默认值0.0000001 (13)power:[1,3],默认值1 (14)仅FullFineTuning支持: checkpointCount:[1,10],默认值1 saveStep:64 或 128 或 256 或 512 或 1024 或 2048 或 4096,默认值256 (15)仅LoRA支持: loraRank: 单选,2 或 4 或 8,默认为8 loraAllLinear: 单选,True 或 False,默认为True |
ERNIE-Lite-8K-0922 | SFT | FullFineTuning、LoRA | (1)epoch:[1,50],默认值1 (2)learningRate: FullFineTuning:[0.000001,0.00004],默认值0.00003 LoRA:[0.000001,0.001],默认值0.0003 (3)maxSeqLen: 单选,512 或 1024 或 2048 或 4096 或 8192,默认值4096 (4)loggingSteps:[1,100],默认值1 (5)warmupRatio:[0.01,0.5],默认值0.1 (6)weightDecay:[0.0001,0.1],默认值0.01 (7)gradientAccumulationSteps:[0,32],默认值0 (8)仅FullFineTuning支持: checkpointCount:[1,10],默认值1 saveStep:64 或 128 或 256 或 512 或 1024 或 2048 或 4096,默认值256 (9)仅LoRA支持: loraRank: 单选,2 或 4 或 8,默认为8 loraAllLinear: 单选,True 或 False,默认为True |
ERNIE-Lite-128K-0419 | SFT | FullFineTuning | (1)epoch:[1,50],默认值1 (2)learningRate: FullFineTuning:[0.000001,0.00004],默认值0.00003 LoRA:[0.000001,0.001],默认值0.0003 (3)maxSeqLen: 单选,16384 或 32768 或 65536 或 131072,默认值32768 (4)loggingSteps:1 (5)warmupRatio:[0.01,0.5],默认值0.1 (6)weightDecay:[0.0001,0.1],默认值0.01 (7)gradientAccumulationSteps:[0,32],默认值0 (8)pseudoSamplingProb:[0,1],默认值0 (9)seed:[1,2147483647],默认值42 (10)lrSchedulerType: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值linear (11)numCycles:[0.1,0.5],默认值0.5 (12)lrEnd:[0.00000001,0.000001],默认值0.0000001 (13)power:[1,3],默认值1 (14)仅FullFineTuning支持: checkpointCount:[1,10],默认值1 saveStep:64 或 128 或 256 或 512 或 1024 或 2048 或 4096,默认值256 (15)仅LoRA支持: loraRank: 单选,2 或 4 或 8,默认为8 loraAllLinear: 单选,True 或 False,默认为True |
ERNIE-Speed-8K | SFT | FullFineTuning、LoRA | (1)epoch:[1,50],默认值1 (2)learningRate: FullFineTuning:[0.000001,0.00004],默认值0.00003 LoRA:[0.000001,0.001],默认值0.0003 (3)maxSeqLen: 单选,512 或 1024 或 2048 或 4096 或 8192,默认值4096 (4)loggingSteps:1 (5)warmupRatio:[0.01,0.5],默认值0.1 (6)weightDecay:[0.0001,0.1],默认值0.01 (7)gradientAccumulationSteps:[0,32],默认值0 (8)pseudoSamplingProb:[0,1],默认值0 (9)seed:[1,2147483647],默认值42 (10)lrSchedulerType: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值linear (11)numCycles:[0.1,0.5],默认值0.5 (12)lrEnd:[0.00000001,0.000001],默认值0.0000001 (13)power:[1,3],默认值1 (14)仅FullFineTuning支持: checkpointCount:[1,10],默认值1 saveStep:64 或 128 或 256 或 512 或 1024 或 2048 或 4096,默认值256 (15)仅LoRA支持: loraRank: 单选,8 或 64,默认为64 loraAllLinear: 单选,True 或 False,默认为True |
ERNIE-Character-8K-0321 | SFT | FullFineTuning、LoRA | (1)epoch:[1,50],默认值1 (2)learningRate: FullFineTuning:[0.000001,0.00004],默认值0.00003 LoRA:[0.000001,0.001],默认值0.0003 (3)maxSeqLen: 单选,512 或 1024 或 2048 或 4096 或 8192,默认值4096 (4)loggingSteps:1 (5)warmupRatio:[0.01,0.5],默认值0.1 (6)weightDecay:[0.0001,0.1],默认值0.01 (7)gradientAccumulationSteps:[0,32],默认值0 (8)pseudoSamplingProb:[0,1],默认值0 (9)seed:[1,2147483647],默认值42 (10)lrSchedulerType: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值linear (11)numCycles:[0.1,0.5],默认值0.5 (12)lrEnd:[0.00000001,0.000001],默认值0.0000001 (13)power:[1,3],默认值1 (14)仅FullFineTuning支持: checkpointCount:[1,10],默认值1 saveStep:64 或 128 或 256 或 512 或 1024 或 2048 或 4096,默认值256 (15)仅LoRA支持: loraRank: 单选,2 或 4 或 8,默认为8 loraAllLinear: 单选,True 或 False,默认为True |
ERNIE-Tiny-8K | SFT | FullFineTuning | (1)epoch:[1,50],默认值1 (2)learningRate: FullFineTuning:[0.000001,0.00004],默认值0.00003 LoRA:[0.000001,0.001],默认值0.0003 (3)maxSeqLen: 单选,512 或 1024 或 2048 或 4096 或 8192,默认值4096 (4)loggingSteps:1 (5)warmupRatio:[0.01,0.5],默认值0.1 (6)weightDecay:[0.0001,0.1],默认值0.01 (7)gradientAccumulationSteps:[0,32],默认值0 (8)pseudoSamplingProb:[0,1],默认值0 (9)seed:[1,2147483647],默认值42 (10)lrSchedulerType: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值linear (11)numCycles:[0.1,0.5],默认值0.5 (12)lrEnd:[0.00000001,0.000001],默认值0.0000001 (13)power:[1,3],默认值1 (14)仅FullFineTuning支持: checkpointCount:[1,10],默认值1 saveStep:64 或 128 或 256 或 512 或 1024 或 2048 或 4096,默认值256 (15)仅LoRA支持: loraRank: 单选,2 或 4 或 8,默认为8 loraAllLinear: 单选,True 或 False,默认为True |
开源系列
model | trainMode | parameterScale | hyperParameterConfig |
---|---|---|---|
Meta-Llama-3-8B | SFT | FullFineTuning | (1)epoch:[1,50],默认值1 (2)learningRate:[0.0000000001,0.0002],默认值0.000001 (3)batchSize:[1,2],默认值1 |
Qianfan-Chinese-Llama-2-1.3B | SFT | FullFineTuning | (1)epoch:[1,50],默认值1 (2)learningRate:[0.0000000001,0.0002],默认值0.000001 (3)batchSize:1 (4)Packing:字符串,true 或 false 或 auto,默认值auto (5)schedulerName: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值cosine (6)warmupRatio:[0.01,0.1],默认值0.03 (7)weightDecay:[0.001,1],默认值0.01 (8)maxSeqLen: 单选,512 或 1024 或 2048 或 4096,默认值4096 (9)checkpointCount:[1,10],默认值1 (10)saveStep:[64,4096],默认值256 (11)仅LoRA支持: loraRank:单选,8 或 16 或 32 或 64,默认值32 loraAlpha:单选,8 或 16 或 32 或 64,默认值32 loraDropout:[0.01,0.5],默认值0.1 |
Qianfan-Chinese-Llama-2-7B | SFT | FullFineTuning、LoRA | (1)epoch:[1,50],默认值1 (2)learningRate:[0.0000000001,0.0002],默认值0.000001 (3)batchSize:[1,4],默认值1 (4)Packing:字符串,true 或 false 或 auto,默认值auto (5)schedulerName: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值cosine (6)warmupRatio:[0.01,0.1],默认值0.03 (7)weightDecay:[0.001,1],默认值0.01 (8)maxSeqLen: 单选,512 或 1024 或 2048 或 4096,默认值4096 (9)checkpointCount:[1,10],默认值1 (10)saveStep:[64,4096],默认值256 (11)仅LoRA支持: loraRank:单选,8 或 16 或 32 或 64,默认值32 loraAlpha:单选,8 或 16 或 32 或 64,默认值32 loraDropout:[0.01,0.5],默认值0.1 |
Qianfan-Chinese-Llama-2-7B-32K | SFT | FullFineTuning、LoRA | (1)epoch:[1,50],默认值3 (2)learningRate:[0.0000000001,0.0002],默认值0.000001 (3)batchSize:1 (4)Packing:字符串,true 或 false 或 auto,默认值auto (5)schedulerName: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值cosine (6)warmupRatio:[0.01,0.1],默认值0.03 (7)weightDecay:[0.001,1],默认值0.01 (8)maxSeqLen:单选,4096 或 8192 或 16384 或 32768,默认值32768 (9)checkpointCount:[1,10],默认值1 (10)saveStep:[64,4096],默认值256 (11)仅LoRA支持: loraRank:单选,8 或 16 或 32 或 64,默认值32 loraAlpha:单选,8 或 16 或 32 或 64,默认值32 loraDropout:[0.01,0.5],默认值0.1 |
Qianfan-Chinese-Llama-2-13B-v1 | SFT | FullFineTuning、LoRA | (1)epoch:[1,50],默认值1 (2)learningRate:[0.0000000001,0.0002],默认值0.000001 (3)batchSize:1 (4)Packing:字符串,true 或 false 或 auto,默认值auto (5)schedulerName: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值cosine (6)warmupRatio:[0.01,0.1],默认值0.03 (7)weightDecay:[0.001,1],默认值0.01 (8)maxSeqLen: 单选,512 或 1024 或 2048 或 4096,默认值4096 (9)checkpointCount:[1,10],默认值1 (10)saveStep:[64,4096],默认值256 (11)仅LoRA支持: loraRank:单选,8 或 16 或 32 或 64,默认值32 loraAlpha:单选,8 或 16 或 32 或 64,默认值32 loraDropout:[0.01,0.5],默认值0.1 |
Qianfan-Chinese-Llama-2-13B-v2 | SFT | FullFineTuning、LoRA | (1)epoch:[1,50],默认值1 (2)learningRate:[0.0000000001,0.0002],默认值0.000001 (3)batchSize:1 (4)Packing:字符串,true 或 false 或 auto,默认值auto (5)schedulerName: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值cosine (6)warmupRatio:[0.01,0.1],默认值0.03 (7)weightDecay:[0.001,1],默认值0.01 (8)maxSeqLen: 单选,512 或 1024 或 2048 或 4096,默认值4096 (9)checkpointCount:[1,10],默认值1 (10)saveStep:[64,4096],默认值256 (11)仅LoRA支持: loraRank:单选,8 或 16 或 32 或 64,默认值32 loraAlpha:单选,8 或 16 或 32 或 64,默认值32 loraDropout:[0.01,0.5],默认值0.1 |
Mixtral-8x7B | SFT | FullFineTuning | (1)epoch:[1,20],默认值1 (2)learningRate:[0.0000000001,0.0002],默认值0.000001 (3)batchSize:[1,4],默认值1 (4)Packing:字符串,true 或 false 或 auto,默认值auto (5)schedulerName: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值cosine (6)warmupRatio:[0.01,0.1],默认值0.03 (7)weightDecay:[0.001,1],默认值0.01 (8)maxSeqLen: 单选,512 或 1024 或 2048 或 4096,默认值4096 (9)checkpointCount:[1,10],默认值1 (10)saveStep:[64,4096],默认值256 (11)仅LoRA支持: loraRank:单选,8 或 16 或 32 或 64,默认值32 loraAlpha:单选,8 或 16 或 32 或 64,默认值32 loraDropout:[0.01,0.5],默认值0.1 |
SQLCoder-7B | SFT | FullFineTuning、LoRA | (1)epoch:[1,50],默认值1 (2)learningRate:[0.0000000001,0.0002],默认值0.000001 (3)batchSize:[1,4],默认值1 (4)Packing:字符串,true 或 false 或 auto,默认值auto (5)schedulerName: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值cosine (6)warmupRatio:[0.01,0.1],默认值0.03 (7)weightDecay:[0.001,1],默认值0.01 (8)maxSeqLen: 单选,512 或 1024 或 2048 或 4096,默认值4096 (9)checkpointCount:[1,10],默认值1 (10)saveStep:[64,4096],默认值256 (11)仅LoRA支持: loraRank:单选,8 或 16 或 32 或 64,默认值32 loraAlpha:单选,8 或 16 或 32 或 64,默认值32 loraDropout:[0.01,0.5],默认值0.1 |
ChatGLM2-6B-32K | SFT | FullFineTuning | (1)epoch:[1,50],默认值1 (2)learningRate:[0.0000000001,0.0002],默认值0.000001 (3)batchSize:1 (4)Packing:字符串,true 或 false 或 auto,默认值auto (5)schedulerName: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值cosine (6)warmupRatio:[0.01,0.1],默认值0.03 (7)weightDecay:[0.001,1],默认值0.01 (8)maxSeqLen: 单选,4096 或 8192 或 16384 或 32768,默认值32768 (9)checkpointCount:[1,10],默认值1 (10)saveStep:[64,4096],默认值256 (11)仅LoRA支持: loraRank:单选,8 或 16 或 32 或 64,默认值32 loraAlpha:单选,8 或 16 或 32 或 64,默认值32 loraDropout:[0.01,0.5],默认值0.1 |
ChatGLM2-6B | SFT | FullFineTuning、LoRA | (1)epoch:[1,50],默认值1 (2)learningRate:[0.0000000001,0.0002],默认值0.000001 (3)batchSize:[1,4],默认值1 (4)Packing:字符串,true 或 false 或 auto,默认值auto (5)schedulerName: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值cosine (6)warmupRatio:[0.01,0.1],默认值0.03 (7)weightDecay:[0.001,1],默认值0.01 (8)maxSeqLen: 单选,512 或 1024 或 2048 或 4096,默认值4096 (9)checkpointCount:[1,10],默认值1 (10)saveStep:[64,4096],默认值256 (11)仅LoRA支持: loraRank:单选,8 或 16 或 32 或 64,默认值32 loraAlpha:单选,8 或 16 或 32 或 64,默认值32 loraDropout:[0.01,0.5],默认值0.1 |
ChatGLM3-6B | SFT | FullFineTuning、LoRA | (1)epoch:[1,50],默认值3 (2)learningRate:[0.0000000001,0.0002],默认值0.000001 (3)batchSize:16 或 32 或 64,默认值16 (4)Packing:字符串,true 或 false 或 auto,默认值auto (5)schedulerName: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值cosine (6)warmupRatio:[0.01,0.1],默认值0.03 (7)weightDecay:[0.001,1],默认值0.01 (8)maxSeqLen:单选,4096 或 8192,默认值4096 (9)checkpointCount:[1,10],默认值1 (10)saveStep:[64,4096],默认值256 (11)仅LoRA支持: loraRank:单选,8 或 16 或 32 或 64,默认值32 loraAlpha:单选,8 或 16 或 32 或 64,默认值32 loraDropout:[0.01,0.5],默认值0.1 |
Baichuan2-7B-Chat | SFT | FullFineTuning、LoRA | (1)epoch:[1,50],默认值1 (2)learningRate:[0.0000000001,0.0002],默认值0.000001 (3)batchSize:[1,4],默认值1 (4)Packing:字符串,true 或 false 或 auto,默认值auto (5)schedulerName: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值cosine (6)warmupRatio:[0.01,0.1],默认值0.03 (7)weightDecay:[0.001,1],默认值0.01 (8)maxSeqLen: 单选,512 或 1024 或 2048 或 4096,默认值4096 (9)checkpointCount:[1,10],默认值1 (10)saveStep:[64,4096],默认值256 (11)仅LoRA支持: loraRank:单选,8 或 16 或 32 或 64,默认值32 loraAlpha:单选,8 或 16 或 32 或 64,默认值32 loraDropout:[0.01,0.5],默认值0.1 |
Baichuan2-13B-Chat | SFT | FullFineTuning、LoRA | (1)epoch:[1,50],默认值1 (2)learningRate:[0.0000000001,0.0002],默认值0.000001 (3)batchSize:[1,4],默认值1 (4)Packing:字符串,true 或 false 或 auto,默认值auto (5)schedulerName: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值cosine (6)warmupRatio:[0.01,0.1],默认值0.03 (7)weightDecay:[0.001,1],默认值0.01 (8)maxSeqLen: 单选,512 或 1024 或 2048 或 4096,默认值4096 (9)checkpointCount:[1,10],默认值1 (10)saveStep:[64,4096],默认值256 (11)仅LoRA支持: loraRank:单选,8 或 16 或 32 或 64,默认值32 loraAlpha:单选,8 或 16 或 32 或 64,默认值32 loraDropout:[0.01,0.5],默认值0.1 |
BLOOMZ-7B | SFT | FullFineTuning、LoRA | (1)epoch:[1,50],默认值1 (2)learningRate:[0.0000000001,0.0002],默认值0.000001 (3)batchSize:[1,4],默认值1 (4)Packing:字符串,true 或 false 或 auto,默认值auto (5)schedulerName: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值cosine (6)warmupRatio:[0.01,0.1],默认值0.03 (7)weightDecay:[0.001,1],默认值0.01 (8)maxSeqLen: 单选,512 或 1024 或 2048 或 4096,默认值4096 (9)checkpointCount:[1,10],默认值1 (10)saveStep:[64,4096],默认值256 (11)仅LoRA支持: loraRank:单选,8 或 16 或 32 或 64,默认值32 loraAlpha:单选,8 或 16 或 32 或 64,默认值32 loraDropout:[0.01,0.5],默认值0.1 |
CodeLlama-7B | SFT | FullFineTuning、LoRA | (1)epoch:[1,50],默认值1 (2)learningRate:[0.0000000001,0.0002],默认值0.000001 (3)batchSize:[1,4],默认值1 (4)Packing:字符串,true 或 false 或 auto,默认值auto (5)schedulerName: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值cosine (6)warmupRatio:[0.01,0.1],默认值0.03 (7)weightDecay:[0.001,1],默认值0.01 (8)maxSeqLen: 单选,512 或 1024 或 2048 或 4096,默认值4096 (9)checkpointCount:[1,10],默认值1 (10)saveStep:[64,4096],默认值256 (11)仅LoRA支持: loraRank:单选,8 或 16 或 32 或 64,默认值32 loraAlpha:单选,8 或 16 或 32 或 64,默认值32 loraDropout:[0.01,0.5],默认值0.1 (12) loraTargetModules:多选,self_attn.q_proj、self_attn.k_proj、self_attn.v_proj、self_attn.o_proj、mlp.gate_proj、mlp.up_proj、mlp.down_proj,默认值self_attn.q_proj + self_attn.v_proj |
Custom-Model(自定义模型) | SFT | FullFineTuning | (1)epoch:[1,50],默认值1 (2)learningRate:[0.0000000001,0.0002],默认值0.000001 (3)schedulerName: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值cosine (4)warmupRatio:[0.01,0.1],默认值0.03 (5)weightDecay:[0.001,1],默认值0.01 |
PostPretrain
model | trainMode | parameterScale | hyperParameterConfig |
---|---|---|---|
ERNIE-Lite-8K-0922 | PostPretrain | - | (1)epoch:[1,10],默认值1 (2)learningRate:[0.000001,0.00004],默认值0.00003 (3)maxSeqLen: 单选,4096 或 8192,默认值4096 (4)checkpointCount:[1,10],默认值1 (5)saveStep:64 或 128 或 256 或 512 或 1024 或 2048 或 4096 或 8192,默认值256 (6)seed:[1,2147483647],默认值42 (7)lrSchedulerType: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值linear (8)numCycles:[0.1,0.5],默认值0.5 (9)lrEnd:[0.00000001,0.000001],默认值0.0000001 (10)power:[1,3],默认值1 |
ERNIE-Speed-8K | PostPretrain | - | (1)epoch:[1,10],默认值1 (2)learningRate:[0.000001,0.00004],默认值0.00003 (3)maxSeqLen: 单选,4096 或 8192,默认值4096 (4)checkpointCount:[1,10],默认值1 (5)saveStep:64 或 128 或 256 或 512 或 1024 或 2048 或 4096 或 8192,默认值256 (6)seed:[1,2147483647],默认值42 (7)lrSchedulerType: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值linear (8)numCycles:[0.1,0.5],默认值0.5 (9)lrEnd:[0.00000001,0.000001],默认值0.0000001 (10)power:[1,3],默认值1 |
ERNIE-Tiny-8K | PostPretrain | - | (1)epoch:[1,10],默认值1 (2)learningRate:[0.000001,0.00004],默认值0.00003 (3)maxSeqLen: 单选,4096 或 8192,默认值4096 (4)checkpointCount:[1,10],默认值1 (5)saveStep:64 或 128 或 256 或 512 或 1024 或 2048 或 4096 或 8192,默认值256 (6)seed:[1,2147483647],默认值42 (7)lrSchedulerType: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值linear (8)numCycles:[0.1,0.5],默认值0.5 (9)lrEnd:[0.00000001,0.000001],默认值0.0000001 (10)power:[1,3],默认值1 |
Qianfan-Chinese-Llama-2-13B-v1 | PostPretrain | - | (1)epoch:1 (2)learningRate:[0.0000002,0.0002],默认值0.00002 (3)batchSize:[48,960],默认值192 (4)weightDecay:[0.0001,0.05],默认值0.01 (5)checkpointCount:[1,10],默认值1 (6)saveStep:[64,8192],默认值256 |
DPO
model | trainMode | parameterScale | hyperParameterConfig |
---|---|---|---|
ERNIE-Lite-8K-0308 | DPO | - | (1)epoch:[1,50],默认值1 (2)learningRate:[0.0000001,0.00001],默认值0.000001 (3)maxSeqLen: 单选,512 或 1024 或 2048 或 4096 或 8192,默认值4096 (4)loggingSteps:[1,100],默认值1 (5)warmupRatio:[0.01,0.5],默认值0.1 (6)weightDecay:[0.0001,0.1],默认值0.01 (7)dpoBeta:[0.01,1],默认值0.1 (8)seed:[1,2147483647],默认值42 (9)lrSchedulerType: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值linear (10)numCycles:[0.1,0.5],默认值0.5 (11)lrEnd:[0.00000001,0.000001],默认值0.0000001 (12)power:[1,3],默认值1 |
ERNIE-Lite-128K-0419 | DPO | - | (1)epoch:[1,50],默认值1 (2)learningRate:[0.0000001,0.00001],默认值0.000001 (3)maxSeqLen:单选,16384 或 32768 或 65536 或 131072,默认值32768 (4)loggingSteps:1 (5)warmupRatio:[0.01,0.5],默认值0.1 (6)weightDecay:[0.0001,0.1],默认值0.01 (7)dpoBeta:[0.01,1],默认值0.1 (8)seed:[1,2147483647],默认值42 (9)lrSchedulerType: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值linear (10)numCycles:[0.1,0.5],默认值0.5 (11)lrEnd:[0.00000001,0.000001],默认值0.0000001 (12)power:[1,3],默认值1 |
ERNIE-Speed-8K | DPO | - | (1)epoch:[1,50],默认值1 (2)learningRate:[0.0000001,0.00001],默认值0.000001 (3)maxSeqLen: 单选,512 或 1024 或 2048 或 4096 或 8192,默认值4096 (4)loggingSteps:[1,100],默认值1 (5)warmupRatio:[0.01,0.5],默认值0.1 (6)weightDecay:[0.0001,0.1],默认值0.01 (7)dpoBeta:[0.01,1],默认值0.1 (8)seed:[1,2147483647],默认值42 (9)lrSchedulerType: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值linear (10)numCycles:[0.1,0.5],默认值0.5 (11)lrEnd:[0.00000001,0.000001],默认值0.0000001 (12)power:[1,3],默认值1 |
ERNIE-Tiny-8K | DPO | - | (1)epoch:[1,50],默认值1 (2)learningRate:[0.0000001,0.00001],默认值0.000001 (3)maxSeqLen: 单选,512 或 1024 或 2048 或 4096 或 8192,默认值4096 (4)loggingSteps:[1,100],默认值1 (5)warmupRatio:[0.01,0.5],默认值0.1 (6)weightDecay:[0.0001,0.1],默认值0.01 (7)dpoBeta:[0.01,1],默认值0.1 (8)seed:[1,2147483647],默认值42 (9)lrSchedulerType: 单选,linear 或 cosine 或 polynomial 或 constant 或 constant_with_warmup,默认值linear (10)numCycles:[0.1,0.5],默认值0.5 (11)lrEnd:[0.00000001,0.000001],默认值0.0000001 (12)power:[1,3],默认值1 |
图像生成类
SFT
model | trainMode | parameterScale | hyperParameterConfig |
---|---|---|---|
WENXIN-YIGE | SFT | FullFineTuning | (1)epoch:[1,100],默认值20 (2)learningRate:[0.00000001,0.01],默认值0.00001 (3)batchSize:[1,8],默认值8 |
Stable-Diffusion-XL-Base-1.0 | SFT | LoRA | epoch:[1,100],默认值20 (2)learningRate:[0.00001,0.0001],默认值0.00005 (3)batchSize:[2,8],默认值8 |