Skip to content

add cupbone model - #3439

Open
learncat163 wants to merge 6 commits into
PaddlePaddle:developfrom
learncat163:feature/add-cupbone
Open

learncat163 wants to merge 6 commits into
PaddlePaddle:developfrom
learncat163:feature/add-cupbone

Conversation

@learncat163

Copy link
Copy Markdown

aistudio项目地址

CPUBone 对齐报告

1. 前向数值对齐

设置:

  • torch 与 paddle 均在同机 GPU
  • torch.backends.cudnn.allow_tf32 = False, torch.backends.cuda.matmul.allow_tf32 = False
  • 两侧同一份 np.random.RandomState(2026).randn(1, 3, size, size) 生成的固定输入
变体 输入尺寸 最大 max_abs_diff 结论
CPUBone_nano 224 9.537e-06 PASS
CPUBone_t0 224 1.097e-05 PASS
CPUBone_s0 224 9.060e-06 PASS
CPUBone_b0_bfrobust 224 6.199e-06 PASS
CPUBone_b1_bfrobust 224 5.722e-06 PASS
CPUBone_b1_dwnorm 256 8.106e-06 PASS
CPUBone_b2_bfrobust 224 6.676e-06 PASS
CPUBone_b2pt5_dwnorm 256 6.385e-06 PASS
CPUBone_b3 224 1.335e-05 PASS

2. ImageNet Top-1 一致率 (全量 50000 张)

设置:

  • 以 timm 预测为参考标签, 指标为严格 Top-1 一致率, 验收线 >= 99.8%
  • 双侧共用同一份预处理产物: 同一 PIL 解码 -> bicubic resize 短边到
    int(size/0.95) (r224: 235, r256: 269) -> center crop -> ImageNet 归一化,
    两侧加载同一张图
变体 一致数/总数 一致率 阈值 结论
CPUBone_nano 50000/50000 100.0000% 99.8% PASS
CPUBone_t0 50000/50000 100.0000% 99.8% PASS
CPUBone_s0 50000/50000 100.0000% 99.8% PASS
CPUBone_b0_bfrobust 50000/50000 100.0000% 99.8% PASS
CPUBone_b1_bfrobust 50000/50000 100.0000% 99.8% PASS
CPUBone_b1_dwnorm 50000/50000 100.0000% 99.8% PASS
CPUBone_b2_bfrobust 50000/50000 100.0000% 99.8% PASS
CPUBone_b2pt5_dwnorm 50000/50000 100.0000% 99.8% PASS
CPUBone_b3 50000/50000 100.0000% 99.8% PASS

@learncat163

Copy link
Copy Markdown
Author

loss复现

固定 batch(4 张随机图 + 固定标签 [0,1,2,3])反复训练 50 步,验证模型能对该 batch 拟合
(loss 下降)。Optimizer 为 AdamW(lr=1e-4, weight_decay=0.05),Loss 为
CrossEntropyLoss,加载转换后的预训练权重,全部 9 个变体在 GPU 上执行。

结果

CPUBone_nano (224x224) 逐步 Loss

Step Loss
0 6.58551598
1 3.69781160
2 1.17306507
3 0.14322652
4 0.00952650
5 0.00638606
6 0.00182910
7 0.00114616
8 0.00082348
9 0.00272816
10 0.00443053
11 0.00124245
12 0.00184661
13 0.00091363
14 0.00210361
15 0.00218385
16 0.00154678
17 0.00121120
18 0.00136079
19 0.00156706
20 0.00217364
21 0.00081747
22 0.00205561
23 0.00116847
24 0.00085303
25 0.00210452
26 0.00125340
27 0.00083646
28 0.00206722
29 0.00175553
30 0.00056145
31 0.00087787
32 0.00051085
33 0.00090591
34 0.00052666
35 0.00042846
36 0.00043752
37 0.00065586
38 0.00059761
39 0.00068646
40 0.00058883
41 0.00101961
42 0.00038941
43 0.00040603
44 0.00035594
45 0.00058409
46 0.00050074
47 0.00067060
48 0.00039296
49 0.00036675

CPUBone_t0 (224x224) 逐步 Loss

Step Loss
0 7.34180975
1 4.70709610
2 2.35248327
3 1.08535862
4 0.24968193
5 0.03607419
6 0.03891234
7 0.00960394
8 0.01450222
9 0.00444068
10 0.00635008
11 0.00403765
12 0.00283411
13 0.00210228
14 0.00118927
15 0.00311285
16 0.00175829
17 0.00092081
18 0.00264425
19 0.00051326
20 0.00152159
21 0.00176269
22 0.00138243
23 0.00095977
24 0.00149708
25 0.00043975
26 0.00055615
27 0.00210044
28 0.00083404
29 0.00230858
30 0.00043057
31 0.00103290
32 0.00243190
33 0.00035379
34 0.00045452
35 0.00135240
36 0.00213140
37 0.00049118
38 0.00142166
39 0.00047858
40 0.00044967
41 0.00023725
42 0.00039710
43 0.00035240
44 0.00022796
45 0.00037632
46 0.00031897
47 0.00034691
48 0.00034726
49 0.00018919

CPUBone_s0 (224x224) 逐步 Loss

Step Loss
0 6.71069193
1 4.41804600
2 2.48709774
3 0.34247181
4 0.21662666
5 0.02346806
6 0.02707148
7 0.02513560
8 0.01561245
9 0.00932950
10 0.01157262
11 0.01157393
12 0.00531803
13 0.00696841
14 0.01097788
15 0.00459060
16 0.00525241
17 0.00644236
18 0.00549703
19 0.00575132
20 0.00635280
21 0.00342484
22 0.00342674
23 0.00259237
24 0.00360700
25 0.00307656
26 0.00362286
27 0.00329980
28 0.00212215
29 0.00171944
30 0.00351810
31 0.00154445
32 0.00107617
33 0.00180922
34 0.00272498
35 0.00150946
36 0.00079776
37 0.00107025
38 0.00135217
39 0.00111195
40 0.00060549
41 0.00063068
42 0.00054341
43 0.00061182
44 0.00162929
45 0.00067066
46 0.00104275
47 0.00052528
48 0.00029842
49 0.00041615

CPUBone_b0_bfrobust (224x224) 逐步 Loss

Step Loss
0 6.97229004
1 2.30422640
2 0.34356487
3 0.02707754
4 0.00746216
5 0.00376179
6 0.00188412
7 0.00084484
8 0.00096389
9 0.00059245
10 0.00075840
11 0.00066804
12 0.00040658
13 0.00033766
14 0.00037314
15 0.00042722
16 0.00022387
17 0.00029951
18 0.00048076
19 0.00038309
20 0.00051802
21 0.00028000
22 0.00031349
23 0.00032350
24 0.00029570
25 0.00030157
26 0.00030678
27 0.00027139
28 0.00029245
29 0.00029877
30 0.00030002
31 0.00028798
32 0.00023856
33 0.00022498
34 0.00020427
35 0.00021103
36 0.00027228
37 0.00020326
38 0.00019712
39 0.00019911
40 0.00019652
41 0.00020066
42 0.00018764
43 0.00016837
44 0.00014432
45 0.00021019
46 0.00014909
47 0.00014167
48 0.00012656
49 0.00017072

CPUBone_b1_bfrobust (224x224) 逐步 Loss

Step Loss
0 6.64760017
1 0.81759399
2 0.06822686
3 0.01191822
4 0.00561605
5 0.00298242
6 0.00091994
7 0.00105344
8 0.00107574
9 0.00068169
10 0.00092946
11 0.00041778
12 0.00047162
13 0.00034859
14 0.00028694
15 0.00038309
16 0.00028607
17 0.00043261
18 0.00025465
19 0.00022340
20 0.00029486
21 0.00033487
22 0.00033093
23 0.00020639
24 0.00030958
25 0.00018952
26 0.00018476
27 0.00026837
28 0.00033496
29 0.00014626
30 0.00020826
31 0.00018163
32 0.00016723
33 0.00026165
34 0.00017245
35 0.00016163
36 0.00019539
37 0.00014471
38 0.00015317
39 0.00023383
40 0.00015404
41 0.00014718
42 0.00012412
43 0.00014364
44 0.00012930
45 0.00014906
46 0.00011852
47 0.00015117
48 0.00012933
49 0.00015150

CPUBone_b1_dwnorm (256x256) 逐步 Loss

Step Loss
0 7.89458370
1 1.29630542
2 0.07204325
3 0.02077942
4 0.00727023
5 0.00374523
6 0.00195218
7 0.00142611
8 0.00096166
9 0.00109912
10 0.00085242
11 0.00081185
12 0.00075154
13 0.00077124
14 0.00070815
15 0.00072096
16 0.00085765
17 0.00083832
18 0.00086237
19 0.00083760
20 0.00071221
21 0.00088965
22 0.00074473
23 0.00065698
24 0.00069964
25 0.00066395
26 0.00069641
27 0.00064063
28 0.00057894
29 0.00047897
30 0.00045949
31 0.00051071
32 0.00046986
33 0.00036932
34 0.00040299
35 0.00035485
36 0.00034259
37 0.00030623
38 0.00026824
39 0.00030432
40 0.00028353
41 0.00023484
42 0.00017546
43 0.00021827
44 0.00017454
45 0.00019387
46 0.00018249
47 0.00019033
48 0.00018917
49 0.00015135

CPUBone_b2_bfrobust (224x224) 逐步 Loss

Step Loss
0 7.19249010
1 1.99727929
2 0.21783894
3 0.05202841
4 0.01999958
5 0.00585234
6 0.00455709
7 0.00314598
8 0.00702090
9 0.00352030
10 0.00274025
11 0.03208156
12 0.00193518
13 0.00665716
14 0.00166979
15 0.00164701
16 0.00225730
17 0.00218381
18 0.00250235
19 0.00105045
20 0.00135838
21 0.00235242
22 0.00153781
23 0.00379914
24 0.00201422
25 0.00256935
26 0.00704574
27 0.00082091
28 0.00223099
29 0.00130413
30 0.00112723
31 0.00158300
32 0.00124127
33 0.00072339
34 0.00069573
35 0.00109467
36 0.00057571
37 0.00178486
38 0.00067544
39 0.00078985
40 0.00041070
41 0.00098354
42 0.00048789
43 0.00043631
44 0.00143820
45 0.00040072
46 0.00103234
47 0.00057494
48 0.00042561
49 0.00091462

CPUBone_b2pt5_dwnorm (256x256) 逐步 Loss

Step Loss
0 7.49304628
1 2.48752570
2 0.71742809
3 0.15288228
4 0.04681477
5 0.01842875
6 0.01024739
7 0.00787554
8 0.00477101
9 0.00399080
10 0.00255333
11 0.00233945
12 0.00243854
13 0.00167857
14 0.00178781
15 0.00138151
16 0.00151785
17 0.00125663
18 0.00135100
19 0.00116375
20 0.00107337
21 0.00099030
22 0.00098110
23 0.00075063
24 0.00088455
25 0.00090697
26 0.00091888
27 0.00076477
28 0.00067768
29 0.00087882
30 0.00066170
31 0.00079646
32 0.00057460
33 0.00056599
34 0.00052930
35 0.00050949
36 0.00051720
37 0.00052283
38 0.00053098
39 0.00039623
40 0.00047277
41 0.00045398
42 0.00044340
43 0.00040647
44 0.00041562
45 0.00043149
46 0.00034514
47 0.00046983
48 0.00034919
49 0.00045751

CPUBone_b3 (224x224) 逐步 Loss

Step Loss
0 7.99523020
1 3.96596146
2 0.85007662
3 0.08752073
4 0.04014708
5 0.02214123
6 0.02129698
7 0.01835546
8 0.01537087
9 0.01379539
10 0.01301311
11 0.02006862
12 0.01720963
13 0.01408002
14 0.01114747
15 0.00963091
16 0.00911144
17 0.00860074
18 0.00779323
19 0.00702792
20 0.00699960
21 0.00636570
22 0.00621024
23 0.00558614
24 0.00559834
25 0.00519889
26 0.00459649
27 0.00520464
28 0.00457724
29 0.00424124
30 0.00431634
31 0.00436194
32 0.00376344
33 0.00374078
34 0.00334241
35 0.00380792
36 0.00314677
37 0.00351800
38 0.00348519
39 0.00310868
40 0.00323708
41 0.00289246
42 0.00300593
43 0.00287377
44 0.00293352
45 0.00286453
46 0.00274729
47 0.00257325
48 0.00270642
49 0.00245950

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant