openEuler-22.03-LTS-SP2上打实时补丁后0001-apply-preempt-RT-patch.patch启动出现warning

> _**请尽量提供详细的信息，如缺乏必要的定位信息，则缺陷不会被定位**_

## 环境信息

【OS版本】openEuler-22.03-LTS-SP2

【内核版本】5.10.0-153.12.0

【硬件平台】飞腾E2000

## 缺陷信息

【问题复现步骤】
1. 选择相应的内核版本5.10.0-153.12.0
2. 打上rt-patch，主要是：
https://gitee.com/src-openeuler/kernel/blob/openEuler-22.03-LTS-SP2/0001-apply-preempt-RT-patch.patch
3. 编译preempt-rt版本内核，并确认CONFIG_SCHED_DEBUG有打开。

【实际结果】请描述出问题的结果和影响
启动时会出现warning calltrace。

【期望结果】请描述出期望的结果和影响
不应该出现这种warning。

【其他相关附件信息】比如 syslog、dmesg、panic、lockup、kdump 信息、图片等

```
[    1.674780] ------------[ cut here ]------------
[    1.674790] rq->balance_callback
[    1.674800] WARNING: CPU: 1 PID: 20 at kernel/sched/sched.h:1610 load_balance+0x610/0xc8c
[    1.674818] Modules linked in:
[    1.674826] CPU: 1 PID: 20 Comm: migration/1 Not tainted 5.10.0+ #31
[    1.674833] Hardware name: Pe2202 DEMO DDR4 (DT)
[    1.674837] Stopper: 0x0 <- 0x0
[    1.674845] pstate: 60000085 (nZCv daIf -PAN -UAO -TCO BTYPE=--)
[    1.674852] pc : load_balance+0x610/0xc8c
[    1.674857] lr : load_balance+0x610/0xc8c
[    1.674863] sp : ffffffc011e03b60
[    1.674865] x29: ffffffc011e03b60 x28: 0000000000000002 
[    1.674871] x27: ffffff9f802a6c00 x26: 0000000000000002 
[    1.674877] x25: ffffffc011b84000 x24: ffffff9fff7d3540 
[    1.674884] x23: 0000000000000002 x22: ffffffc010e9b000 
[    1.674890] x21: ffffffc011c52000 x20: ffffffc011434540 
[    1.674896] x19: 0000000000000000 x18: 0000000000000000 
[    1.674902] x17: 0000000021c2f04a x16: 0000000000000014 
[    1.674908] x15: 000000000000000a x14: 0000000000000000 
[    1.674914] x13: 0000000100000000 x12: 0000000200000004 
[    1.674920] x11: 0000000000000463 x10: 00000000000002f4 
[    1.674925] x9 : ffffffc0100a15a8 x8 : 6c6c61635f65636e 
[    1.674931] x7 : 616c61623e2d7172 x6 : ffffffc011b8c3f9 
[    1.674938] x5 : ffffffc011b8c3f8 x4 : 0000000000000000 
[    1.674944] x3 : ffffffc011425008 x2 : 0000000000000003 
[    1.674950] x1 : 0000000000000000 x0 : 0000000000000000 
[    1.674956] Call trace:
[    1.674958]  load_balance+0x610/0xc8c
[    1.674965]  newidle_balance.isra.0+0x240/0x2cc
[    1.674972]  balance_fair+0x24/0x40
[    1.674978]  __schedule+0x6a4/0x7e0
[    1.674985]  schedule+0xbc/0x110
[    1.674990]  smpboot_thread_fn+0x1d8/0x2a8
[    1.674998]  kthread+0x188/0x198
[    1.675008]  ret_from_fork+0x10/0x30
[    1.675016] ---[ end trace 0000000000000002 ]---
[    2.675144] ------------[ cut here ]------------
[    2.675147] rq->balance_callback
[    2.675153] WARNING: CPU: 0 PID: 142 at kernel/sched/sched.h:1610 try_to_wake_up+0x440/0x52c
[    2.675165] Modules linked in:
[    2.675169] CPU: 0 PID: 142 Comm: irq/24-uart-pl0 Tainted: G        W         5.10.0+ #31
[    2.675175] Hardware name: Pe2202 DEMO DDR4 (DT)
[    2.675178] pstate: 60000085 (nZCv daIf -PAN -UAO -TCO BTYPE=--)
[    2.675184] pc : try_to_wake_up+0x440/0x52c
[    2.675190] lr : try_to_wake_up+0x440/0x52c
[    2.675196] sp : ffffffc011c8bdc0
[    2.675198] x29: ffffffc011c8bdc0 x28: ffffff9f80983300 
[    2.675205] x27: 0000000000000001 x26: ffffffc01255bc80 
[    2.675212] x25: ffffffdfee39f000 x24: ffffff9f80341a60 
[    2.675218] x23: 0000000000000080 x22: 0000000000000000 
[    2.675224] x21: 0000000000000000 x20: ffffff9fff7d3540 
[    2.675230] x19: ffffff9f80341100 x18: 0000000000000000 
[    2.675236] x17: 000000009dcc231e x16: 0000000000000014 
[    2.675242] x15: 000000000000000a x14: 0000000000086a2c 
[    2.675248] x13: ffffffc09235bd17 x12: 0000000000000006 
[    2.675254] x11: ffffffffffffffff x10: ffffffc01198ae40 
[    2.675260] x9 : ffffffc0100a15a8 x8 : 6c6c61635f65636e 
[    2.675266] x7 : 616c61623e2d7172 x6 : ffffffc011b8cad9 
[    2.675272] x5 : ffffffc011b8cad8 x4 : 0000000000000000 
[    2.675278] x3 : ffffffc011425008 x2 : 0000000000010004 
[    2.675284] x1 : 0000000000000000 x0 : 0000000000000000 
[    2.675290] Call trace:
[    2.675293]  try_to_wake_up+0x440/0x52c
[    2.675299]  wake_up_process+0x20/0x2c
[    2.675306]  wake_irq_workd+0x54/0x64
[    2.675315]  irq_work_wake+0x10/0x1c
[    2.675321]  irq_work_single+0x3c/0x84
[    2.675327]  flush_smp_call_function_queue+0x220/0x224
[    2.675335]  generic_smp_call_function_single_interrupt+0x1c/0x28
[    2.675343]  ipi_handler+0x240/0x290
[    2.675351]  handle_percpu_devid_fasteoi_ipi+0x13c/0x1f4
[    2.675359]  generic_handle_irq+0x2c/0x44
[    2.675367]  __handle_domain_irq+0x104/0x108
[    2.675374]  gic_handle_irq+0xc4/0x14c
[    2.675379]  el1_irq+0xac/0x140
[    2.675384]  irq_thread+0xd0/0x1ec
[    2.675390]  kthread+0x188/0x198
[    2.675398]  ret_from_fork+0x10/0x30
[    2.675404] ---[ end trace 0000000000000003 ]---

```

【已分析信息】如已经做过分析和定位，请尽量附上详细的分析结果
WARN报的地方在：rq->balance_callback，这里是说在rq_pin_lock时，balance_callback应该是NULL，应当是已经都被处理掉了。
而queue_balance_callback()的地方很单一。只有rt.c和deadline.c会调用：
```
[    1.353735] [   T14] ------------[ cut here ]------------
[    1.353738] [   T14] rq->balance_callback
[    1.353749] [   T14] WARNING: CPU: 0 PID: 14 at kernel/sched/sched.h:1611 __schedule+0x1e0/0x9ac
[    1.353761] [   T14] Modules linked in:
[    1.353768] [   T14] CPU: 0 PID: 14 Comm: rcuc/0 Not tainted 5.10.0+ #9
[    1.353774] [   T14] Hardware name: Pe2202 DEMO DDR4 (DT)
[    1.353777] [   T14] pstate: 60000085 (nZCv daIf -PAN -UAO -TCO BTYPE=--)
[    1.353784] [   T14] pc : __schedule+0x1e0/0x9ac
[    1.353790] [   T14] lr : __schedule+0x1e0/0x9ac
[    1.353796] [   T14] sp : ffffffc01309bd50
[    1.353798] [   T14] x29: ffffffc01309bd80 x28: 0000000000000000 
[    1.353806] [   T14] x27: ffffffc0122a6000 x26: 0000000000000000 
[    1.353812] [   T14] x25: 0000000000000000 x24: ffffffc011f52c30 
[    1.353819] [   T14] x23: ffffff9fff605f18 x22: ffffffc0122af000 
[    1.353826] [   T14] x21: ffffffc011a87f00 x20: ffffff9f80321e40 
[    1.353833] [   T14] x19: ffffff9fff605f00 x18: 0000000000000010 
[    1.353839] [   T14] x17: 000000006ec2678f x16: 00000000ffffffff 
[    1.353846] [   T14] x15: ffffffc01308b890 x14: ffffffc0100a0cf0 
[    1.353853] [   T14] x13: ffffffffffffffff x12: ffffffffffffffff 
[    1.353860] [   T14] x11: 0000000000000020 x10: 0000000000001990 
[    1.353866] [   T14] x9 : ffffffc0100ceae4 x8 : ffffffc011f745a0 
[    1.353873] [   T14] x7 : ffffffc01309bd50 x6 : ffffffc012d9327c 
[    1.353879] [   T14] x5 : 00000000ffffffc8 x4 : ffffffc012d93268 
[    1.353886] [   T14] x3 : ffffffc0118b6008 x2 : 0000000000000000 
[    1.353892] [   T14] x1 : 0000000000000000 x0 : ffffff9f80321e40 
[    1.353898] [   T14] Call trace:
[    1.353901] [   T14]  __schedule+0x1e0/0x9ac
[    1.353907] [   T14]  schedule+0xb8/0x10c
[    1.353914] [   T14]  smpboot_thread_fn+0x1c4/0x288
[    1.353922] [   T14]  kthread+0x17c/0x18c
[    1.353930] [   T14]  ret_from_fork+0x10/0x30
```
这里的rcuc/0就是rt调度类：
14 rcuc/0          FF
这时情况大概是这样的：
```
前一次schedule
__schedule{ // 切换到task pid 14的前一次
    rq_lock(rq, &rf); -> rq_pin_lock(rq, rf);
    pick_next_task()  // 这里选择了下一个rt task pid 14, 同时给rq->callback_balance注册了一个func。
    if (likely(prev != next)) {
        context_switch(rq, prev, next, &rf); //这里的openeuler 5.10内核(普通内核就没有) 接下来就没有balance_callback。
        --->finish_task_switch(prev)
        --->--->finish_lock_switch(rq);      // 但是在openeuler 6.6内核，和上游的linux 5.10，是会在finish_lock_switch这，执行rq->balance_callback的
    }
    else { // 如果sched是同一个task，会用balance_callback(rq)执行并清掉pick时加的balance_callback。
        rq_unpin_lock(rq, &rf);
        __balance_callbacks(rq);
    }
}
···
本次schedule
···
__schedule{  // task pid 14
    rq_lock(rq, &rf); -> rq_pin_lock(rq, rf); --> 触发打印rq_pin_lock中的打印：

#ifdef CONFIG_SMP
	SCHED_WARN_ON(rq->balance_callback);
#endif
```
从上面简化的流程可以看出，打印WARNING的原因很明显，在finish_task_switch()中，缺少了调用__balance_callbacks的地方。
对此针对rt-patch中的balance_callback相关改动进行了追溯：

相关内容其实是在内核Linux 5.11之后开始增加的：
```
565790d28b1e sched: Fix balance_callback()                                  -> 增加__balance_callbacks(rq);
2558aacff858 sched/hotplug: Ensure only per-cpu kthreads run during hotplug -> 增加balance_switch接口，finish_lock_switch中改成balance_switch(rq);
ae7927023243 sched: Optimize finish_lock_switch()                           -> 又加了其他内容，优化__balance_callbacks(rq);
```
然后在5.10-rt内核中，提前增加了以下两笔：
```
0126b9d415382 sched: Fix balance_callback()
4c1c6261e21f6 sched/hotplug: Ensure only per-cpu kthreads run during hotplug
```
在openEuler-22.03-LTS-SP2的src-openeuler/kernel提供的rt patch中，可以看出来的是5.10-rt的加的这两笔的内容基本都在。就是少了finish_lock_switch中的改动。
从下面的差异，可以看出欧拉内核在5.10版本backport了一些lock相关的改动，这里可能会导致欧拉内核在apply rt-patch时因为冲突，导致遗失了相关改动。
欧拉内核：
```
static inline void finish_lock_switch(struct rq *rq)
{
        spin_acquire(&__rq_lockp(rq)->dep_map, 0, 0, _THIS_IP_);
        raw_spin_rq_unlock_irq(rq);
}
```
linux内核：
```
        spin_acquire(&rq->lock.dep_map, 0, 0, _THIS_IP_);
        balance_switch(rq);
        raw_spin_unlock_irq(&rq->lock);
 }
```
同时做了实验，打上rt-patch后，在finish_lock_switch中补上疑似遗失的`balance_switch(rq);`就正常了。
能否帮忙确认这个问题，并更新的src-openeuler/kernel中用于5.10内核的rt-patch?
**二、缺陷分析结构反馈**
影响性分析说明：

缺陷严重等级:(Critical/High/Moderate/Low)

缺陷根因说明:

受影响版本排查(受影响/不受影响):
openEuler-20.03-LTS-SP4
openEuler-22.03-LTS-SP3
openEuler-22.03-LTS-SP4
openEuler-24.03-LTS
openEuler-24.03-LTS-SP1

修复是否涉及abi变化(是/否):
openEuler-20.03-LTS-SP4
openEuler-22.03-LTS-SP3
openEuler-22.03-LTS-SP4
openEuler-24.03-LTS
openEuler-24.03-LTS-SP1

src-openEuler/kernel

内容风险标识

评论 (5)

src-openEuler/kernel .gitee-modal { width: 500px !important; }

内容风险标识