• ThreadX内核源码分析(SMP) - 核间互斥(arm)


    1、核间互斥介绍(_tx_thread_smp_protection)

            在单核的ThreadX内核中,内核的临界资源互斥通过关中断实现;在多核cpu上,关闭整个cpu的代价比较大,单核上仍然使用关中断实现(其他核的中断仍然开启),但是与其他核之间互斥通过_tx_thread_smp_protection实现:

            首先,单核已经关中断,那么正在访问资源的线程不会被切换出去,在退出临界资源之前,当前核上的线程不会被中断,也就是在退出临界资源前当前核不会有其他线程执行,当前核的线程之间是互斥的;

            其次,_tx_thread_smp_protection标记了占用_tx_thread_smp_protection的核,其他核的线程如果检查到_tx_thread_smp_protection被其他核占用,那么等待_tx_thread_smp_protection被释放,因此实现了核间互斥。

    1.1、TX_THREAD_SMP_PROTECT结构

    1. typedef struct TX_THREAD_SMP_PROTECT_STRUCT
    2. {
    3. ULONG tx_thread_smp_protect_in_force;
    4. ULONG tx_thread_smp_protect_core;
    5. ULONG tx_thread_smp_protect_count;
    6. ULONG tx_thread_smp_protect_pad_0;
    7. ULONG tx_thread_smp_protect_pad_1;
    8. ULONG tx_thread_smp_protect_pad_2;
    9. ULONG tx_thread_smp_protect_pad_3;
    10. } TX_THREAD_SMP_PROTECT;

    tx_thread_smp_protect_in_force: 是否被占用(0没有被占用,1被占用)

    tx_thread_smp_protect_core: 被哪个cpu核占用(0xFFFFFFFF无效)

    tx_thread_smp_protect_count: 被占用次数。

    2、临界资源保护(_tx_thread_smp_protect)

            _tx_thread_smp_protect保护临界资源,实现的功能类似linux内核的自旋锁,主要用于核间的互斥,一个核在访问临界资源的时候,避免其他核访问该临界资源。

            _tx_thread_smp_protect基本原理就是关闭当前核的中断,检查_tx_thread_smp_protection是否被当前核占用,如果是,那么对_tx_thread_smp_protection的占用次数加1,返回关中断之前的中断标志(调用_tx_thread_smp_protect的时候可能就已经是关闭了中断,那么退出临界资源应该恢复关中断的状态,不能简单开启中断;内核的所有临界资源都用_tx_thread_smp_protection保护,存在临界资源嵌套,也就是在临界资源里面再次调用_tx_thread_smp_protect,所以前面对_tx_thread_smp_protection的占用次数加1)。

             _tx_thread_smp_protect主要实现代码如下:

    1. 076 .global _tx_thread_smp_protect
    2. 077 .type _tx_thread_smp_protect, @function
    3. 078 _tx_thread_smp_protect:
    4. 079
    5. 080 /* Disable interrupts so we don't get preempted. */
    6. 081
    7. 082 MRS x0, DAIF // Pickup current interrupt posture
    8. 083 MSR DAIFSet, 0x3 // Lockout interrupts
    9. 084
    10. 085 /* Pickup the CPU ID. */
    11. 086
    12. 087 MRS x2, MPIDR_EL1 // Pickup the core ID
    13. 088 #ifdef TX_ARMV8_2
    14. 089 #if TX_THREAD_SMP_CLUSTERS > 1
    15. 090 UBFX x7, x2, #16, #8 // Isolate cluster ID
    16. 091 #endif
    17. 092 UBFX x2, x2, #8, #8 // Isolate core ID
    18. 093 #else
    19. 094 #if TX_THREAD_SMP_CLUSTERS > 1
    20. 095 UBFX x7, x2, #8, #8 // Isolate cluster ID
    21. 096 #endif
    22. 097 UBFX x2, x2, #0, #8 // Isolate core ID
    23. 098 #endif
    24. 099 #if TX_THREAD_SMP_CLUSTERS > 1
    25. 100 ADDS x2, x2, x7, LSL #2 // Calculate CPU ID
    26. 101 #endif
    27. 102
    28. 103 LDR x1, =_tx_thread_smp_protection // Build address to protection structure
    29. 104 LDR w3, [x1, #4] // Pickup the owning core
    30. 105 CMP w3, w2 // Is it this core?
    31. 106 BEQ _owned // Yes, the protection is already owned
    32. 107
    33. 108 LDAXR w4, [x1, #0] // Pickup the protection flag
    34. 109 CBZ w4, _get_protection // Yes, get the protection
    35. 110 MSR DAIF, x0 // Restore interrupts
    36. 111 ISB //
    37. 112 #ifdef TX_ENABLE_WFE
    38. 113 WFE // Go into standby
    39. 114 #endif
    40. 115 B _tx_thread_smp_protect // On waking, restart the protection attempt
    41. 116
    42. 117 _get_protection:
    43. 118 MOV x4, #1 // Build lock value
    44. 119 STXR w5, w4, [x1] // Attempt to get the protection
    45. 120 CBZ w5, _got_protection // Did it succeed? w5 = 0 means success!
    46. 121 MSR DAIF, x0 // Restore interrupts
    47. 122 B _tx_thread_smp_protect // Restart the protection attempt
    48. 123
    49. 124 _got_protection:
    50. 125 DMB ISH //
    51. 126 STR w2, [x1, #4] // Save owning core
    52. 127 _owned:
    53. 128 LDR w5, [x1, #8] // Pickup ownership count
    54. 129 ADD w5, w5, #1 // Increment ownership count
    55. 130 STR w5, [x1, #8] // Store ownership count
    56. 131 DMB ISH //
    57. 132 RET

    2.1、关当前核的中断

    保存DAIF到x0寄存器(x0也是函数的返回值),设置DAIF的IF标志位,禁止IRQ、FIQ中断。

    1. 082 MRS x0, DAIF // Pickup current interrupt posture // 保存关中断前的DAIF(下一行会修改IF标志位,退出临界资源后需要恢复DAIF;x0在这里除了保存DAIF外,也是_tx_thread_smp_protect的返回值)
    2. 083 MSR DAIFSet, 0x3 // Lockout interrupts // IF位设置为1,禁止当前核的IRQ、FIQ中断(当前核实现互斥,当前核在访问临界资源过程中不会被切换出去,当前核没有中断及线程被调度执行,当前核的临界资源不存在被中断的情况)

    2.2、获取当前核的cpu ID

    通过MPIDR_EL1寄存器读取当前核的cpu ID,保存到x2寄存器里面。

    1. 085 /* Pickup the CPU ID. */
    2. 086
    3. 087 MRS x2, MPIDR_EL1 // Pickup the core ID
    4. 088 #ifdef TX_ARMV8_2
    5. 089 #if TX_THREAD_SMP_CLUSTERS > 1
    6. 090 UBFX x7, x2, #16, #8 // Isolate cluster ID
    7. 091 #endif
    8. 092 UBFX x2, x2, #8, #8 // Isolate core ID
    9. 093 #else
    10. 094 #if TX_THREAD_SMP_CLUSTERS > 1
    11. 095 UBFX x7, x2, #8, #8 // Isolate cluster ID
    12. 096 #endif
    13. 097 UBFX x2, x2, #0, #8 // Isolate core ID
    14. 098 #endif
    15. 099 #if TX_THREAD_SMP_CLUSTERS > 1
    16. 100 ADDS x2, x2, x7, LSL #2 // Calculate CPU ID
    17. 101 #endif

    2.3、比较占用_tx_thread_smp_protection的cpu ID

            获取占用_tx_thread_smp_protection的cpu ID(_tx_thread_smp_protection没有被占用时,cpu ID是一个无效值,不等于任何cpu ID),如果是当前cpu占用了_tx_thread_smp_protection,那么不需要再次占用,跳转到_owned,只需要简单对占用计算器加1即可。

    1. 103 LDR x1, =_tx_thread_smp_protection // Build address to protection structure // x1 = &_tx_thread_smp_protection
    2. 104 LDR w3, [x1, #4] // Pickup the owning core // w3 = _tx_thread_smp_protection.tx_thread_smp_protect_core,通过成员变量的偏移读取成员变量的值
    3. 105 CMP w3, w2 // Is it this core? // w2: 当前核的cpu ID,w3: 占用_tx_thread_smp_protection核的cpu ID
    4. 106 BEQ _owned // Yes, the protection is already owned // 占用_tx_thread_smp_protection的cpu ID等于当前核的cpu ID(在临界资源里面再次调用_tx_thread_smp_protect),跳转到_owned(已经获取到了...)

    2.4、检查_tx_thread_smp_protection是否被占用

            _tx_thread_smp_protection的cpu ID有两种情况,一种是其他cpu的ID,另外一种是无效cpu ID,如果_tx_thread_smp_protection的cpu ID不是当前核的cpu ID,那么要检查这两种情况,主要就是检查tx_thread_smp_protect_in_force,如果tx_thread_smp_protect_in_force为0,那么_tx_thread_smp_protection就没有被占用;

            对tx_thread_smp_protect_in_force的判断改写要实现原子操作,读写过程并没有暂停其他cpu,因此使用LDAXR/STXR实现多核对tx_thread_smp_protect_in_force的互斥;

            尝试获取_tx_thread_smp_protection时,先检查tx_thread_smp_protect_in_force是否为0,如果为0,那么就可以获取_tx_thread_smp_protection,跳转到_get_protection去获取_tx_thread_smp_protection,否则恢复DAIF(主要是中断使能位,允许当前核处理中断),然后跳转到_tx_thread_smp_protect,重新去获取_tx_thread_smp_protection,类似ticket自旋锁。

    1. 103 LDR x1, =_tx_thread_smp_protection // Build address to protection structure // x1 = &_tx_thread_smp_protection
    2. 104 LDR w3, [x1, #4] // Pickup the owning core // w3 = _tx_thread_smp_protection.tx_thread_smp_protect_core,通过成员变量的偏移读取成员变量的值
    3. 105 CMP w3, w2 // Is it this core? // w2: 当前核的cpu ID,w3: 占用_tx_thread_smp_protection核的cpu ID
    4. 106 BEQ _owned // Yes, the protection is already owned // 占用_tx_thread_smp_protection的cpu ID等于当前核的cpu ID(在临界资源里面再次调用_tx_thread_smp_protect),跳转到_owned(已经获取到了...)
    5. 107
    6. 108 LDAXR w4, [x1, #0] // Pickup the protection flag // 读_tx_thread_smp_protection.tx_thread_smp_protect_in_force
    7. 109 CBZ w4, _get_protection // Yes, get the protection // _tx_thread_smp_protection.tx_thread_smp_protect_in_force为0(LDAXR读的时候,_tx_thread_smp_protection没被其他核占用),跳转到_get_protection去获取_tx_thread_smp_protection
    8. 110 MSR DAIF, x0 // Restore interrupts // _tx_thread_smp_protection被其他线程占用,恢复DAIF(对于非嵌套进入临界资源,这里可能恢复中断,如果有中断就绪,那么应该尽快处理中断,所以这里要恢复DAIF)
    9. 111 ISB //
    10. 112 #ifdef TX_ENABLE_WFE
    11. 113 WFE // Go into standby
    12. 114 #endif
    13. 115 B _tx_thread_smp_protect // On waking, restart the protection attempt // 跳转到_tx_thread_smp_protect,重新尝试获取_tx_thread_smp_protection(上面已经恢复DAIF,_tx_thread_smp_protect开始的地方又会关中断,在恢复关中断几条指令间,如果有中断,cpu会优先处理中断)

    2.5、尝试获取_tx_thread_smp_protection

            前面LDAXR并没有阻止其他核获取_tx_thread_smp_protection,LDAXR读取到_tx_thread_smp_protection没被占用之后,可能有其他核也去获取_tx_thread_smp_protection,所以使用STXR指令去写_tx_thread_smp_protection.tx_thread_smp_protect_in_force,写失败就表明别其他核改写了,如果写成功那么就获取到了_tx_thread_smp_protection。

    1. 118 MOV x4, #1 // Build lock value // LDAXR读取到_tx_thread_smp_protection.tx_thread_smp_protect_in_force为0,那么如果在STXR指令之前没有其他核修改_tx_thread_smp_protection.tx_thread_smp_protect_in_force,那么STXR指令就会成功,否则就会失败
    2. 119 STXR w5, w4, [x1] // Attempt to get the protection // 尝试写1到_tx_thread_smp_protection.tx_thread_smp_protect_in_force(LDAXR并没有阻止其他核改写_tx_thread_smp_protection.tx_thread_smp_protect_in_force,如果其他核没有改写的话(读完之后,没有其他cpu获取到_tx_thread_smp_protection),那么STXR将会成功,否则,如果其他核先写_tx_thread_smp_protection.tx_thread_smp_protect_in_force(先获取_tx_thread_smp_protection),那么STXR将会失败)
    3. 120 CBZ w5, _got_protection // Did it succeed? w5 = 0 means success! // 1成功写入到_tx_thread_smp_protection.tx_thread_smp_protect_core,成功获取到_tx_thread_smp_protection,跳转到_got_protection
    4. 121 MSR DAIF, x0 // Restore interrupts // 读取_tx_thread_smp_protection.tx_thread_smp_protect_in_force到写_tx_thread_smp_protection.tx_thread_smp_protect_in_force过程,有其他核先写了_tx_thread_smp_protection.tx_thread_smp_protect_in_force,当前核写失败,需要重新去获取_tx_thread_smp_protection,与前面一样,这里要恢复DAIF,让cpu优先处理中断
    5. 122 B _tx_thread_smp_protect // Restart the protection attempt // 跳转到_tx_thread_smp_protect,重新获取_tx_thread_smp_protection

    2.6、更新_tx_thread_smp_protection

    第一次占用_tx_thread_smp_protection那么需要设置tx_thread_smp_protect_core,否则之需要对占用次数tx_thread_smp_protect_count加1即可,然后返回上一级函数(注意临界资源不能开中断,不能被抢占)。

    1. 124 _got_protection:
    2. 125 DMB ISH //
    3. 126 STR w2, [x1, #4] // Save owning core // 当前核的cpu ID写到_tx_thread_smp_protection.tx_thread_smp_protect_core
    4. 127 _owned:
    5. 128 LDR w5, [x1, #8] // Pickup ownership count
    6. 129 ADD w5, w5, #1 // Increment ownership count
    7. 130 STR w5, [x1, #8] // Store ownership count // 占用次数_tx_thread_smp_protection.tx_thread_smp_protect_count加1
    8. 131 DMB ISH //
    9. 132 RET // 返回,这里没有恢复也不能恢复DAIF,需要在退出临界资源时恢复DAIF(假设在临界资源里面恢复了中断,然后中断服务程序里面唤醒了更高优先级的线程或者时间片用完了调度其他线程,那么_tx_thread_smp_protection就得不到释放,_tx_thread_smp_protect就会进入无线循环!!!)

    3、取消保护(_tx_thread_smp_unprotect)

    退出临界资源。_tx_thread_smp_unprotect与_tx_thread_smp_protect函数一样,也是先关闭了当前核的中断(正常情况,临界资源退出时中断本来就是关闭的)并且读取了当前核的cpu ID(如果不是占用_tx_thread_smp_protection的核释放_tx_thread_smp_protection,那么就是错误的调用,不能释放其他核占用的_tx_thread_smp_protection),然后对占用次数tx_thread_smp_protect_count减1,如果tx_thread_smp_protect_count不为0,那么还要等待其他临界资源退出。

    实现代码如下:

    1. 072 .global _tx_thread_smp_unprotect
    2. 073 .type _tx_thread_smp_unprotect, @function
    3. 074 _tx_thread_smp_unprotect:
    4. 075 MSR DAIFSet, 0x3 // Lockout interrupts
    5. 076
    6. 077 MRS x1, MPIDR_EL1 // Pickup the core ID
    7. 078 #ifdef TX_ARMV8_2
    8. 079 #if TX_THREAD_SMP_CLUSTERS > 1
    9. 080 UBFX x2, x1, #16, #8 // Isolate cluster ID
    10. 081 #endif
    11. 082 UBFX x1, x1, #8, #8 // Isolate core ID
    12. 083 #else
    13. 084 #if TX_THREAD_SMP_CLUSTERS > 1
    14. 085 UBFX x2, x1, #8, #8 // Isolate cluster ID
    15. 086 #endif
    16. 087 UBFX x1, x1, #0, #8 // Isolate core ID
    17. 088 #endif
    18. 089 #if TX_THREAD_SMP_CLUSTERS > 1
    19. 090 ADDS x1, x1, x2, LSL #2 // Calculate CPU ID
    20. 091 #endif
    21. 092
    22. 093 LDR x2,=_tx_thread_smp_protection // Build address of protection structure
    23. 094 LDR w3, [x2, #4] // Pickup the owning core
    24. 095 CMP w1, w3 // Is it this core?
    25. 096 BNE _still_protected // If this is not the owning core, protection is in force elsewhere
    26. 097
    27. 098 LDR w3, [x2, #8] // Pickup the protection count
    28. 099 CMP w3, #0 // Check to see if the protection is still active
    29. 100 BEQ _still_protected // If the protection count is zero, protection has already been cleared
    30. 101
    31. 102 SUB w3, w3, #1 // Decrement the protection count
    32. 103 STR w3, [x2, #8] // Store the new count back
    33. 104 CMP w3, #0 // Check to see if the protection is still active
    34. 105 BNE _still_protected // If the protection count is non-zero, protection is still in force
    35. 106 LDR x2,=_tx_thread_preempt_disable // Build address of preempt disable flag
    36. 107 LDR w3, [x2] // Pickup preempt disable flag
    37. 108 CMP w3, #0 // Is the preempt disable flag set?
    38. 109 BNE _still_protected // Yes, skip the protection release
    39. 110
    40. 111 LDR x2,=_tx_thread_smp_protection // Build address of protection structure
    41. 112 MOV w3, #0xFFFFFFFF // Build invalid value
    42. 113 STR w3, [x2, #4] // Mark the protected core as invalid
    43. 114 DMB ISH // Ensure that accesses to shared resource have completed
    44. 115 MOV w3, #0 // Build release protection value
    45. 116 STR w3, [x2, #0] // Release the protection
    46. 117 DSB ISH // To ensure update of the protection occurs before other CPUs awake
    47. 118
    48. 119 _still_protected:
    49. 120 #ifdef TX_ENABLE_WFE
    50. 121 SEV // Send event to other CPUs, wakes anyone waiting on the protection (using WFE)
    51. 122 #endif
    52. 123 MSR DAIF, x0 // Restore interrupt posture
    53. 124 RET

    3.1、获取当前核的cpu ID

            通过MPIDR_EL1获取当前核的cpu ID,保存到x1寄存器。(需要检查_tx_thread_smp_protection是否是被当前核占用,不是的话,不能释放_tx_thread_smp_protection)

    1. 077 MRS x1, MPIDR_EL1 // Pickup the core ID
    2. 078 #ifdef TX_ARMV8_2
    3. 079 #if TX_THREAD_SMP_CLUSTERS > 1
    4. 080 UBFX x2, x1, #16, #8 // Isolate cluster ID
    5. 081 #endif
    6. 082 UBFX x1, x1, #8, #8 // Isolate core ID
    7. 083 #else
    8. 084 #if TX_THREAD_SMP_CLUSTERS > 1
    9. 085 UBFX x2, x1, #8, #8 // Isolate cluster ID
    10. 086 #endif
    11. 087 UBFX x1, x1, #0, #8 // Isolate core ID
    12. 088 #endif
    13. 089 #if TX_THREAD_SMP_CLUSTERS > 1
    14. 090 ADDS x1, x1, x2, LSL #2 // Calculate CPU ID
    15. 091 #endif

    3.2、检查cpu ID

    如果占用_tx_thread_smp_protection的核不是当前核,那么跳转到_still_protected(_still_protected恢复了DAIF,_tx_thread_smp_unprotect需要interrupt_save这个局部变量,而这个变量是_tx_thread_smp_protect返回的,也是在_tx_thread_smp_protect对应的宏TX_DISABLE定义的,那么调用_tx_thread_smp_unprotect能编译通过的前提是必须定义了interrupt_save,必须调用了_tx_thread_smp_protect,_tx_thread_smp_protection不是被当前核占用的条件就是_tx_thread_smp_unprotect多调用了,因此重复恢复DAIF的值,正常情况下是不影响的)

    1. 093 LDR x2,=_tx_thread_smp_protection // Build address of protection structure
    2. 094 LDR w3, [x2, #4] // Pickup the owning core // 读取_tx_thread_smp_protection.tx_thread_smp_protect_core
    3. 095 CMP w1, w3 // Is it this core? // core比较
    4. 096 BNE _still_protected // If this is not the owning core, protection is in force elsewhere // _tx_thread_smp_protection不是被当前核占用,跳转到_still_protected,不能释放_tx_thread_smp_protection
    5. 097
    6. 098 LDR w3, [x2, #8] // Pickup the protection count
    7. 099 CMP w3, #0 // Check to see if the protection is still active
    8. 100 BEQ _still_protected // If the protection count is zero, protection has already been cleared // 占用计数器已经为0了,实际没有被占用,但是也没有让其他核占用(后面如果禁止抢占的话,是不会修改_tx_thread_smp_protection.tx_thread_smp_protect_core)

    3.3、释放_tx_thread_smp_protection

    _tx_thread_smp_protection占用计数器tx_thread_smp_protect_count减1,如果仍然占用,跳转到_still_protected,还在其他临界资源里面。

    1. 102 SUB w3, w3, #1 // Decrement the protection count
    2. 103 STR w3, [x2, #8] // Store the new count back // 占用计数器_tx_thread_smp_protection.tx_thread_smp_protect_count减1
    3. 104 CMP w3, #0 // Check to see if the protection is still active
    4. 105 BNE _still_protected // If the protection count is non-zero, protection is still in force // 占用计数器_tx_thread_smp_protection.tx_thread_smp_protect_count不为0,仍然占用,跳转到_still_protected

    3.4、抢占检查

            从代码上看,退出临界资源后,如果禁止抢占,那么也不真正释放_tx_thread_smp_protection,也就是不让其他核获取到_tx_thread_smp_protection。

            猜测这个抢占检查应该是为了让更高优先级的线程尽快执行,简单的说就是禁止抢占的情况下,各个cpu核上等待_tx_thread_smp_protection的线程不一定是最高优先级就绪线程(在禁止抢占过程,有中断或者其他线程唤醒更高优先级就绪线程,但是禁止了抢占,所以等待_tx_thread_smp_protection的线程并不会被切换出去),如果禁止抢占期间立即释放_tx_thread_smp_protection,那么别的核就可以获取到_tx_thread_smp_protection并关闭中断,这就会导致禁止抢占期间唤醒的高优先级线程得不到及时的调度,所以禁止抢占期间不立即释放_tx_thread_smp_protection。

    1. 106 LDR x2,=_tx_thread_preempt_disable // Build address of preempt disable flag // 没有占用_tx_thread_smp_protection,已经完全退出了临界资源,检查_tx_thread_preempt_disable
    2. 107 LDR w3, [x2] // Pickup preempt disable flag
    3. 108 CMP w3, #0 // Is the preempt disable flag set?
    4. 109 BNE _still_protected // Yes, skip the protection release // 禁止抢占计数器_tx_thread_preempt_disable不为0,跳过_tx_thread_smp_protection释放(不释放_tx_thread_smp_protection)

    3.5、释放_tx_thread_smp_protection

    主要是恢复_tx_thread_smp_protection的成员变量为未占用状态(tx_thread_smp_protect_in_force、tx_thread_smp_protect_core),然后恢复DAIF寄存器。

    1. 111 LDR x2,=_tx_thread_smp_protection // Build address of protection structure
    2. 112 MOV w3, #0xFFFFFFFF // Build invalid value
    3. 113 STR w3, [x2, #4] // Mark the protected core as invalid
    4. 114 DMB ISH // Ensure that accesses to shared resource have completed
    5. 115 MOV w3, #0 // Build release protection value
    6. 116 STR w3, [x2, #0] // Release the protection
    7. 117 DSB ISH // To ensure update of the protection occurs before other CPUs awake
    8. 118
    9. 119 _still_protected:
    10. 120 #ifdef TX_ENABLE_WFE
    11. 121 SEV // Send event to other CPUs, wakes anyone waiting on the protection (using WFE)
    12. 122 #endif
    13. 123 MSR DAIF, x0 // Restore interrupt posture
    14. 124 RET

  • 相关阅读:
    研究了11种实时聊天软件,我发现都具备这些功能…
    TCP三次握手四次挥手深入
    OFDM Tutorial Series-1: 线性分组码
    第12章-二叉搜索树 12.1 什么是二叉搜索树 12.2 查询二叉搜索树
    常用正则表达式
    备忘录:关于C#生成商品条码
    网络安全的主要威胁及应对方法
    【面试八股总结】MySQL索引(二):B+树数据结构、索引使用场景、索引优化、索引失效
    AndroidStudio离线打包配置汇总
    Springboot项目中Controller层的单元测试
  • 原文地址:https://blog.csdn.net/arm7star/article/details/125608910