• Camera-MTK OpenCamera时序以及耗时

    OpenCamera时序分析

    1.应用层

    08:53:56.087 应用层调用openCamera接口

    2.CameraService层

    08:53:56.099 10ms左右CameraService(进程号:742 线程号:25763)进行相关打印，从应用层到CameraServer之间经过FrameWork层(同一进程)以及ICameraService与CameraService之间的跨进程AIDL通信。此时CameraManagera(进程号:29471 线程号:29508)的相关部分处于阻塞状态，等待CameraService处理完成。

    3.CameraProvider(Camera HAL Interface-CameraProviderImpl SP端)

    08:53:56.116 17ms左右CameraProvider的device部分进行相关的log打印，从CameraService(CameraProviderManager)到CameraProvider(ICameraProviderImpl)之间有着跨进程的HIDL通信，此时CameraProviderManager(进程号:742 线程号:25763)处于阻塞状态，等待ICameraProviderImpl相关处理完成。

     

    4.Camera HAL Interface层

    08:53:56.151 耗时35ms Camera HAL Interface(很多人把这个层级称为CameraProvider层，可能因为它跟CameraServer是通过ICameraProviderImpl进行交互的)(进程号:773 线程号:5314)完成相关操作，之前被阻塞的CameraServer的CameraProviderManager拿到相应的信息，开始执行，但是此时并没有openCamera，才刚刚powerOn拉高camera。

    而且Camera HAL Interface相关的时序以及逻辑是在手机启动之后就已经起来的，与相机有没有打开没有关系，且里面会分成Provider、Commmon、Device、Session等好多模块对相机的open、preview、capture动作进行传递。

    5.Camera HAL层

    08:53:56.151 开始执行Camera HAL层(进程号:773 线程号:29521)相关的时序及逻辑

    08:53:56.220 直到应用层/FrameWork层CameraManagera、ICameraService相关的进程结束阻塞，Camera HAL里面还有很多在执行。

     

    6.Log时序总结

    openCamera是从08:53:56.087-08:53:56.220结束

    CameraProviderManager - ICameraProviderImpl HIDL跨进程通信耗时35ms

    ICameraService - CameraService AIDL跨进程通信中主要的阻塞、等待或者消费方式以及接口需要更进一步研究与明确耗费时间。

    OpenCamera调用接口分析

    1.应用层

     

     

     

    2.Camera FrameWork

    可以在这边打Log，相关进程会hang在这边connectDevice的调用下

    3.CameraService 

    CameraSevice里面与openCamera相关的重要节点都在connectHelper里面了，比如：

    handleEvictionsLocked：这里面会处理所有的camera id冲突场景，且会给不同调用camera id的packages进行打分操作。

     下面的ClientDescriptor相关的还没搞懂。

     

    makeClient：Binder的Bn端，会初始化CameraDeviceClient、Camera2ClientBase、CameraDeviceClientBase以及调用其initialize方法。

     

     

     这边InitializeImpl主要最主要的是run了FrameProcess这个线程，这个线程就在后面的过程中用来接收预览帧数据。

     这边关键节点是startCameraOps、然后又调用了CameraDeviceBase的initialize方法

     可以看到这边定义了很多纯虚函数，需要找到其具体的实现接口

    相关方法的接口具体实现都在Camera3Device里面

     CameraProviderManager里面的deviceInfo3在刚开机时候CameraServer与CameraProvider之间的交互过程中已经获取了。

     

    4.Camera HardWare Interface(CameraProvider)

    这边其实和展锐平台相关的HardWare Interface相关节点都是一样的

    主要就是几个关键的结构体的传递，建立了Camera HardWare Interface和Camera HAL之间的联系。

    1.hw_device_t里面有hw_module_t，可以通过hw_module_t去调用open方法。

    2.hw_module_t这边在里面封装了hw_module_methods_t的函数指针，指向了open的实例化函数接口。

    3.camera3_device_t在MTK camera HAL架构中貌似没有用到，在展讯的平台架构中使用场景还是挺多的。

     

     4.camera_module_t 这个结构体算是链接Camera HardWare Interface和Camera HAL层最为紧密的纽带了。

     5.Camera HAL

    MTK HAL层有三个entry接口，具体咋么实现什么情况对应哪个需要再去研究。

    vendor/mediatek/proprietary/hardware/mtkcam3/main/mtkhal/hidl/depend/entry.cpp

    可以看到对Camera HardWare Interface里面camera_module_t的所有接口说明都在这边进行了实例化。接下来对一些比较重要的接口进行详细说明和解释

    1. extern "C" NSCam::legacy_dev3::LegacyCameraModule* getLegacyCameraModule() {
    2.   static NSCam::CameraDeviceManagerImpl singleton("legacy");
    3.   static bool init = singleton.initialize();
    4.   if (!init) {
    5.     MY_LOGE("CameraDeviceManagerImpl::initialize fail %p", &singleton);
    6.     return nullptr;
    7.   }
    8.   static NSCam::legacy_dev3::LegacyCameraModule* module =
    9.       createLegacyCameraModule(&singleton);
    10.   if (module == nullptr) {
    11.     MY_LOGE("LegacyCameraModule is null: %p", module);
    12.     return nullptr;
    13.   }
    14.   return module;
    15. }
    16.  
    17.  
    18. //  Implementation of hw_module_methods_t
    19.  
    20. static int open_device(hw_module_t const* module,
    21.                        const char* name,
    22.                        hw_device_t** device) {
    23.   // return  getLegacyCameraModule()->open(device, module, name);
    24. #warning "set device_version to 0 to avoid build error"
    25.   // device_version = 0 would query info every time
    26.   return getLegacyCameraModule()->open(device, module, name, 0);
    27. }
    28.  
    29. static hw_module_methods_t* get_module_methods() {
    30.   static hw_module_methods_t _methods = {.open = open_device};
    31.   return &_methods;
    32. }

    vendor/mediatek/proprietary/hardware /mtkcam3/main/mtkhal/devicemgr/depend/CameraDeviceManagerImpl.cpp

    1. class CameraDeviceManagerImpl : public CameraDeviceManagerBase {
    2.   //++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    3.   //  Implementations.
    4.   //++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    5.  public:      Instantiation.
    6.   explicit CameraDeviceManagerImpl(char const* type);
    7.   virtual ~CameraDeviceManagerImpl();
    8.  
    9.  protected:      Operations.
    10.   virtual auto onEnumerateDevicesLocked() -> ::android::status_t;
    11.  
    12.   virtual auto onGetMaxNumOfMultiOpenCameras() const -> uint32_t;
    13.  
    14.   virtual auto onValidateOpenLocked(
    15.       const std::shared_ptr& pVirtualDevice) const
    16.       -> ::android::status_t;
    17.  
    18.   virtual auto onAttachOpenDeviceLocked(
    19.       const std::shared_ptr& pVirtualDevice) -> void;
    20.  
    21.   virtual auto onDetachOpenDeviceLocked(
    22.       const std::shared_ptr& pVirtualDevice) -> void;
    23.  
    24.   virtual auto onEnableTorchLocked(const std::string& deviceName, bool enable)
    25.       -> ::android::status_t;
    26. };

    vendor/mediatek/proprietary/hardware/mtkcam3/main/mtkhal/devicemgr/base/CameraDeviceManagerBase.cpp

    vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/main/mtkhal/devicemgr/ICameraDeviceManager.h

    1. class ICameraDeviceManager {
    2.   //++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    3.   //  Definitions.
    4.   //++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    5.  public:  
    6.   class IVirtualDevice : virtual public ::android::RefBase {
    7.    public:
    8.     /**
    9.      * The device instance names must be of the form
    10.      *      "device@.//" where
    11.      * / is the HIDL version of the interface.
    12.      *  is a small incrementing integer for "internal" device types,
    13.      * with 0 being the main back-facing camera and 1 being the main
    14.      * front-facing camera, if they exist.
    15.      */
    16.  
    17.     struct Info {
    18.       std::string mInstanceName;   //  instance device:
    19.                                    //  "device@.//"
    20.       int32_t mInstanceId;         //  instance id
    21.       int32_t mVirtualInstanceId;  //  virtual instance id
    22.       uint32_t mMajorVersion;      //  major version
    23.       uint32_t mMinorVersion;      //  minor version
    24.       bool mHasFlashUnit;          //  has flash unit
    25.       bool mIsHidden;              //  Is camera hidden
    26.       int32_t mFacing;             //  facing
    27. //      mcam::DeviceType mType;    //  NormalDevice/PostProcDevice/...
    28.     };
    29.  
    30.    public:
    31.     virtual auto getDeviceInterface(std::shared_ptr& rpDevice)
    32.         const -> ::android::status_t = 0;
    33.     virtual auto getDeviceInfo() const -> Info const& = 0;
    34.     virtual auto getInstanceName() const -> char const* {
    35.       return getDeviceInfo().mInstanceName.c_str();
    36.     }
    37.  
    38.     ...
    39.  
    40. }

    vendor/mediatek/proprietary/hardware/mtkcam3/main/hal/device/3.x/device/CameraDevice3Impl.h

            ICameraDeviceManager.h里面并没有相关接口的实例化，真正的接口实例化是在CameraDevice3Impl.cpp里面实现的。

    1. class CameraDevice3Impl
    2.     : public ICameraDeviceManager::IVirtualDevice
    3. {
    4. //++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    5. //  Definitions.
    6. //++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    7. // public:     
    8.  
    9. //     struct  MyDebuggee : public IDebuggee
    10. //     {
    11. //         static const std::string        mName;
    12. //         std::shared_ptrmCookie = nullptr;
    13. //         android::wp  mContext = nullptr;
    14.  
    15. //                         MyDebuggee(CameraDevice3Impl* p) : mContext(p) {}
    16. //         virtual auto    debuggeeName() const -> std::string { return mName; }
    17. //         virtual auto    debug(
    18. //                             android::Printer& printer,
    19. //                             const std::vector& options
    20. //                         ) -> void;
    21. //     };
    22.  
    23. //++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    24. //  Data Members.
    25. //++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    26. protected:  
    27.  
    28.     //  setup during constructor
    29.     int32_t                                 mLogLevel = 0;                  //  log level.
    30.     ICameraDeviceManager*                   mDeviceManager = nullptr;       //  device manager.
    31.     std::shared_ptr                   mStaticDeviceInfo = nullptr;    //  device info
    32.     // std::shared_ptr             mDebuggee = nullptr;
    33.     ::android::sp        mMetadataProvider = nullptr;
    34.     ::android::sp       mMetadataConverter = nullptr;
    35.     ::android::sp    mSession = nullptr;
    36.     std::map> mPhysicalMetadataProviders;
    37.     mutable ::android::Mutex                mGetResourceLock;
    38.  
    39. //++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    40. //  Operations.
    41. //++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    42. public:         Instantiation.
    43.     virtual         ~CameraDevice3Impl();
    44.                     CameraDevice3Impl(
    45.                         ICameraDeviceManager* deviceManager,
    46.                         IMetadataProvider* metadataProvider,
    47.                         std::map>const& physicalMetadataProviders,
    48.                         char const* deviceType,
    49.                         int32_t instanceId,
    50.                         int32_t virtualInstanceId);
    51.  
    52.     virtual auto    initialize(
    53.                         ICameraDevice3Session* session
    54.                         ) -> bool;
    55.  
    56. public:         Operations.
    57.     auto            getLogLevel() const             { return mLogLevel; }
    58.     auto const&     getStaticDeviceInfo() const     { return mStaticDeviceInfo; }
    59.     auto const&     getMetadataConverter() const    { return mMetadataConverter; }
    60.  
    61. //++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    62. //  Interfaces.
    63. //++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    64. public:         ICameraDeviceManager::IVirtualDevice Interfaces.
    65.  
    66.     virtual auto    getDeviceInterface(
    67.                         ::android::sp& rpDevice
    68.                         ) const -> ::android::status_t override;
    69.  
    70.     virtual auto    getDeviceInfo() const -> Info const& override;
    71.  
    72. public:         ICameraDevice Interfaces implement.
    73.  
    74.     virtual auto    getResourceCost(v3::CameraResourceCost& mtkCost) -> int;
    75.  
    76.     virtual auto    setTorchMode(v3::TorchMode mode) -> ::android::status_t;
    77.  
    78.     virtual auto    open() -> ::android::sp;
    79.  
    80.     virtual auto    dumpState(android::Printer& printer, const std::vector& options) -> void;
    81.  
    82.     virtual auto    getCameraCharacteristics() -> const camera_metadata_t*;
    83.  
    84.     virtual auto    getPhysicalCameraCharacteristics(
    85.                         int physicalId,
    86.                         camera_metadata* physicalMetadata) -> int;
    87.  
    88.     virtual auto    isStreamCombinationSupported(
    89.                         const v3::StreamConfiguration& streams,
    90.                         bool& isSupported) -> int;
    91.  
    92. public:         Operations
    93.  
    94.     virtual auto    getDeviceSession() -> const ::android::sp;
    95.  
    96. };

    到这边HAL openCamera相关的动作只算走了一半，仅仅是entry.cpp里面实例化CameraDeviceManagerImpl实例化的过程。

    这边已经开始对sensor进行了相关的init操作

    真正的camera device open操作会在LegacyCameraModule里面，上面的CameraDevice3Impl也有相关的open操作，主要是CameraDeviceSession的调用。

    vendor/mediatek/proprietary/hardware/mtkcam3/main/hal/entry/legacy/device/LegacyCameraDevice3.cpp

    vendor/mediatek/proprietary/hardware/mtkcam3/main/hal/device/3.x/device/CameraDevice3Impl.cpp

    相关的mSession的实例化的操作在initialize这边，调用的是CameraDevice3Session的open操作

     接下来要追踪的便是CameraDeviceImpl.initial里面的session是谁传过来的，是哪边创建的、创建的什么tag。

    vendor/mediatek/proprietary/hardware/mtkcam3/main/hal/device/3.x/device/CameraDevice3Factory.cpp

    1. extern "C"
    2. NSCam::ICameraDeviceManager::IVirtualDevice*
    3. createVirtualCameraDevice(CreateVirtualCameraDeviceParams* params)
    4. {
    5.     MY_LOGD("[hidldc] new createVirtualCameraDevice");
    6.     if ( ! params || ! params->pDeviceManager || ! params->pMetadataProvider ) {
    7.         MY_LOGE("Bad params");
    8.         return nullptr;
    9.     }
    10.  
    11.     auto pDevice = new CameraDevice3Impl(
    12.         params->pDeviceManager,
    13.         params->pMetadataProvider,
    14.         params->physicalMetadataProviders,
    15.         params->deviceType,
    16.         params->instanceId,
    17.         params->virtualInstanceId
    18.     );
    19.  
    20.     if ( ! pDevice ) {
    21.         MY_LOGE("Fail to new CameraDevice3Impl");
    22.         return nullptr;
    23.     }
    24.  
    25.     NSCam::ICameraDevice3Session::CreationInfo const info = {
    26.         .mDeviceManager     = params->pDeviceManager,
    27.         .mStaticDeviceInfo  = pDevice->getStaticDeviceInfo(),
    28.         .mMetadataProvider  = params->pMetadataProvider,
    29.         .mMetadataConverter = pDevice->getMetadataConverter(),
    30.         .mPhysicalMetadataProviders = params->physicalMetadataProviders
    31.     };
    32.  
    33.     bool bInitSuccess = pDevice->initialize(createCameraDevice3Session(info));
    34.     if ( ! bInitSuccess ) {
    35.         delete pDevice;
    36.         pDevice = nullptr;
    37.     }
    38.  
    39.     return pDevice;
    40. }

    createCameraDevice3Session的接口的实现是在CameraDevice3SessionFactory.cpp里面实现的

    1. extern "C"
    2. NSCam::ICameraDevice3Session*
    3. createCameraDevice3Session(
    4.     NSCam::ICameraDevice3Session::CreationInfo const& info
    5. )
    6. {
    7.     //  Based on the information of instanceId + pMetadataProvider,
    8.     //  decide which device session to create.
    9.     return new NSCam::v3::CameraDevice3SessionImpl(info);
    10. }

    然后的主要实现就在CameraDevice3SessionImpl

    vendor/mediatek/proprietary/hardware/mtkcam3/main/hal/device/3.x/device/CameraDevice3SessionImpl.cpp

             这边不在做仔细的剖析了，里面涉及很多接口的调用，直接贴上最关键的节点PipelineMode->open，这样就与MTK 的pipeline mode进行了对接。

            其实在实际解决问题时候，很多人不会去check Camera HAL的前期框架，关注点更会放在MTK Pipeline的框架上，整个流程及框架肯定是越通透对以后的性能、时序分析更加有帮助，能多研究点就多看点呗。

    总结

            MTK Camera HAL性能到这边远没有结束，总的MTK Camera HAL层openCamera架构可以简单的以线程号分成三个大的部分，现在仅仅是对PipelineMode之前的部分进行了流程调用分析，其中的耗时以及等待机制还需要打更多的log去追，以后的优化更不是简简单单能搞定的。

  • 相关阅读:
    删除链表的倒数第n个节点的最优算法实现
    Java学习笔记38——网络编程02
    白杨SEO：2024年短视频怎么做？转型做抖音、快手、视频号等短视频流量难吗？怎么做更好？
    SCS【2】单细胞转录组 之 cellranger
    linux命令：计算的相关命令（expr、let、bc、(())、 $[]）
    编译clashandroid
    猿创征文｜docker 系列： 具名、匿名挂载介绍
    2019 Java面试题
    【学习笔记60】JavaScript原型链的理解
    Conda常用命令和操作
  • 原文地址：https://blog.csdn.net/Cmatrix204/article/details/126526606