• camera预览流程 --- 从HAL到OEM



    我们在前面的文章中主要是单独介绍hal和oem下的某些关键c/cpp文件的内容,比较分散。因为整个预览流程涉及的内容太多,我们不好在一篇文章中讲清楚,只能先把单个的知识点一一说明,然后才能将整个预览流程串起来。
    注:大家阅读本篇前,请先将我之前介绍的 Hal3_2v6 和 OEM 的内容过下。

    下发预览请求

    我在前一篇文章中介绍了SprdCamera3HWI.cpp ,它的三个关键流程中有一个 processCaptureRequest,即发送预览请求的,我们就从这里开始。

    1,SprdCamera3HWI:channel->request

        for (i = 0; i < request->num_output_buffers; i++) {
            const camera3_stream_buffer_t &output = request->output_buffers[i];
            camera3_stream_t *stream = output.stream;
            SprdCamera3Channel *channel = (SprdCamera3Channel *)stream->priv;
            ret = channel->request(stream, output.buffer, frameNumber);
        }
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6

    2,SprdCamera3RegularChannel::request

    我们知道,channel有两类:预览和拍照。分别对应 SprdCamera3RegularChannel 和 SprdCamera3PicChannel (还有一个负责处理metadata的 SprdCamera3MetadataChannel)。
    我们本篇分析预览的流程,那channel走的就是 SprdCamera3RegularChannel,其request的内容不多,我们就全贴出来了。

    int SprdCamera3RegularChannel::request(camera3_stream_t *stream,
                                           buffer_handle_t *buffer,
                                           uint32_t frameNumber) {
        HAL_LOGD("SprdCamera3RegularChannel request frameNumber:%d", frameNumber);
        int ret = NO_ERROR;
        int i;
        char multicameramode[PROPERTY_VALUE_MAX];
    
        for (i = 0; i < CHANNEL_MAX_STREAM_NUM; i++) {
            if (mCamera3Stream[i]) {
                camera3_stream_t *new_stream;
    
                mCamera3Stream[i]->getStreamInfo(&new_stream);
                if (new_stream == stream) {
                    ret = mCamera3Stream[i]->buffDoneQ(frameNumber, buffer);
                    if (ret != NO_ERROR) {
                        return ret;
                    }
                    if (i == 0) {
                        mOEMIf->queueBuffer(buffer, CAMERA_STREAM_TYPE_PREVIEW);
                    } else if (i == (CAMERA_STREAM_TYPE_VIDEO -
                                     REGULAR_STREAM_TYPE_BASE)) {
                       if (mOEMIf->isVideoCopyFromPreview()) {
                           HAL_LOGD("video stream copy preview stream");
                       } else {
                          mOEMIf->queueBuffer(buffer, CAMERA_STREAM_TYPE_VIDEO);
                       }
                    } else if (i == (CAMERA_STREAM_TYPE_CALLBACK -
                                     REGULAR_STREAM_TYPE_BASE))
                        mOEMIf->queueBuffer(buffer, CAMERA_STREAM_TYPE_CALLBACK);
                    else if (i ==
                             (CAMERA_STREAM_TYPE_YUV2 - REGULAR_STREAM_TYPE_BASE))
                        mOEMIf->queueBuffer(buffer, CAMERA_STREAM_TYPE_YUV2);
                    break;
                }
            }
        }
        return ret;
    }
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39

    channel的request流程主要分为如下两个步骤:

    1. SprdCamera3Stream::buffDoneQ
    2. SprdCamera3OEMIf::queueBuffer

    3,SprdCamera3Stream::buffDoneQ

    cam3Stream的buffDoneQ主要是调用 SprdCamera3GrallocMemory 的map,将入参 buffer_handle_t *buffer_handle 传入GraphicBufferMapper 的 lock 中,获取虚拟地址 addr_vir。

    //SprdCamera3Stream
    int SprdCamera3Stream::buffDoneQ(uint32_t frameNumber,buffer_handle_t *buffer) {}    
    hal_buff_list_t *buff_hal = new hal_buff_list_t;
    hal_mem_info_t *buf_mem_info = &(buff_hal->mem_info);
    mMemory->map(buffer, buf_mem_info);
    
    //SprdCamera3GrallocMemory 
    int SprdCamera3GrallocMemory::map(buffer_handle_t *buffer_handle,hal_mem_info_t *mem_info){}
    GraphicBufferMapper &mapper = GraphicBufferMapper::get();
    ret = mapper.lock((const native_handle_t *)*buffer_handle, usage,bounds, &vaddr);
    mem_info->addr_vir = vaddr;
    
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12

    通过SprdCamera3GrallocMemory 的 map之后,拿到了 addr_vir 保存在 cam3Stream的buff_hal下的hal_mem_info_t 中,然后

    mBufferList.add(buff_hal);
    
    • 1

    我们看到cam3Stream中有明显对称的函数:

    • buffDoneQ
    • buffDoneQ2
    • buffDoneDQ
    • buffFirstDoneDQ
      他们分别调用SprdCamera3GrallocMemory 中的map 与 unmap,即 GraphicBufferMapper 的 lock与 unlock。
    SprdCamera3StreamSprdCamera3GrallocMemorySprdCamera3Stream
    buffDoneQ、buffDoneQ2mapmBufferList.add
    buffDoneDQ、buffDoneDQunmapmBufferList.erase

    SprdCamera3Stream中的mBufferList在map时向集合中add、unmap时向集合中erase的数据。
    总之,SprdCamera3Stream::buffDoneQ 的主要目的是获取入参 buffer_handle_t *buffer 的 addr_vir 。

    4,SprdCamera3OEMIf::queueBuffer

    SprdCamera3OEMIf 的 queueBuffer中关于preivew的这条case 代码不多,我们也直接贴出来。也分两步

    • stream->getQBufInfoForHandle
    • mHalOem->ops->queue_buffer
        case CAMERA_STREAM_TYPE_PREVIEW:
            channel = reinterpret_cast<SprdCamera3RegularChannel *>(mRegularChan);
            if (channel == NULL) {
                ret = -1;
                HAL_LOGE("mRegularChan is null");
                goto exit;
            }
    
            ret = channel->getStream(CAMERA_STREAM_TYPE_PREVIEW, &stream);
            if (ret || stream == NULL) {
                HAL_LOGE("getStream failed");
                goto exit;
            }
    
            ret = stream->getQBufInfoForHandle(buff_handle, &buffer);
            if (ret || buffer.addr_vir == NULL) {
                HAL_LOGE("getQBufForHandle failed");
                goto exit;
            }
            mHalOem->ops->queue_buffer(mCameraHandle, buffer,
                                       SPRD_CAM_STREAM_PREVIEW);
            break;
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22

    5,SprdCamera3Stream::getQBufInfoForHandle

    我们在步骤3中介绍了SprdCamera3Stream管理了一个集合 mBufferList,这里就是在该集合中根据入参 buffer_handle_t *buff来获取信息,并将信息封装到 cam_buffer_info_t 对象中,其中最重要的就是 addr_vir。

    int SprdCamera3Stream::getQBufInfoForHandle(buffer_handle_t *buff,
                                                cam_buffer_info_t *bufInfo) {
        Mutex::Autolock l(mLock);
        int ret = NO_ERROR;
        Vector<hal_buff_list_t *>::iterator iter;
    
        for (iter = mBufferList.begin(); iter != mBufferList.end(); iter++) {
            if ((*iter) && (*iter)->buffer_handle == buff) {
                bufInfo->fd = (*iter)->mem_info.fd;
                bufInfo->size = (*iter)->mem_info.size;
                bufInfo->addr_phy = (*iter)->mem_info.addr_phy;
                bufInfo->addr_vir = (*iter)->mem_info.addr_vir;
                bufInfo->width = (*iter)->mem_info.width;
                bufInfo->height = (*iter)->mem_info.height;
                bufInfo->format = (*iter)->mem_info.format;
                bufInfo->frame_number = (*iter)->frame_number;
                return ret;
            }
        }
    
        return BAD_VALUE;
    }
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22

    6,mHalOem->ops->queue_buffer

    我们在SprdCamera3OEMIf中看到很多 mHalOem->ops 调用形式。我们来看下 mHalOem 的实例化,在SprdCamera3OEMIf的构造函数中:

    if (!mHalOem) {
        oem_module_t *omi;
        mHalOem = (oem_module_t *)malloc(sizeof(oem_module_t));
        if (NULL == mHalOem) {
            HAL_LOGE("mHalOem is NULL");
        } else {
            memset(mHalOem, 0, sizeof(*mHalOem));
    
            mHalOem->dso = dlopen(OEM_LIBRARY_PATH, RTLD_NOW);
            if (NULL == mHalOem->dso) {
                char const *err_str = dlerror();
                HAL_LOGE("dlopen error%s ", err_str ? err_str : "unknown");
            }
    
            const char *sym = OEM_MODULE_INFO_SYM_AS_STR;//OMI
            omi = (oem_module_t *)dlsym(mHalOem->dso, sym);
            if (omi) {
                mHalOem->ops = omi->ops;
            }
            HAL_LOGI("loaded libcamoem.so mHalOem->dso = %p", mHalOem->dso);
        }
    }
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    #define OEM_LIBRARY_PATH "libcamoem.so"
    
    • 1

    我们看到是open了一个 OEM_LIBRARY_PATH,即oem的库。在SprdOEMCamera中找到OEM_MODULE_INFO_SYM 的定义:

    struct oem_module OEM_MODULE_INFO_SYM = {
        .tag = 0, .ops = &oem_module_ops, .dso = NULL};
    
    • 1
    • 2

    其中 ops = &oem_module_ops,即SprdOEMCamera中定义的所有操作:

    static oem_ops_t oem_module_ops = {
        camera_init, camera_deinit, camera_release_frame, camera_set_param,
        camera_start_preview, camera_stop_preview, camera_start_autofocus,
        camera_cancel_autofocus, camera_cancel_takepicture,
        // camera_safe_scale_th,
        NULL, camera_take_picture, camera_get_sn_trim, camera_set_mem_func,
        camera_get_redisplay_data, camera_is_change_size,
        NULL, camera_get_preview_rect,
        camera_get_zsl_capability, camera_get_sensor_info_for_raw,
        camera_get_sensor_trim, camera_get_sensor_trim2,
        camera_get_preview_rot_angle, camera_fd_enable, camera_flip_enable,
        camera_fd_start, camera_is_need_stop_preview, camera_takepicture_process,
        camera_get_size_align_page, camera_fast_ctrl, camera_start_preflash,
        camera_get_viewangle, camera_get_sensor_exif_info,
        camera_get_sensor_result_exif_info, camera_get_iommu_status,
        camera_set_preview_buffer, camera_set_video_buffer, camera_set_zsl_buffer,
        queue_buffer, camera_set_video_snapshot_buffer,
        camera_set_zsl_snapshot_buffer, camera_zsl_snapshot_need_pause,
        camera_get_isp_handle, camera_lls_enable, camera_is_lls_enabled,
        camera_vendor_hdr_enable, camera_is_vendor_hdr, camera_set_lls_shot_mode,
        camera_get_lls_shot_mode, camera_get_last_preflash_time,camera_get_isp_info, camera_start_burst_notice,
        camera_end_burst_notice, NULL, NULL, dump_jpeg_file, camera_get_gain_thrs,
        camera_set_sensor_info_to_af, camera_get_sensor_max_fps,
        camera_snapshot_is_need_flash, camera_get_sensor_otp_info,
        camera_get_sensor_vcm_step, camera_set_sensor_close_flag,
        camera_set_reprocess_picture_size, camera_start_capture,
        camera_stop_capture, camera_set_largest_picture_size, camera_set_alloc_picture_size,
        camera_ioctrl,
        camera_reprocess_yuv_for_jpeg, image_sw_algorithm_processing,
        dump_image_with_isp_info,
    #if defined(CONFIG_ISP_2_1)
        camera_get_focus_point, camera_isp_sw_check_buf, camera_isp_sw_proc,
        camera_raw_post_proc, camera_get_tuning_param,
    #endif
    #if defined(CONFIG_ISP_2_3) || defined(CONFIG_ISP_2_4) ||                      \
        defined(CONFIG_CAMERA_3DNR_CAPTURE_SW) ||                                  \
        defined(CONFIG_CAMERA_SUPPORT_ULTRA_WIDE)
        camera_set_gpu_mem_ops,
    #endif
    #ifdef CONFIG_CAMERA_MM_DVFS_SUPPORT
        camera_set_mm_dvfs_policy,
    #endif
        camera_set_original_picture_size,
    };
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44

    那么SprdCamera3OEMIf中调用的queue_buffer 也就走到了SprdOEMCamera中了。

    7,SprdOEMCamera::queue_buffer

    cmr_s32 queue_buffer(cmr_handle camera_handle, cam_buffer_info_t buffer,
                         int steam_type) {
        cmr_int ret = CMR_CAMERA_SUCCESS;
        ret = local_queue_buffer(camera_handle, buffer, steam_type);
        return ret;
    }
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6

    8,cmr_oem::local_queue_buffer

    cmr_oem中的实现,我们只截取preview相关的这一条:

    cmr_s32 local_queue_buffer(cmr_handle oem_handle, cam_buffer_info_t buffer,
                               int steam_type) {
        cmr_int ret = CMR_CAMERA_SUCCESS;
        struct camera_context *cxt;
        struct sensor_exp_info exp_info;
        if (!oem_handle) {
            CMR_LOGE("in parm error");
            ret = -CMR_CAMERA_INVALID_PARAM;
            goto exit;
        }
        cxt = (struct camera_context *)oem_handle;
    
        switch (steam_type) {
        case SPRD_CAM_STREAM_PREVIEW:
            ret = cmr_preview_set_preview_buffer(cxt->prev_cxt.preview_handle,
                                                 cxt->camera_id, buffer);
            if (ret) {
                CMR_LOGE("cmr_preview_set_preview_buffer failed");
                goto exit;
            }
            break;
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21

    9,cmr_preview::cmr_preview_set_preview_buffer

    cmr_preview的流程我们在之前的文章中也介绍了,我们看到 cmr_preview_set_preview_buffer 就是发送了一个message,最终的处理流程是在 prev_set_preview_buffer

    10,cmr_preview::prev_set_preview_buffer

    prev_set_preview_buffer的流程主要包括两方面:

    1. 是将入参 cam_buffer_info_t *buffer 的数据放到如下:
    • prev_cxt->prev_fd_array[valid_num]
    • prev_cxt->prev_phys_addr_array[valid_num]
    • prev_cxt->prev_virt_addr_array[valid_num]
    • prev_cxt->prev_frm[valid_num]
    1. 调用 handle->ops.channel_buff_cfg
      channel_buff_cfg 是在cmr_preview.h中定义的结构体 struct preview_md_ops 下的指针函数,我们看不到具体实现,但是可以猜测 channel_buff_cfg 就是要将我们在 SprdCamera3Mem中map得到的 addr_vir 地址给到更底层的DCAM,用来装下一帧的预览数据的。

    到这里,我们从SprdCamera3HW中的 channel->request 追过来的流程就结束了。总结下,就是要将map到的 addr_vir 设到底层去装预览数据。

    装数据的地址下去了,那么接下来就要去看数据上来的处理流程了, 下面我们再从帧数据上来的地方开始追。

    处理预览帧数据

    我们之前介绍cmr_grab,说它就是hal端能拿到底层帧数据最开始的地方,我们就从cmr_grab中开始。

    1,cmr_grab::cmr_grab_thread_proc

    我们在介绍cmr_grab中说过,他拿到的帧数据就是通过grab_evt_cb向上回调的。

    (*p_grab->grab_evt_cb)(evt_id, &frame,(void *)p_grab->init_param.oem_handle);
    
    • 1

    该回调就为cmr_oem中的 camera_grab_evt_cb

    2,cmr_oem::camera_grab_evt_cb

    我们截取 cmr_grab_evt_reg 的关键内容看下,又调到了 camera_send_channel_data

    cmr_grab_evt_reg(grab_handle, camera_grab_evt_cb);
    
    void camera_grab_evt_cb(cmr_int evt, void *data, void *privdata) {
        switch (evt) {
        case CMR_GRAB_TX_DONE:
            if (frame->is_4in1_frame && frame->fmt != CAM_IMG_FMT_BAYER_MIPI_RAW) {
                camera_4in1_handle(evt, data, privdata);
            }
            camera_send_channel_data((cmr_handle)cxt, receiver_handle, evt, data);
    
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10

    3,cmr_oem::camera_send_channel_data

    if (cxt->prev_cxt.preview_eb && cxt->prev_cxt.preview_channel_id == frm_ptr->channel_id) {
        ret = cmr_preview_receive_data(cxt->prev_cxt.preview_handle,cxt->camera_id, evt, data);
    }
    
    • 1
    • 2
    • 3

    流程又转到 cmr_preview中去了

    4,cmr_preview::cmr_preview_receive_data

    cmr_preivew.c中以cmr_ 开头的函数基本上都是发送一个message,本身没有什么逻辑。其message的处理是prev_receive_data 函数,进一步又会调用 prev_frame_handle。
    cmr_preview_receive_data ==》 prev_receive_data ==》 prev_frame_handle ==》prev_preview_frame_handle

    5,cmr_preview::prev_preview_frame_handle

    prev_preview_frame_handle的主要逻辑分为三步:

    • prev_construct_frame
    • prev_pop_preview_buffer
    • 数据回调

    6,cmr_preview::prev_construct_frame

    prev_construct_frame 主要是根据入参struct frm_info *info找到 prev_cxt->prev_frm[]中对应的frm_id,由此可以得到对应的 prev_cxt->prev_frm[frm_id]数据,即本次上来的预览帧数据保存的地址信息。该数据用作两处:

    • 填充到frame_type
    • 发送到预览跑的算法中:包括 fd、3dnr、ai 等等

    7,cmr_preview::prev_pop_preview_buffer

    prev_pop_preview_buffer 主要是整理 prev_cxt->prev_phys_addr_array[] 和 prev_cxt->prev_frm[]的内容,使数据前移一位,覆盖位置0处的数据。

    for (i = 0; i < (cmr_u32)(valid_num - 1); i++) {
        prev_cxt->prev_phys_addr_array[i] = prev_cxt->prev_phys_addr_array[i + 1];
        prev_cxt->prev_virt_addr_array[i] = prev_cxt->prev_virt_addr_array[i + 1];
        prev_cxt->prev_fd_array[i] = prev_cxt->prev_fd_array[i + 1];
        memcpy(&prev_cxt->prev_frm[i], &prev_cxt->prev_frm[i + 1],sizeof(struct img_frm));
    }
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6

    8,回调 步骤6 中填充的数据

     cb_data_info.cb_type = PREVIEW_EVT_CB_FRAME;
     cb_data_info.func_type = PREVIEW_FUNC_START_PREVIEW;
     cb_data_info.frame_data = &frame_type;
     prev_cb_start(handle, &cb_data_info);
    
    • 1
    • 2
    • 3
    • 4

    prev_cb_start 的处理:

    case PREV_EVT_CB_START:
    cb_data_info = (struct prev_cb_info *)message->data;
    
    if (!handle->oem_cb) {
        CMR_LOGE("oem_cb is null");
        break;
    }
    
    ret = handle->oem_cb(handle->oem_handle, cb_data_info->cb_type,
                         cb_data_info->func_type, cb_data_info->frame_data);
    
    if (cb_data_info->frame_data) {
        free(cb_data_info->frame_data);
        cb_data_info->frame_data = NULL;
    }
    break;
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16

    这里继续调到 cmr_oem中处理

    9,cmr_oem处理回调

    init_param.oem_cb = camera_preview_cb;
    
    • 1

    camera_preview_cb 会对 cb_type 修改

    	if (PREVIEW_FUNC_START_PREVIEW == func) {
            oem_func = CAMERA_FUNC_START_PREVIEW;
        } else if (PREVIEW_FUNC_STOP_PREVIEW == func) {
            oem_func = CAMERA_FUNC_STOP_PREVIEW;
        } else if (PREVIEW_FUNC_START_CAPTURE == func) {
            oem_func = CAMERA_FUNC_TAKE_PICTURE;
        } else {
            CMR_LOGE("err, %d", func);
            goto exit;
        }
    
        case PREVIEW_EVT_CB_FRAME:
            oem_cb_type = CAMERA_EVT_CB_FRAME;
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13

    PREVIEW_FUNC_START_PREVIEW === 》CAMERA_FUNC_START_PREVIEW
    PREVIEW_EVT_CB_FRAME ===》CAMERA_EVT_CB_FRAME

    10,SprdCamera3OEMIf::camera_cb

    case CAMERA_FUNC_START_PREVIEW:
        obj->HandleStartPreview(cb, parm4);
        break;
    
    // HandleStartPreview 中的处理:
    case CAMERA_EVT_CB_FRAME:
        HAL_LOGV("CAMERA_EVT_CB_FRAME");
        switch (getPreviewState()) {
        case SPRD_PREVIEW_IN_PROGRESS:
            receivePreviewFrame((struct camera_frame_type *)parm4);
            break;
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11

    11,SprdCamera3OEMIf::receivePreviewFrame

    关于 receivePreviewFrame 的逻辑,我们在之前介绍SprdCamera3OEMIf的文章中有讲过,cam3OEMIf收到预览帧的处理如下:

    PreviewFrameFaceBeauty(frame, &beautyLevels);
    PreviewFrameCamDebug(frame);
    /* video stream */
    PreviewFrameVideoStream(frame, buffer_timestamp);
    /* preview stream, original code has goto exit */
    ret = PreviewFramePreviewStream(frame, buffer_timestamp);
    /* callback stream */
    PreviewFrameCallbackStream(frame, buffer_timestamp);
    /* yuv2 stream */
    PreviewFrameYuv2Stream(frame, buffer_timestamp);
    /* zsl stream */
    PreviewFrameZslStream(frame, buffer_timestamp);
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12

    与预览流程相关的就是 PreviewFramePreviewStream 函数

    12,SprdCamera3OEMIf::PreviewFramePreviewStream

    PreviewFramePreviewStream 的关键逻辑是通过channel将帧数据callback上去

    channel->channelCbRoutine(frame_num, buffer_timestamp,CAMERA_STREAM_TYPE_PREVIEW);
    
    • 1

    13,SprdCamera3RegularChannel::channelCbRoutine (buffDoneDQ)

    channelCbRoutine的主要逻辑是构建一个 cam_result_data_info_t result_info ,其中当然是包含了帧数据,然后将 result_info 继续向上回调。

    cam_result_data_info_t result_info;
    result_info.is_urgent = false;
    result_info.buffer = buffer;
    result_info.frame_number = frame_number;
    result_info.stream = stream;
    result_info.timestamp = timestamp;
    result_info.buff_status = CAMERA3_BUFFER_STATUS_OK;
    result_info.msg_type = CAMERA3_MSG_SHUTTER;
    
    mChannelCB(&result_info, mUserData);
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10

    channelCbRoutine还有一个非常重要的操作:

    ret = mCamera3Stream[index]->buffDoneDQ(frame_number, &buffer);
    
    • 1

    还记得我们在 下发预览请求 中的第3步中调用了 buffDoneQ么。 这两者buffDoneDQ 和 buffDoneQ是相反的操作。

    • 在预览请求的时候通过 SprdCamera3GrallocMemory 的map 函数得到buffert的虚拟地址
    • 在拿到预览帧数据回调的时候通过SprdCamera3GrallocMemory 的unmap 函数释放掉虚拟地址

    到这里 处理预览帧数 的流程也完成了。我们也将预览请求和预览回调的整个从 hal 到oem 在到 hal 的流程串通了。本篇涉及的逻辑比较多,有些细节部分我们并没有展开,因为之前有对这些关键的c/cpp文件单独的介绍,如有不清楚的可去翻一翻之前的文章。

    log 跟踪

    最后,我们通过一份点击桌面的CameraApp来启动camera预览的log来看下两个关键数据

    • frameNumber
    • addr_vir

    顺序增长的 frameNumber

    在发送预览请求的HWI下的processCaptureRequest 函数中会有如下log

    HAL_LOGD("camId=%d, bufs_num=%d, frame_num=%d, cap_intent=%d, pic_req=%d, "
             "first_regular_req=%d",
             mCameraId, request->num_output_buffers, request->frame_number,
             captureIntent, mPictureRequest, mFirstRegularRequest);
    
    • 1
    • 2
    • 3
    • 4

    在SprdCamera3OEMIf接收帧数据函数 PreviewFramePreviewStream 中有如下根据帧数据y_vir_addr获取 frame_num的log

    cmr_uint buff_vir = (cmr_uint)(frame->y_vir_addr);
    ret = pre_stream->getQBufNumForVir(buff_vir, &frame_num);
    HAL_LOGD("mCameraId=%d, prev:fd=0x%x, vir=0x%lx, num=%d, width=%d, "
             "height=%d, time=0x%llx, iso_value:%d",
             mCameraId, (cmr_u32)frame->fd, buff_vir, frame_num, frame->width,
             frame->height, buffer_timestamp, ae_iso);
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6

    我们来看下log的打印:
    可以看到,基本上一帧请求,一帧回调,并且回调的 frame_num 一定小于 请求的 frame_num

    Line 219697: 08-09 11:34:44.687   532  1000 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=2, cap_intent=1, pic_req=0, first_regular_req=0
    Line 219737: 08-09 11:34:44.695   532  1000 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=3, cap_intent=1, pic_req=0, first_regular_req=0
    Line 219862: 08-09 11:34:44.991   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x16, vir=0xd92d5000, num=0, width=960, height=720, time=0x3c0ec14062d8, iso_value:2500
    Line 219915: 08-09 11:34:45.020   532   999 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=4, cap_intent=1, pic_req=0, first_regular_req=0
    Line 220022: 08-09 11:34:45.085   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x6a, vir=0xd91d7000, num=1, width=960, height=720, time=0x3c0ec7373dd8, iso_value:2500
    Line 220054: 08-09 11:34:45.116   532   999 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=5, cap_intent=1, pic_req=0, first_regular_req=0
    Line 220136: 08-09 11:34:45.189   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x6f, vir=0xd90d9000, num=2, width=960, height=720, time=0x3c0ecd2e43d0, iso_value:1000
    Line 220159: 08-09 11:34:45.219   532   999 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=6, cap_intent=1, pic_req=0, first_regular_req=0
    Line 220299: 08-09 11:34:45.290   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x74, vir=0xd8fdb000, num=3, width=960, height=720, time=0x3c0ed3243c40, iso_value:1000
    Line 220325: 08-09 11:34:45.320   532   999 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=7, cap_intent=1, pic_req=0, first_regular_req=0
    Line 220390: 08-09 11:34:45.330   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x68, vir=0xd8dce000, num=4, width=960, height=720, time=0x3c0ed58722e0, iso_value:1250
    Line 220428: 08-09 11:34:45.350   532   999 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=8, cap_intent=1, pic_req=0, first_regular_req=0
    Line 220498: 08-09 11:34:45.369   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x16, vir=0xd92d5000, num=5, width=960, height=720, time=0x3c0ed7ea28c0, iso_value:1250
    Line 220526: 08-09 11:34:45.401   532   999 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=9, cap_intent=1, pic_req=0, first_regular_req=0
    Line 220579: 08-09 11:34:45.407   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x6a, vir=0xd91d7000, num=6, width=960, height=720, time=0x3c0eda4cffc0, iso_value:1600
    Line 220619: 08-09 11:34:45.435   532   999 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=10, cap_intent=1, pic_req=0, first_regular_req=0
    Line 220649: 08-09 11:34:45.445   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x6f, vir=0xd90d9000, num=7, width=960, height=720, time=0x3c0edcaff600, iso_value:1600
    Line 220712: 08-09 11:34:45.474   532   999 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=11, cap_intent=1, pic_req=0, first_regular_req=0
    Line 220778: 08-09 11:34:45.493   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x74, vir=0xd8fdb000, num=8, width=960, height=720, time=0x3c0edf12d0e8, iso_value:1600
    Line 220811: 08-09 11:34:45.524   532   999 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=12, cap_intent=1, pic_req=0, first_regular_req=0
    Line 220858: 08-09 11:34:45.529   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x68, vir=0xd8dce000, num=9, width=960, height=720, time=0x3c0ee1764fe0, iso_value:1600
    Line 220892: 08-09 11:34:45.540   532   999 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=13, cap_intent=1, pic_req=0, first_regular_req=0
    Line 220950: 08-09 11:34:45.567   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x16, vir=0xd92d5000, num=10, width=960, height=720, time=0x3c0ee3d8adc8, iso_value:1600
    Line 220992: 08-09 11:34:45.591   532   999 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=14, cap_intent=1, pic_req=0, first_regular_req=0
    Line 221062: 08-09 11:34:45.609   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x6a, vir=0xd91d7000, num=11, width=960, height=720, time=0x3c0ee63bc730, iso_value:1250
    Line 221084: 08-09 11:34:45.623   532   999 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=15, cap_intent=1, pic_req=0, first_regular_req=0
    Line 221157: 08-09 11:34:45.652   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x6f, vir=0xd90d9000, num=12, width=960, height=720, time=0x3c0ee89e7b08, iso_value:1250
    Line 221181: 08-09 11:34:45.674   532   999 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=16, cap_intent=1, pic_req=0, first_regular_req=0
    Line 221247: 08-09 11:34:45.690   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x74, vir=0xd8fdb000, num=13, width=960, height=720, time=0x3c0eeb012710, iso_value:1250
    Line 221269: 08-09 11:34:45.708   532   999 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=17, cap_intent=1, pic_req=0, first_regular_req=0
    Line 221339: 08-09 11:34:45.729   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x68, vir=0xd8dce000, num=14, width=960, height=720, time=0x3c0eed644c30, iso_value:1250
    Line 221360: 08-09 11:34:45.741   532   999 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=18, cap_intent=1, pic_req=0, first_regular_req=0
    Line 221436: 08-09 11:34:45.769   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x16, vir=0xd92d5000, num=15, width=960, height=720, time=0x3c0eefc72330, iso_value:640
    Line 221461: 08-09 11:34:45.791   532  1000 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=19, cap_intent=1, pic_req=0, first_regular_req=0
    Line 221560: 08-09 11:34:45.811   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x6a, vir=0xd91d7000, num=16, width=960, height=720, time=0x3c0ef22a30e0, iso_value:640
    Line 221582: 08-09 11:34:45.825   532  1000 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=20, cap_intent=1, pic_req=0, first_regular_req=0
    Line 221619: 08-09 11:34:45.844   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x6f, vir=0xd90d9000, num=17, width=960, height=720, time=0x3c0ef48cfc28, iso_value:640
    Line 221641: 08-09 11:34:45.859   532  1000 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=21, cap_intent=1, pic_req=0, first_regular_req=0
    Line 221757: 08-09 11:34:45.935   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x74, vir=0xd8fdb000, num=18, width=960, height=720, time=0x3c0ef9ead348, iso_value:640
    Line 221779: 08-09 11:34:45.959   532  1000 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=22, cap_intent=1, pic_req=0, first_regular_req=0
    Line 221874: 08-09 11:34:46.025   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x68, vir=0xd8dce000, num=19, width=960, height=720, time=0x3c0eff48c9a8, iso_value:1250
    Line 221896: 08-09 11:34:46.043   532  1000 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=23, cap_intent=1, pic_req=0, first_regular_req=0
    Line 222009: 08-09 11:34:46.120   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x16, vir=0xd92d5000, num=20, width=960, height=720, time=0x3c0f04a69ce0, iso_value:1250
    Line 222042: 08-09 11:34:46.144   532  1000 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=24, cap_intent=1, pic_req=0, first_regular_req=0
    Line 222086: 08-09 11:34:46.156   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x6a, vir=0xd91d7000, num=21, width=960, height=720, time=0x3c0f07097f98, iso_value:1250
    Line 222136: 08-09 11:34:46.178   532  1000 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=25, cap_intent=1, pic_req=0, first_regular_req=0
    Line 222187: 08-09 11:34:46.196   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x6f, vir=0xd90d9000, num=22, width=960, height=720, time=0x3c0f096d3158, iso_value:1250
    Line 222231: 08-09 11:34:46.211   532   999 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=26, cap_intent=1, pic_req=0, first_regular_req=0
    Line 222318: 08-09 11:34:46.244   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x74, vir=0xd8fdb000, num=23, width=960, height=720, time=0x3c0f0bcf9710, iso_value:1250
    Line 222381: 08-09 11:34:46.263   532  1000 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=27, cap_intent=1, pic_req=0, first_regular_req=0
    Line 222447: 08-09 11:34:46.282   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x68, vir=0xd8dce000, num=24, width=960, height=720, time=0x3c0f0e325e70, iso_value:1250
    Line 222477: 08-09 11:34:46.297   532  1000 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=28, cap_intent=1, pic_req=0, first_regular_req=0
    Line 222546: 08-09 11:34:46.320   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x16, vir=0xd92d5000, num=25, width=960, height=720, time=0x3c0f109525d0, iso_value:1250
    Line 222567: 08-09 11:34:46.329   532  1000 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=29, cap_intent=1, pic_req=0, first_regular_req=0
    Line 222648: 08-09 11:34:46.363   532 26402 D Cam3OEMIf: 4437, PreviewFramePreviewStream: mCameraId=0, prev:fd=0x6a, vir=0xd91d7000, num=26, width=960, height=720, time=0x3c0f12f80c70, iso_value:1250
    Line 222678: 08-09 11:34:46.380   532  1000 D Cam3HWI : 1708, processCaptureRequest: camId=0, bufs_num=1, frame_num=30, cap_intent=1, pic_req=0, first_regular_req=0
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56

    循序使用的fd & addr_vir

    SprdCamera3GrallocMemory::map 到的 fd & addr_vir :在这里插入图片描述

    SprdCamera3OEMIf在PreviewFramePreviewStream中收到的帧数据的 fd & addr_vir :
    在这里插入图片描述

    我们看到 fd 和 addr_vir 只有5个在一直循环:
    fd=0x6f,offset=0x0,addr_vir=0xd90d9000
    fd=0x74,offset=0x0,addr_vir=0xd8fdb000
    fd=0x68,offset=0x0,addr_vir=0xd8dce000
    fd=0x16,offset=0x0,addr_vir=0xd92d5000
    fd=0x6a,offset=0x0,addr_vir=0xd91d7000

    这与我们在SprdCamera3OEMIf中看到得 mZslNum 的值是一致的,也就是说hal申请了5个buffer,预览帧数据一帧一帧的请求,然后在一帧一帧的回调上去,就是在不停地往这5个buffer写、读。

    log也进一步验证的我们之前跟踪的预览请求和回调流程。

    到这里,hal 和 oem 模块公同完成的预览请求和回调的流程我们就讲完了。

  • 相关阅读:
    Java异常-Exception
    思科网络设备常用命令
    全链路自研:腾讯混元大模型释放企业全新可能性
    数据库查询优化器,RBO优化规则介绍及示例
    java117-list迭代器和包含方法
    【数据结构与算法(C语言)】离散事件模拟- 单链表和队列的混合实际应用
    Python基础——函数(二)
    31-jwt认证
    从下载镜像到装系统(MSDN 和软碟通)
    Spring MVC HandlerAdapter原理解析
  • 原文地址:https://blog.csdn.net/u010869159/article/details/126232790