Android 12(S) 图像显示系统 - Surface 一点补充知识（十二）

---

必读：

Android 12(S) 图像显示系统 - 开篇

---

 

一、前言

---

因为个人工作主要是Android多媒体播放的内容，在工作中查看源码或设计程序经常会遇到调用API：

static inline int native_window_api_connect(struct ANativeWindow* window, int api)
static inline int native_window_api_disconnect(struct ANativeWindow* window, int api)

所以也一直好奇这两个方法都做了什么事情？这篇文章就来一探究竟。

 

二、native_window_api_connect 解析

---

Android系统中，开始播放视频并设置Surface后，都会做一次 connectToSurface 的操作，比如MediaCodec中，在初始化阶段setSurface后就会调用方法：

status_t MediaCodec::connectToSurface(const sp<Surface> &surface) {
    ...
    err = nativeWindowConnect(surface.get(), "connectToSurface");
    ...
}

这里就是去调用了 /frameworks/av/media/libstagefright/SurfaceUtils.cpp  中的方法：

status_t nativeWindowConnect(ANativeWindow *surface, const char *reason) {
    ALOGD("connecting to surface %p, reason %s", surface, reason);

    status_t err = native_window_api_connect(surface, NATIVE_WINDOW_API_MEDIA);
    ALOGE_IF(err != OK, "Failed to connect to surface %p, err %d", surface, err);

    return err;
}

是不是看到了 native_window_api_connect ，其中参数 NATIVE_WINDOW_API_MEDIA 表明 video decoder会作为生产者来生成buffer数据。

再接着往下走，看看到底做了什么？

大体的调用流程如下：

 /frameworks/native/libs/gui/Surface.cpp

 /frameworks/native/libs/gui/BufferQueueProducer.cpp

---

>>> static inline int native_window_api_connect(struct ANativeWindow* window, int api)

>>> int Surface::hook_perform(ANativeWindow* window, int operation, ...)

>>> int Surface::perform(int operation, va_list args)

>>> int Surface::dispatchConnect(va_list args)

>>> int Surface::connect(int api)

>>> int Surface::connect(int api, const sp<IProducerListener>& listener)

>>> int Surface::connect(int api, const sp<IProducerListener>& listener, bool reportBufferRemoval)

>>> status_t BufferQueueProducer::connect(const sp<IProducerListener>& listener,
                int api, bool producerControlledByApp, QueueBufferOutput *output) 

---

最终进入到了BufferQueueProducer::connect函数中，看起来这里应该就是做具体事情的地方了

老规矩，看源码：

status_t BufferQueueProducer::connect(const sp<IProducerListener>& listener,
        int api, bool producerControlledByApp, QueueBufferOutput *output) {
    ATRACE_CALL();
    std::lock_guard<std::mutex> lock(mCore->mMutex);
    mConsumerName = mCore->mConsumerName; // 消费者名字
    BQ_LOGV("connect: api=%d producerControlledByApp=%s", api,
            producerControlledByApp ? "true" : "false");
    // 对一些条件进行判断，必要时直接返回return
    if (mCore->mIsAbandoned) {
        BQ_LOGE("connect: BufferQueue has been abandoned");
        return NO_INIT;
    }

    if (mCore->mConsumerListener == nullptr) {
        BQ_LOGE("connect: BufferQueue has no consumer");
        return NO_INIT;
    }

    if (output == nullptr) {
        BQ_LOGE("connect: output was NULL");
        return BAD_VALUE;
    }

    if (mCore->mConnectedApi != BufferQueueCore::NO_CONNECTED_API) {
        BQ_LOGE("connect: already connected (cur=%d req=%d)",
                mCore->mConnectedApi, api);
        return BAD_VALUE;
    }

    int delta = mCore->getMaxBufferCountLocked(mCore->mAsyncMode,
            mDequeueTimeout < 0 ?
            mCore->mConsumerControlledByApp && producerControlledByApp : false,
            mCore->mMaxBufferCount) -
            mCore->getMaxBufferCountLocked();
    if (!mCore->adjustAvailableSlotsLocked(delta)) {
        BQ_LOGE("connect: BufferQueue failed to adjust the number of available "
                "slots. Delta = %d", delta);
        return BAD_VALUE;
    }

    int status = NO_ERROR;
    switch (api) {
        case NATIVE_WINDOW_API_EGL:
        case NATIVE_WINDOW_API_CPU:
        case NATIVE_WINDOW_API_MEDIA:
        case NATIVE_WINDOW_API_CAMERA:
            mCore->mConnectedApi = api;

            output->width = mCore->mDefaultWidth;
            output->height = mCore->mDefaultHeight;
            output->transformHint = mCore->mTransformHintInUse = mCore->mTransformHint;
            output->numPendingBuffers =
                    static_cast<uint32_t>(mCore->mQueue.size());
            output->nextFrameNumber = mCore->mFrameCounter + 1;
            output->bufferReplaced = false;
            output->maxBufferCount = mCore->mMaxBufferCount;

            if (listener != nullptr) {
                // Set up a death notification so that we can disconnect
                // automatically if the remote producer dies
#ifndef NO_BINDER
                if (IInterface::asBinder(listener)->remoteBinder() != nullptr) {
                    status = IInterface::asBinder(listener)->linkToDeath(
                            static_cast<IBinder::DeathRecipient*>(this));
                    if (status != NO_ERROR) {
                        BQ_LOGE("connect: linkToDeath failed: %s (%d)",
                                strerror(-status), status);
                    }
                    mCore->mLinkedToDeath = listener;
                }
#endif
                mCore->mConnectedProducerListener = listener; // 设置producer listener
                mCore->mBufferReleasedCbEnabled = listener->needsReleaseNotify();
            }
            break;
        default:
            BQ_LOGE("connect: unknown API %d", api);
            status = BAD_VALUE;
            break;
    }
    mCore->mConnectedPid = BufferQueueThreadState::getCallingPid();
    mCore->mBufferHasBeenQueued = false;
    mCore->mDequeueBufferCannotBlock = false;
    mCore->mQueueBufferCanDrop = false;
    mCore->mLegacyBufferDrop = true;
    if (mCore->mConsumerControlledByApp && producerControlledByApp) {
        mCore->mDequeueBufferCannotBlock = mDequeueTimeout < 0;
        mCore->mQueueBufferCanDrop = mDequeueTimeout <= 0;
    }

    mCore->mAllowAllocation = true; // 允许分配 graphic buffer
    VALIDATE_CONSISTENCY();
    return status;
}

我才疏学浅，按我理解，就是完成了一些初始化的操作，貌似也没啥了，这之后应该produder就可以 dequeue buffer 了

另外一点，这里有设置producer listener，之前文章中也讲过，貌似也没啥作用(也许我错了)

    mCore->mConnectedProducerListener = listener;
    mCore->mBufferReleasedCbEnabled = listener->needsReleaseNotify();

三、native_window_api_disconnect 解析

---

disconnect的调用流程和connect的流程类似。

先看 Surface::disconnect 中做了啥

int Surface::disconnect(int api, IGraphicBufferProducer::DisconnectMode mode) {
    ATRACE_CALL();
    ALOGV("Surface::disconnect");
    Mutex::Autolock lock(mMutex);
    mRemovedBuffers.clear();
    mSharedBufferSlot = BufferItem::INVALID_BUFFER_SLOT;
    mSharedBufferHasBeenQueued = false;
    freeAllBuffers();
    int err = mGraphicBufferProducer->disconnect(api, mode);
    if (!err) {
        mReqFormat = 0;
        mReqWidth = 0;
        mReqHeight = 0;
        mReqUsage = 0;
        mCrop.clear();
        mScalingMode = NATIVE_WINDOW_SCALING_MODE_FREEZE;
        mTransform = 0;
        mStickyTransform = 0;
        mAutoPrerotation = false;
        mEnableFrameTimestamps = false;
        mMaxBufferCount = NUM_BUFFER_SLOTS;

        if (api == NATIVE_WINDOW_API_CPU) {
            mConnectedToCpu = false;
        }
    }
    return err;
}

看起来主要是把reset一些变量和release一些资源，看到调用

    freeAllBuffers();
    int err = mGraphicBufferProducer->disconnect(api, mode)

freeAllBuffers是把mSlots里面的元素都置为 nullptr

void Surface::freeAllBuffers() {
    if (!mDequeuedSlots.empty()) {
        ALOGE("%s: %zu buffers were freed while being dequeued!",
                __FUNCTION__, mDequeuedSlots.size());
    }
    for (int i = 0; i < NUM_BUFFER_SLOTS; i++) {
        mSlots[i].buffer = nullptr;
    }
}

---

本文作者@二的次方  2022-03-24 发布于博客园

---

BufferQueueProducer::disconnect源码如下

status_t BufferQueueProducer::disconnect(int api, DisconnectMode mode) {
    ATRACE_CALL();
    BQ_LOGV("disconnect: api %d", api);

    int status = NO_ERROR;
    sp<IConsumerListener> listener;
    { // Autolock scope
        std::unique_lock<std::mutex> lock(mCore->mMutex);

        if (mode == DisconnectMode::AllLocal) {
            if (BufferQueueThreadState::getCallingPid() != mCore->mConnectedPid) {
                return NO_ERROR;
            }
            api = BufferQueueCore::CURRENTLY_CONNECTED_API;
        }

        mCore->waitWhileAllocatingLocked(lock);

        if (mCore->mIsAbandoned) {
            // It's not really an error to disconnect after the surface has
            // been abandoned; it should just be a no-op.
            return NO_ERROR;
        }

        if (api == BufferQueueCore::CURRENTLY_CONNECTED_API) {
            if (mCore->mConnectedApi == NATIVE_WINDOW_API_MEDIA) {
                ALOGD("About to force-disconnect API_MEDIA, mode=%d", mode);
            }
            api = mCore->mConnectedApi;
            // If we're asked to disconnect the currently connected api but
            // nobody is connected, it's not really an error.
            if (api == BufferQueueCore::NO_CONNECTED_API) {
                return NO_ERROR;
            }
        }

        switch (api) {
            case NATIVE_WINDOW_API_EGL:
            case NATIVE_WINDOW_API_CPU:
            case NATIVE_WINDOW_API_MEDIA:
            case NATIVE_WINDOW_API_CAMERA:
                if (mCore->mConnectedApi == api) {
                    mCore->freeAllBuffersLocked();

#ifndef NO_BINDER
                    // Remove our death notification callback if we have one
                    if (mCore->mLinkedToDeath != nullptr) {
                        sp<IBinder> token =
                                IInterface::asBinder(mCore->mLinkedToDeath);
                        // This can fail if we're here because of the death
                        // notification, but we just ignore it
                        token->unlinkToDeath(
                                static_cast<IBinder::DeathRecipient*>(this));
                    }
#endif
                    mCore->mSharedBufferSlot =
                            BufferQueueCore::INVALID_BUFFER_SLOT;
                    mCore->mLinkedToDeath = nullptr;
                    mCore->mConnectedProducerListener = nullptr;
                    mCore->mConnectedApi = BufferQueueCore::NO_CONNECTED_API;
                    mCore->mConnectedPid = -1;
                    mCore->mSidebandStream.clear();
                    mCore->mDequeueCondition.notify_all();
                    mCore->mAutoPrerotation = false;
                    listener = mCore->mConsumerListener;
                } else if (mCore->mConnectedApi == BufferQueueCore::NO_CONNECTED_API) {
                    BQ_LOGE("disconnect: not connected (req=%d)", api);
                    status = NO_INIT;
                } else {
                    BQ_LOGE("disconnect: still connected to another API "
                            "(cur=%d req=%d)", mCore->mConnectedApi, api);
                    status = BAD_VALUE;
                }
                break;
            default:
                BQ_LOGE("disconnect: unknown API %d", api);
                status = BAD_VALUE;
                break;
        }
    } // Autolock scope

    // Call back without lock held
    if (listener != nullptr) {
        listener->onBuffersReleased();
        listener->onDisconnect();
    }

    return status;
}

看样子也是reset一些变量和release资源。

 

调用了mCore->freeAllBuffersLocked()

void BufferQueueCore::freeAllBuffersLocked() {
    for (int s : mFreeSlots) {
        clearBufferSlotLocked(s);
    }

    for (int s : mFreeBuffers) {
        mFreeSlots.insert(s);
        clearBufferSlotLocked(s);
    }
    mFreeBuffers.clear();

    for (int s : mActiveBuffers) {
        mFreeSlots.insert(s);
        clearBufferSlotLocked(s);
    }
    mActiveBuffers.clear();

    for (auto& b : mQueue) {
        b.mIsStale = true;
        b.mAcquireCalled = false;
    }

    VALIDATE_CONSISTENCY();
}

 

通知了消费者 listener->onBuffersReleased()  and listener->onDisconnect()

消费者响应 onBuffersReleased 也是去free buffer

void ConsumerBase::onBuffersReleased() {
    Mutex::Autolock lock(mMutex);

    CB_LOGV("onBuffersReleased");

    if (mAbandoned) {
        // Nothing to do if we're already abandoned.
        return;
    }

    uint64_t mask = 0;
    mConsumer->getReleasedBuffers(&mask);
    for (int i = 0; i < BufferQueue::NUM_BUFFER_SLOTS; i++) {
        if (mask & (1ULL << i)) {
            freeBufferLocked(i);
        }
    }
}

消费者响应 onDisconnect

void BLASTBufferItemConsumer::onDisconnect() {
    Mutex::Autolock lock(mMutex);
    mPreviouslyConnected = mCurrentlyConnected;
    mCurrentlyConnected = false;
    if (mPreviouslyConnected) {
        mDisconnectEvents.push(mCurrentFrameNumber);
    }
    mFrameEventHistory.onDisconnect();
}

到此，就讲完了，感觉好乏味...，没有什么实质的东西

 

♦ Android 12 Google将buffer queue组件从SurfaceFlinger端移动到了客户端（旧版本是在BufferQueueLayer中去createBufferQueue的）。

buffer queue组件的创建和初始化也放在BLASTBufferQueue中。通过类名可以看出BLASTBufferQueue更像是buffer queue组件的一层封装或装饰。

♦ 通过前面系列文章的分析，可以看到，整个生产消费模型都在客户端，图形缓冲区的出队、入队、获取等操作都在客户端完成，预示着producer -- buffer queue -- consumer 间的通信都变成了本地通信。

♦ BLASTBufferQueue需要通过事务Transaction来向SurfaceFlinger端提交Buffer与图层的属性。

 

♦ 如本文讲的 disconnect event 在12平台就无法传递到SurfaceFlinger了。如要传递信息，就要使用 Transaction 传递过去。

 

---

不过我还是有个疑问：disconnect过程中，可以看到producer/consumer/surface都有去free buffer，此时，为GraphicBuffer分配的内存真的就释放了吗？

(mSlots[slot].mGraphicBuffer.clear() or  mSlots[slotIndex].mGraphicBuffer = nullptr)

 

四、小结

---

感觉没啥说的了，就此结束吧