• gstreamer协商negoation

    首先说明，这是自己的一些笔记，比较凌乱，还没有好好整理，不要留言骂人啊，我会慢慢整理出来，我本人是做gsrremer插件工作，所以明白gstreamer真的很难，不是看一些博客就能了解的，看了了解到的大部分也是错的，所以你真的想要学习gstreamer的话，一定要把gstreamer代码结构先看明白，那些是基础代码，那些是插件代码，当然还要多多少少了解一些meson的构建方式。
    达到这个程度后，最好在linux下编译一个最简单的gstreamer库，然后准备一个非常简单的案例，打开gstreamer的log，当你跑通一个案例的时候，跟着日志看代码的流程，不要怕麻烦，一点一点来，先从最简单的插件加载方式看，然后看add,然后link，然后看state的改变，中途如果对某些代码调用步确定，可以用gdb跟踪下代码，这一套流程下来，你就会对gstreamer的整个框架有个非常宏观的了解了。

    所以与其翻各种错误百出的帖子还不如自己静下心来好好看看代码，看看官方文档

    gstreamer关键步骤有下面三个：

     gst_bin_add_many(GST_BIN(data.pipeline), data.source, data.h264parse, data.omx264dec, data.convert, data.sink, nullptr);
 
gst_element_link_many(data.source, data.h264parse, data.omx264dec, data.convert, data.sink, nullptr)

gst_element_set_state(data.pipeline, GST_STATE_PLAYING);

    • 1
    • 2
    • 3
    • 4
    • 5

    • add最简单，主要将element添加到pipeline中，但是这里还有一个操作，就是将pipeline的child bus设置为所有element中的bus，各个element的parent设置为pipeline.

    • link阶段核心就是将上下element的sinkpad和srcpad通过peerpad连接在一起，但是连接之前会查询双方pad的caps,并且进行交集计算，如果有交集才进行link.查询的时候发送的是caps query

    • gst_element_set_state(data.pipeline, GST_STATE_PLAYING)这是pipeline状态改变的重点函数，这里，状态首先从NULl改变到READY，然后再把状态从READY改变为PAUSED,其中到PAUSED的过程中会触发激活函数，比如src element的loop线程开始启动，解码器开始打开等动作。注意的是这个过程中的element是link状态下element倒着来的，比如先是sink,接着covert，然后omx264dec，然后。。。，

    • 当pipeline的t状态改变为AREADY的时候，会从source发送一个STREAM_START的event，这个event一直传递到最后的sink element，每个element接收到STREAM_START后会做一些事情。
      下图就是event从上床底到下的一个流程图，不一定每个event都从上传递到下，个别event可能从中途就返回了。
      在这里插入图片描述当sink element收到一个stream-start event后会发送给一个msg到bus总线。

    • 协商就发生在stream start之后，发起者是source element就是第一个element，所以当你还没有了解到整个gstreamer运转机制的时候，先写一个简单的pipeline，然后从第一个element开始看协商函数。

    下面函数是整个pipeline的动力起源：

    
/* Called with STREAM_LOCK */
static void
gst_base_src_loop (GstPad * pad)
{
  GstBaseSrc *src;
  GstBuffer *buf = NULL;
  GstFlowReturn ret;
  gint64 position;
  gboolean eos;
  guint blocksize;
  GList *pending_events = NULL, *tmp;

  eos = FALSE;

  src = GST_BASE_SRC (GST_OBJECT_PARENT (pad));

  /* Just leave immediately if we're flushing */
  GST_LIVE_LOCK (src);
  if (G_UNLIKELY (src->priv->flushing || GST_PAD_IS_FLUSHING (pad)))
    goto flushing;
  GST_LIVE_UNLOCK (src);

  /* Just return if EOS is pushed again, as the app might be unaware that an
   * EOS have been sent already */
  if (GST_PAD_IS_EOS (pad)) {
    GST_DEBUG_OBJECT (src, "Pad is marked as EOS, pause the task");
    gst_pad_pause_task (pad);
    goto done;
  }
	//首先发送一个stream start的event
  gst_base_src_send_stream_start (src);

  /* The stream-start event could've caused something to flush us */
  GST_LIVE_LOCK (src);
  if (G_UNLIKELY (src->priv->flushing || GST_PAD_IS_FLUSHING (pad)))
    goto flushing;
  GST_LIVE_UNLOCK (src);

  /* check if we need to renegotiate */
  if (gst_pad_check_reconfigure (pad)) {
  //这里进行第一次协商
    if (!gst_base_src_negotiate_unlocked (src)) {
      gst_pad_mark_reconfigure (pad);
      if (GST_PAD_IS_FLUSHING (pad)) {
        GST_LIVE_LOCK (src);
        goto flushing;
      } else {
        goto negotiate_failed;
      }
    }
  }

  GST_LIVE_LOCK (src);

  if (G_UNLIKELY (src->priv->flushing || GST_PAD_IS_FLUSHING (pad)))
    goto flushing;

  blocksize = src->blocksize;

  /* if we operate in bytes, we can calculate an offset */
  if (src->segment.format == GST_FORMAT_BYTES) {
    position = src->segment.position;
    /* for negative rates, start with subtracting the blocksize */
    if (src->segment.rate < 0.0) {
      /* we cannot go below segment.start */
      if (position > src->segment.start + blocksize)
        position -= blocksize;
      else {
        /* last block, remainder up to segment.start */
        blocksize = position - src->segment.start;
        position = src->segment.start;
      }
    }
  } else
    position = -1;

  GST_LOG_OBJECT (src, "next_ts %" GST_TIME_FORMAT " size %u",
      GST_TIME_ARGS (position), blocksize);

  /* clean up just in case we got interrupted or so last time round */
  if (src->priv->pending_bufferlist != NULL) {
    gst_buffer_list_unref (src->priv->pending_bufferlist);
    src->priv->pending_bufferlist = NULL;
  }

  ret = gst_base_src_get_range (src, position, blocksize, &buf);
  if (G_UNLIKELY (ret != GST_FLOW_OK)) {
    GST_INFO_OBJECT (src, "pausing after gst_base_src_get_range() = %s",
        gst_flow_get_name (ret));
    GST_LIVE_UNLOCK (src);
    goto pause;
  }
  }

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67
    • 68
    • 69
    • 70
    • 71
    • 72
    • 73
    • 74
    • 75
    • 76
    • 77
    • 78
    • 79
    • 80
    • 81
    • 82
    • 83
    • 84
    • 85
    • 86
    • 87
    • 88
    • 89
    • 90
    • 91
    • 92
    • 93
    • 94

    下面先看：gst_base_src_negotiate_unlocked（）

    static gboolean
gst_base_src_negotiate_unlocked (GstBaseSrc * basesrc)
{
  GstBaseSrcClass *bclass;
  gboolean result;

  bclass = GST_BASE_SRC_GET_CLASS (basesrc);

  GST_DEBUG_OBJECT (basesrc, "starting negotiation");

  if (G_LIKELY (bclass->negotiate))
    result = bclass->negotiate (basesrc);
  else
    result = TRUE;

  if (G_LIKELY (result)) {
    GstCaps *caps;
//首先查询caps
    caps = gst_pad_get_current_caps (basesrc->srcpad);
//开始分配buf
    result = gst_base_src_prepare_allocation (basesrc, caps);

    if (caps)
      gst_caps_unref (caps);
  }
  return result;
}

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27

    首先看第一个：

    caps = gst_pad_get_current_caps (basesrc->srcpad);

    • 1

    下面是日志：

    [Level:5] ../subprojects/gstreamer/libs/gst/base/gstbasesrc.c:gst_base_src_negotiate_unlocked:3439  starting negotiation
[Level:5] ../subprojects/gstreamer/gst/gstutils.c:gst_pad_query_caps:3102  get pad caps with filter (NULL)
[Level:5] ../subprojects/gstreamer/gst/gstquery.c:gst_query_new_custom:679  creating new query 0x5630fc4bf6d0 caps
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query:4170  doing query 0x5630fc4bf6d0 (caps)
[Level:5] ../subprojects/gst-plugins-base/gst-libs/gst/app/gstappsrc.c:gst_app_src_internal_get_caps:860  caps: (NULL)
[Level:5] ../subprojects/gstreamer/libs/gst/base/gstbasesrc.c:gst_base_src_default_query:1381  query caps returns 0
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query:4193  sent query 0x5630fc4bf6d0 (caps), result 0
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query:4239  query failed
[Level:5] ../subprojects/gstreamer/libs/gst/base/gstbasesrc.c:gst_base_src_default_negotiate:3369  caps of src: ANY
[Level:5] ../subprojects/gstreamer/libs/gst/base/gstbasesrc.c:gst_base_src_default_negotiate:3415  no negotiation needed
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_get_current_caps:2734  get current pad caps (NULL)

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11

    从日志可以看到，source app返回的是ANY。
    接着看如何allocation:

    result = gst_base_src_prepare_allocation (basesrc, caps);

//这个函数总体就发送了一个caps的event，然后根据decoder返回的allocation来创建了一个bufpool，然后启动部分pool；
static gboolean
gst_base_src_prepare_allocation (GstBaseSrc * basesrc, GstCaps * caps)
{
  GstBaseSrcClass *bclass;
  gboolean result = TRUE;
  GstQuery *query;
  GstBufferPool *pool = NULL;
  GstAllocator *allocator = NULL;
  GstAllocationParams params;

  bclass = GST_BASE_SRC_GET_CLASS (basesrc);

  /* make query and let peer pad answer, we don't really care if it worked or
   * not, if it failed, the allocation query would contain defaults and the
   * subclass would then set better values if needed */
  //发送了一个allocation event ,下一个element是h264parse，所以接下来看h264中关于allocation的query处理，
  query = gst_query_new_allocation (caps, TRUE);
  if (!gst_pad_peer_query (basesrc->srcpad, query)) {
    /* not a problem, just debug a little */
    GST_DEBUG_OBJECT (basesrc, "peer ALLOCATION query failed");
  }

  g_assert (bclass->decide_allocation != NULL);
  result = bclass->decide_allocation (basesrc, query);

  GST_DEBUG_OBJECT (basesrc, "ALLOCATION (%d) params: %" GST_PTR_FORMAT, result,
      query);

  if (!result)
    goto no_decide_allocation;

  /* we got configuration from our peer or the decide_allocation method,
   * parse them */
  if (gst_query_get_n_allocation_params (query) > 0) {
    gst_query_parse_nth_allocation_param (query, 0, &allocator, &params);
  } else {
    allocator = NULL;
    gst_allocation_params_init (&params);
  }

  if (gst_query_get_n_allocation_pools (query) > 0)
    gst_query_parse_nth_allocation_pool (query, 0, &pool, NULL, NULL, NULL);
//这里把pool激活
  result = gst_base_src_set_allocation (basesrc, pool, allocator, &params);

  if (allocator)
    gst_object_unref (allocator);
  if (pool)
    gst_object_unref (pool);

  gst_query_unref (query);

  return result;

  /* Errors */
no_decide_allocation:
  {
    GST_WARNING_OBJECT (basesrc, "Subclass failed to decide allocation");
    gst_query_unref (query);

    return result;
  }
}

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66

    下面看如何发送allocation的event:

    Level:5] ../subprojects/gstreamer/gst/gstquery.c:gst_query_new_custom:679  creating new query 0x5630fc4bf770 allocation
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_peer_query:4297  peer query 0x5630fc4bf770 (allocation)
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query:4170  doing query 0x5630fc4bf770 (allocation)
[Level:5] ../subprojects/gstreamer/libs/gst/base/gstbaseparse.c:gst_base_parse_sink_query:1637  allocation query
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query_default:3511  forwarding 0x5630fc4bf770 (allocation) query
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_iterate_internal_links_default:2965  Making iterator

//h264parse不处理，直接向下转发
[Level:6] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_forward:3074  calling forward function on pad h264-parse:src
[Level:6] ../subprojects/gstreamer/gst/gstpad.c:query_forward_func:3447  query peer 0x5630fc4bf770 (allocation) of h264-parse:src
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_peer_query:4297  peer query 0x5630fc4bf770 (allocation)
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query:4170  doing query 0x5630fc4bf770 (allocation)
//videodecoder处理这个query，然后直接返回
[Level:5] ../subprojects/gst-plugins-base/gst-libs/gst/video/gstvideodecoder.c:gst_video_decoder_sink_query:2178  received query 35846, allocation
[Level:6] ../subprojects/gst-plugins-base/gst-libs/gst/video/gstvideodecoder.c:gst_video_decoder_sink_query_default:2087  handling query: allocation query: 0x5630fc4bf770, GstQueryAllocation, caps=(GstCaps)"NULL", need-pool=(boolean)true;
//这里返回了allocation
[Level:5] ../subprojects/gst-omx/omx/gstomxvideodec.c:gst_omx_video_dec_propose_allocation:3517  request at least 3 buffers of size 32768

[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query:4193  sent query 0x5630fc4bf770 (allocation), result 1
[Level:2] ../subprojects/gstreamer/gst/gststructure.c:priv_gst_structure_append_to_gstring:2091  No value transform to serialize field 'pool' of type 'GArray'
[Level:6] ../subprojects/gstreamer/libs/gst/base/gstbaseparse.c:gst_base_parse_sink_query:1644  allocation query result: 1 allocation query: 0x5630fc4bf770, GstQueryAllocation, caps=(GstCaps)"NULL", need-pool=(boolean)true, pool=(GArray)NULL;
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query:4193  sent query 0x5630fc4bf770 (allocation), result 1

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22

    接着执行：

      g_assert (bclass->decide_allocation != NULL);
  result = bclass->decide_allocation (basesrc, query);

    • 1
    • 2

    下面是decide_allocation日志：

    
[Level:5] ../subprojects/gstreamer/libs/gst/base/gstbasesrc.c:gst_base_src_decide_allocation_default:3235  no pool, making new pool
[Level:5] ../subprojects/gstreamer/gst/gstpoll.c:gst_poll_new:681  0x7f3478003000: new controllable : 1
[Level:5] ../subprojects/gstreamer/gst/gstpoll.c:gst_poll_add_fd_unlocked:848  0x7f3478003000: fd (fd:9, idx:0)
[Level:5] ../subprojects/gstreamer/gst/gstpoll.c:gst_poll_fd_ctl_read_unlocked:1014  0x7f3478003000: fd (fd:9, idx:0), active : 1
[Level:6] ../subprojects/gstreamer/gst/gstpoll.c:raise_wakeup:290  0x7f3478003000: raise
[Level:5] ../subprojects/gstreamer/gst/gstbufferpool.c:gst_buffer_pool_init:179  created
[Level:5] ../subprojects/gstreamer/gst/gstbufferpool.c:gst_buffer_pool_new:231  created new buffer pool
[Level:2] ../subprojects/gstreamer/gst/gststructure.c:priv_gst_structure_append_to_gstring:2091  No value transform to serialize field 'params' of type 'GstAllocationParams'

[Level:5] ../subprojects/gstreamer/gst/gstbufferpool.c:default_set_config:622  config GstBufferPoolConfig, caps=(GstCaps)"NULL", size=(uint)32768, min-buffers=(uint)3, max-buffers=(uint)0, allocator=(GstAllocator)"NULL", params=(GstAllocationParams)NULL;
[Level:2] ../subprojects/gstreamer/gst/gststructure.c:priv_gst_structure_append_to_gstring:2091  No value transform to serialize field 'pool' of type 'GArray'
[Level:2] ../subprojects/gstreamer/gst/gststructure.c:priv_gst_structure_append_to_gstring:2091  No value transform to serialize field 'allocator' of type 'GArray'
[Level:5] ../subprojects/gstreamer/libs/gst/base/gstbasesrc.c:gst_base_src_prepare_allocation:3315  ALLOCATION (1) params: allocation query: 0x5630fc4bf770, GstQueryAllocation, caps=(GstCaps)"NULL", need-pool=(boolean)true, pool=(GArray)NULL, allocator=(GArray)NULL;
[Level:5] ../subprojects/gstreamer/libs/gst/base/gstbasesrc.c:gst_base_src_set_allocation:3143  activate pool
[Level:6] ../subprojects/gstreamer/gst/gstbufferpool.c:gst_buffer_pool_set_active:514  active 1

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16

    gst_base_src_decide_allocation_default（）

    static gboolean
gst_base_src_decide_allocation_default (GstBaseSrc * basesrc, GstQuery * query)
{
  GstCaps *outcaps;
  GstBufferPool *pool;
  guint size, min, max;
  GstAllocator *allocator;
  GstAllocationParams params;
  GstStructure *config;
  gboolean update_allocator;
//首先解析query
  gst_query_parse_allocation (query, &outcaps, NULL);

  /* we got configuration from our peer or the decide_allocation method,
   * parse them */
   * //解析allocation params
  if (gst_query_get_n_allocation_params (query) > 0) {
    /* try the allocator */
    gst_query_parse_nth_allocation_param (query, 0, &allocator, &params);
    update_allocator = TRUE;
  } else {
    allocator = NULL;
    gst_allocation_params_init (&params);
    update_allocator = FALSE;
  }

//解析allocation pool
  if (gst_query_get_n_allocation_pools (query) > 0) {
    gst_query_parse_nth_allocation_pool (query, 0, &pool, &size, &min, &max);

    if (pool == NULL) {
      /* no pool, we can make our own */
      GST_DEBUG_OBJECT (basesrc, "no pool, making new pool");
      //创建pool
      pool = gst_buffer_pool_new ();
    }
  } else {
    pool = NULL;
    size = min = max = 0;
  }

  /* now configure */
  if (pool) {
    config = gst_buffer_pool_get_config (pool);
    gst_buffer_pool_config_set_params (config, outcaps, size, min, max);
    gst_buffer_pool_config_set_allocator (config, allocator, &params);

    /* buffer pool may have to do some changes */
    if (!gst_buffer_pool_set_config (pool, config)) {
      config = gst_buffer_pool_get_config (pool);

      /* If change are not acceptable, fallback to generic pool */
      if (!gst_buffer_pool_config_validate_params (config, outcaps, size, min,
              max)) {
        GST_DEBUG_OBJECT (basesrc, "unsupported pool, making new pool");

        gst_object_unref (pool);
        pool = gst_buffer_pool_new ();
        gst_buffer_pool_config_set_params (config, outcaps, size, min, max);
        gst_buffer_pool_config_set_allocator (config, allocator, &params);
      }

      if (!gst_buffer_pool_set_config (pool, config))
        goto config_failed;
    }
  }

  if (update_allocator)
    gst_query_set_nth_allocation_param (query, 0, allocator, &params);
  else
    gst_query_add_allocation_param (query, allocator, &params);
  if (allocator)
    gst_object_unref (allocator);

  if (pool) {
    gst_query_set_nth_allocation_pool (query, 0, pool, size, min, max);
    gst_object_unref (pool);
  }

  return TRUE;

config_failed:
  GST_ELEMENT_ERROR (basesrc, RESOURCE, SETTINGS,
      ("Failed to configure the buffer pool"),
      ("Configuration is most likely invalid, please report this issue."));
  gst_object_unref (pool);
  return FALSE;
}

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67
    • 68
    • 69
    • 70
    • 71
    • 72
    • 73
    • 74
    • 75
    • 76
    • 77
    • 78
    • 79
    • 80
    • 81
    • 82
    • 83
    • 84
    • 85
    • 86
    • 87
    • 88

    接下来看协商,最难的协商也是发生在这里，下面是调用栈，可以看到发生在gst_app_src_create()这里
    在这里插入图片描述
    今天继续，我们从gst_app_src_create（）开始吧，

    
gst_app_src_create ()
	gst_app_src_do_negotiate (bsrc)
			gst_base_src_set_caps (basesrc, caps);
					current_caps = gst_pad_get_current_caps (GST_BASE_SRC_PAD (src))
							gst_pad_push_event (src->srcpad, gst_event_new_caps (caps))


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7

    所以整个pipeline的协商起始位置在这里，发送了一个caps event，下面看日志，可以看出创建的event发送到了下方的peerpad h264-parse:sink，对于gst_pad_push_event（）这个函数，这里不展开，请单独看，非常简单的，同时也能解决你的一个疑问，sticky event.

    [Level:5] ../subprojects/gstreamer/gst/gstpad.c:ginf:2734  get current pad caps (NULL)
[Level:4] ../subprojects/gstreamer/gst/gstevent.c:gst_event_new_caps:892  creating caps event video/x-h264, alignment=(string)au, stream-format=(string)byte-stream, width=(int)176, height=(int)144
[Level:5] ../subprojects/gstreamer/gst/gstevent.c:gst_event_new_custom:310  creating new event 0x5630fc4c3560 caps 12814
[Level:6] ../subprojects/gstreamer/gst/gstpad.c:store_sticky_event:5354  stored sticky event caps
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:store_sticky_event:5360  notify caps
[Level:6] ../subprojects/gstreamer/gst/gstobject.c:gst_object_dispatch_properties_changed:472  deep notification from src (caps)
[Level:6] ../subprojects/gstreamer/gst/gstobject.c:gst_object_dispatch_properties_changed:472  deep notification from src (caps)
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:check_sticky:4102  pushing all sticky events
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:push_sticky:4031  event stream-start was already received

[Level:6] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_push_event_unchecked:5538  sending event 0x5630fc4c3560 (caps) to peerpad <h264-parse:sink>
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_send_event_unchecked:5827  have event type caps event: 0x5630fc4c3560, time 99:99:99.999999999, seq-num 31, GstEventCaps, caps=(GstCaps)"video/x-h264\,\ alignment\=\(string\)au\,\ stream-format\=\(string\)byte-stream\,\ width\=\(int\)176\,\ height\=\(int\)144";


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13

    大致调用流程是：

    gst_pad_push_event_unchecked（）
	gst_pad_send_event_unchecked (peerpad, event, type)
			pre_eventfunc_check (pad, event);
					eventfunc (pad, parent, event)


    • 1
    • 2
    • 3
    • 4
    • 5

    上面pre_eventfunc_check (pad, event)做了很多事情：

    static GstFlowReturn
pre_eventfunc_check (GstPad * pad, GstEvent * event)
{
  GstCaps *caps;

  switch (GST_EVENT_TYPE (event)) {
    case GST_EVENT_CAPS:
    {
      /* backwards compatibility mode for caps */
      gst_event_parse_caps (event, &caps);
			//这里查询，也就是发送caps事件的时候先发送caps query
			//如果有accept caps，才可以发送caps event
      if (!gst_pad_query_accept_caps (pad, caps))
        goto not_accepted;
      break;
    }
    default:
      break;
  }
  return GST_FLOW_OK;
}

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21

    下面日志可以看出先构建了一个accept-capsquery，然后查询

    [Level:5] ../subprojects/gstreamer/gst/gstutils.c:gst_pad_query_accept_caps:3185  accept caps of video/x-h264, alignment=(string)au, stream-format=(string)byte-stream, width=(int)176, height=(int)144
[Level:5] ../subprojects/gstreamer/gst/gstquery.c:gst_query_new_custom:679  creating new query 0x7f3478003050 accept-caps
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query:4170  doing query 0x7f3478003050 (accept-caps)

    • 1
    • 2
    • 3

    source element发送了查询后，接下来肯定是h264parse接受到了啊，下面是日志，

    //h264parese接收到这个查询
[Level:5] ../subprojects/gstreamer/libs/gst/base/gstbaseparse.c:gst_base_parse_sink_query:1637  accept-caps query

[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query_accept_caps_default:3197  query accept-caps accept-caps query: 0x7f3478003050, GstQueryAcceptCaps, caps=(GstCaps)"video/x-h264\,\ alignment\=\(string\)au\,\ stream-format\=\(string\)byte-stream\,\ width\=\(int\)176\,\ height\=\(int\)144", result=(boolean)false;
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query_accept_caps_default:3222  allowed caps intersect video/x-h264, caps video/x-h264, alignment=(string)au, stream-format=(string)byte-stream, width=(int)176, height=(int)144
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query_default:3511  not forwarding 0x7f3478003050 (accept-caps) query

[Level:6] ../subprojects/gstreamer/libs/gst/base/gstbaseparse.c:gst_base_parse_sink_query:1644  accept-caps query result: 1 accept-caps query: 0x7f3478003050, GstQueryAcceptCaps, caps=(GstCaps)"video/x-h264\,\ alignment\=\(string\)au\,\ stream-format\=\(string\)byte-stream\,\ width\=\(int\)176\,\ height\=\(int\)144", result=(boolean)true;
[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query:4193  sent query 0x7f3478003050 (accept-caps), result 1
[Level:5] ../subprojects/gstreamer/gst/gstutils.c:gst_pad_query_accept_caps:3191  query returned 1

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10

    根据日志提示，我们来看gst_base_parse_sink_query（）函数：

    gst_base_parse_sink_query（）
	gst_base_parse_sink_query_default（）
		gst_pad_query_default (pad, GST_OBJECT_CAST (parse), query)
			gst_pad_query_accept_caps_default (pad, query)
				st_query_parse_accept_caps (query, &caps)
						gst_caps_can_intersect (caps, allowed)
							gst_query_set_accept_caps_result (query, result)
gst_base_parse_sink_query（）//最终返回到这里

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8

    接下来到了eventfunc (pad, parent, event)函数：

    
static GstFlowReturn
gst_pad_send_event_unchecked (GstPad * pad, GstEvent * event,
    GstPadProbeType type)
{
//也就是pre函数成功了才开发送事件函数，根据上面我们日志和代码的跟踪，发现本次pre就是查看了
//source srcpad和h264parse sinkpad是否有交集，如果有交集才发送event caps。
 ret = pre_eventfunc_check (pad, event);
  if (G_UNLIKELY (ret != GST_FLOW_OK))
    goto precheck_failed;

  if (sticky)
    gst_event_ref (event);

  if (eventfullfunc) {
    ret = eventfullfunc (pad, parent, event);
  } else if (eventfunc (pad, parent, event)) {
    ret = GST_FLOW_OK;
  } else {
    /* something went wrong */
    switch (event_type) {
      case GST_EVENT_CAPS:
        ret = GST_FLOW_NOT_NEGOTIATED;
        break;
      default:
        ret = GST_FLOW_ERROR;
        break;
    }
  }

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29

    这里开始eventfunc()函数：

    static gboolean
gst_base_parse_sink_event (GstPad * pad, GstObject * parent, GstEvent * event)
{
  GstBaseParse *parse = GST_BASE_PARSE (parent);
  GstBaseParseClass *bclass = GST_BASE_PARSE_GET_CLASS (parse);
  gboolean ret;

  ret = bclass->sink_event (parse, event);

  return ret;
}

//起始内部执行的是这个函数
static gboolean
gst_base_parse_sink_event_default (GstBaseParse * parse, GstEvent * event)
{
  GstBaseParseClass *klass = GST_BASE_PARSE_GET_CLASS (parse);
  gboolean ret = FALSE;
  gboolean forward_immediate = FALSE;

  GST_DEBUG_OBJECT (parse, "handling event %d, %s", GST_EVENT_TYPE (event),
      GST_EVENT_TYPE_NAME (event));

  switch (GST_EVENT_TYPE (event)) {
    case GST_EVENT_CAPS:
    {
      GstCaps *caps;

      gst_event_parse_caps (event, &caps);
      GST_DEBUG_OBJECT (parse, "caps: %" GST_PTR_FORMAT, caps);

      if (klass->set_sink_caps)
        ret = klass->set_sink_caps (parse, caps);
      else
        ret = TRUE;

      /* will send our own caps downstream */
      gst_event_unref (event);
      event = NULL;
      break;
    }
    }

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42

    接着是set_sink_caps (parse, caps)，实际执行的是gst_h264_parse_set_caps，这个函数太长了，大致做了一些解析query的工作，然后

    static gboolean
gst_h264_parse_set_caps (GstBaseParse * parse, GstCaps * caps)
{
  GstH264Parse *h264parse;
  GstStructure *str;
  const GValue *codec_data_value;
  GstBuffer *codec_data = NULL;
  gsize size;
  guint format, align, off;
  GstH264NalUnit nalu;
  GstH264ParserResult parseres;
  GstCaps *old_caps;

  h264parse = GST_H264_PARSE (parse);

  /* reset */
  h264parse->push_codec = FALSE;

  old_caps = gst_pad_get_current_caps (GST_BASE_PARSE_SINK_PAD (parse));
  if (old_caps) {
    if (!gst_caps_is_equal (old_caps, caps))
      gst_h264_parse_reset_stream_info (h264parse);
    gst_caps_unref (old_caps);
  }

  str = gst_caps_get_structure (caps, 0);

  /* accept upstream info if provided */
  gst_structure_get_int (str, "width", &h264parse->width);
  gst_structure_get_int (str, "height", &h264parse->height);
  gst_structure_get_fraction (str, "framerate", &h264parse->fps_num,
      &h264parse->fps_den);
  gst_structure_get_fraction (str, "pixel-aspect-ratio",
      &h264parse->upstream_par_n, &h264parse->upstream_par_d);

  /* get upstream format and align from caps */
  gst_h264_parse_format_from_caps (caps, &format, &align);

  codec_data_value = gst_structure_get_value (str, "codec_data");

  /* fix up caps without stream-format for max. backwards compatibility */
  if (format == GST_H264_PARSE_FORMAT_NONE) {
    /* codec_data implies avc */
    if (codec_data_value != NULL) {
      GST_ERROR ("video/x-h264 caps with codec_data but no stream-format=avc");
      format = GST_H264_PARSE_FORMAT_AVC;
    } else {
      /* otherwise assume bytestream input */
      GST_ERROR ("video/x-h264 caps without codec_data or stream-format");
      format = GST_H264_PARSE_FORMAT_BYTE;
    }
  }

...

  /* bytestream caps sanity checks */
  if (format == GST_H264_PARSE_FORMAT_BYTE) {
    /* should have SPS/PSS in-band (and/or oob in streamheader field) */
    if (codec_data_value != NULL)
      goto bytestream_caps_with_codec_data;
  }
 
 if (codec_data_value != NULL) {
 ...
}else if (format == GST_H264_PARSE_FORMAT_BYTE) {
    GST_DEBUG_OBJECT (h264parse, "have bytestream h264");
    /* nothing to pre-process */
    h264parse->packetized = FALSE;
    /* we have 4 sync bytes */
    h264parse->nal_length_size = 4;
  } 
  
{
    GstCaps *in_caps;

    /* prefer input type determined above */
    in_caps = gst_caps_new_simple ("video/x-h264",
        "parsed", G_TYPE_BOOLEAN, TRUE,
        "stream-format", G_TYPE_STRING,
        gst_h264_parse_get_string (h264parse, TRUE, format),
        "alignment", G_TYPE_STRING,
        gst_h264_parse_get_string (h264parse, FALSE, align), NULL);
    /* negotiate with downstream, sets ->format and ->align */
    //继续向下协商，h264parse里面有前面解析到的width,hight,framerate等
    gst_h264_parse_negotiate (h264parse, format, in_caps);
    gst_caps_unref (in_caps);
  }
}


    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67
    • 68
    • 69
    • 70
    • 71
    • 72
    • 73
    • 74
    • 75
    • 76
    • 77
    • 78
    • 79
    • 80
    • 81
    • 82
    • 83
    • 84
    • 85
    • 86
    • 87
    • 88
    • 89

    gst_h264_parse_negotiate (h264parse, format, in_caps);做了些什么呢：

    tatic void
gst_h264_parse_negotiate (GstH264Parse * h264parse, gint in_format,
    GstCaps * in_caps)
{
  GstCaps *caps;
  guint format = h264parse->format;
  guint align = h264parse->align;

  g_return_if_fail ((in_caps == NULL) || gst_caps_is_fixed (in_caps));
  //这里是个重点GST_BASE_PARSE_SRC_PAD (h264parse)获取到了src pad
  caps = gst_pad_get_allowed_caps (GST_BASE_PARSE_SRC_PAD (h264parse));
  GST_DEBUG_OBJECT (h264parse, "allowed caps: %" GST_PTR_FORMAT, caps);

  /* concentrate on leading structure, since decodebin parser
   * capsfilter always includes parser template caps */
  if (caps) {
    caps = gst_caps_truncate (caps);
    GST_DEBUG_OBJECT (h264parse, "negotiating with caps: %" GST_PTR_FORMAT,
        caps);
  }

  h264parse->can_passthrough = FALSE;

  if (in_caps && caps) {
    if (gst_caps_can_intersect (in_caps, caps)) {
      GST_DEBUG_OBJECT (h264parse, "downstream accepts upstream caps");
      gst_h264_parse_format_from_caps (in_caps, &format, &align);
      gst_caps_unref (caps);
      caps = NULL;
      h264parse->can_passthrough = TRUE;
    }
  }

  /* FIXME We could fail the negotiation immediately if caps are empty */
  if (caps && !gst_caps_is_empty (caps)) {
    /* fixate to avoid ambiguity with lists when parsing */
    caps = gst_caps_fixate (caps);
    gst_h264_parse_format_from_caps (caps, &format, &align);
  }

  /* default */
  if (!format)
    format = GST_H264_PARSE_FORMAT_BYTE;
  if (!align)
    align = GST_H264_PARSE_ALIGN_AU;

  GST_DEBUG_OBJECT (h264parse, "selected format %s, alignment %s",
      gst_h264_parse_get_string (h264parse, TRUE, format),
      gst_h264_parse_get_string (h264parse, FALSE, align));

  h264parse->format = format;
  h264parse->align = align;

  h264parse->transform = in_format != h264parse->format ||
      align == GST_H264_PARSE_ALIGN_AU;

  if (caps)
    gst_caps_unref (caps);
}

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59

    接着继续看gst_pad_get_allowed_caps()：

    GstCaps *gst_pad_get_allowed_caps (GstPad * pad)
{
  GstCaps *mycaps;
  GstCaps *caps = NULL;
  GstQuery *query;

  g_return_val_if_fail (GST_IS_PAD (pad), NULL);

  GST_OBJECT_LOCK (pad);
  if (G_UNLIKELY (GST_PAD_PEER (pad) == NULL))
    goto no_peer;
  GST_OBJECT_UNLOCK (pad);

  GST_CAT_DEBUG_OBJECT (GST_CAT_PROPERTIES, pad, "getting allowed caps");
//查询src pad的caps
//[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_get_allowed_caps:2819  getting allowed caps
//[Level:5] ../subprojects/gstreamer/gst/gstutils.c:gst_pad_query_caps:3102  get pad caps with filter (NULL)
//[Level:5] ../subprojects/gstreamer/gst/gstquery.c:gst_query_new_custom:679  creating new query 0x7f34780030f0 caps
//[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query:4170  doing query 0x7f34780030f0 (caps)
//[Level:5] ../subprojects/gstreamer/libs/gst/base/gstbaseparse.c:gst_base_parse_src_query:1660  caps query: caps query: 0x7f34780030f0, GstQueryCaps, filter=(GstCaps)"NULL", caps=(GstCaps)"NULL";
//[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query_caps_default:3256  query caps caps query: 0x7f34780030f0, GstQueryCaps, filter=(GstCaps)"NULL", caps=(GstCaps)"NULL";

//[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query_caps_default:3274  fixed pad caps: trying pad caps
//[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query_caps_default:3280  trying pad template caps
//[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query_caps_default:3309  using caps 0x5630fc4a2a30 video/x-h264, parsed=(boolean)true, stream-format=(string){ avc, avc3, byte-stream }, alignment=(string){ au, nal }
//[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query_default:3511  not forwarding 0x7f34780030f0 (caps) query
//[Level:6] ../subprojects/gstreamer/libs/gst/base/gstbaseparse.c:gst_base_parse_src_query:1668  caps query result: 1 caps query: 0x7f34780030f0, GstQueryCaps, filter=(GstCaps)"NULL", caps=(GstCaps)"video/x-h264\,\ parsed\=\(boolean\)true\,\ stream-format\=\(string\)\{\ avc\,\ avc3\,\ byte-stream\ \}\,\ alignment\=\(string\)\{\ au\,\ nal\ \}";
//[Level:5] ../subprojects/gstreamer/gst/gstpad.c:gst_pad_query:4193  sent query 0x7f34780030f0 (caps), result 1
//[Level:5] ../subprojects/gstreamer/gst/gstutils.c:gst_pad_query_caps:3109  query returned video/x-h264, parsed=(boolean)true, stream-format=(string){ avc, avc3, byte-stream }, alignment=(string){ au, nal }
  mycaps = gst_pad_query_caps (pad, NULL);

  /* Query peer caps */
  //继续向下查询
  query = gst_query_new_caps (mycaps);
  if (!gst_pad_peer_query (pad, query)) {
    GST_CAT_DEBUG_OBJECT (GST_CAT_CAPS, pad, "Caps query failed");
    goto end;
  }

  gst_query_parse_caps_result (query, &caps);
  if (caps == NULL) {
    g_warn_if_fail (caps != NULL);
    goto end;
  }
  gst_caps_ref (caps);

  GST_CAT_DEBUG_OBJECT (GST_CAT_CAPS, pad, "allowed caps %" GST_PTR_FORMAT,
      caps);

end:
  gst_query_unref (query);
  gst_caps_unref (mycaps);

  return caps;

no_peer:
  {
    GST_CAT_DEBUG_OBJECT (GST_CAT_PROPERTIES, pad, "no peer");
    GST_OBJECT_UNLOCK (pad);

    return NULL;
  }
}

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63

    gst_pad_peer_query（）这个函数，会从h264parse开始查询，逐步进入：

    gst_video_decoder_sink_getcaps()
	st_video_decoder_proxy_getcaps (decoder, NULL, filter)
		__gst_video_element_proxy_getcaps()
				gst_pad_peer_query_caps (srcpad, NULL);//这里查询covert的sink了
			
	

    • 1
    • 2
    • 3
    • 4
    • 5
    • 6

    （未完待续）

    h264parse srcpad查询，应该是decoder的sinkpad 收到这个pad

    所以接下来我们直接看h264-parse:sink pad的event函数就可以了，

      parse_class->sink_event = GST_DEBUG_FUNCPTR (gst_h264_parse_event);
  parse_class->src_event = GST_DEBUG_FUNCPTR (gst_h264_parse_src_event);

    • 1
    • 2

    总结：gstbasesrc从gst_app_src_do_negotiate (bsrc)，发送caps event结束后，再次进入到gst_app_src_create（）函数，
    接着到：gst_app_src_emit_need_data（）告诉app用户需要数据，用户开始发送数据。

  • 相关阅读:
    java火焰图收集
    Zabbix使用手册
    手把手教你分析MySQL查询性能瓶颈，包教包会
    java毕业设计软式装修设计网站mp4（附源码、数据库）
    计算机毕业设计springboot+vue基本微信小程序的健康管理系统
    【C语言练习——打印空心正方形及其变形】
    vxe-table 打包部署上线，校验样式失效
    Linux安装jdk1.8教程（服务器可以访问网络）
    MyBatis的一级缓存和二级缓存
    【手把手教你写服务器】监听端口功能的实现、epoll技术概述
  • 原文地址：https://blog.csdn.net/weixin_43360707/article/details/127587197