音视频从入门到精通——FFmpeg结构体：AVStream分析

FFmpeg结构体：AVStream分析

概述

AVStream是存储每一个视频/音频流信息的结构体。本文分析一下该结构体里重要变量的含义和作用。该结构体位于avformat.h中，首先看一下结构体的定义：

代码

/**
 * Stream structure.
 * New fields can be added to the end with minor version bumps.
 * Removal, reordering and changes to existing fields require a major
 * version bump.
 * sizeof(AVStream) must not be used outside libav*.
 */
typedef struct AVStream {
#if FF_API_AVSTREAM_CLASS
    /**
     * A class for @ref avoptions. Set on stream creation.
     */
    const AVClass *av_class;
#endif

    int index;    /**< stream index in AVFormatContext */
    /**
     * Format-specific stream ID.
     * decoding: set by libavformat
     * encoding: set by the user, replaced by libavformat if left unset
     */
    int id;

    void *priv_data;

    /**
     * This is the fundamental unit of time (in seconds) in terms
     * of which frame timestamps are represented.
     *
     * decoding: set by libavformat
     * encoding: May be set by the caller before avformat_write_header() to
     *           provide a hint to the muxer about the desired timebase. In
     *           avformat_write_header(), the muxer will overwrite this field
     *           with the timebase that will actually be used for the timestamps
     *           written into the file (which may or may not be related to the
     *           user-provided one, depending on the format).
     */
    AVRational time_base;

    /**
     * Decoding: pts of the first frame of the stream in presentation order, in stream time base.
     * Only set this if you are absolutely 100% sure that the value you set
     * it to really is the pts of the first frame.
     * This may be undefined (AV_NOPTS_VALUE).
     * @note The ASF header does NOT contain a correct start_time the ASF
     * demuxer must NOT set this.
     */
    int64_t start_time;

    /**
     * Decoding: duration of the stream, in stream time base.
     * If a source file does not specify a duration, but does specify
     * a bitrate, this value will be estimated from bitrate and file size.
     *
     * Encoding: May be set by the caller before avformat_write_header() to
     * provide a hint to the muxer about the estimated duration.
     */
    int64_t duration;

    int64_t nb_frames;                 ///< number of frames in this stream if known or 0

    /**
     * Stream disposition - a combination of AV_DISPOSITION_* flags.
     * - demuxing: set by libavformat when creating the stream or in
     *             avformat_find_stream_info().
     * - muxing: may be set by the caller before avformat_write_header().
     */
    int disposition;

    enum AVDiscard discard; ///< Selects which packets can be discarded at will and do not need to be demuxed.

    /**
     * sample aspect ratio (0 if unknown)
     * - encoding: Set by user.
     * - decoding: Set by libavformat.
     */
    AVRational sample_aspect_ratio;

    AVDictionary *metadata;

    /**
     * Average framerate
     *
     * - demuxing: May be set by libavformat when creating the stream or in
     *             avformat_find_stream_info().
     * - muxing: May be set by the caller before avformat_write_header().
     */
    AVRational avg_frame_rate;

    /**
     * For streams with AV_DISPOSITION_ATTACHED_PIC disposition, this packet
     * will contain the attached picture.
     *
     * decoding: set by libavformat, must not be modified by the caller.
     * encoding: unused
     */
    AVPacket attached_pic;

    /**
     * An array of side data that applies to the whole stream (i.e. the
     * container does not allow it to change between packets).
     *
     * There may be no overlap between the side data in this array and side data
     * in the packets. I.e. a given side data is either exported by the muxer
     * (demuxing) / set by the caller (muxing) in this array, then it never
     * appears in the packets, or the side data is exported / sent through
     * the packets (always in the first packet where the value becomes known or
     * changes), then it does not appear in this array.
     *
     * - demuxing: Set by libavformat when the stream is created.
     * - muxing: May be set by the caller before avformat_write_header().
     *
     * Freed by libavformat in avformat_free_context().
     *
     * @see av_format_inject_global_side_data()
     */
    AVPacketSideData *side_data;
    /**
     * The number of elements in the AVStream.side_data array.
     */
    int            nb_side_data;

    /**
     * Flags indicating events happening on the stream, a combination of
     * AVSTREAM_EVENT_FLAG_*.
     *
     * - demuxing: may be set by the demuxer in avformat_open_input(),
     *   avformat_find_stream_info() and av_read_frame(). Flags must be cleared
     *   by the user once the event has been handled.
     * - muxing: may be set by the user after avformat_write_header(). to
     *   indicate a user-triggered event.  The muxer will clear the flags for
     *   events it has handled in av_[interleaved]_write_frame().
     */
    int event_flags;
/**
 * - demuxing: the demuxer read new metadata from the file and updated
 *     AVStream.metadata accordingly
 * - muxing: the user updated AVStream.metadata and wishes the muxer to write
 *     it into the file
 */
#define AVSTREAM_EVENT_FLAG_METADATA_UPDATED 0x0001
/**
 * - demuxing: new packets for this stream were read from the file. This
 *   event is informational only and does not guarantee that new packets
 *   for this stream will necessarily be returned from av_read_frame().
 */
#define AVSTREAM_EVENT_FLAG_NEW_PACKETS (1 << 1)

    /**
     * Real base framerate of the stream.
     * This is the lowest framerate with which all timestamps can be
     * represented accurately (it is the least common multiple of all
     * framerates in the stream). Note, this value is just a guess!
     * For example, if the time base is 1/90000 and all frames have either
     * approximately 3600 or 1800 timer ticks, then r_frame_rate will be 50/1.
     */
    AVRational r_frame_rate;

    /**
     * Codec parameters associated with this stream. Allocated and freed by
     * libavformat in avformat_new_stream() and avformat_free_context()
     * respectively.
     *
     * - demuxing: filled by libavformat on stream creation or in
     *             avformat_find_stream_info()
     * - muxing: filled by the caller before avformat_write_header()
     */
    AVCodecParameters *codecpar;

    /**
     * Number of bits in timestamps. Used for wrapping control.
     *
     * - demuxing: set by libavformat
     * - muxing: set by libavformat
     *
     */
    int pts_wrap_bits;
} AVStream;


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182

分析

AVStream 主要成员

我们来分析几个参数看看

第一个，index

int index;    /**< stream index in AVFormatContext */
1

AVFormatContext中的流索引，用来标识是视频还是音频 一般0是视频，1是音频。实际上没什么用。
判断流是视频还是音频通过，AVMEDIA_TYPE_AUDIO音频，AVMEDIA_TYPE_VIDEO视频来判断。

第二个，time_base

    /**
     * This is the fundamental unit of time (in seconds) in terms
     * of which frame timestamps are represented.
     * 这是表示帧时间戳的基本时间单位（秒）。
     *
     * decoding: set by libavformat
     * encoding: May be set by the caller before avformat_write_header() to
     *           provide a hint to the muxer about the desired timebase. In
     *           avformat_write_header(), the muxer will overwrite this field
     *           with the timebase that will actually be used for the timestamps
     *           written into the file (which may or may not be related to the
     *           user-provided one, depending on the format).
     */
    AVRational time_base;
1
2
3
4
5
6
7
8
9
10
11
12
13
14

看这个AVRational结构体，它在AVStream内大量使用。
有理数是整数和分数的集合 ; 有理数可以用两个整数相除 ( 分数 ) 来表示。所以，AVRational是一个有理数。

/**
 * @defgroup lavu_math_rational AVRational
 * @ingroup lavu_math
 * Rational number calculation.
 *
 * While rational numbers can be expressed as floating-point numbers, the
 * conversion process is a lossy one, so are floating-point operations. On the
 * other hand, the nature of FFmpeg demands highly accurate calculation of
 * timestamps. This set of rational number utilities serves as a generic
 * interface for manipulating rational numbers as pairs of numerators and
 * denominators.
 *
 * Many of the functions that operate on AVRational's have the suffix `_q`, in
 * reference to the mathematical symbol "ℚ" (Q) which denotes the set of all
 * rational numbers.
 *
 */

/**
 * Rational number (pair of numerator and denominator).
 */
typedef struct AVRational{
    int num; ///< Numerator---分子
    int den; ///< Denominator---分母
} AVRational;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

所以，AVStream中的time_base 是一个分数，它比float，double类似更加精确。

第三个，duration

/**
 * Decoding: duration of the stream, in stream time base.
 * If a source file does not specify a duration, but does specify
 * a bitrate, this value will be estimated from bitrate and file size.
 *
 * Encoding: May be set by the caller before avformat_write_header() to
 * provide a hint to the muxer about the estimated duration.
 */
解码：流的持续时间，流时基。
如果源文件未指定持续时间，但指定
比特率，该值将根据比特率和文件大小估计。

编码：可以由调用方在avformat_write_header（）之前设置为
向多路复用器提供有关估计持续时间的提示。

    int64_t  duration;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

流的持续时间，这个数的值不是秒，而是一个以time base为基准的统计上的数量。大概是可以通过这个数算出这个视频，或者音频的时长。
计算方法如下

视频总时长(毫秒) 
duration * ( (double)time_base.num / (double)time_base.den ) * 1000

视频总时长(秒) 
duration * ( (double)time_base.num / (double)time_base.den ) 
1
2
3
4
5

注意，time_base.den有可能为0，这里需要写一个函数来判断。

第四个，codec

AVCodecContext *codec ，指向该视频/音频流的AVCodecContext（它们是一一对应的关系）。
这个参数在AVStream代码里没有找到，它已经被第五个参数codecpar代替了。

第五个，codecpar

    /**
     * Codec parameters associated with this stream. Allocated and freed by
     * libavformat in avformat_new_stream() and avformat_free_context()
     * respectively.
     *
     * - demuxing: filled by libavformat on stream creation or in
     *             avformat_find_stream_info()
     * - muxing: filled by the caller before avformat_write_header()
     */
    AVCodecParameters *codecpar;
1
2
3
4
5
6
7
8
9
10

音视频参数，这个参数是用来替代上面的codec的。

因为AVCodecContext结构体包含的参数太多，AVCodecParameters将编码器的参数从AVCodecContext分离出来，AVCodecParameters结构体中部分重要的参数如下：

enum AVMediaType codec_type; 编解码器的类型 
const struct AVCodec *codec; 编解码器,初始化后不可更改
enum AVCodecID codec_id; 编解码器的id
int64_t bit_rate; 平均比特率
uint8_t *extradata; int extradata_size; 针对特定编码器包含的附加信息
AVRational time_base; 根据该参数可以将pts转化为实践
int width, height; 每一帧的宽和高
int gop_size; 一组图片的数量，编码时用户设置，解码时不使用
enum AVPixelFormat pix_fmt; 像素格式，编码时用户设置，解码时可由用户指定，但是在分析数据会被覆盖用户的设置
int refs; 参考帧的数量
enum AVColorSpace colorspace; YUV色彩空间类型
enum AVColorRange color_range; MPEG JPEG YUV范围
int sample_rate; 采样率 仅音频
int channels; 声道数（音频）
enum AVSampleFormat sample_fmt; //采样格式
int frame_size; 每个音频帧中每个声道的采样数量
int profile;配置类型
int level;级别
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

第六个,avg_frame_rate

avg_frame_rate表示，平均帧率。

    /**
     * Average framerate
     *
     * - demuxing: May be set by libavformat when creating the stream or in
     *             avformat_find_stream_info().
     * - muxing: May be set by the caller before avformat_write_header().
     */
    AVRational avg_frame_rate;
1
2
3
4
5
6
7
8

帧率 ( FPS ) : 单位时间内 ( 1 秒 ) , 需要显示的图像个数 , 单位是 Hz ;
计算方法

//获取视频的 FPS 帧率 ( 1秒中播放的帧数 )

AVRational frame_rate = stream->avg_frame_rate;

// AVRational 结构体由一个分子和分母组成 , 分子 / 分母就是 fps
//  也可以使用 av_q2d() 方法传入 AVRational 结构体进行计算
//  上面两种方法都可以获取 帧率 ( FPS )
//      FPS 的值不是固定的 , 随着视频播放 , 其帧率也会随之改变
int fps = frame_rate.num / frame_rate.den;
//int fps = av_q2d(frame_rate);
1
2
3
4
5
6
7
8
9
10

如果 FPS 为 100Hz , 那么1 秒钟绘制 100 张画面 , 每隔 10ms 绘制一张图像。

第七个，attached_pic

附带的图片。比如说一些MP3，AAC音频文件附带的专辑封面。

    /**
     * For streams with AV_DISPOSITION_ATTACHED_PIC disposition, this packet
     * will contain the attached picture.
     *
     * decoding: set by libavformat, must not be modified by the caller.
     * encoding: unused
     */
    AVPacket attached_pic;
1
2
3
4
5
6
7
8

其他成员

看附录

附录

int index; 标识该音频/视频流
int id; 流的标识，依赖于具体的容器格式。解码libavformat,编码由用户设置，用户不设置则使用libavformat
AVCodecContext *codec; 流对应与AVCodecContext结构，调用avformat_open_input设置
AVRational time_base; 表示帧时间戳的基本单位。通过该值可以把PTS,DTS转化为真正的时间，其他结构体中也有这个字段，只有AVStream 中的time_base为真正的时间。
int64_t start_time; 流的起始时间，以流的基准时间为单位
int64_t duration; 流的持续时间，如果源文件未指定持续时间，但指定了比特率，则将根据比特率和文件大小估计该值。
int64_t nb_frames; 流中帧的数据，为0或者已知
AVDictionary *metadata; 元数据
AVRational sample_aspect_ratio; 样本长宽比
AVRational avg_frame_rate; 评价帧率，解封装时：在创建流时设置或者在avformat_find_stream_info()中设置；封装时调用avformat_write_header()设置
AVCodecParameters *codecpar; 编解码器参数
int64_t first_dts; 第一个dts
int64_t cur_dts; 当前dts
int probe_packets; 编码器用户probe的包的个数
int codec_info_nb_frames; 在avformat_find_stream_info()期间已经解封装的帧数
AVPacket attached_pic;附带的一些图片
int request_probe; 流探测状态，-1表示完成，0表示没有探测请求，rest执行探测
int skip_to_keyframe; 是否丢弃所有内容直接跳到关键帧
int skip_samples;在下一个数据表解码的帧开始要跳过的采样数
int64_t start_skip_samples;从流的开始要跳过的采样的数量
int64_t first_discard_sample; 如果不是0，从流中丢弃第一个音频的样本
int nb_decoded_frames; 内部解码的帧数

int64_t pts_reorder_error[MAX_REORDER_DELAY+1];
uint8_t pts_reorder_error_count[MAX_REORDER_DELAY+1]; 内部数据从pts到dts

int64_t last_dts_for_order_check; 内部数据用于分析dts和探测mpeg流的错误
uint8_t dts_ordered;
uint8_t dts_misordered;

AVRational display_aspect_ratio; 显示的宽高比
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

参考

FFmpeg时间戳
FFmpeg结构体：AVStream
雷霄骅FFMPEG结构体分析：AVStream
某乎-FFMPEG结构体：AVStream
Android FFMPEG 开发FFMPEG 音视频同步