• 音视频开发：MediaCodec录制MP4文件

    概述

    • 完成音频的硬编码和硬解码

    • 完成视频的硬编码和硬解码

    • 完成音视频的录制MP4

    MediaCodec介绍

    在Android 4.1版本提供了MediaCodec来访问设备的编解码器，它采用的是硬件编解码，所以在速度上比软解码更有优势

    MediaCodec的工作流程

    俩边的Client分别代表输入端和输出端

    使用者输入端用MediaCodec请求一个一个空的ByteBuffer，填充数据后将他传递给MediaCodec去处理

    MediaCodec处理完之后将处理后的数据输出到一个空的ByteBuffer中

    使用者从MediaCodec中获取输出的ByteBuffer，消耗掉里面的数据，使用完输出的ByteBuffer后

    MediaCodec的生命周期

    MediaCodec的声明周期有三种：Stopped、Executing、Released

    Stopped包含三种状态：Uninitialized、Configured、Error

    Executing包含三种状态：Flushed、Running、End-of-Stream

    本文福利， 免费领取C++音视频学习资料包、技术视频，内容包括（音视频开发，面试题，FFmpeg ，webRTC ，rtmp ，hls ，rtsp ，ffplay ，srs）↓↓↓↓↓↓见下面↓↓文章底部点击免费领取↓↓

    Stopped的三种子状态

    Uninitialized：当创建了一个MediaCodec对象，此时处于 Uninitialized状态，可以在任何状态下调用reset方法，返回Uninitialized状态

    Configured：当使用configure(…)方法对MediaCodec进行配置是进入Configured状态

    Error：MediaCodec当遇到错误时进入Error状态，

    Executing的三种子状态

    Flushed：在调用start方法MediaCodec进入Flushed状态，此时MediaCodec拥有所有的缓存，可以在Executing状态的任何时候调用flush()方法进入Flushed状态

    Running：一旦第一个输入队列input buffer被移出队列，MediaCodec就转入Running状态，这种状态占据了MediaCodec的大部分的声明周期，可以通过调用stop返回Uninitialized状态

    End-of-Stream：讲一个带有end-of-stream标志的输入buffer入队列时，MediaCodec将进入End-of-Stream状态，这种状态下MediaCodec不在接收之后的输入buffer，但他依然产生输出buffer，直到End-of-Stream标记输出

    Released

    当使用完MediaCodec之后，调用release方法释放资源，进入最终的Released状态

    MediaCodec API简介

    createDecoderByType/createEncoderByType

    根据特定的MIME类型比如（video/avc）来创建MediaCodec

    createByCodecName

    直到组件的确切名称（OMX.google.mp3.decoder）根据确切的名称创建MediaCodec，组件名称可以根据MediaCodecList类来获取

    configure

    1.  public void configure(
    2.             MediaFormat format,
    3.             Surface surface, MediaCrypto crypto, int flags);

    MediaFormat format：数据所需要的格式，

    Surface surface：用于解码器输出的渲染,如果解码器不生成原始视频输出，或想配置输出解码器的ByteBuffer，则传null

    MediaCrypto crypto：指定一个crypto对象，对媒体数据进行安全解密，对于非安全的解码器传null，

    int flags：当组件是解码器时指定为CONFIGURE_FLAG_ENCODE

    MediaFormat

    封装描述灭提数据格式的信息，以及可选特性的元数据

    媒体数据格式为key/value，key是字符串，值可以是integer、long、float、String或ByteBuffer

    特性元数据指定为string/boolean

    dequeueInputBuffer

    public final int dequeueInputBuffer(long timeoutUs)

    返回用于填充有效数据输入buffer的索引，如果当前没有可用buffer则返回-1

    long timeoutUs：用于等待返回可用buffer的时间

    timeoutUs == 0立马返回

    timeoutUs < 0无限期等待可用buffer

    timeoutUs > 0等待timeoutUs时间

    queueInputBuffer

    将填充好的buffer发给MediaCodec

    1.     public native final void queueInputBuffer(
    2.             int index,
    3.             int offset, int size, long presentationTimeUs, int flags)

     int index:dequeueInputBuffer方法返回的索引

    int offset:开始输入时buffer的偏移量

    int size:有效的输入字节数

    long presentationTimeUs：此buffer的PTS(以微秒为单位)。

    int flags:这个目前我也不太理解,我直接传 0

    dequeueOutputBuffer

    从MediaCodec中获取输出buffer

    1.     public final int dequeueOutputBuffer(
    2.             @NonNull BufferInfo info, long timeoutUs)

    返回值

    返回INFO_TRY_AGAIN_LATER而timeoutUs指定为了非负值，表示超时了。

    返回INFO_OUTPUT_FORMAT_CHANGED表示输出格式已更改，后续数据将遵循新格式。

    参数：

    BufferInfo info：输出buffer的metadata

    long timeoutUs：跟上方一样超时时间

    releaseOutputBuffer

    使用此方法把输出的buffer返回给codec或者渲染到surface上

    1. public void releaseOutputBuffer (int index, 
    2.                 boolean render)

    boolean render:如果codec配置了有效的surface，当这个参数为true时，则把数据渲染到surface上，一旦不使用此buffer，surafce将buffer返还给codec

    实现

    直接上代码了

    视频数据nv21硬编码为h264

    1. public class CameraToH264 {
    2.  
    3.     public ArrayBlockingQueue<byte[]> yuv420Queue = new ArrayBlockingQueue<>(10);
    4.     private boolean isRuning;
    5.     private byte[] input;
    6.     private int width;
    7.     private int height;
    8.     private MediaCodec mediaCodec;
    9.     private MediaMuxer mediaMuxer;
    10.     private int mVideoTrack=-1;
    11.     private long nanoTime;
    12.  
    13.  
    14.     public void init(int width, int heigth) {
    15.         nanoTime = System.nanoTime();
    16.         this.width = width;
    17.         this.height = heigth;
    18.         MediaFormat videoFormat = MediaFormat.createVideoFormat("video/avc", width, heigth);
    19.         videoFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420SemiPlanar);
    20.         videoFormat.setInteger(MediaFormat.KEY_FRAME_RATE, 30);
    21.         videoFormat.setInteger(MediaFormat.KEY_BIT_RATE, width * heigth * 5);
    22.         videoFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 1);
    23.  
    24.         try {
    25.             mediaCodec = MediaCodec.createEncoderByType("video/avc");
    26.             mediaCodec.configure(videoFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
    27.             mediaCodec.start();
    28.  
    29.             mediaMuxer = new MediaMuxer("sdcard/aaapcm/camer1.mp4", MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
    30.         } catch (IOException e) {
    31.             e.printStackTrace();
    32.         }
    33.     }
    34.  
    35.  
    36.     public void putData(byte[] buffer) {
    37.         if (yuv420Queue.size() >= 10) {
    38.             yuv420Queue.poll();
    39.         }
    40.         yuv420Queue.add(buffer);
    41.     }
    42.  
    43.  
    44.     public void startEncoder() {
    45.         new Thread(new Runnable() {
    46.             @Override
    47.             public void run() {
    48.                 isRuning = true;
    49.                 while (isRuning) {
    50.                     if (yuv420Queue.size() > 0) {
    51.                         input = yuv420Queue.poll();
    52.                         byte[] yuv420sp = new byte[width * height * 3 / 2];
    53.                         // 必须要转格式，否则录制的内容播放出来为绿屏
    54.                         NV21ToNV12(input, yuv420sp, width, height);
    55.                         input = yuv420sp;
    56.                     } else {
    57.                         input = null;
    58.                     }
    59.  
    60.                     if (input != null) {
    61.                         ByteBuffer[] inputBuffers = mediaCodec.getInputBuffers();
    62.                         ByteBuffer[] outputBuffers = mediaCodec.getOutputBuffers();
    63.                         int inputBufferIndex = mediaCodec.dequeueInputBuffer(0);
    64.                         if (inputBufferIndex >= 0) {
    65.                             ByteBuffer inputBuffer = inputBuffers[inputBufferIndex];
    66.                             inputBuffer.clear();
    67.                             inputBuffer.put(input);
    68.                             mediaCodec.queueInputBuffer(inputBufferIndex, 0, input.length, (System.nanoTime() - nanoTime) / 1000, 0);
    69.                         }
    70.  
    71.                         MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
    72.                         int outputBufferIndex = mediaCodec.dequeueOutputBuffer(bufferInfo, 0);
    73.  
    74.                         if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
    75.                             mVideoTrack = mediaMuxer.addTrack(mediaCodec.getOutputFormat());
    76.                             Log.d("mmm", "改变format");
    77.                             if (mVideoTrack >= 0) {
    78.                                 mediaMuxer.start();
    79.                                 Log.d("mmm", "开始混合");
    80.                             }
    81.                         }
    82.                         while (outputBufferIndex > 0) {
    83.                             ByteBuffer outputBuffer = outputBuffers[outputBufferIndex];
    84.                             if (mVideoTrack >= 0) {
    85.                                 mediaMuxer.writeSampleData(mVideoTrack, outputBuffer, bufferInfo);
    86.                                 Log.d("mmm", "正在写入");
    87.                             }
    88.  
    89.                             mediaCodec.releaseOutputBuffer(outputBufferIndex, false);
    90.                             outputBufferIndex = mediaCodec.dequeueOutputBuffer(bufferInfo, 0);
    91.                             if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
    92.                                 Log.e("mmm", "video end");
    93.                             }
    94.                         }
    95.  
    96.                     }
    97.                 }
    98.  
    99.                 Log.d("mmm", "停止写入");
    100.                 mediaMuxer.stop();
    101.                 mediaMuxer.release();
    102.                 mediaCodec.stop();
    103.                 mediaCodec.release();
    104.             }
    105.         }).start();
    106.     }
    107.  
    108.     public void stop() {
    109.         isRuning = false;
    110.     }
    111.  
    112.  
    113.     private void NV21ToNV12(byte[] nv21, byte[] nv12, int width, int height) {
    114.         if (nv21 == null || nv12 == null) return;
    115.         int framesize = width * height;
    116.         int i = 0, j = 0;
    117.         System.arraycopy(nv21, 0, nv12, 0, framesize);
    118.         for (i = 0; i < framesize; i++) {
    119.             nv12[i] = nv21[i];
    120.         }
    121.         for (j = 0; j < framesize / 2; j += 2) {
    122.             nv12[framesize + j - 1] = nv21[j + framesize];
    123.         }
    124.         for (j = 0; j < framesize / 2; j += 2) {
    125.             nv12[framesize + j] = nv21[j + framesize - 1];
    126.         }
    127.     }
    128.  
    129.  
    130. }

    putData方法，是外部输入视频数据，我们这里是从相机拿到的数据，其格式为nv21

    init方法：初始化MediaFormat，为编码器设置帧率，码率等关键参数，初始化MediaCodec用于编码，初始化MediaMuxer用于合成MP4文件

    NV21ToNV12：转换nv21格式为nv12格式

    音频数据pcm编码为aac

    1. public class PcmToAAC {
    2.  
    3.     private MediaCodec encoder;
    4.     private MediaMuxer mediaMuxer;
    5.     private boolean isRun;
    6.     private int mAudioTrack;
    7.     private long prevOutputPTSUs;
    8.     private Queue queue;
    9.  
    10.  
    11.     public void init() {
    12.  
    13.         queue = new Queue();
    14.         queue.init(1024 * 100);
    15.         try {
    16.             MediaFormat audioFormat = MediaFormat.createAudioFormat(MediaFormat.MIMETYPE_AUDIO_AAC, 16000, 1);
    17.             //设置比特率
    18.             audioFormat.setInteger(MediaFormat.KEY_BIT_RATE, 64000);
    19.             audioFormat.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectLC);
    20.             audioFormat.setInteger(MediaFormat.KEY_CHANNEL_COUNT, 1);
    21.             audioFormat.setInteger(MediaFormat.KEY_SAMPLE_RATE, 16000);
    22.  
    23.             encoder = MediaCodec.createEncoderByType(MediaFormat.MIMETYPE_AUDIO_AAC);
    24.             encoder.configure(audioFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
    25.             encoder.start();
    26.  
    27.             mediaMuxer = new MediaMuxer("sdcard/pcm.aac", MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
    28.  
    29.  
    30.             Log.d("mmm", "accinit");
    31.         } catch (IOException e) {
    32.             e.printStackTrace();
    33.         }
    34.  
    35.     }
    36.  
    37.     public void putData(byte[] buffer, int len) {
    38.         queue.addAll(buffer);
    39.     }
    40.  
    41.     public void start() {
    42.         isRun = true;
    43.         Log.d("mmm", "aacstart");
    44.         new Thread(new Runnable() {
    45.             @Override
    46.             public void run() {
    47.                 ByteBuffer[] inputBuffers = encoder.getInputBuffers();
    48.                 ByteBuffer[] outputBuffers = encoder.getOutputBuffers();
    49.                 while (isRun) {
    50.                     byte[] bytes = new byte[640];
    51.                     int all = queue.getAll(bytes, 640);
    52.                     if (all < 0) {
    53.                         try {
    54.                             Thread.sleep(50);
    55.                             continue;
    56.                         } catch (InterruptedException e) {
    57.                             e.printStackTrace();
    58.                         }
    59.                     }
    60.                     int inputBufferindex = encoder.dequeueInputBuffer(0);
    61.                     if (inputBufferindex > 0) {
    62.                         ByteBuffer inputBuffer = inputBuffers[inputBufferindex];
    63.                         inputBuffer.clear();
    64.                         inputBuffer.put(bytes);
    65.                         encoder.queueInputBuffer(inputBufferindex, 0, bytes.length, getPTSUs(), 0);
    66.                     }
    67.  
    68.                     MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
    69.  
    70.                     int outputBufferindex = encoder.dequeueOutputBuffer(bufferInfo, 0);
    71.                     if (outputBufferindex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
    72.                         Log.d("mmm", "改变format");
    73.                         mAudioTrack = mediaMuxer.addTrack(encoder.getOutputFormat());
    74.                         if (mAudioTrack >= 0) {
    75.                             mediaMuxer.start();
    76.                             Log.d("mmm", "开始混合");
    77.                         }
    78.                     } else if (outputBufferindex == MediaCodec.INFO_TRY_AGAIN_LATER) {
    79.                         Log.d("mmm", "try-later");
    80.                     }
    81.                     while (outputBufferindex > 0) {
    82.  
    83.                         if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0) {
    84.                             bufferInfo.size = 0;
    85.                         }
    86.                         bufferInfo.presentationTimeUs = getPTSUs();
    87.                         ByteBuffer outputBuffer = outputBuffers[outputBufferindex];
    88. //                            outputBuffer.position(bufferInfo.offset);
    89.                         if (mAudioTrack >= 0) {
    90.                             mediaMuxer.writeSampleData(mAudioTrack, outputBuffer, bufferInfo);
    91.                             Log.d("mmm", "写入文件");
    92.                         }
    93.                         prevOutputPTSUs = bufferInfo.presentationTimeUs;
    94.                         encoder.releaseOutputBuffer(outputBufferindex, false);
    95.                         outputBufferindex = encoder.dequeueOutputBuffer(bufferInfo, 0);
    96.                     }
    97.  
    98.                 }
    99.                 Log.d("mmm", "aacstop");
    100.                 encoder.stop();
    101.                 encoder.release();
    102.                 mediaMuxer.stop();
    103.                 mediaMuxer.release();
    104.             }
    105.         }).start();
    106.     }
    107.  
    108.     public void stop() {
    109.         isRun = false;
    110.     }
    111.  
    112.  
    113.     private long getPTSUs() {
    114.         long result = System.nanoTime() / 1000L;
    115.         // presentationTimeUs should be monotonic
    116.         // otherwise muxer fail to write
    117.         if (result < prevOutputPTSUs)
    118.             result = (prevOutputPTSUs - result) + result;
    119.         return result;
    120.     }
    121. }

    putData方法，是外部输入pcm音频数据，我们这里是从AudioRecoder拿到的pcm数据

    init方法：初始化MediaFormat，为编码器设置声道数和采样率等关键参数，初始化MediaCodec用于编码，初始化MediaMuxer用于合成wav文件

    本文福利， 免费领取C++音视频学习资料包、技术视频，内容包括（音视频开发，面试题，FFmpeg ，webRTC ，rtmp ，hls ，rtsp ，ffplay ，srs）↓↓↓↓↓↓见下面↓↓文章底部点击免费领取↓↓

    采集音频和视频数据，合成MP4音视频文件

    1.  
    2.  
    3. public class AudioThread extends Thread {
    4.  
    5.     private static final int TIMEOUT_USEC = 10000;
    6.     private static final String MIME_TYPE = "audio/mp4a-latm";
    7.     private static final int SAMPLE_RATE = 16000;
    8.     private static final int BIT_RATE = 64000;
    9.  
    10.     private MediaFormat audioFormat;
    11.     private MediaCodec mMediaCodec;
    12.     private final Queue queue;
    13.     private boolean isRun;
    14.     private long prevOutputPTSUs;
    15.     private MuxerThread muxerThread;
    16.     private MediaMuxer mediaMuxer;
    17.     private int audiotrack;
    18.  
    19.     public AudioThread(MuxerThread muxerThread) {
    20.         Log.d("mmm", "AudioThread");
    21.         this.muxerThread = muxerThread;
    22.         queue = new Queue();
    23.         queue.init(1024 * 100);
    24.         preper();
    25.     }
    26.  
    27.     private void preper() {
    28.         audioFormat = MediaFormat.createAudioFormat(MIME_TYPE, SAMPLE_RATE, 1);
    29.         audioFormat.setInteger(MediaFormat.KEY_BIT_RATE, BIT_RATE);
    30.         audioFormat.setInteger(MediaFormat.KEY_CHANNEL_COUNT, 1);
    31.         audioFormat.setInteger(MediaFormat.KEY_SAMPLE_RATE, SAMPLE_RATE);
    32.  
    33.         try {
    34.             mMediaCodec = MediaCodec.createEncoderByType(MIME_TYPE);
    35.             mMediaCodec.configure(audioFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
    36.             mMediaCodec.start();
    37.             Log.d("mmm", "preper");
    38.  
    39.         } catch (IOException e) {
    40.             e.printStackTrace();
    41.         }
    42.  
    43.         isRun = true;
    44.     }
    45.  
    46.     public void addAudioData(byte[] data) {
    47.         if (!isRun) return;
    48.         queue.addAll(data);
    49.     }
    50.  
    51.     public void audioStop() {
    52.         isRun = false;
    53.     }
    54.  
    55.  
    56.     @Override
    57.     public void run() {
    58.         while (isRun) {
    59.             byte[] bytes = new byte[640];
    60.             int all = queue.getAll(bytes, 640);
    61.             if (all < 0) {
    62.                 try {
    63.                     Thread.sleep(50);
    64.                     continue;
    65.                 } catch (InterruptedException e) {
    66.                     e.printStackTrace();
    67.                 }
    68.             }
    69.  
    70.             ByteBuffer[] inputBuffers = mMediaCodec.getInputBuffers();
    71.             ByteBuffer[] outputBuffers = mMediaCodec.getOutputBuffers();
    72.  
    73.             int inputBufferIndex = mMediaCodec.dequeueInputBuffer(TIMEOUT_USEC);
    74.             if (inputBufferIndex > 0) {
    75.                 ByteBuffer inputBuffer = inputBuffers[inputBufferIndex];
    76.                 inputBuffer.clear();
    77.                 inputBuffer.put(bytes);
    78.                 mMediaCodec.queueInputBuffer(inputBufferIndex, 0, bytes.length, getPTSUs(), 0);
    79.             }
    80.  
    81.  
    82.             MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
    83.             int outputBufferIndex = mMediaCodec.dequeueOutputBuffer(bufferInfo, TIMEOUT_USEC);
    84.  
    85.             do {
    86.                 if (outputBufferIndex == MediaCodec.INFO_TRY_AGAIN_LATER) {
    87.                 } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
    88.                     outputBuffers = mMediaCodec.getOutputBuffers();
    89.                 } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
    90.                     Log.e("mmm", "audioINFO_OUTPUT_FORMAT_CHANGED");
    91.                     MediaFormat format = mMediaCodec.getOutputFormat(); // API >= 16
    92.                     if (muxerThread != null) {
    93.                         Log.e("mmm", "添加音轨 INFO_OUTPUT_FORMAT_CHANGED " + format.toString());
    94.                         muxerThread.addTrackIndex(MuxerThread.TRACK_AUDIO, format);
    95.                     }
    96.  
    97.  
    98.  
    99.                 } else if (outputBufferIndex < 0) {
    100.                     Log.e("mmm", "encoderStatus < 0");
    101.                 } else {
    102.                     final ByteBuffer encodedData = outputBuffers[outputBufferIndex];
    103.                     if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0) {
    104.                         bufferInfo.size = 0;
    105.                     }
    106.  
    107.  
    108.                     if (bufferInfo.size != 0 && muxerThread != null && muxerThread.isStart()) {
    109.                         bufferInfo.presentationTimeUs = getPTSUs();
    110.                         muxerThread.addMuxerData(new MuxerData(MuxerThread.TRACK_AUDIO, encodedData, bufferInfo));
    111.                         prevOutputPTSUs = bufferInfo.presentationTimeUs;
    112.                     }
    113.                     mMediaCodec.releaseOutputBuffer(outputBufferIndex, false);
    114.                 }
    115.                 outputBufferIndex = mMediaCodec.dequeueOutputBuffer(bufferInfo, TIMEOUT_USEC);
    116.  
    117.             } while (outputBufferIndex >= 0);
    118.         }
    119.  
    120.         mMediaCodec.stop();
    121.         mMediaCodec.release();
    122.         Log.d("mmm", "audiomMediaCodec");
    123.     }
    124.  
    125.     private long getPTSUs() {
    126.         long result = System.nanoTime() / 1000L;
    127.         // presentationTimeUs should be monotonic
    128.         // otherwise muxer fail to write
    129.         if (result < prevOutputPTSUs)
    130.             result = (prevOutputPTSUs - result) + result;
    131.         return result;
    132.     }
    133. }
    1.  
    2.  
    3. public class VideoThread extends Thread {
    4.  
    5.     private final MuxerThread muxerThread;
    6.     private final int mWidth;
    7.     private final int mHeigth;
    8.  
    9.     public static final int IMAGE_HEIGHT = 1080;
    10.     public static final int IMAGE_WIDTH = 1920;
    11.  
    12.     // 编码相关参数
    13.     private static final String MIME_TYPE = "video/avc"; // H.264 Advanced Video
    14.     private static final int FRAME_RATE = 25; // 帧率
    15.     private static final int IFRAME_INTERVAL = 10; // I帧间隔（GOP）
    16.     private static final int TIMEOUT_USEC = 10000; // 编码超时时间
    17.  
    18.     private static final int COMPRESS_RATIO = 256;
    19.     private static final int BIT_RATE = IMAGE_HEIGHT * IMAGE_WIDTH * 3 * 8 * FRAME_RATE / COMPRESS_RATIO; // bit rate CameraWrapper.
    20.     private final Vector<byte[]> frameBytes;
    21.     private final byte[] mFrameData;
    22.     private MediaFormat mediaFormat;
    23.     private MediaCodec mMediaCodec;
    24.     private boolean isRun;
    25.     private int videoTrack;
    26.  
    27.  
    28.     public VideoThread(int width, int heigth, MuxerThread muxerThread) {
    29.         Log.d("mmm", "VideoThread");
    30.         this.muxerThread = muxerThread;
    31.         this.mWidth = width;
    32.         this.mHeigth = heigth;
    33.         mFrameData = new byte[this.mWidth * this.mHeigth * 3 / 2];
    34.         frameBytes = new Vector<byte[]>();
    35.         preper();
    36.     }
    37.  
    38.     private void preper() {
    39.         mediaFormat = MediaFormat.createVideoFormat(MIME_TYPE, mWidth, mHeigth);
    40.         mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, BIT_RATE);
    41.         mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, FRAME_RATE);
    42.         mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420SemiPlanar);
    43.         mediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, IFRAME_INTERVAL);
    44.  
    45.         try {
    46.             mMediaCodec = MediaCodec.createEncoderByType("video/avc");
    47.             mMediaCodec.configure(mediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
    48.             mMediaCodec.start();
    49.             Log.d("mmm", "preper");
    50.         } catch (IOException e) {
    51.             e.printStackTrace();
    52.         }
    53.  
    54.     }
    55.  
    56.     public void add(byte[] data) {
    57.         if (!isRun) return;
    58.         if (frameBytes.size() > 10) {
    59.             frameBytes.remove(0);
    60.         }
    61.         frameBytes.add(data);
    62.     }
    63.  
    64.  
    65.     @Override
    66.     public void run() {
    67.         isRun = true;
    68.         while (isRun) {
    69.             if (!frameBytes.isEmpty()) {
    70.                 byte[] bytes = this.frameBytes.remove(0);
    71.                 Log.e("ang-->", "解码视频数据:" + bytes.length);
    72.  
    73.                 NV21toI420SemiPlanar(bytes, mFrameData, mWidth, mHeigth);
    74.                 ByteBuffer[] outputBuffers = mMediaCodec.getOutputBuffers();
    75.                 ByteBuffer[] inputBuffers = mMediaCodec.getInputBuffers();
    76.  
    77.                 int inputBufferindex = mMediaCodec.dequeueInputBuffer(TIMEOUT_USEC);
    78.                 if (inputBufferindex > 0) {
    79.                     ByteBuffer inputBuffer = inputBuffers[inputBufferindex];
    80.                     inputBuffer.clear();
    81.                     inputBuffer.put(mFrameData);
    82.                     mMediaCodec.queueInputBuffer(inputBufferindex, 0, mFrameData.length, System.nanoTime() / 1000, 0);
    83.                 }
    84.  
    85.                 MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
    86.  
    87.                 int outputBufferIndex = mMediaCodec.dequeueOutputBuffer(bufferInfo, TIMEOUT_USEC);
    88.  
    89.                 do {
    90.                     if (outputBufferIndex == MediaCodec.INFO_TRY_AGAIN_LATER) {
    91.  
    92.                     } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
    93.                         outputBuffers = mMediaCodec.getOutputBuffers();
    94.                     } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
    95.                         Log.e("mmm", "videoINFO_OUTPUT_FORMAT_CHANGED");
    96.                         MediaFormat newFormat = mMediaCodec.getOutputFormat();
    97.                         muxerThread.addTrackIndex(MuxerThread.TRACK_VIDEO, newFormat);
    98.                     } else if (outputBufferIndex < 0) {
    99.                         Log.e("mmm", "outputBufferIndex < 0");
    100.                     } else {
    101.                         ByteBuffer outputBuffer = outputBuffers[outputBufferIndex];
    102.                         if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0) {
    103.                             Log.d("mmm", "ignoring BUFFER_FLAG_CODEC_CONFIG");
    104.                             bufferInfo.size = 0;
    105.                         }
    106.  
    107.                         if (bufferInfo.size != 0 && muxerThread.isStart()) {
    108.  
    109.                             muxerThread.addMuxerData(new MuxerData(MuxerThread.TRACK_VIDEO, outputBuffer, bufferInfo));
    110.                         }
    111.                         mMediaCodec.releaseOutputBuffer(outputBufferIndex, false);
    112.                     }
    113.                     outputBufferIndex = mMediaCodec.dequeueOutputBuffer(bufferInfo, TIMEOUT_USEC);
    114.  
    115.                 } while (outputBufferIndex > 0);
    116.             }
    117.         }
    118.  
    119.         mMediaCodec.stop();
    120.         mMediaCodec.release();
    121.         Log.d("mmm","videomMediaCodec");
    122.  
    123.     }
    124.  
    125.     public void stopvideo() {
    126.         isRun = false;
    127.     }
    128.  
    129.     private static void NV21toI420SemiPlanar(byte[] nv21bytes, byte[] i420bytes, int width, int height) {
    130.         System.arraycopy(nv21bytes, 0, i420bytes, 0, width * height);
    131.         for (int i = width * height; i < nv21bytes.length; i += 2) {
    132.             i420bytes[i] = nv21bytes[i + 1];
    133.             i420bytes[i + 1] = nv21bytes[i];
    134.         }
    135.     }
    136. }
    1.  
    2.  
    3. public class MuxerThread extends Thread {
    4.  
    5.     public static final String TRACK_AUDIO = "TRACK_AUDIO";
    6.     public static final String TRACK_VIDEO = "TRACK_VIDEO";
    7.     private Vector<MuxerData> muxerDatas;
    8.     private MediaMuxer mediaMuxer;
    9.     private boolean isAddAudioTrack;
    10.     private boolean isAddVideoTrack;
    11.     private static MuxerThread muxerThread = new MuxerThread();
    12.     private int videoTrack;
    13.     private int audioTrack;
    14.     private VideoThread videoThread;
    15.     private AudioThread audioThread;
    16.     private boolean isRun;
    17.  
    18.  
    19.     private MuxerThread() {
    20.         Log.d("mmm", "MuxerThread");
    21.     }
    22.  
    23.     public static MuxerThread getInstance() {
    24.         return muxerThread;
    25.     }
    26.  
    27.  
    28.     public synchronized void addTrackIndex(String track, MediaFormat format) {
    29.         if (isAddAudioTrack && isAddVideoTrack) {
    30.             return;
    31.         }
    32.  
    33.         if (!isAddVideoTrack && track.equals(TRACK_VIDEO)) {
    34.             Log.e("mmm", "添加视频轨");
    35.             videoTrack = mediaMuxer.addTrack(format);
    36.             if (videoTrack >= 0) {
    37.                 isAddVideoTrack = true;
    38.                 Log.e("mmm", "添加视频轨完成");
    39.             }
    40.         }
    41.  
    42.         if (!isAddAudioTrack && track.equals(TRACK_AUDIO)) {
    43.             Log.e("mmm", "添加音频轨");
    44.             audioTrack = mediaMuxer.addTrack(format);
    45.             if (audioTrack >= 0) {
    46.                 isAddAudioTrack = true;
    47.                 Log.e("mmm", "添加音频轨完成");
    48.             }
    49.         }
    50.  
    51.         if (isStart()) {
    52.             mediaMuxer.start();
    53.         }
    54.     }
    55.  
    56.     public boolean isStart() {
    57.         return isAddAudioTrack && isAddVideoTrack;
    58.     }
    59.  
    60.  
    61.     public void addMuxerData(MuxerData muxerData) {
    62.         muxerDatas.add(muxerData);
    63.     }
    64.  
    65.     public void addVideoData(byte[] data) {
    66.         if (!isRun) return;
    67.         videoThread.add(data);
    68.     }
    69.  
    70.     public void addAudioData(byte[] data) {
    71.         if (!isRun || audioThread == null) return;
    72.         audioThread.addAudioData(data);
    73.     }
    74.  
    75.     public void startMuxer(int width, int height) {
    76.         Log.d("mmm", "startMuxer");
    77.         try {
    78.             mediaMuxer = new MediaMuxer("sdcard/camer111.mp4", MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
    79.             muxerDatas = new Vector<>();
    80.         } catch (IOException e) {
    81.             e.printStackTrace();
    82.         }
    83.         isRun = true;
    84.         videoThread = new VideoThread(width, height, this);
    85.         videoThread.start();
    86.         audioThread = new AudioThread(this);
    87.         audioThread.start();
    88.         start();
    89.     }
    90.  
    91.     public void exit() {
    92.         isRun = false;
    93.         videoThread.stopvideo();
    94.         audioThread.audioStop();
    95.     }
    96.  
    97.     @Override
    98.     public void run() {
    99.         while (isRun) {
    100.             if (!muxerDatas.isEmpty() && isStart()) {
    101.                 MuxerData muxerData = muxerDatas.remove(0);
    102.                 if (muxerData.trackIndex.equals(TRACK_VIDEO) && videoTrack >= 0) {
    103.                     Log.d("mmm", "写入视频" + muxerData.bufferInfo.size);
    104.                     mediaMuxer.writeSampleData(videoTrack, muxerData.byteBuf, muxerData.bufferInfo);
    105.                 }
    106.  
    107.                 if (muxerData.trackIndex.equals(TRACK_AUDIO) && audioTrack >= 0) {
    108.                     Log.d("mmm", "写入音频" + muxerData.bufferInfo.size);
    109.                     mediaMuxer.writeSampleData(audioTrack, muxerData.byteBuf, muxerData.bufferInfo);
    110.                 }
    111.             }
    112.         }
    113.  
    114.         mediaMuxer.stop();
    115.         mediaMuxer.release();
    116.  
    117.         Log.d("mmm", "mediaMuxerstop");
    118.     }
    119.  
    120. }
    121.  
    1.  
    2.  
    3. public class Queue {
    4.  
    5.     private byte[] buffer;
    6.     private int head;
    7.     private int tail;
    8.     private int count;
    9.     private int size;
    10.  
    11.  
    12.     public void init(int n) {
    13.         buffer = new byte[n];
    14.         size = n;
    15.         head = 0;
    16.         tail = 0;
    17.         count = 0;
    18.     }
    19.  
    20.  
    21.     public void add(byte data) {
    22.         if (size == count) {
    23. //            Log.d("mmm", "队列已满");
    24.             get();
    25.         }
    26.  
    27.         if (tail == size) {
    28.             tail = 0;
    29.         }
    30.  
    31.         buffer[tail] = data;
    32.         tail++;
    33.         count++;
    34.     }
    35.  
    36.     public byte get() {
    37.         if (count == 0) {
    38.             Log.d("mmm", "队列为空");
    39.             return -1;
    40.         }
    41.  
    42.         if (head == size) {
    43.             head = 0;
    44.         }
    45.         byte data = buffer[head];
    46.         head++;
    47.         count--;
    48.         return data;
    49.     }
    50.  
    51.  
    52.     public void addAll(byte[] data) {
    53.         synchronized (this) {
    54.             for (byte b : data) {
    55.                 add(b);
    56.             }
    57.         }
    58.     }
    59.  
    60.  
    61.     public int getAll(byte[] data, int len) {
    62.         synchronized (this) {
    63.             if (count < len) {
    64.                 return -1;
    65.             }
    66.             int j = 0;
    67.             for (int i = 0; i < len; i++) {
    68.                 byte b = get();
    69.                 data[i] = b;
    70.                 j++;
    71.             }
    72.  
    73.             return j;
    74.         }
    75.  
    76.     }
    77.  
    78.  
    79. }
    80.  
    1.  
    2. public class MuxerData {
    3.     String trackIndex;
    4.     ByteBuffer byteBuf;
    5.     MediaCodec.BufferInfo bufferInfo;
    6.  
    7.     public MuxerData(String trackIndex, ByteBuffer byteBuf, MediaCodec.BufferInfo bufferInfo) {
    8.         this.trackIndex = trackIndex;
    9.         this.byteBuf = byteBuf;
    10.         this.bufferInfo = bufferInfo;
    11.     }
    12. }

    介绍一下各个类的作用

    AudioThread：负责收集音频pcm，然后硬编码为aac，最后把编码完的数据传递给合成器MuxerThread

    VideoThread：负责收集视频yuv，然后硬编码为h264,最后把编码完的数据给合成器MuxerThread

    Queue：就是一个普通的循环队列，负责装音频数据pcm

    MuxerData：负责封装编码完成的aac和h264

    MuxerThread:负责把编码好的aac和h264,合成音视频Mp4

    硬解码AAC

    1.  public void init1() {
    2.         //音频
    3.         audioExtractor = new MediaExtractor();
    4.         try {
    5.             audioExtractor.setDataSource("/sdcard/camer111.mp4");
    6.             int audioTrack = -1;
    7.  
    8.             int trackCount1 = audioExtractor.getTrackCount();
    9.             for (int i = 0; i < trackCount1; i++) {
    10.                 MediaFormat trackFormat1 = audioExtractor.getTrackFormat(i);
    11.                 String string = trackFormat1.getString(MediaFormat.KEY_MIME);
    12.                 if (string.startsWith("audio/")) {
    13.                     audioTrack = i;
    14.                     Log.d("mmm", "找到轨道" + audioTrack);
    15.                 }
    16.             }
    17.  
    18.             audioExtractor.selectTrack(audioTrack);
    19.  
    20.             audiodecoder = MediaCodec.createDecoderByType(MediaFormat.MIMETYPE_AUDIO_AAC);
    21.             MediaFormat trackFormat1 = audioExtractor.getTrackFormat(audioTrack);
    22.             audiodecoder.configure(trackFormat1, null, null, 0);
    23.             audiodecoder.start();
    24.  
    25.             int bufferSize = AudioTrack.getMinBufferSize(16000, AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT);
    26.             audioTrackplay = new AudioTrack(AudioManager.STREAM_MUSIC, 16000, AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT, bufferSize, AudioTrack.MODE_STREAM);
    27.             audioTrackplay.play();
    28.         } catch (IOException e) {
    29.             e.printStackTrace();
    30.         }
    31.  
    32.     }
    33.     
    34.     
    35.     
    36.      public void start1() {
    37.         new Thread(new Runnable() {
    38.             @Override
    39.             public void run() {
    40.                 while (true) {
    41.                     int inputBufferindex = audiodecoder.dequeueInputBuffer(0);
    42.                     if (inputBufferindex > 0) {
    43.                         ByteBuffer inputBuffer = audiodecoder.getInputBuffer(inputBufferindex);
    44.                         int sampleSize = audioExtractor.readSampleData(inputBuffer, 0);
    45.                         if (sampleSize > 0) {
    46.                             Log.d("mmm", "找到数据渲染pcm");
    47.                             audiodecoder.queueInputBuffer(inputBufferindex, 0, sampleSize, audioExtractor.getSampleTime(), 0);
    48.                             audioExtractor.advance();
    49.                         }
    50.                     }
    51.                     MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
    52.                     int outputBufferIndex = audiodecoder.dequeueOutputBuffer(bufferInfo, 0);
    53.  
    54.  
    55.                     while (outputBufferIndex > 0) {
    56.                         Log.d("mmm", "开始渲染pcm");
    57.                         ByteBuffer outputBuffer = audiodecoder.getOutputBuffer(outputBufferIndex);
    58.                         byte[] bytes = new byte[bufferInfo.size];
    59.                         outputBuffer.get(bytes);
    60.                         outputBuffer.clear();
    61.                         audioTrackplay.write(bytes, 0, bufferInfo.size);
    62.                         audiodecoder.releaseOutputBuffer(outputBufferIndex, true);
    63.                         outputBufferIndex = audiodecoder.dequeueOutputBuffer(bufferInfo, 0);
    64.                     }
    65.  
    66.  
    67.                 }
    68.             }
    69.         }).start();
    70.     }

    逻辑很简单，从MP4文件取出音轨，然后硬解码AAC为pcm，最后用AudioTrack播放pcm

    硬解码h264,并且显示在surface上

    1.     public void init(SurfaceHolder surfaceHolder) {
    2.  
    3.         try {
    4.             //视频
    5.             videoExtractor = new MediaExtractor();
    6.             videoExtractor.setDataSource("/sdcard/camer111.mp4");
    7.             int videoTrack = -1;
    8.  
    9.             int trackCount = videoExtractor.getTrackCount();
    10.             for (int i = 0; i < trackCount; i++) {
    11.                 MediaFormat trackFormat = videoExtractor.getTrackFormat(i);
    12.                 String string = trackFormat.getString(MediaFormat.KEY_MIME);
    13.                 Log.d("mmm", "找到轨道" +string );
    14.                 if (string.startsWith("video/")) {
    15.                     videoTrack = i;
    16.                     Log.d("mmm", "找到轨道video" + videoTrack);
    17.                 }
    18.  
    19.         }
    20.         videoExtractor.selectTrack(videoTrack);
    21.  
    22.         videodecoder = MediaCodec.createDecoderByType(MediaFormat.MIMETYPE_VIDEO_AVC);
    23.         MediaFormat trackFormat = videoExtractor.getTrackFormat(videoTrack);
    24.         videodecoder.configure(trackFormat, surfaceHolder.getSurface(), null, 0);
    25.         videodecoder.start();
    26.  
    27.  
    28.     } catch(
    29.     IOException e)
    30.  
    31.     {
    32.         e.printStackTrace();
    33.     }
    34.  
    35. }
    36.  
    37.  
    38.  
    39.    public void start() {
    40.         isRunning = true;
    41.         new Thread(new Runnable() {
    42.             @Override
    43.             public void run() {
    44.                 while (isRunning) {
    45.                     int inputBufferindex = videodecoder.dequeueInputBuffer(0);
    46.                     if (inputBufferindex > 0) {
    47.                         ByteBuffer inputBuffer = videodecoder.getInputBuffer(inputBufferindex);
    48.                         int sampleSize = videoExtractor.readSampleData(inputBuffer, 0);
    49.                         long sampleTime = videoExtractor.getSampleTime();
    50.                         if (sampleSize > 0) {
    51.                             Log.d("mmm", "找到数据渲染");
    52.                             videodecoder.queueInputBuffer(inputBufferindex, 0, sampleSize, videoExtractor.getSampleTime(), 0);
    53.                             videoExtractor.advance();
    54.                             try {
    55.                                 sleep(30);
    56.                             } catch (InterruptedException e) {
    57.                                 e.printStackTrace();
    58.                             }
    59.                         }
    60.                     }
    61.                     MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
    62.                     int outputBufferIndex = videodecoder.dequeueOutputBuffer(bufferInfo, 0);
    63.  
    64.  
    65.                     while (outputBufferIndex > 0) {
    66.                         Log.d("mmm", "开始渲染");
    67.                         videodecoder.releaseOutputBuffer(outputBufferIndex, true);
    68.                         outputBufferIndex = videodecoder.dequeueOutputBuffer(bufferInfo, 0);
    69.                     }
    70.  
    71.  
    72.                 }
    73.             }
    74.         }).start();
    75.     }

    逻辑很简单，先从MP4中分离出视频轨道，然后取出h264数据解码然后直接显示在surface上

    h264转换为图片

    这部分代码也是网上扒的，做下记录

    1. public class H264ToBitmap {
    2.  
    3.     private final int decodeColorFormat = MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Flexible;
    4.  
    5.     public static final int FILE_TypeI420 = 1;
    6.     public static final int FILE_TypeNV21 = 2;
    7.     public static final int FILE_TypeJPEG = 3;
    8.  
    9.     private int outputImageFileType = FILE_TypeJPEG;
    10.  
    11.     private static final int COLOR_FormatI420 = 1;
    12.     private static final int COLOR_FormatNV21 = 2;
    13.  
    14.  
    15.     public void videoDecode() {
    16.         try {
    17.             MediaExtractor mediaExtractor = new MediaExtractor();
    18.             mediaExtractor.setDataSource("/sdcard/eee.mp4");
    19.             int vidioTrack = selectTrack(mediaExtractor);
    20.             if (vidioTrack < 0) {
    21.                 Log.d("mmm", "未找到正确轨道");
    22.             }
    23.             mediaExtractor.selectTrack(vidioTrack);
    24.             MediaFormat mediaFormat = mediaExtractor.getTrackFormat(vidioTrack);
    25.             String MIME = mediaFormat.getString(MediaFormat.KEY_MIME);
    26.             MediaCodec decoder = MediaCodec.createDecoderByType(MIME);
    27.             showSupportedColorFormat(decoder.getCodecInfo().getCapabilitiesForType(MIME));
    28.  
    29.             if (isColorFormatSupported(decodeColorFormat, decoder.getCodecInfo().getCapabilitiesForType(MIME))) {
    30.                 mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, decodeColorFormat);
    31.             } else {
    32.                 Log.d("mmm", "不支持此类型");
    33.             }
    34.  
    35.             decodeFramesToImage(decoder, mediaExtractor, mediaFormat);
    36.  
    37.         } catch (IOException e) {
    38.             e.printStackTrace();
    39.         }
    40.     }
    41.  
    42.  
    43.     private int selectTrack(MediaExtractor mediaExtractor) {
    44.         int trackCount = mediaExtractor.getTrackCount();
    45.         for (int i = 0; i < trackCount; i++) {
    46.             MediaFormat trackFormat = mediaExtractor.getTrackFormat(i);
    47.             String string = trackFormat.getString(MediaFormat.KEY_MIME);
    48.             if (string.startsWith("video/")) {
    49.                 return i;
    50.             }
    51.         }
    52.  
    53.         return -1;
    54.     }
    55.  
    56.  
    57.     private void showSupportedColorFormat(MediaCodecInfo.CodecCapabilities capabilitiesForType) {
    58.         for (int a : capabilitiesForType.colorFormats) {
    59.             Log.d("mmm", "支持格式" + a + "/");
    60.         }
    61.     }
    62.  
    63.     private boolean isColorFormatSupported(int decodeColorFormat, MediaCodecInfo.CodecCapabilities capabilitiesForType) {
    64.         for (int a : capabilitiesForType.colorFormats) {
    65.             if (a == decodeColorFormat) {
    66.                 return true;
    67.             }
    68.         }
    69.         return false;
    70.     }
    71.  
    72.  
    73.     private void decodeFramesToImage(MediaCodec decoder, MediaExtractor mediaExtractor, MediaFormat mediaFormat) {
    74.         MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
    75.         boolean sawInputEOS = false;
    76.         boolean sawOutputEOS = false;
    77.         decoder.configure(mediaFormat, null, null, 0);
    78.         decoder.start();
    79.         final int width = mediaFormat.getInteger(MediaFormat.KEY_WIDTH);
    80.         final int height = mediaFormat.getInteger(MediaFormat.KEY_HEIGHT);
    81.         int outputFrameCount = 0;
    82.  
    83.         while (!sawOutputEOS) {
    84.             if (!sawInputEOS) {
    85.                 int inputBufferId = decoder.dequeueInputBuffer(0);
    86.                 if (inputBufferId >= 0) {
    87.                     ByteBuffer inputBuffer = decoder.getInputBuffer(inputBufferId);
    88.                     int sampleSize = mediaExtractor.readSampleData(inputBuffer, 0);
    89.                     if (sampleSize >= 0) {
    90.                         long sampleTime = mediaExtractor.getSampleTime();
    91.                         decoder.queueInputBuffer(inputBufferId, 0, sampleSize, sampleTime, 0);
    92.                         mediaExtractor.advance();
    93.                     } else {
    94.                         decoder.queueInputBuffer(inputBufferId, 0, 0, 0L, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
    95.                         sawInputEOS = true;
    96.                     }
    97.  
    98.                 }
    99.             }
    100.  
    101.  
    102.             int outputBufferId = decoder.dequeueOutputBuffer(bufferInfo, 0);
    103.  
    104.             if (outputBufferId >= 0) {
    105.                 if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
    106.                     sawOutputEOS = true;
    107.                 }
    108.  
    109.                 if (bufferInfo.size != 0) {
    110.                     outputFrameCount++;
    111.                     Image outputImage = decoder.getOutputImage(outputBufferId);
    112.                     Log.d("mmm", "imageformat" + outputImage.getFormat());
    113.  
    114.                     if (outputImageFileType != -1) {
    115.                         String fileName;
    116.                         switch (outputImageFileType) {
    117.                             case FILE_TypeI420:
    118.                                 fileName = "/sdcard/aaapcm" + String.format("frame_%05d_I420_%dx%d.yuv", outputFrameCount, width, height);
    119.                                 dumpFile(fileName, getDataFromImage(outputImage, COLOR_FormatI420));
    120.                                 break;
    121.                             case FILE_TypeNV21:
    122.                                 fileName = "/sdcard/aaapcm" + String.format("frame_%05d_NV21_%dx%d.yuv", outputFrameCount, width, height);
    123.                                 dumpFile(fileName, getDataFromImage(outputImage, COLOR_FormatNV21));
    124.                                 break;
    125.                             case FILE_TypeJPEG:
    126.                                 fileName = "/sdcard/aaapcm/" + String.format("frame_%05d.jpg", outputFrameCount);
    127.                                 compressToJpeg(fileName, outputImage);
    128.                                 break;
    129.                         }
    130.  
    131.                     }
    132.  
    133.                     outputImage.close();
    134.                     decoder.releaseOutputBuffer(outputBufferId, false);
    135.                 }
    136.  
    137.             }
    138.         }
    139.     }
    140.  
    141.  
    142.     private byte[] getDataFromImage(Image image, int colorFormat) {
    143.         if (colorFormat != COLOR_FormatI420 && colorFormat != COLOR_FormatNV21) {
    144.             Log.d("mmm", "only support COLOR_FormatI420");
    145.             throw new IllegalArgumentException("only support COLOR_FormatI420 " + "and COLOR_FormatNV21");
    146.         }
    147.         if (!isImageFormatSupported(image)) {
    148.             Log.d("mmm", "不支持图片类型");
    149.             throw new RuntimeException("can't convert Image to byte array, format " + image.getFormat());
    150.         }
    151.         Rect crop = image.getCropRect();
    152.         int format = image.getFormat();
    153.         int width = crop.width();
    154.         int height = crop.height();
    155.         Image.Plane[] planes = image.getPlanes();
    156.         byte[] data = new byte[width * height * ImageFormat.getBitsPerPixel(format) / 8];
    157.         byte[] rowData = new byte[planes[0].getRowStride()];
    158.         int channelOffset = 0;
    159.         int outputStride = 1;
    160.         for (int i = 0; i < planes.length; i++) {
    161.             switch (i) {
    162.                 case 0:
    163.                     channelOffset = 0;
    164.                     outputStride = 1;
    165.                     break;
    166.                 case 1:
    167.                     if (colorFormat == COLOR_FormatI420) {
    168.                         channelOffset = width * height;
    169.                         outputStride = 1;
    170.                     } else if (colorFormat == COLOR_FormatNV21) {
    171.                         channelOffset = width * height + 1;
    172.                         outputStride = 2;
    173.                     }
    174.                     break;
    175.                 case 2:
    176.                     if (colorFormat == COLOR_FormatI420) {
    177.                         channelOffset = (int) (width * height * 1.25);
    178.                         outputStride = 1;
    179.                     } else if (colorFormat == COLOR_FormatNV21) {
    180.                         channelOffset = width * height;
    181.                         outputStride = 2;
    182.                     }
    183.                     break;
    184.             }
    185.             ByteBuffer buffer = planes[i].getBuffer();
    186.             int rowStride = planes[i].getRowStride();
    187.             int pixelStride = planes[i].getPixelStride();
    188.  
    189.             int shift = (i == 0) ? 0 : 1;
    190.             int w = width >> shift;
    191.             int h = height >> shift;
    192.             buffer.position(rowStride * (crop.top >> shift) + pixelStride * (crop.left >> shift));
    193.             for (int row = 0; row < h; row++) {
    194.                 int length;
    195.                 if (pixelStride == 1 && outputStride == 1) {
    196.                     length = w;
    197.                     buffer.get(data, channelOffset, length);
    198.                     channelOffset += length;
    199.                 } else {
    200.                     length = (w - 1) * pixelStride + 1;
    201.                     buffer.get(rowData, 0, length);
    202.                     for (int col = 0; col < w; col++) {
    203.                         data[channelOffset] = rowData[col * pixelStride];
    204.                         channelOffset += outputStride;
    205.                     }
    206.                 }
    207.                 if (row < h - 1) {
    208.                     buffer.position(buffer.position() + rowStride - length);
    209.                 }
    210.             }
    211.         }
    212.         return data;
    213.     }
    214.  
    215.     private static boolean isImageFormatSupported(Image image) {
    216.         int format = image.getFormat();
    217.         switch (format) {
    218.             case ImageFormat.YUV_420_888:
    219.             case ImageFormat.NV21:
    220.             case ImageFormat.YV12:
    221.                 return true;
    222.         }
    223.         return false;
    224.     }
    225.  
    226.     private void dumpFile(String fileName, byte[] data) {
    227.         FileOutputStream outStream;
    228.         try {
    229.             outStream = new FileOutputStream(fileName);
    230.         } catch (IOException ioe) {
    231.             throw new RuntimeException("Unable to create output file " + fileName, ioe);
    232.         }
    233.         try {
    234.             outStream.write(data);
    235.             outStream.close();
    236.         } catch (IOException ioe) {
    237.             throw new RuntimeException("failed writing data to file " + fileName, ioe);
    238.         }
    239.     }
    240.  
    241.     private void compressToJpeg(String fileName, Image image) {
    242.         FileOutputStream outStream;
    243.         try {
    244.             outStream = new FileOutputStream(fileName);
    245.         } catch (IOException ioe) {
    246.             throw new RuntimeException("Unable to create output file " + fileName, ioe);
    247.         }
    248.         Rect rect = image.getCropRect();
    249.         YuvImage yuvImage = new YuvImage(getDataFromImage(image, COLOR_FormatNV21), ImageFormat.NV21, rect.width(), rect.height(), null);
    250.         yuvImage.compressToJpeg(rect, 100, outStream);
    251.  
    252.         Log.d("mmm", "写入图片" + fileName);
    253.     }
    254. }

    播放器音视频同步

    上方已经可以单独播放MP4的音频和视频

    本文福利， 免费领取C++音视频学习资料包、技术视频，内容包括（音视频开发，面试题，FFmpeg ，webRTC ，rtmp ，hls ，rtsp ，ffplay ，srs）↓↓↓↓↓↓见下面↓↓文章底部点击免费领取↓↓

  • 相关阅读:
    node使用jsonwebtoken生成token与验证是否过期
    Rocky Linux 运维工具 mv
    利用APT技术实现安卓组件化的解耦（上）
    .。。。。。。。。。。
    kilo TextEditor-1
    技术分享 | 接口自动化测试如何处理 Header cookie
    SpringBoot配置全局异常处理
    ADE Explorer和Assembler的仿真技巧
    Impala计算日期差datediff
    OceanMind海睿思入选弯弓研究院《2023中国营销技术生态图谱8.0》
  • 原文地址：https://blog.csdn.net/m0_60259116/article/details/126560206