High Quality Video Compression: Integrating an H.265/HEVC Solution for Intel® Atom™-Based Android* Platforms

Abstract

According to the International Data Corporation, the total amount of global data for 2012 reached 2.7 zettabytes [1], an increase of 48% from 2011. Ninety percent of the global data was videos. Video apps consumed 66% of the total Internet data flow, and that number continues to increase rapidly. End users want to watch high quality videos but, for online video providers, costs of purchasing the network bandwidth and the storage devices grow increasingly expensive every year. How can video content providers meet the challenge of large data sources and their increasing storage requirements? Is it possible to achieve high quality videos with smaller bandwidth? The solution to these challenges is found in utilizing a video core compression technique known as High Efficiency Video Coding (H.265/HEVC).

Some open source communities such as FFmpeg [2] are developing an H.265/HEVC decoder, but the performance has not yet reached commercial use capability. Strongene Ltd. [3], a producer of video codec core technology, has developed advanced H.265/HEVC encoder/decoder solutions, including optimized H.265/HEVC encoder/decoder libraries and demonstration codes for Intel® Atom™ processor-based Android platforms. This article introduces Strongene's H.265/HEVC solution on Intel Atom processor-based Android tablet (condenamed Bay Trail).

Strongene's H.265/HEVC Solution

To evaluate a good video standard, we generally use efficiency and compatibility standards. The video standard evolution over the past 20 years is shown in the following graph:

H.265/HEVC is the successor codec to H.264/AVC (Advanced Video Coding), both were developed jointly by the ISO/IEC Moving Picture Experts Group [4] and ITU-T Video Coding Experts Group (VCEG) [5]. The primary goal of the new codec is to provide 50% better compression efficiency than H.264 and support resolutions up to 8192 x 4320.

As shown in Figure 1, it took 9 years to evolve from MPEG-2 to H.264/AVC, so more challenges are expected to be encountered in moving from the H.264/AVC standard to the H.265/HEVC standard. Gaps between the technical concept and the actual quality of the product still exist for H.265/HEVC. However, H.265/HEVC does a good job of balancing the efficiency and compatibility requirements, which make up the next level of video standards.

Strongene's H.265/HEVC solutions were optimized using YASM [6] assembler compiler, Intel® C++ Compiler [7], Intel® Streaming SIMD Extensions (Intel® SSE) [8], and Intel® Threading Building Blocks (Intel® TBB) [9], with OpenGL* [10] used for rendering. By directly uploading the decoded YUV420 data to the GPU, it finishes the transcoding of YUV data to RGB data and renders the RGB data to LCD. This reduces the CPU's workload and improves performance. The data stream is shown in the following diagram:

On an Intel Atom processor-based tablet (codenamed Bay Trail), after having been tested by Intel® Graphics Performance Analyzers (Intel® GPA) [11] tool, the optimized H.265/HEVC decoder refresh rate can reach 90 FPS (frames per second) when playing a 1080p HEVC video. If we set the refresh rate to 24 FPS on the Bay Trail tablet when playing the 1080p video, the CPU workload is less than 25%. Thus, Strongene's H.265/HEVC solution achieves commercial use capability.

Analysis of the APIs of Strongene's H.265/HEVC Solution

The APIs of Strongene's H.265/HEVC solution are easily used directly with or integrated into FFmpeg open source. Strongene defined five functions and one structure for the H.265/HEVC decoder, as listed below:

Structure:

lenthevcdec_ctx;

Description: It is the context of decoder for identifying different decoders.

Functions:
1. Unit32_t lenthevcdec_version(void);

Description: For getting API version of current library.

2. lenthevcdec_ctx lenthevcdec_create(int threads, int comoatibility, void* reserved);

Description: Create decoder with specified parameters.

Example:

lenthevcdec_ctx ctx;
ctx = lenthevcdec_create(2, 0x7fffffff, NULL);
if ( NULL == ctx ) {
fprintf(stderr, "call lenthevcdec_create failed!n");
exit(1);}

3. Void lenthevcdec_destroy(lenthevcdec_ctx ctx);
Description： Close decoder and release any resources.

4. Void lenthevcdec_flush(lenthevcdec_ctx ctx);
Description: flush decoder and clean the buffer.

5. int LENTAPI lenthevcdec_decode_frame( lenthevcdec_ctx ctx, const void* bs, intbs_len, int64_tpts, int* got_frame, int* width, int* height, intline_stride[3], void* pixels[3], int64_t* got_pts);

Description: Decode one frame. Input one frame's bit stream to decoder, and get one frame's pixel data from decoder if any frame has been decoded.

Example:

int32_t got_frame, width, height, stride[3], ret, i;
uint8_t* pixels[3];
int64_tpts, got_pts;
for ( i = 0; i <au_count; i++ ) {
pts = i * 40;
got_frame = 0;
ret = lenthevcdec_decode_frame(ctx, au_buf + au_pos[i], au_pos[i + 1] -
au_pos[i], pts,&got_frame, &width, &height, stride, (void**)pixels,
&got_pts);
if ( ret < 0 ) {
fprintf(stderr, 
"calllenthevcdec_decode_frame failed! ret=%dn", ret);
exit(1);
}
if ( got_frame> 0 ) {
printf("decode frame, %dx%d, pts is %" PRId64 "n", 
width, height, got_pts);
/* got frame, do something ... */
}
}

How to Integrate Strongene's H.265/HEVC Solution

Developers can easily integrate these five functions when developing HEVC video players. The documentation and sample code are available from Strongene's download web site [12].

Developers can directly call the Strongene's H.265/HEVC APIs to develop their video players, or they can merge Strongene's H.265/HEVC patch for FFmpeg, and then use the merged FFmpeg APIs to develop their video players. The following paragraph introduces the two methods separately.

Directly Use Strongene's APIs to Develop Video Players

On Strongene's web site, developers can download the:

• sample code testdec.c/lenthevcdec.h
• decoder: liblenthevcdec.so
• documents: lenthevcdec_en.pdf, the Makefile

Copy the sample code to Ubuntu* build machine into separate folders:

wangsy@ubuntu:~/Desktop cp testdec.c Makefille ~/hevc/src
wangsy@ubuntu:~/Desktop cp lenthevcdec.h  ~/hevc/include
wangsy@ubuntu:~/Desktop cp liblenthevcdec.so ~/hevc/lib/Android_x86

Set up ANDROID_NDK_HOME environment with the "export" command:

export ANDROID_NDK_HOME= $ ANDROID_NDK_HOME :/~/android-ndk-r9c

Then run the "make" command.

Then build the sample code and get the output file testdec. Copy the output file and the decoder libs into the root Android devices under /data/hevc_test.

Set up LD_LIBRARY_PATH environment with the "export" command:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/data/hevc_test

Then you can run the demo on the Android devices directly.

Use the H.265/HEVC Patch for FFmpeg to Develop the Video Players

In fact, most developers will use the open source FFmpeg to develop their video players. Strongene's web site also provides the patch for FFmpeg, which can be downloaded from: lentoid_ffmpeg2.0_patch_2014_01_23_new_encoder_interface.patch

To merge this patch, you should also download and decompress the FFmpeg 2.0.4 version [13]. Then add the patch:

wangsy@ubuntu:~/ffmpeg-2.0.4
patch –p1 < ./lentoid_ffmpeg2.0_patch_2014_01_23_new_encoder_interface.patch

Copy the downloaded headfile and the libs code to ~/ffmpeg-2.0.4/thirdparty

cp lenthevcdec.h lenthevcenc.h liblenthevcdec.so liblenthevcenc.so ~/ffmpeg-2.0.4/thirdparty

Copy the following configure file to ~/ffmpeg-2.0.4 and add the "run" permissions:

wangsy@ubuntu: cp build_x86.sh ~/ffmpeg-2.0.4
wangsy@ubuntu: sudo chmod a+x ~/ffmpeg-2.0.4/build_x86.sh

Download build_x86.txt

Run the configure file "build_x86.sh" (embedded above), then run the "make" and "make install" commands. The merged FFmpeg with Strongene's H.265/HEVC patch should now be built successfully. The output libs will be under ~/FFmpeg2.0.4/android/x86/lib. You can use these output libs in your video players.

Summary

The H.265/HEVC standard is becoming more and more popular in the mobile market. Online video providers, mobile Internet users, and broadcast/TV operators are the beneficiaries of this new video revolution, along with users. To produce the results described in this article, Strongene's H.265/HEVC solutions were fully optimized for Intel® Atom™ processor-based Android platforms, so don't hesitate to adopt their solutions to develop HEVC players.

Related Articles

• Optimizing H.265/HEVC Decoder on Intel® Atom™ Processor-Based Platforms: http://software.intel.com/en-us/articles/optimizing-h265hevc-decoder-on-intel-atom-processor-based-platforms
• Real-time End-to-End H.265/HEVC Solution for Intel® Architecture-based Platforms: https://software.intel.com/en-us/android/articles/real-time-end-to-end-h265hevc-solution-for-intel-architecture-based-platforms

References

[1] International Data Corporation (IDC). "Top 10 Prediction/IDC Predictions 2012: Competing for 2020." IDC web site: http://cdn.idc.com/research/Predictions12/Main/downloads/IDCTOP10Predictions2012.pdf

[2] FFmpeg: http://www.ffmpeg.org/index.html

[3] Strongene Ltd.: http://xhevc.com/en/about/about-shijun.jsp

[4] ISO/IEC Moving Picture Experts Group (MPEG): http://mpeg.chiariglione.org/

[5] ITU-T Video Coding Experts Group (VCEG): http://www.itu.int/en/ITU-T/studygroups/com16/video/Pages/default.aspx

[6] Yasm Modular Assembler Project: http://yasm.tortall.net/

[7] Intel® C++ Compiler (Intel® ICC): https://software.intel.com/en-us/c-compilers

[8] Intel® Streaming SIMD Extensions (Intel® SSE): https://software.intel.com/en-us/articles/performance-tools-for-software-developers-intel-compiler-options-for-sse-generation-and-processor-specific-optimizations.

[9] Intel® Threading Building Blocks (Intel® TBB): https://software.intel.com/sites/default/files/m/d/4/1/d/8/tutorial.pdf

[10] OpenGL: http://www.opengl.org/

[11] Intel® Graphics Performance Analyzers (Intel® GPA): https://software.intel.com/en-us/articles/gpa-faq

[12] Strongene Ltd. Downloads: http://www.strongene.com/en/downloads/downloadCenter.jsp

[13] FFmpeg Downloads: http://www.ffmpeg.org/olddownload.html

About the Author

Songyue Wang is a senior application engineer in the Intel Software and Solutions Group (SSG), Developer Relations Division, Intel® Atom™ Processor Mobile Enabling Team. Songyue is responsible for Android app enabling on Intel Atom processors. He focuses on optimizing multimedia performance on the Bay Trail platform, working closely with the most popular online video providers in the PRC region to enable the H.265/HEVC encoder and decoder solution and Intel® Wireless Display differentiation features on Android for x86 platforms.

[1]Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark* and MobileMark*, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products.

Configurations: Intel^® Atom™ processor Bay Trail tablet FFRD8 with 2GB RAM and Android 4.4, Decoding Frame Rate. For more information, go to http://www.intel.com/performance