Great opportunity exists in porting games and other applications that make extensive use of 3D graphics through OpenGL standards to Google Android devices, including those built on the Intel® Atom™ microarchitecture because of the availability of games, game engines, and other legacy software based on OpenGL; the portability that OpenGL provides; and Android’s evolving support for OpenGL ES and C/C++. Many OpenGL-based game titles and engines are even available as open source, like Id Software’s Quake series. This two-part article shows how to begin such a project by detailing the barriers that exist to porting the rendering components of apps built on earlier versions of OpenGL to Android on Intel Atom processors. These applications can be games, game engines, or any software that uses OpenGL to construct 3D scenes or a graphical user interface (GUI). The porting of OpenGL code from desktop operating systems like Windows* and Linux* as well as apps that use the embedded versions of OpenGL ES—with or without a windowing system—is also covered.
Part 1 of this article introduces how to use OpenGL ES on the Android platform through either the software development kit (SDK) or the Android Native Development Toolkit (NDK) and how to decide which approach to use. The various OpenGL ES sample apps in the SDK and NDK are described as well as the Java* Native Interface (JNI), which allows you to combine Java and C/C++ components. How you decide whether you should target OpenGL ES version 1.1 or 2.0 is also discussed.
Part 2 will discuss the barriers that exist to porting OpenGL games you should know before undertaking such a project, including differences in OpenGL extensions, floating-point support, texture compression formats, and the GLU library. Setting up your Android development system for Intel Atom processors and getting the best possible performance from the Android virtual device emulation tools with OpenGL ES will also be covered.
Graphics on Android
You can render graphics on the Android platform in four distinct ways, each with its own strengths and limitations (see Table 1). Covering all of these options is beyond the scope of this article. Only two approaches make sense for porting OpenGL code from other platforms: the SDK Wrapper classes for OpenGL ES and the NDK for OpenGL ES development in C/C++. Regarding the other approaches, the SDK Canvas application programming interface (API) is a powerful 2D API that you can use in combination with OpenGL ES but is otherwise limited to 2D and requires new code to use.
Android’s Renderscript API did support OpenGL ES initially, but that has been deprecated in API level 16 (Jelly Bean) and should not be used for new projects. The best application for Renderscript now is to improve the performance of compute-intensive algorithms that do not require much memory allocation or data transfer, such as calculations for simulated game physics.
Table 1. The Four Ways to Render Graphics on Android
| Method | Stipulations |
|---|
| SDK Canvas API | Java and 2D graphics only |
| SDK Wrapper classes for OpenGL ES | OpenGL ES 1.1 and 2.0 callable from Java but with JNI overhead |
| NDK OpenGL ES | OpenGL ES 1.1 and 2.0, with native C/C++ callable from Java |
| Renderscript for OpenGL ES | OpenGL ES support has been deprecated in API level 16 |
Porting OpenGL applications to early releases of Android was difficult because most legacy OpenGL code was written in C or C++ and Android only supported Java until the release of the NDK in Android 1.5 (Cupcake). OpenGL ES 1.0 and 1.1 have been supported from the beginning, but the performance was inconsistent because acceleration was optional. But Android has made tremendous strides in recent years. Support for OpenGL ES 2.0 was added to the SDK in Android 2.2 (Froyo) and to the NDK in revision 3, with expanded support for OpenGL ES extensions in NDK revision 7. OpenGL ES 2.0 is now used so extensively by Android’s own graphics framework that it’s no longer considered optional. Accelerated OpenGL ES 1.1 and 2.0 are now essential on all new Android devices, particularly as screen sizes continue to grow. Today, Android delivers consistent and reliable performance for 3D-intensive applications built on either OpenGL ES 1.1 or 2.0 code in either Java or C/C++, and developers have several options to make the porting process easier.
Using the Android Framework SDK with the OpenGL ES Wrapper Classes
The Android SDK framework provides a set of wrapper classes for the three versions of OpenGL ES that Android supports (1.0, 1.1, and 2.0). These classes allow Java code to easily call the OpenGL ES drivers in the Android system, even though the drivers execute natively. If you are creating a new OpenGL ES Android game from scratch or are willing to convert your legacy C/C++ code to Java, then this is probably the easiest path. Even though Java is designed for portability, it can be difficult to port Java apps because Android does not support the full set of established Java Platform, Standard Edition (Java SE) or Java Platform, Micro Edition (Java ME) classes, libraries, or APIs. Android’s Canvas API is a capable 2D API, but it’s unique to Android and offers no compatibility for legacy code.
Several additional classes provided by the Android SDK make using OpenGL ES easier, such as GLSurfaceView and TextureView. GLSurfaceView is similar to the SurfaceView class used with the Canvas API but with additional features specifically for OpenGL ES. It handles the required initialization of the Embedded System Graphics Library (EGL), allocates a rendering surface that Android displays at a fixed position on the screen, and threading. It also has some useful features for tracing and debugging OpenGL ES calls. You can create a new OpenGL ES application quickly by implementing just three methods for the GLSurfaceView.Renderer() interface, as detailed in Table 2.
Table 2. The Minimum Required Methods for GLSurfaceView.Renderer
| Method | Description |
|---|
| onSurfaceCreated() | Called once when the application starts for initialization |
| onSurfaceChanged() | Called whenever a change occurs to the GLSurfaceView size or orientation |
| onDrawFrame() | Called repeatedly to render each frame of the graphics scene |
Beginning with Android 4.0, you can use the TextureView class instead of GLSurfaceView to provide rendering surfaces for OpenGL ES with additional capabilities, but it requires more code to use. TextureView surfaces behave as regular Views and can be used to render to off-screen surfaces. This functionality allows you to move, transform, animate, or blend TextureViews when they are composited to the screen. Or, you can use a TextureViewto combine OpenGL ES rendering with the Canvas API.
The Bitmap and GLUtils classes make it easy to create textures for OpenGL ES using the Android Canvas API or to load textures from PNG, JPG, or GIF files. Bitmaps are used to allocate rendering surfaces for Android’s Canvas API. The GLUtils class can convert images from bitmaps into OpenGL ES textures. This integration allows you to use the Canvas API to render 2D images that can then be used as textures with OpenGL ES. This is particularly useful for creating graphics elements that OpenGL ES does not provide, like GUI widgets and text fonts. But, of course, new Java code is required to exploit these features.
The Bitmap class was designed primarily for use with the Canvas API, and it has some serious limitations when used to load textures for OpenGL ES. The Canvas API follows the Porter-Duff specification for alpha blending, and Bitmap optimizes images with per-pixel alpha by storing them in the premultiplied format (A, R*A, G*A, B*A). This is great for Porter-Duff but not for OpenGL, which requires a non-premultiplied (ARGB) format. This means that the Bitmap class can only be used with textures that are completely opaque (or have no per-pixel alpha). Three-dimensional games typically require textures with per-pixel alpha, in which case you must avoid using bitmaps and load textures from a byte array or through the NDK, instead.
Another issue is that Bitmaps only support loading images from the PNG, JPG, or GIF formats, whereas OpenGL games typically use compressed texture formats that are decoded by the GPU and are often proprietary to the GPU architecture, such as ETC1 and PVRTC. Bitmap and GLUtils do not support any proprietary compressed texture formats or mipmapping. Because these features are used heavily by most 3D games, this represents a serious barrier to porting legacy OpenGL games to Android with the SDK. Until Google resolves these problems, the best solution is to avoid using theBitmap and GLUtils classes to load textures. Texture formats are discussed further in the section, “Texture Compression Formats.”
The Android ApiDemos contain a sample app named StaticTriangleRenderer that demonstrates how to load an opaque texture from a PNG resource file using the GLES10 wrapper for OpenGL ES 1.0, GLSurfaceView, Bitmap, and GLUtils classes. A similar version named GLES20TriangleRendereruses the GLES20 wrapper class for OpenGL ES 2.0, instead. These sample apps make good starting points for developing OpenGL ES games if you are targeting the Android Framework SDK using the wrapper classes. Don’t use the original version, named TriangleRenderer, because it uses a wrapper for an older version of the OpenGL ES bindings for Java named javax.microedition.khronos.opengles. Google created the new bindings to provide a static interface for OpenGL ES specifically for Android. These static bindings offer better performance, expose more OpenGL ES features, and provide a programming model that is closer to how OpenGL ES is used with C/C++, which may help with code reuse.
The Android Framework SDK provides support for OpenGL ES through several bindings for Java from Google and Khronos, as detailed in Table 3.
Table 3. Summary of Wrapper Classes for OpenGL ES and Sample Apps
| Java Bindings for OpenGL ES | Description | Sample Apps |
|---|
| javax.microedition.khronos.egl | Khronos standard definition | – |
| javax.microedition.khronos.opengles | Khronos standard definition | TriangleRenderer, Kube, CubeMap |
| android.opengl | Android-specific static interfaces | – |
The Android-specific static bindings offer better performance and should be used instead of the Khronos bindings whenever possible. The static bindings provide corresponding wrapper classes for all versions of OpenGL ES available for application development on Android. Table 4 summarizes these classes.
Table 4. Summary of Android Wrapper Classes for OpenGL ES and Sample Apps
| API Version | Java Class | Sample Apps |
|---|
| OpenGL ES 1.0 | android.opengl.GLES10 | StaticTriangleRenderer, CompressedTexture |
| OpenGL ES 1.0 | android.opengl.GLES10Ext | – |
| OpenGL ES 1.1 | android.opengl.GLES11 | – |
| OpenGL ES 1.1 | android.opengl.GLES11Ext | – |
| OpenGL ES 2.0 | android.opengl.GLES20 | GLES20TriangleRenderer, BasicGLSurfaceView, HelloEffects |
These wrapper classes are designed such that most OpenGL ES calls in legacy C/C++ code can be converted to Java by simply prefixing OpenGL ES function and symbol names with a wrapper class of the appropriate API version. See the examples in Table 5.
Table 5. Examples of Editing OpenGL ES Calls from C to Java
| C Language | Java Language |
|---|
| glDrawArrays(GL_TRIANGLES, 0, count) | GLES11.glDrawArrays(GLES11.GL_TRIANGLES, 0, count) |
| glDrawArrays(GL_TRIANGLES, 0, count) | GLES20.glDrawArrays(GLES20.GL_TRIANGLES, 0, count) |
Porting OpenGL games to Android using these wrapper classes has three significant limitations: the overhead of the Java Virtual Machine and the JNI and the work required to convert legacy C/C++ code to Java. Java is an interpreted language, and all Java code on Android runs on the Dalvik virtual machine, which executes more slowly than compiled C/C++ code. Because the OpenGL ES drivers always execute natively, every call to an OpenGL ES function through these wrappers incurs the overhead of the JNI, limiting the performance of your game’s graphics rendering. The more OpenGL ES calls your application makes, the greater the impact of the JNI overhead. Fortunately, OpenGL ES is well designed to help minimize the number of calls typically required in performance-critical rendering loops. If performance is a problem, you always have the option of moving performance-critical code to C/C++ with the NDK. But, of course, if your code began as C/C++, it might be best to start with the NDK in the first place.
Whether you should convert C/C++ code to Java depends on the details of your particular project. If the amount of C/C++ code is relatively small and well understood and you don’t want to push the performance envelope, then converting it to Java may be reasonable. Otherwise, if the amount of C/C++ code is large and not well understood and you do want to push the envelope, then consider using the NDK.
Using the Android NDK
Google added the NDK to Android in June 2009 to allow apps to use C/C++ code running natively, which offers better performance than Java. And because most legacy OpenGL code is written in C or C++, the NDK provides an easier path for porting applications, especially when the amount of C/C++ code is so large that converting it all to Java is not practical. This is actually the main reason Google decided to release the NDK publicly—to make it easier to port OpenGL games to Android. Because of these advantages, the NDK is becoming the dominate approach for implementing apps that require fast graphics on Android.
The NDK lets you compile C/C++ code to Linux shared object libraries, which are statically linked to your Android app. Libraries are built using GNU tools, which are provided in the NDK release package from Google, and you can run the tools on Windows, Mac OS* X, or Linux development systems using the Eclipse* integrated development environment or a command-line interface. This toolchain supports three processor architectures: ARM, Intel Atom (x86), and MIPS. Although the full power of C/C++ is available, most Linux APIs are not. In fact, the APIs that are supported directly include little more than OpenGL ES, OpenMAX* AL, OpenSL ES*, zlib, and Linux file I/O, which Google terms the Stable APIs. However, documentation is provided on how to port other Linux libraries into your NDK project as needed.
The NDK gives you the flexibility to partition code between Java and C/C++ as appropriate for your application. The NDK supports a mechanism for calling C/C++ code from Java called the Java Native Interface. But, significant overhead is associated with JNI calls, so it’s important to partition applications carefully using native code to minimize the number of calls through the JNI. Typically, most OpenGL ES code should remain in C/C++ for the best performance and ease of porting, while new Java code can be written to use GLSurfaceView and other SDK classes to manage app life cycle events and support other game functions. Android began supporting the JNI for Intel Atom processors beginning with NDK revision 6b.
The NDK supports OpenGL ES 1.1 and 2.0 and provides sample apps for both versions that also demonstrate how to combine C functions with Java using the JNI. These apps differ in how their code is partitioned between Java and C and in how they are threaded. They all use the NDK and native C code, but the native-media sample app retains all of its OpenGL ES code in Java, whereas san-angeles and native-activity have all of their OpenGL ES code in C and hello-gl2 splits its EGL and OpenGL ES code between Java and C. It’s best to avoid using the hello-gl2 sample because of this split and because it does not use the preferred method to configure GLSurfaceView for OpenGL ES 2.0 surfaces, which is to callsetEGLContextClientVersion(2). See Table 6.
Table 6. Summary of OpenGL ES Sample Apps in the NDK
| API Used | Sample App | SDK/NDK Partitioning |
|---|
| OpenGL ES 1.1 | san-angeles | All EGL and OpenGL ES code is C. |
| OpenGL ES 1.1 | native-activity | All code is in C and uses the NativeActivity class. |
| OpenGL ES 2.0 | hello-gl2 | EGL setup is in Java, and OpenGL ES code is C. |
| OpenGL ES 2.0 | native-media | All EGL and OpenGL ES code is Java. |
Even though it does not use OpenGL ES, the bitmap-plasma sample is also interesting because it demonstrates how to use the jnigraphics library to implement native functions that directly access the pixels of Android bitmaps.
Activity Life Cycle Events and Threading
Android requires that all calls to OpenGL ES execute from a single thread because the EGL context can only be associated with a single thread and rendering graphics on the main UI thread is strongly discouraged. So the best approach is to create a separate thread specifically for all of your OpenGL ES code that will always execute from this same thread. If your app uses GLSurfaceView, it creates this dedicated OpenGL ES rendering thread automatically. Otherwise, your app must create the rendering thread itself.
The san-angeles and native-activity sample apps retain all of their OpenGL ES code in C, but san-angeles uses some Java and GLSurfaceViewto create the rendering thread and manage the Activity life cycle, but the native-activity sample has no Java code at all. Instead of usingGLSurfaceView, it manages Activity life cycle events in C and uses a rendering thread that the NativeActivity class provides. NativeActivity is a convenience class provided by the NDK that allows you to implement Activity life cycle event handlers in native code, such as onCreate(), onPause(), and onResume(). Some Android services and content providers cannot be accessed directly from native code but can still be reached through the JNI.
The native-activity sample is a good starting point for porting OpenGL ES games because it demonstrates how to use the NativeActivity class and the android_native_app_glue static library to handle life cycle events in native code. These classes provide a separate rendering thread for your OpenGL ES code, a rendering surface, and a window on the screen, so you don’t need the GLSurfaceView or TextureView classes. The main entry point of this native application is android_main(), which runs in its own thread and has its own event loop for receiving input events. Unfortunately, the NDK does not provide a version of this sample for OpenGL ES 2.0, but you can replace all of the 1.1 code in this sample with your 2.0 code.
Apps that use NativeActivity must be run on Android 2.3 or later and make special declarations in their manifest file, as described in part 2 of this article series.
The Java Native Interface
If you choose to implement your app mostly in C/C++, it will be hard for you to avoid using some Java classes for larger and more professional projects. For example, the Android AssetManager and Resources APIs are only available in the SDK, and this is the preferred way to handle internationalization and different screen sizes, among other things. However, the JNI provides the solution because it not only allows Java code to call C/C++ functions but also allows C/C++ code to call Java classes. So, even though some overhead is associated with the JNI, don’t avoid using it completely. It’s the best solution for accessing important system functionality that’s only available in the SDK, especially when these functions are not performance critical. A full description of using the JNI is beyond the scope of this article, but here are the three basic steps required to make a call from Java to C/C++ using the JNI:
- Add a declaration for the C/C++ function in your Java class file as a native type.
- Add a static initializer for the shared object library that contains that native function.
- Add the function of the appropriate name following a specific naming scheme to the native source file.
OpenGL ES 1.1 or 2.0?
So, which version of OpenGL ES should you target on Android? Version 1.0 of OpenGL ES has been superseded by version 1.1, so the choice is really between versions 1.1 and 2.0. Khronos and Google may support both of these versions indefinitely, but OpenGL ES 2.0 is superior to 1.1 in most ways. It is much more versatile and offers higher performance because of its OpenGL Shading Language (GLSL) ES shader programming features. It can even require less code and less memory for textures. But Khronos and Google still have at least one good reason for continuing to support version 1.1—it looks a lot more like the original OpenGL 1.x used in the desktop and console gaming worlds for decades. Therefore, it may be easier to port an older game to OpenGL ES 1.1 than to 2.0; and the older the game, the more likely that is to be true.
If the game you are porting has no shader code, then you have the choice of targeting either OpenGL ES 1.1 or 2.0, but version 1.1 will probably be the easier path. If your game does have shader code already, then you should certainly target OpenGL ES 2.0, especially when you consider that recent releases of Android now use version 2.0 predominately. According to Google, more than 90 percent of Android devices that have accessed the Google Play website have support for both OpenGL ES 1.1 and 2.0 as of October 2012.
Conclusion
You can implement graphics rendering on Android with OpenGL ES through the Android SDK, the NDK, or a combination of both using the JNI. The SDK approach requires coding in Java and is best suited to new app development, whereas the NDK makes porting legacy OpenGL code in C/C++ much more practical. Most game porting projects will require a combination of both SDK and NDK components. For new projects, OpenGL ES 2.0 should be targeted rather than version 1.1, unless your legacy OpenGL code is so old that it does not use any GLSL shader code.
Part 2 of this series will discuss the barriers that exist to porting OpenGL games that you must recognize before undertaking such a project, including differences in OpenGL extensions, floating-point support, texture compression formats, and the GLU library. Setting up your Android development system for Intel Atom processors and getting the best possible performance from the Android virtual device emulation tools with OpenGL ES will also be covered.
About the Author
Clay D. Montgomery is a leading developer of drivers and apps for OpenGL on embedded systems. His experience includes the design of graphics accelerator hardware, graphics drivers, APIs, and OpenGL applications across many platforms at STB Systems, VLSI Technology, Philips Semiconductors, Nokia, Texas Instruments, AMX, and as an independent consultant. He was instrumental in the development of some of the first OpenGL ES, OpenVG*, and SVG drivers and applications for the Freescale i.MX and TI OMAP* platforms and the Vivante, AMD, and PowerVR* graphics cores. He has developed and taught workshops on OpenGL ES development on embedded Linux and represented several companies in the Khronos Group.
For More Information
