Vulkan part 1

In my faculty as a Developer Advocate for Qualcomm, I’ve gotten the awesome chance of working with Vulkan for Android for a few months now.  Being able to work with such a new technology this closely is an awesome chance and I’m so excited about it I felt the need to create an example program and share it with the world via this blog.

Let’s start by talking a bit about what this whole Vulkan thing is and why it is important.  Back at the beginning of the graphics world, hackers found ways of creating cursors of ASCII art images decorated by text.  There were no GUI or concept of a window or any other form of User interface beyond a keyboard.  To be honest that level of technology is perfectly serviceable to the business world whose needs range from a text editor to a spreadsheet.  Thus the interest in creating more was chiefly a pursuit by “hackers” a term truly defined as those that would tinker in their free time, making things like games and creatively pursuing the very limits of what was possible.

At first these early maverick game makers and hackers got by with text for graphics and create interfaces requiring text input and output.  This era was the text adventure game era with Zork, and Hitchhikers Guide.  Then came the idea of allowing refresh draws over the entire screen and use text placed on a cartesian plane so it can move around in predictable ways.  Games like Pac Man, Pong and Space Invaders came from this next era.  Riffing on the idea of a cartesian plane, a standard was created by the Kronos group called “OpenGL” for Open Graphics Language.  OpenGL described XY coordinates with commands like GLVertex2( 10.0, 20.0 );  Then describe the color of that coordinate with GLColor3f( 1.0f, 1.0f, 1.0f );  which would go to position 10, 20 and draw that location white as the variables in Color means Red Green Blue values between 0 and 1 so full value Red Green and Blue will give the color white.

Now you can’t get very far being able to draw a vertex a single color as even monitors at the time were had 480 x 640 pixels, so each picture would be an impossible amount of code to write long hand like that.  Instead one would need to be able to describe full geometry.  So the creative graphics guys remembered their high school geometry and came up with some basic shapes that can be described like this:

GLColor3f(1.0f, 1.0f, 1.0f);

GLBegin(GL_TRIANGLES);

GLVertex3f(0.0f, 10.0f, 0.0f);

GLVertex3f(0.0f, 0.0f, 0.0f);

GLVertex3f(10.0f; 0.0f; 0.0f);

GLEnd();

The above would give you one white triangle.  Now we’re in to the awesomeness of being able to render very complex scenes as any artist will tell you, everything in the world can be formed by simple shapes being joined together in large volumes.  To this day, games, Operating systems, and Word processors still draw everything you see on the screen with lots of little triangles.  3d modeling software such as Maya and 3dsMax actually work by taking the object described by the artist and creating triangles then the programmer takes that output and puts it in a graphics language like OpenGL to draw those shapes.  Cool aint it?

Now 2d flat images will get you a very long way, but eventually you want to draw things on top of other things, or triangles that are closer or further away from the viewer.  So let’s think about this problem for a second, How do you say something is in front of something else?  That sentence assumes that there is an observer able to relate to the two objects in a definitive way.  Why would you need the observer? Because something in front of something else is exactly the opposite if your behind both objects.  Thus their state in depth space is dependent upon the frame of reference we view them from.

Ah, so now we need to first know what a view is and what an observer is before we can say something is in front or behind or establish depth space at all and break out of our 2d plane world.  So let’s use terms that kind of make sense,  What we want is to observe from a single point all other points visible as if we were standing in that location with our physical eye.  Well there’s a real world object that stands in perfectly called a Camera.  The way a camera works is by observing a scene projected by the light entering the lens.  Right there we have the ability to talk about a 3d world being able to be represented by a 2d image, hey progress!  So we need to describe a coordinate system related to the camera.  Well the camera “sees” in a conic shape whose center of the cone is the center of the lens and grows out to infinity by in the direction that the camera is pointing, which we’ll call the Z-axis, or Z-vector.  Now we have something really interesting, how to point our camera at an object?

Well we just described the Z-vector, and given the XYZ location of the camera and the related XYZ location of the object, you can build a vector that shows the direction the camera must point to have the camera look along the Z-Axis towards the object.  To get the camera to rotate towards another object, you’d take the current Z-Vector and dot product it against the vector you’re wanting to change it to in order to get the angle that you’d want to rotate your original vector by in order to get the two vectors looking along the same vector.

Knowledge of all this gives you a 3d scene you can place any object in, look at that object and relate what’s in from or behind as it relates to the negative Z-Axis of the camera.

So now that we have a very basic understanding of how graphics libraries work and the expectations at a very high level.  What about all this news about Vulkan?  What’s new and why do we care?  OpenGL was created when the systems I just described were just being invented.  It is an open standard that involves making these kind of expressions easy and straight forward to think about.  However, it was written in the 80’s and backwards compatibility means that if you write a program using OpenGL in the 90’s you want to expect that it still works today.  So you can’t make changes that could break backwards compatibility.  This leads to a few problems that aren’t really easily addressable in OpenGL.

One of the easiest problems to point out is OpenGL was created before many programmers understood threads or thought that parallel processing would be important.  Thus it was designed as a global state machine which means that when OpenGL is set to a state it remains in that state until it is changed.   A corollary of this state management system is that in the preceding code example, all three vertices are set to the color white.  We told OpenGL to be in the white color state at the very first line, and never change it away from that state.  If we wanted a different state we would have to change to that state prior to drawing.

This state management leads to an issue when doing parallel code with Threads.  The graphics card is a global resource so placing it into a certain state in one thread would wipe out what state was set in another thread.  That restriction has led to some rather crafty techniques in modern code whereby we are careful about setting up a texture’s draw state and actual render state to never happen at the same time.  To explain that, let’s say you work on a large image that you want to load from the hard drive and display on screen.  However, while loading that texture, you want to display something on screen until it’s ready.  This means that you’ll be wanting parallel processing with Threads.  Thread A would render the scene while Thread B in the background loads the texture, places it into graphics card memory and then labels it as ready to render so when Thread A can logically render it, it’s ready and does all this through a shared context.

This synchronization leads to some scheduling challenges that can better be handled with synchronization at the graphics API level however, OpenGL has no facility to help with this.  Now one of the greatest facets of Vulkan is the ability to run your own scheduler at the API layer thus making the synchronization challenges easier to deal with.  Both threads in their own contexts can inform the custom written (i.e. you write it) Vulkan to render at the same time.  Your scheduler handles the nastiness and Application layer threading gets faster and easier.

That’s the good news.  Now for the bad, Vulkan gives you NOTHING for free.  In order to get that very useful optimization, you must write your own scheduler, handle the states, memory management and graphics Pipeline that OpenGL provides for free in well tested decades mature code.  Thus this blog post seeks to provide a warning of sometimes the devil you know is better than the devil you don’t.  Next Time, I’ll take you through drawing a Triangle with pure Vulkan solution and provide a code sample with Android Studio 2.2 that gives some improvements over what Qualcomm provides in the sample project.  *hint* you don’t have to bother with GLSLValdator anymore as the functionality is embedded in 2.2.

Note that if you do proceed to use Vulkan in the embedded world, and I highly encourage you to take the journey with me, then I recommend using Qualcomm’s libraries as you have the ability to target 820 chipsets running any flavor of Android and not just Android N (note N is not yet named by Google let alone released!!!).