One of the coolest and most powerful features of Skytap is the ability to create a virtual data center and start interacting with its virtual machines (VMs) in just a few clicks, all from the comfort of your favorite browser. It takes a lot of complicated technology to make this simplicity possible, and that’s what I’ll be talking about in this blog.
To enable in-browser desktop access we’ve created a few technologies, collectively known as SmartClient. SmartClient consists of both a server and a client component. The server, known as a “muxer,” is responsible for exposing the frame buffer of, and sending inputs to, Skytap VMs in a secure manner. It is called a “muxer” because it is capable of multiplexing several client connections to a single VM. The muxer implements the Remote Desktop Protocol (RDP) to communicate with clients. In this post, I’ll be primarily discussing the Skytap custom RDP client, known as the SmartClient applet.
The SmartClient applet is built upon Oracle’s Java Applet technology. It implements the client portion of RDP to communicate with the Skytap muxer. The muxer in turn transcodes RDP to other communication protocols supported by hypervisors or VM operating systems. At its core, the SmartClient applet is a Java Swing application that allows you to both interact with your VM’s desktop as well as control their state from a browser. The applet is invoked from a web page, and is initialized with a JSON object with all of the details necessary to interact with your virtual data center. This web page also contains an HTML representation of your virtual data center and the applet allows you to quickly switch back and forth between an individual VM’s desktop and the full virtual data center view.
Once the applet has loaded the JSON representing the virtual data center, it opens a connection to the muxer and negotiates an RDP connection. Our applet primarily uses the RemoteFX codec to send screen updates to the applet. In order to support RemoteFX inside our muxer, we needed a RemoteFX encoder. Rather than attempt to implement it ourselves, we spoke with the developer who implemented the RemoteFX decoder in the FreeRDP project and agreed to sponsor his effort to implement the encoder and commit it back to the project.
Decoding RemoteFX screen updates is a mathematically intensive process. It requires the manipulation of matrices and various bit twiddling operations. We initially tried to implement this in pure Java but found the performance to be unsatisfactory (the applet was using too much CPU and screen updates were falling behind). Decoding RemoteFX updates can benefit greatly from using SSE2 instructions. In order to leverage these instructions and use the hardware more efficiently, we implemented a small C library that wraps the FreeRDP RemoteFX decoder. We use JNI to invoke this C library on each update to decode each RemoteFX packet. Since we support Linux, Mac and Windows, we built this library for all those platforms in both 32-bit and 64-bit flavors. This gave us a considerable speed up over the Java implementation. The FreeRDP project has published the speed gains achieved by using SSE2. RemoteFX decoding is fast enough that we routinely test the applet while watching full screen video from inside a virtual machine.
Using JNI did introduce its own set of drawbacks though. We had to be much more careful about how we managed memory. Each update is read off the wire into a ByteBuffer and then passed as a byte array into our decode method. However, care must be taken to tell the Java garbage collector when it can collect the memory once we are finished with it. You also must make sure to free all allocated memory inside the native object, otherwise you can wind up with some difficult to track down memory leaks. These leaks won’t show up in VisualVM or other Java memory examination tools.
The other cool part of the SmartClient applet is its use of LiveConnect to interact with the Skytap application to control VMs. We have implemented a Swing toolbar that hovers over the top of a VMs desktop that allows you to run/suspend your VM, toggle which VM you are connected to, alter the desktop size, switch to full screen mode, and launch our help documentation. All of these actions require interacting with the Skytap application, and in order to accomplish this, we implemented a JavaScript API that is invoked in the applet via LiveConnect. This is the same API that our HTML client uses to control VMs.
SmartClient is a big leap forward in remote desktop access for Skytap. It has allowed us to offer increased performance and functionality to our customers, while also giving us the ability to start work on the next generation of remote desktop clients. HTML5 canvas is in our future! In a follow up post we’ll spend some time examining the internals of the Skytap muxer and how it makes accessing Skytap desktop so seamless.
Want to talk about SmartClient? Feel free to leave me a question or comment below.
