Part 3: Introduction to WinRT, the new ‘Windows Runtime’ in Windows 8

To install Visual Studio 2011 Developer Preview + Window 8 beta, refer to post here.

To get started working with Windows 8 Operating System, see post here.

Windows Runtime, or shortly WinRT, is Microsoft’s new programming model that makes the backbone of the new Metro-style apps (also known as Immersive) in the new Windows 8 operating system. WinRT solves many of the problems of Win32, from an apps perspective. Apps created for WinRT are safe, secure, and sandboxed, can’t wreck other apps and all install in just 2-3 seconds. They feature isolated storage, single folder installs (as per the Mac), and require user consent to access the general file system. This is part of it, a modern, truly different app platform that is far more different, and far more capable, than many people understand right now.

WinRT isn’t another abstraction layer; it’s a new runtime sitting directly on top of the Windows Kernel Services just like the good old Win32 API as shown in the above figure that allows developers to write Metro style applications for Windows 8, using a variety of languages including C++ (with WRL or  with Component Extensions – C++/CX), the managed languages C# and VB.NET, as well as JavaScript. In the above figure, right on top of NT kernel (like Win32) is WinRT. This WinRT has an application model and three boxes of capabilities that are expressed by APIs (Communications & Data, Graphics & Media, and Devices & Printing). "Above" WinRT is the two sets of presentation layer/programming language couples: XAML and various high level programming languages (C#, VB, and so on) and HTML/CSS and JavaScript, respectively. (DirectX is left out, but sits on top of WinRT too.) WinRT is an unmanaged native layer and its API is object oriented that can be consumed both from native or managed languages, as well as JavaScript.

In short, WinRT is the Object Oriented Programming replacement for Win32.

This marks the first major break in the core of Windows since Win32 was introduced in 1993 with Windows NT. WinRT represents a new application execution environment with semantics that are very different than Win32. Unlike Win32, which was designed with C in mind, the WinRT APIs are written in C++ and designed from the beginning to be object oriented. Consistency, ease of use, and performance are key aspects of the new runtime API. Every object in the WinRT API supports reflection so that even dynamic languages such as JavaScript can use them efficiently. Along with this comes a unified object model, a rarity for C++ based libraries. The new Windows SDK for Metro style apps also includes a subset of traditional Win32, Component Object Model (COM), and .NET Framework APIs, as well as HTML5 and CSS3 APIs that are accessible to Metro style app developers. I.e. Metro style apps can use a subset of the Win32 and COM API. This subset of APIs was chosen to support key scenarios for Metro style apps that were not already covered by the Windows Runtime, HTML/CSS, or other supported languages or standards. The Windows App Certification Kit ensures that your app uses only this subset of the Win32 and COM API. The subset of the Win32 and COM API that can be used in a Metro style app is indicated in the header files in the Windows Software Development Kit (SDK) for Metro style Apps. Look for the following statements in the header files:

#pragma region Application Family
#pragma region Desktop Family

The compiler and object browser in Microsoft Visual Studio 11 Express for Windows Developer Preview use these statements to determine whether to show or hide a Win32 or COM API element.

One other big change is that all WinRT components have metadata available for them, just like .NET assemblies. Well, the Windows Runtime is exposed using API metadata stored in .winmd files as shown in the following figure 2. In COM you were able to achieve that with type libraries (Type libraries contain metadata that represent COM types), but not every COM component had them.  The metadata (API definitions) of WinRT contained in .winmd files are encoded in ECMA 335 metadata format, the same format that .NET uses but with a few modifications. The underlying binary contract makes it easy for you to access the Windows Runtime APIs directly in the development language of your choice. The shape and structure of the Windows Runtime APIs can be understood by both static languages such as C# and dynamic languages such as JavaScript. IntelliSense is available in JavaScript, C#, Visual Basic, and C++. This common metadata format allows for significantly less overhead when invoking WinRT from .NET applications compared to a P/Invoke, and much simpler syntax. Regular C++ (with COM-specific discipline) can also be used to program with WinRT components, with the help of new template library called Windows Runtime C++ Template Library (WRL), which is similar in purpose to what Active Template Library provides for COM. The MSDN documentation, however, recommends using C++/CX instead of WRL.

Fig showing the “C:\Program Files (x86)\Windows Kits\8.0\Windows Metadata\” folder in Windows 8 Developer Preview. You can observe that they are actually CLI assemblies with no code, just metadata tables.

As shown in the following Fig 3, you can open the .winmd file with ILDASM. Note, this doesn’t mean that WinRT itself is managed – it simply reuses the file format.

For the structure of the Windows Runtime, see the Microsoft site.

There are a number of libraries implemented in terms of the new object model, i.e. defining WinRT interfaces and classes. Look at “Windows Metadata” folder mentioned above to see what’s there; or just fire up Object Browser in VS and select “Windows 8.0? in the framework selector, to see what’s covered. There’s a lot there, and it doesn’t deal with UI alone – you also get namespaces such as Windows.Data.Json, or Windows.Graphics.Printing, or Windows.Networking.Sockets. Then you get several libraries, which are specifically dealing with UI – mostly these would be various namespaces under Windows.UI or Windows.UI.Xaml. A lot of them are very similar to WPF/Silverlight namespaces – e.g., Windows.UI.Xaml.Controls is closely matching System.Windows.Controls; also for Windows.UI.Xaml.Documents etc.

The WinRT type system relies strongly on WinRT components, which are COM objects implementing a specific set of interfaces and adhering to a certain ABI (Application Binary Interface). WinRT is essentially a COM-based API, relying on an enhanced COM that implements the IInspectable interface. The IInspectable interface enables projecting Win32 and COM features into JavaScript and other languages, such as C# and Visual Basic. Implement the IInspectable interface when you want your class to be available in other programming environments. The IInspectable interface is the base interface for all Windows Runtime classes. All Windows Runtime classes must implement the IInspectable interface. The IInspectable interface inherits from the IUnknown interface. so every WinRT object implements IUnknown and does refcounting (WinRT objects are reference counted like COM for memory management, with weak references to avoid circularity), and builds from there. It does add quite a lot of new concepts in comparison to COM of old, most of which come directly from .NET – for example, WinRT object model has delegates, and events are done .NET-style (with delegates and add/remove subscriber methods, one per event) rather than the old COM model of event sources and sinks. Of other notable things, WinRT also has parameterized (“generic”) interfaces. 

In short, Windows Runtime is a new set of libraries exposing Windows functionality and available to JavaScript/C#/VB/C++ as shown in figure 4. Each language has been made to understand and be able to call them directly rather than having to go through some thunking layer. Silverlight and WPF are flavors of XAML that run on the CLR. Among other functionality, Windows Runtime exposes a version of XAML very similar to Silverlight, but does so in a native way, not via the CLR. It can be accessed from the CLR, but also from C++.

Relation between .NET and WinRT

The important concept to understand is that the .NET framework now has two profiles.

1. .NET Metro profile (CLR that deal with Metro application)
2. .NET Client profile (CLR runtime for C#,VB.NET applications)

The .NET APIs for Metro style apps provide a set of managed types that you can use to create Metro style apps for Windows using C# or Visual Basic. The .NET APIs for Metro style apps include a subset of types from the .NET Framework and enable .NET developers to create Metro style apps within a familiar programming framework. You use these types with types from the Windows Runtime API to create Metro style apps.

In some cases, a type that you used in a .NET Framework application does not exist within the .NET APIs for Metro style apps; instead, you must use a type from the Windows Runtime. For example, the System.IO.IsolatedStorage.IsolatedStorageSettings class is not included in the .NET APIs for Metro style apps; instead, you use the Windows.Storage.ApplicationDataContainer class for app settings. You retrieve an instance of the ApplicationDataContainer class through the Windows.Storage.ApplicationData.Current.LocalSettings property.

• The .NET Framework 4.5 is used in both Desktop Mode apps and in Metro style apps. There is a difference though. Metro style apps use what is best described as a different .NET Profile (e.g. Desktop apps use the .NET Client Profile and Metro style apps use the .NET Metro Profile). There is NOT actually a different profile, but the implementation of .NET in Metro style apps is LIKE a different profile. Don’t go looking for this profile – its basically rules built into the .NET Framework and CLR that define what parts of the framework are available.
• There is only one CLR. Each application or application pool (depending on the app type) spins up a process and the CLR is used within that process. Meaning, a Metro style app and a Desktop Mode app running at the same time are using the same CLR bits, but separate instances of the CLR.
• Whether a Desktop Mode app or a Metro style app, if it is a .NET app, it is compiled to the same MSIL. There isn’t a special Windows 8 Metro IL – there is, like the CLR, only one MSIL.

In the above diagram (Fig 4) you can see that the CLR and the .NET Framework 4.5 are used for C# and Visual Basic apps in either Desktop Mode apps (blue side) or Metro style apps (green side). Silverlight is still only available in Desktop Mode as a plug-in to Internet Explorer (yes, out of browser is still supported in Desktop Mode). Another addition in this diagram is DirectX, which was strangely absent from the original diagram. DirectX is the defacto technology for high-polygon count applications, such as immersive games. DirectX leverages the power of C++.

This biggest confusion, has been around the use of the .NET Framework across the blue side and green side. The reason for the .NET Metro Profile is because the Metro style apps run in an app container that limits what the application can have access to in order to protect the end user from potentially malicious apps. As such, the Metro Profile is a subset of the .NET Client Profile and simply takes away some of the capabilities that aren’t allowed by the app container for Metro style apps. Developers used to .NET will find accessing the WinRT APIs very intuitive – it works similarly to having an assembly reference and accessing the members of said referenced assembly.

Additionally, some of the changes in the Metro Profile are to ensure Metro style apps are constructed in the preferred way for touch-first design and portable form factors. An example is File.Create(). Historically if you were using .NET to create a new file you would use File.Create(string fileLocation) to create the new file on the disk, then access a stream reader to create the contents of the file as a string. This is a synchronous operation – you make the call and the process stalls while you wait for the return. The idea of modern, Metro style apps is that asynchronous programming practices should be used to cut down on things like IO latency, such as that created by file system operations. What this means is that the .NET Metro Profile doesn’t provide access to File.Create() as a synchronous operation.

Ultimately all of this means that you have some choice, but you don’t have to sacrifice much if anything along the way. You can still build .NET and Silverlight apps the way you are used to, and they will run on Windows for years to come. If you want to build a new Metro style app, you have four options to choose from:

1. XAML and .NET (C# or VB): You don’t have to giving up too much in the .NET Framework (remember, you only give up what is forbidden by the Application Container), and you get access to WinRT APIs for sensor input and other system resources.
2. XAML and C++: You can use your skills in XAML and C++ in order to leverage (or even extend) WinRT. Of course you don’t get the benefit of the .NET Framework. You can use WRL or  with Component Extensions – C++/CX
3. HTML and JavaScript: You can leverage the skills you have in UI layout, and make calls from JavaScript to WinRT for access to system resources, and sensor input.
4. DirectX and C++: If you’re building an immersive game you can use DirectX and access the device sensors and system resources through C++ and WinRT.

.NET with WinRT

C#/VB: The end of P/Invoke : Calling native functions from .NET usually involves building up structures and manipulating pointers. Under WinRT all APIs are exposed as objects that C# and VB can consume directly. This puts .NET developers on level footing with C++ developers.

Application responsiveness is very important to Microsoft. In order to convey that to developers all OS-level API calls that take longer than 50 milliseconds will be exposed as an asynchronous operation.

A managed Metro application (e.g. written in C#) will still load the CLR.  The CLR version bundled with the preview is 4.0.30319.17020, and that’s the version that gets loaded inside a Metro app.  Symmetrically, a “fully” .NET application (e.g. a WinForms app) can reference the WinRT metadata assemblies and use WinRT APIs. This will be a necessity in some cases, for example to tap into the WinRT sensor APIs.

.NET has the ability to directly reference WinRT components as if they were .NET assemblies. This works differently from COM Interop – you don’t need any intermediate artifacts such as interop assemblies, you just read a .winmd file, and all types and their members in its metadata become visible to you as if they were .NET objects. Note that native C++ programs that use WinRT do not require CLR at all (WinRT libraries themselves are fully native) – the magic to expose all that stuff as managed is inside the CLR itself, and is fairly low level. If you ildasm a .NET program that references a .winmd, you’ll see that it actually looks like an extern assembly reference – there’s no sleight of hand trickery such as type embedding there.

It’s not a blunt mapping, either – CLR tries to adapt WinRT types to their equivalents, where possible. So e.g. GUIDs, dates and URIs become System.Guid, System.DateTime, and System.Uri, respectively; WinRT collection interfaces such as IIterable and IVector become IEnumerable and IList; and so on. This goes both ways – if you have a .NET object that implements IEnumerable, and pass it back to WinRT, it’ll see it as IIterable.

Some developers are confused as to whether .NET is there or not in the first place, as not all of the .NET APIs are present (File I/O, Sockets), many were moved and others were introduced to integrate with WinRT. When you use C# and VB, you are using the full .NET framework. But they have chosen to expose a smaller subset of the API to developers to push the new vision for Windows 8. And this new vision includes safety/sandboxed systems and asynchronous programming. This is why you do not get direct file system access or socket access and why synchronous APIs that you were used to consuming are not exposed. What they did was that they only exposed to the compiler a set of APIs when you target the Metro profile. So your application will not accidentally call File.Create for example. At runtime though, the CLR will load the full class library, the very one that contains File.Create, so internally, the CLR could call something like File.Create, it is just you that will have no access to it. So for managed applications, the Metro projects in Visual Studio are targeting a special subset of the .NET framework, and will not have access to all the classes – not even all the classes of the .NET Client Profile. For example, the System.IO.FileStream class is not available.

Although these limitations may seem artificial, annoying, and counterproductive, there is probably a good reason for having them. The Metro UI and Metro apps are a fresh start for Windows applications and their developers. There is no trivial way to port an existing UI app to Metro without a considerable UI redesign. It’s also a great opportunity to enforce additional limitations that make sure Metro apps respect the user’s privacy, conserve battery power, integrate well with the UI guidelines, and live in harmony within the application sandbox..

Ultimately, what this means is that your .NET Metro apps get access to a subset of the existing standard .NET libraries, and also to (native) WinRT libraries, some of which – particularly Windows.UI – look very similar to Silverlight, API-wise. You still have XAML to define your UI, and you still deal with the same basic concepts as in Silverlight – data bindings, resources, styles, templates etc. In many cases, it is possible to port a Silverlight app simply by using the new namespaces, and tweaking a few places in code where the API was adjusted. WinRT itself doesn’t have anything to do with HTML and CSS, and it bears relation to JavaScript only in a sense that it is also exposed there, similar to how it is done for .NET. You don’t need to deal with HTML/CSS/JS when you use WinRT UI libraries in your .NET Metro app.

C++11 with WinRT: C++ Development

A good news for C++ developers is that User interfaces in C++ will be written primarily in XAML for immersive applications. However, this is not available for classical, Win32 applications. This libraries for working with XAML have all been ported to C++ and are compiled to native x86. Metro applications written with XAML and C++ do not run on top of .NET, they get compiled directly to x86 just like any other Visual C++ application. Calling methods on UI controls is just like calling methods on any other object in C++.

Microsoft has created a new language called C++ Component Extension, or simply C++/CX. While the syntax is very similar to C++/CLI, the language is not managed, it’s still native. WinRT components built in C++/CX do not compile to managed code, but to 100% native code.

App Container and Application Permissions : Metro applications run in what’s known as the “app container”. This appears to replace the windowing environment used by Win32 based applications.

Most API calls are sent directly to the underlying kernel. Some, however, some will be routed through the system broker. The system broker ensures that applications can only access features that the user has approved. For example, the first time an application tries to access the camera the service broker will prompt the user for their approval. Applications are required to include a manifest that indicates all of the restricted services the application may need. This model will be very familiar to mobile device developers.

All Metro applications run within WinRT’s app container and thus are monitored by the system broker, even those written in C++. The idea is to limit the ability of applications to damage the system. While probably not impossible, building malware with WinRT will be much harder than it is in Win32.

Overlapping Windows No Longer Exist : In Metro, there are no overlapping top-level windows. There’s still Popup (http://msdn.microsoft.com/en-us/library/windows/apps/windows.ui.xaml.controls.primitives.popup). Dialog boxes, a core concept from previous versions of Windows, does not exist in WinRT. The performance cost and usability concerns are simply no longer justifiable to Microsoft. Applications that wish to use this pattern will have to develop other ways to express things such as message boxes. Another library that didn’t make it into WinRT is GDI. If an application is going to use the Metro interface it needs to do so from top to bottom, it appears that mixing Metro and classic user interfaces is not possible.

PlayTo Contract : Another contract that is being exposed is PlayTo. This allows applications to send media files such as audio and video to a charm. The charm will then allow the user to choose which application they want to use to view the file. Presumably these aren’t just physical files, but rather any form of media that can be expressed as a data stream.

All Metro Applications Must be Digitally Signed: Anonymous applications are not allowed. Applications can be self-signed for testing, but by the time they appear in the app store they will need to be signed using a real certificate.

JavaScript

Another major language for Windows 8 is JavaScript. While it doesn’t use XAML, it does have direct access to the underlying WinRT API just like native and .NET applications. This isn’t just a container like PhoneGap, JavaScript developers get the same rich API that other developers use. Since this is JavaScript the UI toolkit of choice is HTML and CSS rather than XAML. The same rendering engine used by Internet Explorer 10 us used by Metro JavaScript applications, though they don’t actually run in a browser. A JavaScript application looks just like any other Metro application. Expression Blend 5 now supports editing HTML5, much like you use it for editing XAML.

User controls in JavaScript are nearly on par with those in C++ and .NET. Some controls are intrinsic to the HTML rendering engine, others are written in JavaScript. These JavaScript based ones are div based much like controls built using jQuery.

The IInspectable interface contains three methods: GetIids, which returns the interfaces implemented by the component; GetRuntimeClassName, which returns the fully-qualified name of the component; and GetTrustLevel, which returns the component’s trust level that can be used to control component activation.

IInspectable is used by the dynamic language bindings from JavaScript. Given a reference to a WinRT component, the JavaScript interpreter can call the IInspectable::GetRuntimeClassName method and obtain the fully-qualified class name of the WinRT component. Using the namespace name the interpreter can ask Windows to load the metadata file (.winmd) describing the component, and from the metadata determine how to use the interfaces implemented by the component.

WinRT Projections

What we call "bindings" Microsoft now calls "projections". Projections are the process of exposing APIs to three environments: Native (C and C++), HTML/JavaScript and .NET.  If you author a component in C++ or a .NET language, its API will be stored in a WinMD file and you will be able to consume it from all three environments (Native, JavaScript and .NET). Even in C++ you are not exposed to COM. The use of COM is hidden behind the C++ projection tools. You use what looks and feels like a C++ object oriented API. To support the various constructs of WinRT, the underlying platform defines a basic set of types and their mappings to various environment. In particular, collection objects in WinRT are mapped to constructs that are native to each environment.

Conclusion

WinRT solves many of the problems of Win32, from an apps perspective. Apps created for WinRT are safe, secure, and sandboxed, can’t wreck other apps, can’t cause "Windows rot," and all install in just 2-3 seconds. They feature isolated storage, single folder installs (as per the Mac), and require user consent to access the general file system.

To summarize, WinRT does not replace .NET, nor does it compete with .NET – and the same applies to Win32. You can write server and client applications using .NET, including WPF and Silverlight. You can write server and client applications using C++, including Win32 and MFC. However, if you choose to believe in the Metro way, you can write Metro applications for Windows 8 using .NET languages or using C++ or using JavaScript, with a consistent set of APIs exposed through WinRT. At the same time the Win32 applications will continue to run just as before and you can still (and most certainly will) develop Win32 applications.

Finally, I am very impressed with the details of Windows 8! It’s fantastic. There are three ways to write WinRT applications. One is C++, in which case you write directly to the “projection” of WinRT into your language. The second is .NET, in which case your code goes via the CLR. The third is HTML and JavaScript, in which case your code goes via the “Chakra” JavaScript engine also used by Internet Explorer 9 and higher. There is no message loop in WinRT. There is no GDI in WinRT. All graphics are via DirectX. XNA, the .NET games framework, is not supported. The WinMD (Windows Metadata – which makes the whole Windows Metro API available to any language – hence the ability to use JavaScript, for example) is great. The new object-oriented API, with namespaces and interfaces etc., is great. The language projections are great. Oh and the native support in Microsoft C++!! for the Windows Runtime, and the fact that C++ is ideal for it (since it is, at the core, an extension of COM), is great. So much to learn!!

A great leader’s courage to fulfill his vision comes from passion, not position. - Paul John Maxwell