Emmy award: Bringing premium media to the web

On April 8th, the National Academy of Television, Arts & Science recognized Microsoft and other industry leaders with a Technology & Engineering Emmy award for our contributions in standardizing HTML5, Encrypted Media Extensions (EME), and Media Sources Extensions (MSE) for a Full TV Experience.

The 70th Technology & Engineering Emmy Awards, Las Vegas; 4/7/2019

John Simmons and I accepting the Emmy on behalf of the Windows and Microsoft Edge teams, respectively.

Today, premium media sites, like Netflix, Amazon Prime, Hulu, and others, use the HTML5 Video, Encrypted Media Extensions (EME), and Media Sources Extensions (MSE) web standards to deliver premium movies and TV experience on web sites, web apps, and TV set boxes using web technologies. But before these web standards, most premium media sites would rely on plug-ins, like Flash, Sliverlight, and others, to deliver these experiences on the web. Unfortunately, plug-ins had many flaws: they require the user to install them before they can view the video and had notoriously poor performance, reliability, and suffered from many security issues.

Starting in 2013, Microsoft worked with other industry partners, like W3C, Netflix, Comcast, and Google, to start developing several web standards that would support these same experiences on the web natively without plug-ins. We first helped standardize the HTML5 Video element, so browsers would natively support a video player without requiring the user install a plug-in. This video element was great for most videos but had some limitations: it used progressive playback where the video would have to be fully downloaded first and the video source was available for anyone to see. While this is okay for most media, content owners that stream premium video want to stream video as quickly and with the highest quality as possible and make sure that only users authorized by the streaming service should be able to view the video.

To solve the first problem, we helped standardize Media Source Extensions to support adaptive streaming. With this web standard and a streaming server, sites could adaptively switch between different media streams so users with good bandwidth would get highest quality and data rate video streams and users with lower bandwidth would get lower quality streams. This ensured that users are less likely to see buffering as their network conditions change or need to wait for the full video to download.

To solve the second problem, we helped standardize Encrypted Media Extension so web sites could leverage content protection systems, like Digital Rights Management (DRM), for web media. Content owners will not stream premium content if it can be easily saved and shared outside the service. With EME, web sites can ensure their premium media content is protected.

On the browser media team, we worked closely with the Windows team to bring these technologies to IE11 and Microsoft Edge. While IE11 was the first browser to implement many of these early web standards, Microsoft Edge continues to provide the best-in-class protected media support. Microsoft Edge often gets highest resolution and bitrate video because it’s the only browser on Windows to use the robust hardware-backed Microsoft PlayReady DRM – as video quality goes up, so does the need for better protection. Sites that rely on hardware-backed PlayReady DRM on Microsoft Edge can stream 1080p or 4k with high dynamic range (HDR) with confidence that their content cannot be stolen, while also giving their users the best battery life because we’re leveraging hardware. To provide developers with technology choice and highest level of compatibility, Microsoft Edge also supports multiple DRM systems, including both Microsoft PlayReady and Google Widevine DRM systems.

Emmy2

One of the Emmy awards given to Microsoft innovators.

I’m very appreciative of the National Academy of Television, Arts & Science for recognizing Microsoft’s contributions in helping make the web better for premium video experiences. It’s great to see how far the web has come in just a few years and I’m looking forward to seeing how we continue to make it even better with more powerful capabilities in the future.

Build 2019: Moving the web forward with Microsoft Edge

Last year, we announced that the next major version of Microsoft Edge will be based off of the Chromium open source project and that we intended to become significant contributors to that open source project in a way that would not just make Microsoft Edge, but other browsers as well, better on both PCs and other devices.

At Build 2019, Gaurav Seth and I went deeper into our plans for how we intended to move the web forward with Microsoft Edge, including how we intended to contribute to Chromium, our ongoing work in web standards, and delivering a consistent set of developers tools and application experiences built using web technologies.

Build2019Tali

Video link to the Build 2019 session: Moving the web forward with Microsoft Edge

What to expect in the new Microsoft Edge Insiders Channel

John and I share what to expect in the new Microsoft Edge Insiders Channel on the Microsoft Edge Dev blog.

Today we are shipping the first Dev and Canary channel builds of the next version of Microsoft Edge, based on the Chromium open-source project. We’re excited to be sharing this work at such an early stage in our development process. We invite you to try out the preview today on your devices, and we look forward to working together with the Microsoft Edge Insider community to make browsing the best experience possible for everyone.

In this post, we’ll walk you through how the new release channels work, and share a closer look at our early work in the Chromium open source project, as
well as what’s coming next.

Introducing the Microsoft Edge Insider Channels

The new Microsoft Edge builds are available through preview channels that we call “Microsoft Edge Insider Channels.” We are starting by launching the first two Microsoft Edge Insider Channels, Canary and Dev, which you can download and try at the
Microsoft Edge Insider site. These channels are available starting today on all supported versions of Windows 10, with more platforms coming soon.


Screenshot of download page showing three Microsoft Edge Insider Channels - Beta Channel, Dev Channel, and Canary Channel

Canary channel will be updated daily, and Dev channel will be updated weekly. You can even choose to install multiple channels side-by-side for testing—they will have separate icons and names so you can tell them apart. Support for other platforms, like Windows 7, Windows 8.1, macOS, and other channels, like Beta and Stable, will come later.

Every night, we produce a build of Microsoft Edge―if it passes automated testing, we’ll release it to the Canary channel. We use this same channel internally to validate bug fixes and test brand new features. The Canary channel is truly the bleeding edge, so you may discover bugs before we’ve had a chance to discover and fix them. If you’re eager for the latest bits and don’t mind risking a bug or two, this is the channel for you.

If you prefer a build with slightly more testing, you might be interested in the Dev channel. The Dev channel is still relatively fresh―it’s the best build of the week from the Canary channel. We look at several sources, like user feedback, automated test results, performance metrics, and telemetry, to choose the right Canary build to promote to the Dev channel. If you want to use the latest development version of Microsoft Edge as a daily driver, this is the channel for you. We expect most users will be on the Dev channel.

Later, we will also introduce the Beta and Stable channels. The Beta channel reflects a significantly more stable release and will be a good target for Enterprises and IT Pros to start piloting the next version of Microsoft Edge.

We are not changing the existing version of Microsoft Edge installed on your devices at this time – it will continue to work side by side with the builds from any of the Microsoft Edge Insider Channels.

Adopting and contributing to the Chromium open source project

When we initially announced our decision to adopt Chromium as the foundation for future versions of Microsoft Edge, we published a set of open source principles and declared our intent to contribute to the Chromium project to make Microsoft Edge and other Chromium-based browsers better on PCs and other devices.

While we will continue to focus on delivering a world class browsing experience with Microsoft Edge’s user experience and connected services, when it comes to improving the web platform, our default position will be to contribute to the Chromium project.

We still have a lot to learn as we increase our use of and contributions to Chromium, but we have received great support from Chromium engineers in helping us get involved in this project, and we’re pleased to have landed some modest but meaningful contributions already. Our plan is to continue working in Chromium rather than creating a parallel project, to avoid any risk of fragmenting the community.

Our early contributions include landing over 275 commits into the Chromium project since we joined this community in December. We also have started to make progress on some of the initial areas of focus we had shared:

Accessibility

We are committed to building a more accessible web platform for all users. Today, Microsoft Edge is the only browser to earn a perfect score on the HTML5Accessibility browser benchmark, and we’re hoping to bring those contributions to the Chromium project and improve web experiences for all users.

  • Modern accessibility APIs. To enable a better accessibility experience for screen readers, like Windows Narrator, magnifiers, braille displays, and other accessibility tools, we’ve shared our intent to implement support for the Microsoft UI Automation interfaces, a more modern and secure Windows accessibility framework, in Chromium. We’re partnering with Google’s Accessibility team and other Chromium engineers to land commits and expect the full feature to be completed later this year.
  • High contrast. To ensure our customers have the best readability experience, we’re working in the W3C CSS working group to standardize the high-contrast CSS Media query and have shared our intent to implement it in Chromium. This will allow customers to use the Windows Ease of Access settings to select their
    preferred color contrast settings to improve content readability on Windows devices.
  • HTML video caption styling. We’ve partnered with Chromium engineers to land support for Windows Ease of Access settings to improve caption readability on Windows 10.
  • Caret browsing. For customers who use their keyboard to navigate the web and select text, we’ve shared our intent to implement caret browsingin Chromium.
  • We’re starting to work with our Chromium counterparts to improve the accessibility of native web controls, like media and input controls. Over time we expect this work will help Chromium earn a perfect score on the HTML5Accessibility browser benchmark.

ARM64

We’ve been collaborating with Google engineers to enable Chromium to run natively on Windows on ARM devices starting with Chromium 73. With these contributions, Chromium-based browsers will soon be able to ship native implementations for ARM-based Windows 10 PCs, significantly improving their performance and battery life.

Touch

To help our customers with touch devices get the best possible experience, we’ve implemented better support for Windows touch keyboard in Chromium, now supporting touch text suggestions as you type and “shape writing” that lets you type by swiping over keys without releasing your finger.

Scrolling

Microsoft Edge is known for class-leading scrolling experiences on the web today, and we’re collaborating closely with Chromium engineers to make touchpad, touch, mouse wheel, keyboard, and sidebar scrolling as smooth as possible. We’re still early in this investigation, but have started sharing some ideas in this area.

Media

Premium media sites use the encrypted media extensions (EME) web standard and digital rights management (DRM) systems to protect streaming media content so that it can only be played by users authorized by the streaming service. In fact, Microsoft and other industry partners were recognized with a Technology & Engineering Emmy award yesterday for helping bring premium media to the web through this and other web standards. To provide users with the highest level of compatibility and web developers with technology choice, Microsoft Edge now supports both Microsoft PlayReady and Google Widevine DRM systems.

While Microsoft Edge often gets highest resolution and bitrate video because it uses the robust hardware-backed Microsoft PlayReady DRM, there are some sites that only support the Google Widevine DRM system. Sites that rely on hardware-backed PlayReady DRM on Microsoft Edge will be able to continue to stream 1080p or 4k with high dynamic range (HDR) or Dolby Vision, while those that only support Widevine will just work in Microsoft Edge for the first time.

We also want to help contribute improvements to video playback power efficiency that many of our Microsoft Edge users have come to expect. We’re early in these investigations but will be partnering closely with the Chromium team on how we can help improve this space further.

Windows Hello

Microsoft Edge supports the Windows Hello authenticator as a more personal and secure way to use biometrics authentication on the web for password-less and two-factor authentication scenarios. We’ve worked with the Chromium team to land Windows Hello support in the Web Authentication API in Chromium 73+ on the latest Windows 10 Insider Preview release.

Evolving the web through standards

While we’re participating in the Chromium open source project, we still believe the evolution of the open web is best served though the standards communities, and the open web benefits from open debate from a wide variety of perspectives.

We are continuing to remain deeply engaged in standards discussions where the perspectives of vendors developing different browsers and the larger web community can be heard and considered. You can keep track of all Microsoft explainer documents on the Microsoft Edge Explainers GitHub.

HTML Modules

For example, we recently introduced the HTML Modules proposal, which is now being developed in the W3C and WHATWG Web Components Incubation Groups.

We’ve heard from web developers that while ES6 Script Modules are a great way for developers to componentize their code and create better dependency management systems, the current approach doesn’t help developers who use declarative HTML markup. This has forced developers to re-write their code to generate markup dynamically.

We’ve taken lessons learned from HTML Imports to introduce an extension of the ES6 Script Modules system to include HTML Modules. Considering the early support we’ve received on this feature from standards discussions, we’ve also shared our intent to implement this feature in Chromium.

User Agent String

With Microsoft Edge adopting Chromium, we are changing our user agent string to closely resemble that of the Chromium user agent string with the addition of the “Edg” token. If you’re blocking site access on user agent strings, please update your logic to treat this string as another Chromium-based browser. Below is the user agent string for the latest Dev Channel build of Microsoft Edge:

 

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.48 Safari/537.36 Edg/74.1.96.24

 

We’ve selected the “Edg” token to avoid compatibility issues that may be caused by using the string “Edge,” which is used by the current version of Microsoft Edge based on EdgeHTML. The “Edg” token is also consistent with existing tokens used on iOS and Android. We recommend that developers use feature detection where possible and avoid browser version detection through the user agent string, as it results in more maintenance and fragile code.

User Experience

We are committed to building a world class browser with Microsoft Edge through differentiated user experience features and connected services. With this initial release, we have made a number of changes to the user interface to make our product feel more like Microsoft Edge.

However, you will continue to see the look and feel of the browser evolve in future releases as we iterate and listen to customer feedback. We do not plan to contribute user experience or Microsoft service changes to Chromium, since browser vendors generally like to make their own decisions in these areas.

We know that this initial release is still missing a few features that are available in the current version of Microsoft Edge. We’re in the early stages and are intentionally focusing on fundamentals as we continue to work towards a complete feature set.

Over time, we will roll out new features and run experiments to gauge user interest and satisfaction, and to assess the quality of each new feature or improvement. This will help us ensure that all new features address our customers’ needs in the best possible way and meet our quality standards.

Integration with Microsoft services

While the next version of Microsoft Edge will be based on Chromium, we intend to use the best of Microsoft wherever we can, including our services integrations. Some of these services integrations include:

  • Bing Search powers search and address bar suggestions by default.
  • Windows Defender SmartScreen delivers best-in-class phishing and malware protection when navigating to sites and downloading content.
  • Microsoft Account service and Azure Active Directory can now be used to sign-in to the browser to help you manage both your personal and work accounts. You can even use multiple identities at the same time in different browser sessions.
  • Microsoft Activity Feed Service synchronizes your data across Microsoft Edge preview builds. We currently synchronize your favorites across your Windows 10 desktop devices running Microsoft Edge preview builds. In future builds, we
    will also sync passwords, browsing history, and other settings across all supported platforms, including Microsoft Edge on iOS and Android.
  • Microsoft News powers the new tab experience, giving you the choice of an inspirational theme with vivid Bing images, a focused theme that helps you get straight to work, or a more news focused informational theme.

Feedback

Getting your feedback is an important step in helping us make a better browser – we consider it essential to create the best possible browsing experience. If you run into any issues or have feedback, please use the “Send Feedback” tool in Microsoft Edge. Simply click the smiley face next to the Menu button and let us know what you like or if there’s something we can improve.

For web developers, if you encounter an issue that reproduces in Chromium, it’s best to file a Chromium bug. For problems in the existing version of Microsoft Edge, please continue to use the EdgeHTML Issue Tracker.

You can also find the latest information on the next version of Microsoft Edge and get in touch with the product team to share feedback or get help on the Microsoft Edge Insider site.

We’re delighted to share our first Canary and Dev builds of the next version of Microsoft Edge! We hope you’ll try the preview out today, and we look forward to hearing your feedback in the Microsoft Edge Insider community.

Jatinder Mann, Group Program Manager, Web Platform
John Hazen, Group Program Manager, Operations

Service Workers: Going beyond the page

Ali and I announce Service Worker and Progressive Web App support on the Microsoft Edge blog:

We’re thrilled to announce that today’s Windows Insider build enables Service Workers by default in Microsoft Edge for the first time.

This is an exciting milestone for us and for the web! These new APIs allow the web to offer better experiences when devices are offline or have limited connectivity, and to send push notifications even when the page or browser isn’t open.

This milestone also establishes the foundation for full-featured Progressive Web App experiences in Microsoft Edge and the Microsoft Store. We’ve carefully tailored our implementation together with Windows and are excited to be the only browser on Windows 10 to provide push handling in the background to optimize for better battery usage. We’ll have more to share about PWAs on Windows in the weeks ahead, so be sure to stay tuned!

We believe Service Workers are a foundational new tool for the web with the potential to usher in a new golden age of web apps. You can try out web experiences powered by Service Workers in Microsoft Edge on starting with today’s Windows Insider release, 17063.

The state of the web

Not too long ago, the web’s capabilities were lagging behind what native apps could do. Browser vendors, standards bodies, and the web community have relentlessly attacked this gap over the last decade, with the goal of enabling richer web experiences.

A particularly egregious sore spot for the web has always been how it handled—or failed to handle—the lack of an internet connection, or a poor-quality connection. Offline and the web never really went well together—in most cases, we were given a frustrating error page that only made it clearer that the web’s greatest asset was also its greatest weakness: the internet. In contrast, native apps are typically designed to provide a good experience even while offline or experiencing spotty service.

On top of that, native apps can re-engage their users with push notifications. On the web, after the browser or the app disappears, so does your ability to deliver relevant information or updates to your users.

In the rearview: Application Cache

By 2012, we were able to do offline on the web by introducing a new standard: Application Cache (or App Cache for short). A site author could list URLs for the browser to keep in a special cache called the App Cache so that when you visited the page you would see something other than an infuriating error page that would make you want to smash your keyboard and throw it against the wall.

Unfortunately, this wasn’t the silver bullet we were looking for in terms of bringing offline to the web. There are more than a few well-documented limitations for App Cache that made it confusing and error prone for many users and web developers. The sum of it all was that there was little control in how the App Cache would work since most of the logic occurred behind-the-scenes in the browser.

That meant if you ran into an issue, it would be exceedingly difficult to understand how to resolve it. There was an obvious need for something that gave greater control to developers by offering the capabilities of App Cache but made it more programmatic and dynamic while doing away with many of its limitations that made it difficult to debug.

Hit Rewind

App Cache left a lot to be desired. For the next swing at enabling offline scenarios, it was clear that browsers needed to provide web developers true control over what would happen when a page and its sub-resources were downloaded, rather than having it automatically and transparently handled in the browser.

With the ability to intercept network requests from a page and to prescribe what to do with each, site authors would be able to respond back to the page with a resource that it could use. Before we get there, it seems that we would need to revisit one of the most fundamental aspects of the web: fetching a resource.

How fetching!

As you may recall from my last post on Fetch, we now have the fetch() method as well as the Request and Response primitives. As a refresher, here’s how you might retrieve some JSON data using the fetch API:

Every request that happens on a page (including the initial navigation, CSS, images, scripts and XHR) is defined as a fetch. The fetch() method (as shown in the code sample) is just a way to explicitly initiate a fetch, while implicit fetches occur when loading a page and all of its sub-resources.

Since we’ve unified the concepts of fetching resources across the web platform, we can provide site authors the chance to define their own behavior via a centralized algorithm. So, how do we pass that control over to you?

Service worker: the worker that serves

Web workers have long been a great tool to offload intensive JavaScript to a separate execution context that doesn’t block the UI nor interaction with the page. Given the right conditions, we can repurpose the concept of a web worker to allow a developer to write logic in response to a fetch occurring on a page. This worker wouldn’t just be a web worker, though. It deserves a new name: Service worker.

A service worker, like a web worker, is written in a JavaScript file. In the script, you can define an event listener for the fetch event. This event is special, in that it gets fired every time the page makes a request for a resource. In the handler for the fetch event, a developer will have access to the actual Request being made.

You can choose to respond with a fetch for the provided Request using the JavaScript APIs which returns a Response back to the page. Doing this essentially follows the typical browser behavior for that request—it’s not intrinsically useful to just do a fetch for the request, ideally it would be more useful if we save previous Response objects for later.

Cache it!

The Cache API allows us to look up a specific Request and get its associated Response object. The APIs give access to a new underlying key/value storage mechanism, where the key is a Request object and the value is a Response object. The underlying caches are separate from the browser’s HTTP cache and are origin-bound (meaning that they are isolated based on scheme://hostname:port) so that you cannot access caches outside of your origin. Each origin can define multiple different caches with different names. The APIs allow you asynchronously open and manipulate the caches by making use of Promises:

These caches are completely managed by the developer, including updating the entries and purging them when they’re no longer needed – this allows you to rely on what will be there when you may not necessarily be connected to the internet.

Although the Caches API is defined as part of the Service Worker spec, it can also be accessed from the main page.

So now you have two asynchronous storage APIs to choose from: Indexed DB and the Caches API. In general, if what you’re trying to store is URL-addressable, use the Caches API; for everything else, use Indexed DB.

Now that we have a way to save those Response objects for later use, we’re in business!

Back to the worker

With a service worker, we can intercept the request and respond from cache. This gives us the ability to improve page load performance and reliability, as well as to offer an offline experience. You can choose to let the fetch go through to the internet as is, or to get something from the cache using the Cache API.

The first step to using a service worker is to register it on the page. You can do this by first feature-detecting and then calling the necessary APIs:

As part of the registration, you’ll need to specify the location of the service worker script file and define the scope. The scope is used to define the range of URLs that you want the service worker to control. After a service worker is registered, the browser will keep track of the service worker and the scope it is registered to.

Upon navigating to a page, the browser will check if a service worker is registered for that page based on the scope. If so, the page will go on to use that service worker until it is navigated away or closed. In such a case, the page is said to be controlled by that service worker. Otherwise, the page will instead use the network as usual, and will not be controlled by a service worker.

Upon registration, the service worker won’t control the page that registered it. It will take control if you refresh the page or you open a new page that’s within its scope.

After initiating the registration of a service worker, it will go through the registration process. That will involve going through the different phases of its lifecycle.

The service worker lifecycle

Let’s unpack the different phases of the service worker’s lifecycle, starting with what happens once you try to register it:

  • Installing: This is the first step that any service worker goes through. After the JavaScript file has been downloaded and parsed by the browser, it will run the install event of your script. That’s when you’ll want to get everything ready such as priming your caches.

In the following example, the oninstall event handler in the service worker will create a cache called “static-v1” and add all the static resources of the page to the cache for later use by the fetch handler.

  • Installed: At this point, the setup is complete, and the service worker is awaiting all pages/iframes (clients) that are controlled by this service worker registration to be closed so that it can be activated. It could be potentially problematic to change the service worker for pages that are still actively using a previous version of the service worker, so the browser will instead wait until they’ve been navigated away or closed.
  • Activating: Once no clients are controlled by the service worker registration (or if you called the skipWaiting API), the service worker goes to the activating phase. This will run the activate event in the service worker which will give you the opportunity to clean up after the previous workers that may have left things behind, such as stale caches.

In this example, the onactivate event handler in the service worker will remove all caches that are not named “static-v1.”

  • Activated: Once it’s been activated, the service worker can now handle fetch and other events as well!

In this example, the onfetch event handler in the service worker will respond back to the page with a match from the cache if it exists and if there isn’t an entry in the cache, it will defer to making a fetch to the internet instead. If that fetch fails, it will resort to returning a fallback.

  • Redundant: The final phase of the service worker is when it is being replaced by another service worker because there’s a new one available that is going to take its place.

There’s more to it: the big picture

So far, we’ve explored the following service worker events: install, activate, and fetch. Install and activate are considered lifetime events while fetch is considered a functional event. What if we could expand on the service worker’s programming model and introduce other functional events that could plug in to it? Given that service workers are event-driven and are not tied down to the lifetime of a page, we could add other events such as push and notificationclick which would present the necessary APIs to enable push notifications on the web.

Push it to the limit

Push notifications provide a mechanism for developers to inform their users in a timely, power-efficient and dependable way, that re-engages them with customized and relevant content. Compared to current web notifications, a push notification can be delivered to a user without needing the browser/app or page to be opened.

The W3C Push API and Notification API go hand-in-hand to enable push notifications in modern browsers. The Push API is used to set up a push subscription and is invoked when a message is pushed to the corresponding service worker. The service worker then is responsible for showing a notification to the user using the Notification API and reacting to user interaction with the notification.

A standardized method of message delivery is also important for the W3C Push API to work consistently across all major browsers where application servers will need to use multiple push services. For instance, Google Chrome and Mozilla Firefox use Firebase Cloud Messaging (FCM) and Mozilla Cloud Services (MCS), respectively while Microsoft Edge relies on the Windows Push Notification Service (WNS) to deliver push messages. To reach reasonable interoperability with other browsers’ messaging services, WNS has now deployed support for the Web Push protocols being finalized within IETF, as well as the Message Encryption spec and the Voluntary Application Server Identification (VAPID) spec for web push. Web developers can now use the Web Push APIs and service workers to provide an interoperable push service on the web.

To start, you’ll first need to make sure your web server is setup to send pushes. The Web-Push open-source library is a great reference for anyone new to web push. The contributors have done a reasonable job in keeping up with the IETF specs. After starting up a node.js server based on the web-push library, you’ll need to setup the VAPID keys. Keep in mind that you’ll need to use HTTPS as it is required for service workers and push. You only need to set up the VAPID keys once which can be generated easily using the corresponding function in the web-push library.

Once that’s all sorted out, it’s time to take advantage of push in your site or app. Once the page loads, the first thing you’ll want to do is get the public key from the application server so that you can set up the push subscription.

With the public key in hand, as before, we’ll need to install the service worker, but this time, we’ll also create a push subscription.

Before a new push subscription is created, Microsoft Edge will check whether a user granted permission to receive notifications. If not, the user will be prompted by the browser for permission. You can read more about permission management in an earlier post about Web Notifications in Microsoft Edge. From a user’s perspective, it’s not obvious whether a notification will be shown via the page or through a push service, so we are using the same permission for both types of notifications.

To create a push subscription, you’ll need to set the userVisibleOnly option to “true” – meaning a notification must be shown as a result of a push – and provide a valid applicationServerKey. If there is already a push subscription, there is no need to subscribe again.

At any point when a push is received by the client, a corresponding service worker is run to handle the event. As part of this push handling, a notification must be shown so that the user understands that something is potentially happening in the background.

Of course, after a notification is shown, there is still the matter of dealing with when its been clicked. As such, we need to have another event listener in the service worker that would handle this case.

In this case, we first dismiss the notification and then we can choose to open a window to the intended destination. You’re also able to sort through the already open windows and focus one of those, or perhaps even navigate an existing window.

Push: The Next Generation

As part of our ongoing commitment to expanding the possibilities of the web, Microsoft Edge and PWAs in Windows will handle these service worker push event handlers in the background. That’s right, there’s no need for Microsoft Edge or your PWA to be running for the push to be handled. That’s because we’ve integrated with Windows to allow for a more holistic approach to push notifications. By leveraging Windows’ time-tested process lifetime management, we’re able to offer a system that reacts appropriately to system pressures such as low battery or high CPU and memory usage.

For our users it means better resource management and battery life expectations. For our developers, it means a push event handler that will get to run to completion without interruption from a user action such as closing the browser window or app. Note that a service worker instance that is running in the foreground for the fetch event will not be the same as the one in the background handling the push event.

Notifications in Microsoft Edge and PWAs will be integrated in the Windows Action Center. If you receive a notification and didn’t get the chance to act on it, it will get tucked away into the Action Center for later. That means that notifications never get left unseen. On top of that, the Action Center will group multiple notifications coming from the same domain so that users have an easier time sorting through them.

Service worker: properties

I’d like to take a moment to go over some things you should keep in mind when using service workers in your web app or site. In no particular order, here they are:

  • HTTPS-only. Service workers will not work in HTTP; you will need to use HTTPS. Fortunately, if you’re testing locally, you’re allowed to register service workers on localhost.
  • No DOM access is allowed. As with web workers, you don’t get access to the page’s object model. This means that if you need to change something about the page, you’ll need to use postMessage from the service worker to the page so that you can handle it DOM changes from the page.
  • Executes separate from page. Because these scripts are not tied to the lifetime of a page, it’s important to understand that they do not share the same context as the page. Aside from not having access to the DOM (as stated earlier), they won’t have access to the same variables available on the page.
  • Trumps App Cache. Service workers and App Cache don’t play well together. App Cache will be ignored when service worker is in use. Service workers were meant to give more control to the web developer. Imagine if you had to deal with the magic of App Cache while you’re trying to step through the logic of your service worker.
  • Script can’t be on CDN. The JavaScript file for the service worker can’t be hosted on a Content Distribution Network (CDN), it must be on the same domain as the page. However, if you like, you can import scripts from your CDN.
  • Can be terminated any time. Remember that service workers are meant to be short-lived and their lifetime is tied to events. In particular, service workers have a time limit in which they must finish executing their event handlers. In other cases, the browser or the operating system may choose to terminate a service worker that impacts the battery, CPU, or memory consumption. In either case, it would be prudent to not rely on global variables in the service worker in case a different service worker instance is used on a subsequent event that’s being handled.
  • Only asynchronous requests allowed. Synchronous XHR is not allowed here! Neither is localStorage, so it’s best to make use of Indexed DB and the new Caches API described earlier.
  • Service worker to scope is 1:1. You’ll only be able to have one service worker per scope. That means if you try to register a different service worker for a scope that already has a service worker, that service worker will be updated.

Recap

As you can see, service workers are so much more than an HTTP proxy, they are in fact a web app model that enable event-driven JavaScript to run independent of web pages. Service workers were brought in to the web platform as a necessity to solve offline, but it’s clear that they can do so much more as we continue to extend their capabilities to solve other scenarios. Today we have push, but in the future, there will be other exciting capabilities that will bring the web that much closer to offering the captivating and reliable experiences we’ve always wanted.

Go put a worker to work!

So, what are you waiting for? Go and install the latest windows insider preview build and test out service workers in Microsoft Edge today. We’d love to hear your feedback, so please file bugs as you see them!

— Ali Alabbas, Program Manager, Microsoft Edge
— Jatinder Mann, Program Manager, Microsoft Edge

Get better quality video with Microsoft Edge

My team and I wrote this post on the Window Experience blog:

When it comes to video, the closer to the hardware, the better.  From video hardware acceleration to PlayReady Content Protection and the Protected Media Path, Windows 10 is designed to provide the highest quality, most secure, and most power-efficient video playback available on any version of Windows.  Microsoft Edge has been engineered to optimize for and take advantage of these Windows 10 built-in media capabilities, providing the best video experience of any browser on Windows 10 based on our data and testing. So go ahead, binge watch your favorite shows on Microsoft Edge!

Most Power Efficient Video Playback

As we recently blogged, you get more out of your battery with Microsoft Edge.  This time-lapse video test shows Microsoft Edge lasting three hours longer than Google Chrome streaming the same content side by side on identical Surface Book machines.  Our results have shown that Microsoft Edge outlasts the rest, delivering 17%-70% more battery life than the competition. Today, we are publishing details on the test methodology that was used and in this post, we’ll dig into the technologies that make Microsoft Edge so much more efficient.

Battery life comparison

Microsoft Edge has the most power efficient video playback because it was engineered to take advantage of Windows 10 platform features that keep the CPU in low power states during video playback.  It does this by offloading CPU intensive video processing operations to power efficient peripheral hardware found in modern PCs and mobile devices. This starts with the use of Microsoft DirectX video acceleration (DXVA) to offload decoding of compressed video.  For rendering, Microsoft Edge also works with Multiplane overlay display hardware and sophisticated graphics and UI compositing features to offload video rendering operations. This significantly reduces memory bandwidth required for video processing and compositing at the display.

CPU management in the Windows 10 media stack keeps the CPU running in the lowest power states possible, without compromising UI responsiveness.  It is also able to run the display at lower refresh rates during full screen playback of film based content.  This saves power by reducing memory bandwidth and improves the quality of film playback by reducing video judder caused by the conversion of the film frame rate (24 Hz, for example) on displays running at 60 Hz.  And Microsoft Edge also takes advantage of a feature of the Windows 10 audio stack to offload audio stream processing from the main CPU to dedicated power-efficient audio processing hardware.

Power savings from these features are available to other browsers, but it requires other browser vendors to optimize performance on Windows devices, while Microsoft Edge was designed to provide these power savings innately.  And to be clear, the power difference playing higher quality content, like 1080p, becomes even greater.  Tight integration with Windows media, graphics, and composition stacks allows Microsoft Edge to render the highest quality content with minimal power draw.

Higher Quality Video

In our video tests, not only was Microsoft Edge the most power efficient, but the premium video site we used also sent higher resolution and bitrate video to Microsoft Edge compared to the other browsers.  Here are the details:

Maximum Resolution Maximum Bitrate
Microsoft Edge (EdgeHTML 13.10586) 1080p 7500
Opera 38 720p 4420
Mozilla Firefox 46 720p 4420
Google Chrome 51 720p 4420

The fact that Microsoft Edge received 1080p content in our power test means it actually ran a somewhat higher power draw than it otherwise would have playing 720p content like the other browsers.  Microsoft Edge provided the highest quality content and also delivered the longest battery life.

Content owners that stream premium video on the web need to make choices that balance providing the best quality possible, while also ensuring that the content is protected.  As quality increases, so does the need for strong Digital Rights Management (DRM), systems that protect media streams so that they can only be played by users authorized by the streaming service.  This is important now as companies make decisions to stream 1080p, and will become even more important as video resolutions increase. Content owners will not stream premium content if it can be easily saved and shared outside the service.

Microsoft Edge was built to take advantage of platform features in Windows 10.  It is optimized to use PlayReady Content Protection and the media engine’s Protected Media Path, whereas Chrome and Opera implement Widevine, and Firefox implements both Adobe Access and Widevine.  Like video decode efficiency, content protection in the platform and closer to the hardware can offer superior performance.  Likewise, the better the content protection, the better the video quality the service is likely to provide to that browser.

What Does the Future Hold?

Displays have already moved beyond Full HD/1080p to UltraHD/4K video.  Upcoming improvements to audio and video quality go beyond just more audio channels and more pixels.  We will see an increase in the number and intensity of colors displayed (color gamut), higher frame rate, high dynamic range (HDR) and broad availability of immersive, spatial or 3-D audio. In the Alliance for Open Media, Microsoft and other leading internet companies are developing next-generation media formats, codecs and other technologies for UltraHD video.

Microsoft is working with industry leading graphics chipset companies to expand support for hardware acceleration of new higher quality content.  We are also working with chipset companies to support Enhanced Content Protection that moves the Protected Media Path into peripheral hardware for an even higher level of security.  The code running in the peripheral is isolated from the main OS, which provides an increased level of protection, and will likely enable content owners to stream 4k or higher resolution video with confidence.  We call the improved security from hardware based DRM “Security Level 3000”.

Chipsets that support hardware based DRM have been shipping in newer PCs.  The full support system awaits completion of software components that are coming in Fall of 2016.  The Windows 10 platform has significant power and security advantages for media playback available to any application.  Paired with the media features built into Windows 10, Microsoft Edge has best-in-class battery life and video playback quality on Windows 10 devices, including PCs, Mobile, Xbox, HoloLens, and other devices.

Let us know what you think over at @MSEdgeDev!

  • Jatinder Mann, Senior Program Manager Lead, Microsoft Edge
  • Jerry Smith, Senior Program Manager, Microsoft Edge
  • John Simmons, Principal Program Manager, Windows Media

Managing Microsoft Edge in the enterprise

In this Microsoft Edge blog post, we discuss Microsoft Edge manageability options for Enterprise customers:

At last year’s Microsoft Ignite conference, we introduced the enterprise story for the web on Windows 10. Microsoft Edge is designed from the ground up to provide a modern, interoperable, and secure browsing experience; in addition, Internet Explorer 11 is also a part of Windows 10 to help bring all your legacy line of business (LOB) applications forward.

Microsoft Edge and Internet Explorer 11 work together to help ease IT management overhead, and also provide a seamless user experience for your users. In this post, we’ll walk through the policies you can use to manage Microsoft Edge in the enterprise for both PCs and mobile devices, including some new policies coming in the Windows 10 Anniversary Update.

Policies currently supported in Microsoft Edge

With Microsoft Edge, we set out to provide a simple, consistent set of scenario-driven management policies to help manage Windows 10 browser deployments on both desktop and mobile. The policies for Microsoft Edge on desktop are available as both Group Policy settings and MDM settings. On mobile they are available as MDM settings.

Here is a summary of all the policies supported by Microsoft Edge grouped by Windows 10 releases:

  • Available in Windows 10 version 1507 or later:
    • Configure Autofill
    • Configure Cookies
    • Configure Do Not Track
    • Configure Password Manager
    • Configure Pop up Blocker
    • Configure search suggestions in the Address bar
    • Configure the Enterprise Mode Site List
    • Configure the SmartScreen Filter
    • Send all intranet sites to Internet Explorer 11
  • Available in Windows 10 version 1511 or later:
    • Allow Developer Tools
    • Allow InPrivate browsing
    • Allow web content on New Tab page
    • Configure Favorites
    • Configure Home pages (see additional note below)
    • Prevent bypassing SmartScreen prompts for files
    • Prevent bypassing SmartScreen prompts for sites
    • Prevent sharing LocalHost IP address for WebRTC

What’s new in Windows 10 Anniversary update

We have added support for the following new Microsoft Edge management policies as a part of the Windows 10 Anniversary Update:

  • Allow access to the about:flags page
  • Allow usage of extensions
  • Show a transitional message when opening Internet Explorer sites

We’ve made a few updates to existing policies based on feedback from customers.  First, all of the Microsoft Edge Group Policy settings on desktop are now available in both the User and Machine policy hives. Second, the home page policy configured on a domain-joined device will no longer allow the user to override the setting.

You can find further details on all Microsoft Edge policies on TechNet, including info about Windows 10 policies that also apply to Microsoft Edge, such as Cortana and Sync settings. Your feedback is important to us, so please let us know what you think or if you have any questions about these changes!

– Dalen Abraham, Principal Program Manager Lead

– Jatinder Mann, Senior Program Manager Lead

– Josh Rennert, Program Manager

A world without passwords: Windows Hello in Microsoft Edge


In this Microsoft Edge blog post, my team and I discuss bringing biometric authentication to the web:

At Build 2016, we announced that Microsoft Edge is the first browser to natively support Windows Hello as a more personal, seamless, and secure way to authenticate on the web. This experience is powered by an early implementation of the Web Authentication (formerly FIDO 2.0) specification, and we are working closely with industry leaders in both the FIDO Alliance and W3C Web Authentication working group to standardize these APIs. Try out this Test Drive demo in Microsoft Edge on recent Windows Insider builds to experience Windows Hello on the web today!

WindowsHello

Passwords can be a hassle. Most people don’t create strong passwords or make sure to maintain a different one for every site. People create easy-to-remember passwords and typically use the same passwords across all of their accounts. Surprisingly – and if it’s not surprising to you, you may want to change your password – passwords like “123456” and “password” are very common. Malicious actors can use social engineering, phishing, or key logging techniques to steal passwords from your machine, or they can compromise the server where the passwords are stored. When the same password is used across several sites, compromising one account can expose many others to abuse.

We look forward to a web where the user doesn’t need to remember a password, and the server doesn’t need to store a password in order to authenticate that user. Windows Hello, combined with Web Authentication, enables this vision with biometrics and asymmetric cryptography. In order to authenticate a user, the server sends down a plain text challenge to the browser. Once Microsoft Edge is able to verify the user through Windows Hello, the system will sign the challenge with a private key previously provisioned for this user and send the signature back to the server. If the server can validate the signature using the public key it has for that user and verify the challenge is correct, it can authenticate the user securely.

Screen Capture showing Windows Hello prompt to log in to a web page

These keys are not only stronger credentials – they also can’t be guessed and can’t be re-used across origins. The public key is meaningless on its own and the private key is never shared. Not only is using Windows Hello a delightful user experience, it’s also more secure by preventing password guessing, phishing, and keylogging, and it’s resilient to server database attacks.

Web Authentication: Passwordless and Two Factor Authentication

We’ve been working at the FIDO Alliance with organizations from across the industry to enable strong credentials and help move the web off of passwords. The main goal of the FIDO Alliance is to standardize these interfaces, so websites can use Windows Hello and other biometric devices across browsers. The FIDO Alliance had recently submitted the FIDO 2.0 proposal to the W3C and the newly formed Web Authentication working group is standardizing these APIs in the W3C Web Authentication specification.

FIDO Alliance logo

The Web Authentication specification defines two authentication scenarios: passwordless and two factor. In the passwordless case, the user does not need to log into the web page using a user name or password – they can login solely using Windows Hello. In the two factor case, the user logs in normally using a username and password, but Windows Hello is used as a second factor check to make the overall authentication stronger.

In traditional password authentication, a user creates a password and tells the server, which stores a hash of this password. The user, or an attacker who obtains the password, can then use the same password from any machine to authenticate to the server. Web Authentication instead uses asymmetric key authentication. In asymmetric key authentication, the user’s computer creates a strong cryptographic key pair, consisting of a private key and a public key. The public key is provided to the server, while the private key can be held by the computer in dedicated hardware such as a TPM, so that it cannot be moved from that computer. This protects the users against attacks on both the client and the server – client attacks cannot be used to let an attacker authenticate from elsewhere, and server attacks will only give the attacker a list of public keys.

Microsoft Edge supports an early implementation of the Web Authentication spec – in fact, the latest editor’s draft has already been updated beyond our implementation and we expect the spec to continue to change as it goes through the standardization process. We have implemented our APIs with the ms-prefix to indicate that these APIs are very likely to change in the future. We’ll continue to update the APIs in future releases as the standard finalizes – so be sure to watch for changes.

The Web Authentication API is very simple – it supports two methods: window.webauthn.makeCredential and window.webauthn.getAssertion. You will need to make both server and client sides changes to enable Web Auth authentication in your web application. Let’s talk through how to use these methods.

Registering the user

To use Web Auth, you, the identity provider, will first need to create a Web Auth credential for your user using the window.webauthn.makeCredential method.

When you use the makeCredential method, Microsoft Edge will first ask Windows Hello to use face or fingerprint identification to verify that the user is the same user as the one logged into the Windows account. Once this step is completed, Microsoft Passport will generate a public/private key pair and store the private key in the Trusted Platform Module (TPM), the dedicated crypto processor hardware used to store credentials. If the user doesn’t have a TPM enabled device, these keys will be stored in software. These credentials are created per origin, per Windows account, and will not be roamed because they are tied to the device. This means that you’ll need to make sure the user registers to use Windows Hello for every device they use. This makes the credentials even stronger – they can only be used by a particular user on a particular origin on a particular device.

Before registering the credential to a user on your server, you will need to confirm the identity of the user. This can be done by sending the user an email confirmation or asking them to use their traditional login method.

The below code sample shows how you would use the makeCredential API. When you call the makeCredential API, you will need to supply as parameters data structures containing the user account information, crypto parameters, and an attestation challenge. The user account information contains information on the user’s name, profile image, site in which the user is logging into, and user identifier information on the site. We’ll cover later how this information is used. The crypto parameter is a data structure that contains the crypto algorithm you which to use. The attestation challenge is used by the authenticator to produce an attestation statement, which tells the server what security measures the authenticator implements for its credentials. There are a number of other optional parameters, which we’ll ignore here. The methods are all implemented as promises. When the promise returns, it will include an object that contains information on the credential ID, public key, crypto algorithm, and the attestation challenge. The credential ID will be used to identify the public and private key pairs. You will then send this information back to the server for validating future authentications.

function makeCredential() {
  try {
    var accountInfo = {
      rpDisplayName: 'Contoso',  // Name of relying party
      displayName: 'John Doe',  // Name of user account in relying partying
      name: 'johndoe@contoso.com',// Detailed name of account
      id: 'joed',                 // Account identifier
      imageUri: imgUserProfile,  // user’s account image
    }; 
    
    var cryptoParameters = [
      {
        type: 'ScopedCred',
        algorithm: 'RSASSA-PKCS1-v1_5'
      }      
    ];
 
    var timeout = { };
    var denyList = { };
    var ext = { };
 
    var attestationChallenge = getChallengeFromServer();
 
    window.webauthn.makeCredential(
           accountInfo, 
                  cryptoParameters, 
                  attestationChallenge, 
                  timeout, 
                  denyList, 
                  ext)
      .then(function (creds) {
       // If promised succeeds, send credential information to the server
        sendToServer(creds);
      })
      .catch(function(reason) {
        // User may have cancelled the Windows Hello dialog
      });
  } catch(ex) {
     // The user may not have setup Windows Hello, show instructions
   }
}

The Microsoft Edge implementation is ms-prefixed, so you’ll need to call window.msCredentials.makeCredential instead of window.webauthn.makeCredential. The Microsoft Edge implementation is also based on an earlier draft of the specification, so there are a number of other differences in implementation as well, like the credential type is “FIDO_2_0” instead of “ScopedCred”, we don’t yet implement the optional timeout, denylist, or ext parameters, or require the attestation challenge to make the credential. In Microsoft Edge, you would instead make this call using the following code:

    var accountInfo = {
      rpDisplayName: 'Contoso',        // Name of relying party
      userDisplayName: 'John Doe'      // Name of user account in relying partying
    }; 
    
    var cryptoParameters = [
      {
        type: 'FIDO_2_0',
        algorithm: 'RSASSA-PKCS1-v1_5'
      }      
    ];
 
    window.msCredential.makeCredential(accountInfo, cryptoParameters)
      .then(function (cred) {
       // If promised succeeds, send credential information to the server
        sendToServer({
                   credential: {type: 'ScopedCred', id: cred.id},
                   algorithm: cred.algorithm,
                   publicKey: JSON.parse(cred.publicKey),
                   attestation: cred.attestation
               }
     );
      });

Authenticating the user

Once the credential is created on the client, the next time the user attempts to log into the site, you can offer to sign them in using Windows Hello instead of a password. You will authenticate the user using the window.webauthn.getAssertion call.

The getAssertion call has a number of optional parameters, but the only required parameter is the challenge. This is the challenge that the server will send down to the client. This challenge is a random quantity generated by the server. Since the challenge is not predictable by an attacker, the server can be assured that any assertions it receives were freshly generated in response to this challenge and are not replays of earlier assertions. The allowList parameter also takes an optional list of credential ID information to locate the correct private key. This information is useful if you’re doing two factor auth and you can share the id from the server, where it is stored. In the passwordless case, you don’t want to share the id from the server because the user hasn’t yet authenticated.

If the user has multiple credentials for an origin in the TPM, the browser will show a user experience allowing the user to pick the account they meant to use (assuming they have multiple accounts with an application, and you don’t provide a credential ID). This is why we collect the user profile picture and account name upon registration.

Once the getAssertion call is made, Microsoft Edge will show the Windows Hello prompt, which will verify the identity of the user using biometrics. After the user is verified, the challenge will be signed within the TPM and the promise will return with an assertion object that contains the signature and other metadata. You will then send this data to the server. We’ll check the server code next to see how you verify the challenge is the same that you had sent.

function getAssertion() {
  try {
    var challenge = getChallengeFromServer(); 
    var allowList = 
          [
            {
              type: 'ScopedCred',
              id:  getCredentialID()
            }
          ];
    var timeout = { };
    var ext = { };
 
    window.webauthn.getAssertion(challenge, timeout, allowList, ext)
      .then(function(assertion) {
         // Send signed challenge and meta data to server
        sendToServer(assertion);
      }, function (e) {
        // No credential in the store. Fallback to password
      });    
  } catch (ex) {
    // Log failure
  }
}

In Microsoft Edge, you will need to call window.msCredentials.getAssertion instead of window.webauthn.getAssertion. The Microsoft Edge implementation also requires the credential ID and we don’t yet support the account picker experience. A side effect of this is that for the passwordless case, you’ll need to store your credential ID information in local storage on the client, either in indexDB or localStorage when making your credential. This mean that if a user deletes their browsing history, including local storage, they will need to re-register to use Windows Hello the next time they log in. We will very likely fix this issue in a future release.

Here’s how you would make the getAssertion call in Microsoft Edge. Note how the accept object is required for the filter parameter.

var filters = {
        accept:
          [
            {
              type: 'FIDO_2_0',
              id:  getCredentialIDFromLocalStorage()
            }
          ]
        };
window.msCrendentials.getAssertion(challenge, filters)
.then(function(attestation) {
    // Send signed challenge and meta data to server
    sendToServer({
                   credential: {type: 'ScopedCred', id: attestation.id},
                   clientData: attestation.signature.clientData,
                   authnrData: attestation.signature.authnrData,
                   signature: attestation.signature.signature
                   };
    );
  });   

Server side authentication

Once you receive the assertion on the server, you will need to validate the signature. The below Node.JS code shows how you would validate the signature to authenticate the user on the server. We also have the same code available in C# and PHP.

var jwkToPem = require('jwk-to-pem')
var crypto = require('crypto');
 
var webAuthAuthenticator = {
   validateSignature: function (publicKey, clientData, authnrData, signature, challenge) {
       // Make sure the challenge in the client data 
       // matches the expected challenge
       var c = new Buffer(clientData, 'base64');
       var cc = JSON.parse(c.toString().replace(/\0/g,''));
       if(cc.challenge != challenge) return false;
 
       // Hash data with sha256
       const hash = crypto.createHash('sha256');
       hash.update(c);
       var h = hash.digest();
 
       // Verify signature is correct for authnrData + hash
       var verify = crypto.createVerify('RSA-SHA256');
       verify.update(new Buffer(authnrData,'base64'));
       verify.update(h);
       return verify.verify(jwkToPem(JSON.parse(publicKey)), signature, 'base64');
   }
};   

Evolving Web Authentication standard and Microsoft Edge implementation

As mentioned above, Microsoft Edge has an early implementation of Web Authentication and there are a number of differences between our implementation and the April 2016 spec.

  • Microsoft Edge APIs are ms- prefixed
  • Microsoft Edge does not yet support external credentials like USB keys or Bluetooth devices. The current API is limited to embedded credentials stored in the TPM.
  • The currently logged in Windows user account must be configured to support at least a PIN, preferably face or fingerprint biometrics. This is to ensure that we can authenticate the access to the TPM.
  • We do not support all of the options in the current Web Auth spec draft, like extensions or timeouts.
  • As mentioned earlier, our implementation requires that the list of acceptable credential IDs be included in every getAssertion call.

The specification is also going through the W3C standardization process and we expect a number of changes in the specification, like the object name recently changing from window.fido to window.webauthn in the latest’s editor’s draft.

We have a number of resources available to help you prototype and experiment with these APIs:

    • Webauthn.js polyfill. Using this polyfill, you can code to the standard instead of our early implementation. We’ll update this polyfill for every major published version of the specification.
    • Windows Hello in Microsoft Edge test drive sample. This test drive sample shows you the typical client side registration and assertion flow.
    • Server and client side WebAuth This sample code shows the end to end client and server side flow for registration and assertion.
    • C#, PHP, and JS server side sample. These code samples show how could implement your server side logic in a number of language options.
    • Web Authentication MSDN documentation and dev guide.
    • Edge Summit talk on Windows Hello in Microsoft Edge.

Call to Action

We’re excited to support Windows Hello and Web Authentication in Microsoft Edge and innovate in the biometric authentication and passwordless space in the web. Now is a great time to start prototyping and experimenting with these APIs and sharing your feedback with us in the comments below or on Twitter. We look forward to hearing from you!

Rob Trace, Program Manager, Microsoft Edge
Jatinder Mann, Program Manager Lead, Microsoft Edge
Vijay Bharadwaj, Software Engineering Lead, Security
Anoosh Saboori, Program Manager, Security

Continuing to make it easier for Enterprise customers to upgrade to Internet Explorer 11 — and Windows 10

Fred and I discuss Enterprise Mode on the Microsoft Edge blog:

Helping our Enterprise customers stay up-to-date with the latest version of the browser is a top priority for Microsoft. This is particularly important for Windows 7 customers who should upgrade to Internet Explorer 11 by January 12, 2016 to continue receiving security updates and technical support. We understand many customers have web apps and services designed specifically for older versions of Internet Explorer, so we’re continuing to improve our set of Enterprise Mode tools to help you run those applications in Internet Explorer 11. Upgrading to Internet Explorer 11 can ease the upgrade to Windows 10, too, since Internet Explorer 11 is supported on Windows 7, Windows 8.1, and Windows 10.

Continuing to improve Enterprise Mode

We continue to make significant improvements to Enterprise Mode, helping customers upgrade more easily to Internet Explorer 11 while also extending the ability to run older web apps. Customer feedback has told us that we’re on the right track with these tools, and that IE upgrades are going more smoothly than ever before.

Here are some features and resources available for you today:

IE8 Enterprise Mode
IE8 Enterprise Mode provides a higher fidelity emulation of the Internet Explorer 8 browser by turning on functionality not available in other document modes. You can use the Enterprise Mode Site List Manager to specify any web path to load in this mode.

IE7 Enterprise Mode
Internet Explorer 8 shipped with Compatibility View, a compatibility option that looks for the DOCTYPE tag, rendering a web page in IE7 document mode if it exists, or in IE5 document mode if it doesn’t.

While IE11 still has Compatibility View, you can use IE7 Enterprise Mode to provide a higher fidelity emulation of how the IE8 browser acts when running in Compatibility View. You can use the Enterprise Mode Site List Manager to specify any web path to load in this mode.

IE5, IE7, IE8, IE9, IE10, and IE11 Document Modes
Internet Explorer 11 supports document modes for emulation of the IE5, IE7, IE8, IE9, IE10, and IE11 rendering engines. While site markup can request the browser to load in these modes, you can also use the Enterprise Mode Site List to override the site to load in any document mode that makes the site work.

Enterprise Site Discovery
If you want to prioritize your testing, you can use Enterprise Site Discovery with IE8, IE9, IE10, and IE11 to discover which web apps your users are visiting and how the apps are built, like what document mode or ActiveX Controls are used. The Enterprise Mode Site List Manager tool can even import this data to help you seed your Enterprise Mode Site List. Of course, using the F12 developer tools for manual testing is always an option, but Enterprise Site Discovery can help you quickly gather information from a broad set of users.

(New!) Support for HTTP ports in Enterprise Mode
We’ve also heard from many customers that they want the ability to apply Enterprise Mode compatibility features to sites with HTTP ports, such as http://contoso.com:8080, and we’ve listened: Enterprise Mode now supports HTTP ports. You can specify a HTTP port directly in your Enterprise Mode Site List XML, such as <domain>contoso.com:8080</domain>, or you can use the new Enterprise Mode Site List Manager tool to add HTTP ports. To use this feature on Windows 7 or Windows 8.1, you’ll need to also install the IE11 October Cumulative Update or later.

(New!) Web Application Compatibility Lab Kit
To help customers learn how to use Enterprise Mode and Enterprise Site Discovery, today we are introducing the Web Application Compatibility Lab Kit. This Lab Kit provides a walk-through of how to configure and set up Enterprise Mode in addition to Enterprise Site Discovery, the F12 developer tools, and the Enterprise Mode Site List Manager. The Lab Kit comes with VMs for Windows 7 SP1 Enterprise (Evaluation) and Windows 10 Enterprise (Evaluation), or you can download a “lite” version without the VMs.

For more information or assistance:

New improvements to Enterprise Mode in Windows 10

We’re also making it easier for customers to upgrade to Windows 10, because Windows 10 supports the same version of Internet Explorer 11—along with all of the Enterprise Mode features—available on Windows 7 and Windows 8.1. At the same time, we’ve continued to make improvements to Enterprise Mode in the Windows 10 Fall Update and higher to simplify management and better support Microsoft Edge. Many of these improvements will also be made available to Internet Explorer 11 on Windows 7 and Windows 8.1 early next year.

Microsoft Edge and IE11 work better together on Windows 10

Windows 10 features Microsoft Edge, a new browser that goes beyond browsing with features like Web Note and Cortana integration. Microsoft Edge is built from the ground up to improve productivity, to be more secure, and to correctly, quickly, and reliably render web pages. As we announced a few months ago, you can use Enterprise Mode with Microsoft Edge to open Internet Explorer 11 for your business’s sites that require IE proprietary technologies. This approach enables your users to run a modern browser designed for better productivity, security, and rendering web pages—without sacrificing compatibility with legacy line of business applications.

However, we also recognize that some customers have a significant number of sites that require IE proprietary technologies, and still need to use Internet Explorer 11 as their primary browser. For those customers, we have introduced a new capability in Enterprise Mode for Internet Explorer 11 to open Microsoft Edge for modern sites that need the latest web platform features. This feature has a very similar user experience to the analogous feature in Microsoft Edge to open IE11.

In this example below, we’ve setup Enterprise Mode with IE11 to open http://www.bing.com in Microsoft Edge.

Screen capture showing Internet Explorer 11 prompting the user to open a site in Internet Explorer 11.

With this change, customers now have the following options for browsers on Windows 10:

Use Microsoft Edge as your primary browser If your business sites primarily use modern web technologies, we recommend that you use Microsoft Edge as your primary browser.
Use Microsoft Edge as your primary browser and use Enterprise Mode to open sites in IE11 that use IE proprietary technologies If your business sites primarily use modern web technologies but you still have older sites that use IE proprietary technologies, we recommend that you use Microsoft Edge as your primary browser and use the Enterprise Mode Site List to open older sites automatically in Internet Explorer 11. This approach helps ensure your new site development continues to happen on a modern web platform, but that you can still use your older apps that were designed for older versions of IE. Your Enterprise Mode Site List also becomes a to-do list for sites that you need to modernize. Many customers tell us they plan to use this approach. We use this option at Microsoft, since we still have some older applications built for older IE technologies.
Use Microsoft Edge as your primary browser and open all intranet sites in IE11 If you have external sites that use modern web technologies, but still have a large number of internal business sites that use IE proprietary technologies, you can choose to use Microsoft Edge as your primary browser and open all intranet sites in IE11 using the Sends all intranet traffic over to Internet Explorer Microsoft Edge Group Policy or MDM setting. You can still use the Enterprise Mode Site List to ensure the internal sites that work with Microsoft Edge don’t open in IE11. This approach helps ensure your new site development continues to happen on a modern web platform, but balances it with your need to continue using older apps that were designed for older versions of IE.
Use IE11 as your primary browser and use Enterprise Mode to open sites in Microsoft Edge that use modern web technologies If you have more business sites built on IE proprietary technologies than modern web technologies, you can choose to use IE11 as your primary browser and use the Enterprise Mode Site List to explicitly define the modern web sites that should open automatically in Microsoft Edge. This approach helps ensure your new site development continues to happen on a modern web platform engine in Microsoft Edge, while balancing the fact that most of your web apps are still based on IE.
Use IE11 as your primary browser If you want the exact same environment on Windows 10 as on Windows 7 or Windows 8.1, you can choose to use IE11 as your primary browser. You can set IE11 as your default browser by using the Set a default associations configuration file in the File Manager Group Policy. While this is a great approach to quickly get to Windows 10, we recommend that customers consider using Microsoft Edge as a more secure, reliable browser instead. Developing new business applications using IE proprietary technologies could also make it harder for you to adopt modern web technologies in the future.

Simpler, cleaner, and more scalable Enterprise Mode XML schema

We’ve heard that most customers are using the Enterprise Mode Site List to upgrade to IE11 on Windows 7, and will likely continue to use the site list going forward. We’ve also heard feedback that the existing site list XML schema isn’t always very easy to understand. Starting in the Windows 10 Fall Update, we’re now supporting a new v.2 Enterprise Mode XML schema, which is designed to be simpler, cleaner, more scalable, and to help ease list management. Internet Explorer 11 and Microsoft Edge on Windows 10 will continue to support the existing v.1 XML schema that is supported on Windows 7 and Windows 8.1 today, as well as the new v.2 XML schema.

Below is an example of an Enterprise Mode Site List based on the existing v.1 XML schema. Customer feedback included:

  • It’s not clear that if the same entry appears in the emie and docMode sections, the emie section wins.
  • The double negative of doNotTransition=”false” is confusing.
  • There are many ways to put a site in a document mode: entry in emie section, entry in docMode section, or forceCompatView attribute.
  • You have to break up an URL into the domain and paths separately for the emie and docMode sections.

EMIE v1 schema

Below, we have the same Enterprise Mode Site List using the v.2 XML schema. Here are some of the changes:

  • All entries are just URLs under the site element. This reduces the hierarchy and the need to split sites into domains and paths.
  • We’ve introduced a new compat-mode element where you can specify any compatibility mode: IE8Enterprise, IE7Enterprise, IE5, IE7, IE8, IE9, IE10, IE11, and default. This attribute replaces the emie and docMode sections, and forceCompatView and exclude attributes.
  • We’ve introduced a new open-in element where you can specify which browser the site should open in: IE11, MSEdge, or none (don’t open in another browser). This replaces the double negative of doNotTransition=”false”.
  • You can use a self-closing element with no children to declare a site should just use the browser defaults. This replaces the exclude attribute.
  • We include the date the list was created and the Enterprise Mode Site List Manager tool version number for easier site list version management. This information doesn’t impact how the browser interprets the site list.
  • This format makes it very easy for us to introduce new Enterprise Mode features as children elements to the site element.

EMIE v2 schema

We encourage customers to adopt the easier to read and manage v.2 version of the XML schema, starting with Windows 10 and coming in early 2016 to IE11 on Windows 7 and Windows 8.1. Going forward, we plan to bring new features only to the v.2 XML schema. For example, the new Enterprise Mode feature for IE11 to open sites in Microsoft Edge is only supported with the new v.2 XML schema.

Here is an example of how you can specify that IE11 open a site in Microsoft Edge:

OpenInEdge

In addition to the new v.2 version of the XML schema, we’re also supporting a new version of the Enterprise Mode Site List Manager tool for Windows 10. This version of the tool lets you import an existing v.1 Enterprise Mode Site List XML file and automatically convert it to the v.2 XML schema.

Important note:
To avoid adding extra work to migrate from the v.1 to v.2 XML schema while customers are trying to upgrade to IE11 on Windows 7 and Windows 8.1 before the January 12, 2016 support policy deadline, we’ve decided to wait before supporting the v.2 XML schema on Windows 7 or Windows 8.1 today. We recommend customers working on IE11 upgrades on Windows 7 and Windows 8.1 continue to use the existing v.1 XML schema and the existing Enterprise Mode Site List Manager tool for Windows 7 and Windows 8.1 Update. The existing schema continues to be supported on all versions of Windows where IE11 is supported, including Windows 10. We will make the v.2 Enterprise Mode XML available in Internet Explorer 11 for Windows 7 and Windows 8.1 early next year, and customers shouldn’t consider moving to the v.2 XML schema until that patch lands on all of their devices.

To learn more about this Enterprise Mode XML schema changes, see the following:

Better diagnostics and management with about:compat

In the Windows 10 Fall Update, we’ve also introduced a new about:compat page in Microsoft Edge and Internet Explorer 11 to help customers better manage their Enterprise Mode Site List. You can use about:compat to see all of the compatibility features you or Microsoft have applied to sites on the client machine. This tool is not only an easy way to visualize and search the Enterprise Mode Site List, but a great way to diagnose problems on a client machine, like whether the latest Enterprise Mode Site List is being applied.

On Internet Explorer 11 for Windows 10, you can view and search the Enterprise Mode Site List, local additions to the Enterprise Mode list, the Microsoft Compatibility List, and local additions to Compatibility View.

Screen capture showing the about:compat interface in Internet Explorer 11 on Windows 10.

On Microsoft Edge, you can view and search the Enterprise Mode Site List and Microsoft Compatibility List, as only those lists are applicable.

Screen capture showing the about:compat interface in Microsoft Edge

Upgrade guidance

Migrating to Internet Explorer 11 is easier than ever before, thanks to backward compatibility features like Enterprise Mode. Please start with the resources in this post, such as the Web Application Compatibility Lab Kit, to learn how to use Enterprise Mode and Enterprise Site Discovery effectively. If you still have a technical or business impediment that prevents you from upgrading, please reach out to your Microsoft account team or Microsoft partner, as we may be able to help.

We made these product improvements and upgrade resources based on customer feedback, and hope the new features help you manage browser compatibility more easily than ever before. Like other customers, you may also find that upgrading to Internet Explorer 11 with Enterprise Mode is easier and less costly than previous upgrades. Best of all, the upgrade to Internet Explorer 11 should help ease your migration to Windows 10 and Microsoft Edge.

– Jatinder Mann, Senior Program Manager Lead
– Fred Pullen, Product Marketing Director

Bringing automated testing to Microsoft Edge through WebDriver

My team has been working very hard to deliver WebDriver support in Microsoft Edge. Here’s a blog we wrote on the Microsoft Edge Blog.

Today we’re announcing support for automated testing of Microsoft Edge through the W3C WebDriver standard. To use WebDriver with Microsoft Edge, you need the MicrosoftWebDriver server on a Windows Insiders build of 10240 or newer. You can also out our test drive to learn more7 about WebDriver!

WebDriver is an emerging standard through which Web developers can write tests to automate Web browsers for site testing. It provides a programmable remote control for developing complex user scenarios and running them in an automated fashion against your website in a browser. WebDriver is used by top web properties like Bing, Azure, SharePoint, Facebook, Google, and others to automate testing their sites within a browser. With this new capability, Microsoft Edge can be run through the same regression testing as other browsers, helping developers identify issues with less effort and making sites just work for our end users.

As we described in “Building a more interoperable Web with Microsoft Edge,” the Web is built on the principle of multiple independent, interoperable implementations of web standards, and true interoperability means that all web content, browsers, and specifications should align to the same well-defined behavior. Keeping that principle in mind, our implementation supports both the W3C WebDriver specification and the JSON Wire Protocol. The Browser Testing and Tools working group has been doing a great job of standardizing automation testing interfaces as a part of the emerging WebDriver specification. For backwards compatibility with existing tests, we have implemented the JSON Wire Protocol, so existing tests should also just work. We’re not done yet, as the WebDriver specification is still in an early stage of standardization, and we still have more features to implement. You can find out more about all of the WebDriver interfaces we have implemented on our Platform Status page here.

Borland contributions to Microsoft Edge

MicroFocus Borland logoMicrosoft has a long history of collaborating with industry technology leaders to bring the best-in-class experiences. In order to truly build an interoperable implementation that our customers can use, we have partnered with the Borland Silk team from Micro Focus, an industry leader in automation testing tools, to help contribute code to the WebDriver implementation in Microsoft Edge. The Borland team is also bringing their expertise in automation testing to help inform the next level of changes we should pursue in the W3C standard. Our thanks to the Borland team for all of their help in making the Web better!

How WebDriver works

To get started using WebDriver, you will need to download a testing framework of your choice along with an appropriate language binding and the MicrosoftWebDriver server. We have submitted updates to the C# and Java Selenium language bindings and are hoping to add support to other languages in the future.

WebDriver is disabled by default for security. In order to enable using WebDriver, you will need to download and install the MicrosoftWebDriver in a location with your test repository. You should be able to use Microsoft Edge’s WebDriver implementation just like you would use any other browser’s implementation.

Diagram of WebDriver and Microsoft Edge

Here’s an example of C# code opening a new browser Window, navigating to http://www.bing.com and searching webdriver. For more example commands, check out our test drive.

We’re excited to deliver WebDriver in Microsoft Edge and look forward to hearing what you think! Take it for a spin and share your feedback in the comments below or @MSEdgeDev on Twitter.

Clay Martin, Program Manager, Microsoft Edge
John Jansen, Principle Software Engineer, Microsoft Edge
Jatinder Mann, Senior Program Manager Lead, Microsoft Edge
– Mark Conway, Director, Micro Focus

Making it easier for Enterprise customers to upgrade to Internet Explorer 11 — and Windows 10

In this IE Blog Post, I talk about making it easier for Enterprises to stay up-to-date with Internet Explorer.

As we shared last year, a top priority for Microsoft is helping our Enterprise customers stay up-to-date with the latest version of Internet Explorer. This is particularly important for Windows 7 customers who are upgrading to Internet Explorer 11 by January 12, 2016 to continue receiving security updates and technical support. We understand many customers have Web apps and services that were designed specifically for older versions of Internet Explorer and we provide a set of tools, like Enterprise Mode, the Enterprise Mode Site List, and Enterprise Site Discovery, to help you run these applications in Internet Explorer 11 — and ease the upgrade to Windows 10.

Enterprise Mode helps customers extend their investments in older Web apps through higher compatibility with the IE8 rendering engine in a more modern browser like Internet Explorer 11. In the ten months since we released Enterprise Mode, we’ve heard from our customers that it is very effective at improving legacy app compatibility and that the upgrade to Internet Explorer 11 was easier than ever before. To help us tell this story, Microsoft commissioned Forrester Consulting to interview and survey large customers in the US, UK, Germany, and Japan who have started their IE11 upgrades. Customers found that:

  • Upgrading from IE8 to IE11 was 1.8 times faster than expected, thanks to Enterprise Mode.
  • The effort to rewrite applications to modern browser standards was reduced by 75%.
  • Ongoing browser support and critical applications testing were significantly reduced.
  • Many business users saw improved productivity from using a single browser.

Organizations save money, reduce risk, and experience higher productivity by upgrading to IE11. Best of all, upgrading to Internet Explorer 11 now can help ease your migration to Windows 10. Download The Total Economic Impact of Microsoft Internet Explorer 11.

Better Backward Compatibility with the Enterprise Mode Site List

Enterprise Mode can be very effective in providing backward compatibility for older Web apps. In January, for example, Microsoft held an Enterprise Customer Summit with about 100 large customers, and we found that every broken site brought by a customer to our IE App Compatibility Workshop was fixed by using Enterprise Mode. Depending on your environment, you may not need to use Enterprise Mode for better emulation of the IE8 engine, but may still benefit from using the Enterprise Mode Site List and its new <docMode> functionality.

In November, we expanded the functionality of the Enterprise Mode Site List to include the ability to put any Web app in any document mode, without changing a single line of code on the Web site. This new functionality adds a <docMode> section to the Enterprise Mode XML file, separate from the <emie> section for Enterprise Mode sites. Using this new functionality, Microsoft’s own IT department saw our internal line of business application pass rate go from 93% to 100% with 24 document mode entries in our Enterprise Mode Site List and only a single Enterprise Mode entry.

Web paths added to the Enterprise Mode Site List can now be rendered either in Enterprise Mode—which provides higher-fidelity emulation for IE8—or any of the document modes listed below:

Enterprise Mode Site List diagram

Enterprise Mode (in green above), with its higher-fidelity emulation for IE8, was added to Internet Explorer 11 in April, 2014. The blue capabilities were added to the Enterprise Mode Site List in November, 2014, and can particularly help IE9 or IE10 customers upgrade more easily. Enterprise Mode can be further improved by using it in combination with Compatibility View, a mode added in IE8 for better compatibility for sites designed for IE7, as indicated in orange.

Compatibility View is basically a switch: If a web page has no DOCTYPE, the page will be rendered in IE5 mode. If there is a DOCTYPE, the page will be rendered in IE7 mode. You can effectively get Compatibility View by specifying IE7 in the <docMode> section—as this falls back to IE5 automatically if there’s no DOCTYPE—or you can use Enterprise Mode with Compatibility View for even better emulation. See below for details.

Speaking with customers, we found that IT Pros aren’t always the ones doing the app remediation work. Much of this work is often done by IT Developers, who don’t always understand all of the IE backward compatibility offerings. In this post, we want to provide a clearer set of mitigations for IT Pros and IT Developers to help make your upgrade as easy as possible.

Call to action for IT Pros

We know that upgrading to a new browser can be a time-consuming and potentially costly venture. To help reduce these costs, we introduced the Enterprise Site Discovery toolkit to help you prioritize which sites you should be testing based on their usage in your enterprise. For example, if the data shows that no one is visiting a particular legacy Web app anymore, you may not need to test or fix it. This tool also gives you information on what document mode the page runs in your current browser, so you can better understand how to fix that site if it breaks in a newer version of the browser. This tool is currently only supported in IE11, but we are bringing Enterprise Site Discovery support to IE8, IE9, and IE10 very soon.

Once you know which sites to test and fix, the following remediation methods may help fix your app compatibility issues in IE11 and Windows 10.

If you’re on IE8 and upgrading to IE11…

  • Use the Enterprise Mode Site List to add sites to IE5, IE7, and IE8 modes.
  • Sites with x-ua-compatible meta tag or HTTP header set to “IE=edge” may break in IE11 and need to be set to IE8 mode. This is because Edge in IE8 meant IE8 mode, but Edge in IE11 means IE11 mode.
  • Sites without a DOCTYPE in zones other than Intranet will default to QME (or “interoperable quirks) rather than IE5 Quirks and may need to be set to IE5 mode.
  • If you have enabled Turn on Internet Explorer Standards Mode for local intranet group policy setting, sites with a DOCTYPE in the Intranet zone will open in IE11 mode instead of IE7 mode, and may need to be set to IE7 mode. Sites without a DOCTYPE will open in QME and may need to be set to IE5 mode.
  • Some IE5, IE7, and IE8 sites may need to be added to Enterprise Mode to work.
  • Some sites may need to be added to both Enterprise Mode and Compatibility View to work. You can do this by adding the site both to the Enterprise Mode section of the Enterprise Mode Site List and to the Use Policy List of Internet Explorer 7 sites group policy.

If you’re on IE9 and upgrading to IE11…

  • Use the Enterprise Mode Site List to add sites to IE5, IE7, and IE9 modes.
  • Sites with x-ua-compatible meta tag or HTTP header set to “IE=edge” may break in IE11 and need to be set to IE9 mode. This is because Edge in IE9 meant IE9 mode, but Edge in IE11 means IE11 mode.
  • Sites without a DOCTYPE in zones other than Intranet will default to QME rather than IE5 Quirks and may need to be set to IE5 mode.
  • If you have enabled Turn on Internet Explorer Standards Mode for local intranet group policy setting, sites with a DOCTYPE in the Intranet zone will open in IE11 mode instead of IE7 mode, and may need to be set to IE7 mode. Sites without a DOCTYPE will open in QME and may need to be set to IE5 mode.
  • If your sites worked in IE9, you won’t need Enterprise Mode but can still take advantage of the newer <docMode> section of the Enterprise Mode Site List.

If you’re on IE10 and upgrading to IE11…

  • Use Enterprise Mode Site List to add sites to IE5, IE7, and IE10 modes.
  • Sites with x-ua-compatible meta tag or HTTP header set to “IE=edge” may break in IE11 and need to be set to IE10 mode. This is because Edge in IE10 meant IE10 mode, but Edge in IE11 means IE11 mode.
  • If you have enabled Turn on Internet Explorer Standards Mode for local intranet group policy setting, sites with a DOCTYPE in the Intranet zone will open in IE11 mode instead of IE7 mode, and may need to be set to IE7 mode. Sites without a DOCTYPE will open in QME and may need to be set to IE5 mode.
  • If your sites worked in IE10, you won’t need Enterprise Mode but can still take advantage of the newer <docMode> section of the Enterprise Mode Site List.

If you’re on IE11 and upgrading to Windows 10…

  • Use Enterprise Mode Site List to add sites to IE5, IE7, IE8, IE9, IE10, and IE11 modes as needed.
  • The x-ua-compatible meta tag and HTTP header will be ignored for all sites not in the Intranet zone. If you enable Turn on Internet Explorer Standards Mode for local intranet group policy, all sites, including those in the Intranet zone, will ignore x-ua-compatible. Look at your Enterprise Site Discovery data to see which modes your sites loaded in IE11 and add those sites to the same modes using the Enterprise Mode Site List tool.

We recommend that Enterprise customers focus their new development on established, modern Web standards for better performance and interoperability across devices, and avoid developing sites in older IE document modes. We often hear that due to the Intranet zone defaults to Compatibility View, IT Developers inadvertently create new sites in IE7 or IE5 modes in the Intranet zone, depending on whether they used a DOCTYPE. As you move your Web apps to modern standards, you can enable the Turn on Internet Explorer Standards Mode for local intranet group policy and add sites that need IE5 or IE7 modes to the Site List. Of course, testing is always a good idea to ensure these settings work for your environment.

Call to action for IT Developers

An IT Pro may ask you to update your site if it worked in an older IE version but no longer works in IE11. Here are the set of steps you should follow to find the right remediation:

Try Document Modes

Try to see if the site works in one of the following document modes: IE5, IE7, IE8, IE9, IE10, or IE11.

  • Open the site in IE11, load the F12 tools by pressing the ‘F12’ key or selecting ‘F12 Developer Tools’ from the ‘Tools’ menu, and select the ‘Emulation’ Tab.

'Emulation' tab in the IE11 F12 Developer tools

  • Try running the site in each document mode until you find one in which it works. You will need to make sure the user agent string dropdown matches the same browser version as the document mode dropdown. For example, if you were testing if the site works in IE10, you should update the document mode dropdown to “10” and the user agent string drop down to “Internet Explorer 10.”
  • If you find a mode where your site works, inform your IT Pro to add the site domain, sub-domain, or URL to the Enterprise Mode Site List in the document mode where the site works. While you can add the x-ua-compatible meta tag or HTTP header, this approach will only work in Windows 10 for sites in the Intranet zone when the Turn on Internet Explorer Standards Mode for local intranet group policy is not enabled.

Try Enterprise Mode

If a document mode didn’t fix your site, try Enterprise Mode. Enterprise Mode only benefits sites written for IE5, IE7, and IE8 document modes.

  • Enable the Let users turn on and use Enterprise Mode from the Tools menu group policy setting locally on your machine. You can do this by searching and running gpedit.msc, going to ‘Computer Configuration’ ‘Administrative Template’ ‘Windows Components’ ‘Internet Explorer’, and enabling the Let users turn on and use Enterprise Mode from the Tools menu group policy setting. After making this change, run gpupdate.exe /force to make sure the setting is applied locally. Make sure to disable this setting once you’re done testing. Alternately, you can use a regkey; see Turn on Enterprise Mode and use a site list for more information.
  • Restart IE11 and open the site you’re testing, then go to Emulation Tab in F12 tools and select “Enterprise” from the Browser profile dropdown. If the site works, inform your IT Pro that the site needs to be added to the Enterprise Mode section.

Try Compatibility View with Enterprise Mode

If Enterprise Mode doesn’t work, setting Compatibility View with Enterprise Mode will give you the Compatibility View behavior that shipped with IE8.

  • While browsing in Enterprise Mode, go to the ‘Tools’ menu and select Compatibility View Settings, and add the site to the list.
  • If this works, inform your IT Pro to add the site to both the Enterprise Mode section and the Use Policy List of Internet Explorer 7 sites group policy setting. Please note that adding the same Web path to the Enterprise Mode and docMode sections of the Enterprise Mode Site List will not work, but we are addressing this in a future update.

Update site for modern Web standards

If you have the time and budget, you should update your site for established, modern Web standards, so you don’t need to use compatibility offerings to make your site continue to work.

More Resources

For more information on all of these tools, please see the following resources:

As always, we suggest that consumers upgrade to the latest version and enable automatic updates for more secure browsing. If you use an older version of Internet Explorer at work, encourage your IT department to learn more the new backward-compatible features of Internet Explorer 11. Like many of our other customers, you may find that upgrading to the latest version of Internet Explorer is easier and less costly than previous upgrades. Best of all, the upgrade to Internet Explorer 11 can help ease your migration to Windows 10.

– Jatinder Mann, Senior Program Manager Lead

– Fred Pullen, Senior Product Marketing Manager

Understanding the Real-World Performance of your Web Application Across IE11 and Other Browsers


In this IE Blog article I talk about how to use the Timing APIs to understand the real-world performance of your web applications.

Together with Google, Mozilla, Microsoft, and other community leaders, the W3C Web Performance working group has standardized the Navigation Timing, Resource Timing, User Timing, and Performance Timeline interfaces to help you understand the real-world performance of navigating, fetching resources, and running scripts in your Web application. You can use these interfaces to capture and analyze how your real-world customers are experiencing your Web application, instead of relying on synthetic testing, which tests the performance of your application in an artificial environment. With this timing data, you can identify opportunities to improve the real-world performance of your Web applications. All of these interfaces are supported in IE11. Check out the Performance Timing Test Drive to see these interfaces in action.

The Performance Timing testdrive lets you try out the Timing APIs.
The Performance Timing Test Drive lets you try out the Timing APIs.

Performance Timeline

The Performance Timeline specification has been published as a W3C Recommendation and is fully supported in IE11 and Chrome 30. Using this interface, you can get an end to end view of time spent during navigating, fetching resources, and executing scripts running in your application. This specification defines both the minimum attributes all performance metrics need to implement and the interfaces developers can use to retrieve any type of performance metric.

All performance metrics must support the following four attributes:

  • name. This attribute stores a unique identifier for the performance metric. For example, for a resource, it will be the resolved URL of the resource.
  • entryType. This attribute stores the type of performance metric. For example, a metric for a resource would be stored as “resource.”
  • startTime. This attribute stores the first recorded timestamp of the performance metric.
  • duration. This attribute stores the end-to-end duration of the event being recorded by the metric.

All of the timing data is recorded in high resolution time using the type DOMHighResTimeStamps, defined in the High Resolution Time specification. Unlike DOMTimeStamps which measure time values in milliseconds from 01 January, 1970 UTC, the high resolution time value is measured in at least microsecond resolution from the start of navigation of the document. For example, if I check the current time using performance.now(), the high resolution time analogous for Date.now(), I would get the following interpretation of the current time:

> performance.now();

4038.2319370044793

 

> Date.now()

1386797626201

This time value also has the benefit of not being impacted by clock skew or adjustments. You can explore the What Time Is It Test Drive to understand the use of high resolution time.

You can use the following interfaces to retrieve a list of the performance metrics recorded at the time of the call. Using the startTime and duration, as well as any other attributes provided by the metric, you can obtain an end-to-end timeline view of your page performance as your customers had experienced.

PerformanceEntryList getEntries();

PerformanceEntryList getEntriesByType(DOMString entryType);

PerformanceEntryList getEntriesByName(DOMString name, optional DOMString entryType);

The getEntries method returns a list of all of the performance metrics on the page, whereas the other methods return specific items based on the name or type. We expect most developers will just use JSON stringify on the entire list of metrics and send the results to their server for analysis rather than processing the information on the client.

Let’s take a closer look at each of the different performance metrics: navigation, resource, marks, and measures.

Navigation Timing

The Navigation Timing interfaces provide accurate timing measurements for each of the phases of navigating to your Web application. The Navigation Timing L1 specification has been published as a W3C Recommendation, with full support since IE9, Chrome 28, and Firefox 23. The Navigation Timing L2 specification is a First Public Working Draft and is supported by IE11.

With Navigation Timing, developers can not only get the accurate end-to-end page load time, including the time it takes to get the page from the server, but also get the breakdown of where that time was spent in each of the networking and DOM processing phases: unload, redirect, app cache, DNS, TCP, request, response, DOM processing, and the load event. The script below uses Navigation Timing L2 to get this detailed information. The entry type for this metric is “navigation,” while the name is “document.” Check out a demo of Navigation Timing on the IE Test Drive site.

<!DOCTYPE html>

<html>

<head></head>

<body>

<script>

 functionsendNavigationTiming() {

   var nt = performance.getEntriesByType(‘navigation’)[0];

   var navigation = ‘Start Time: ‘ + nt.startTime;

   navigation += ‘Duration: ‘ + nt.duration;

   navigation += ‘Unload: ‘ + (nt.unloadEventEnd – nt.unloadEventStart);

   navigation += ‘Redirect: ‘ + (nt.redirectEnd – nt.redirectStart);

   navigation += ‘App Cache: ‘ + (nt. domainLookupStart – nt.fetchStart);

   navigation += ‘DNS: ‘ + (nt.domainLookupEnd – nt.domainLookupStart);

   navigation += ‘TCP: ‘ + (nt.connectEnd – nt.connectStart);

   navigation += ‘Request: ‘ + (nt.responseStart – nt.requestStart);

   navigation += ‘Response: ‘ + (nt.responseEnd – nt.responseStart);

   navigation += ‘Processing: ‘ + (nt.domComplete – nt.domLoading);

   navigation += ‘Load Event: ‘ + (nt.loadEventEnd – nt.loadEventStart);

   sendAnalytics(navigation);

 }

</script>

</body>

</html>

By looking at the detailed time spent in each of the network phases, you can better diagnose and fix your performance issues. For example, you may consider not using a redirection if you find redirect time is high, use a DNS caching service if DNS time is high, use a CDN closer to your users if request time is high, or GZip your content if response time is high. Check out this video for tips and tricks on improving your network performance.

The main difference between the two Navigation Timing specification versions is in how timing data is accessed and in how time is measured. The L1 interface defines these attributes under the performance.timing object and in milliseconds since 01 January, 1970. The L2 interface allows the same attributes to be retrieved using the Performance Timeline methods, enables them to be more easily placed in a timeline view, and records them with high resolution timers.

Prior to Navigation Timing, developers would commonly try to measure the page load performance by writing JavaScript in the head of the document, like the below code sample. Check out a demo of this technique on the IE Test Drive site.

<!DOCTYPE html>

<html>

<head>

<script>

 var start = Date.now();

 

 function sendPageLoad() {

   var now = Date.now();

   var latency = now – start;

   sendAnalytics(‘Page Load Time: ‘ + latency);

 }

 

</script>

</head>

<body onload=’sendPageLoad()’>

</body>

</html>

However, this technique does not accurately measure page load performance, because it does not include the time it takes to get the page from the server. Additionally, running JavaScript in the head of the document is generally a poor performance pattern.

Resource Timing

Resource Timing provides accurate timing information on fetching resources in the page. Similar to Navigation Timing, Resource Timing provides detailed timing information on the redirect, DNS, TCP, request, and response phases of the fetched resources. The Resource Timing specification has been published as a W3C Candidate Recommendation with support since IE10 and Chrome 30.

The following sample code uses the getEntriesByType method to obtain all resources initiated by the element. The entry type for resources is “resource,” and the name will be the resolved URL of the resource. Check out a demo of Resource Timing on the IE Test Drive site.

<!DOCTYPE html>

<html>

<head></head>

<body onload=’sendResourceTiming()’>

<img src=’http://some-server/image1.png’&gt;

<img src=’http://some-server/image2.png’&gt;

<script>

 function sendResourceTiming()

 {

  var resourceList = window.performance.getEntriesByType(‘resource’);

  for (i = 0; i < resourceList.length; i++)

  {

   if(resourceList[i].initiatorType == ‘img’)

   {

    sendAnalytics(‘Image Fetch Time: ‘ + resourceList[i].duration);

   }

  }

 }

</script>

</body>

</html>

For security purposes, cross-origin resources only show their start time and duration; the detailed timing attributes are set to zero. This helps avoid issues of statistical fingerprinting, where someone can try to determine your membership in an organization by confirming whether a resource is in your cache by looking at the detailed network time. The cross origin server can send the timing-allow-origin HTTP header if it wants to share timing data with you.

User Timing

User Timing provides detailed timing information on the execution of scripts in your application, complementing Navigation Timing and Resource Timing which provide detailed network timing information. User Timing allows you to display your script timing information in the same timeline view as your network timing data to get an end to end understanding of your app performance. The User Timing specification has been published as a W3C Recommendation, with support since IE10 and Chrome 30.

The User Timing interface defines two metrics used to measure script timing: marks and measures. A mark represents a high resolution time stamp at a given point in time during your script execution. A measure represents the difference between two marks.

The following methods can be used to create marks and measures:

void mark(DOMString markName);

void measure(DOMString measureName, optional DOMString startMark, optional DOMString endMark);

Once you have added marks and measures to your script, you can retrieve the timing data by using the getEntry, getEntryByType, or getEntryByName methods. The mark entry type is “mark,” and the measure entry type is “measure.”

The following sample code uses the mark and measure methods to measure the amount of time it takes to execute the doTask1() and doTask2() methods. Check out a demo of User Timing on the IE Test Drive site.

<!DOCTYPE html>

<html>

<head></head>

<body onload=’doWork()’>

<script>

 function doWork()

 {

  performance.mark(‘markStartTask1′);

  doTask1();

  performance.mark(‘markEndTask1′);

  performance.mark(‘markStartTask2′);

  doTask2();

  performance.mark(‘markEndTask2′);

 

  performance.measure(‘measureTask1′, ‘markStartTask1′, ‘markEndTask1′);

  performance.measure(‘measureTask2′, ‘markStartTask2′, ‘markEndTask2′);

  sendUserTiming(performance.getEntries());

 }

</script>

</body>

</html>

We want to thank everyone in the W3C Web Performance Working Group for helping design these interfaces and browsers vendors for working on implementing this interface with an eye towards interoperability. With these interfaces, Web developers can truly start to measure and understand what they can do to improve the performance of their applications.

Try out the performance measurement interfaces with your Web apps in IE11, and as always, we look forward to your feedback via Connect.

Thanks,

Jatinder Mann, Internet Explorer Program Manager

25 HTML Speed Tips

net244feathtml5

From minifying your JavaScript to using image sprites, here are 25 top tips for high performance web apps. My article on 25 HTML5 speed tips originally appeared in .Net Magazine Issue 244.

For the past few years I've been part of the IE team at Microsoft looking into ways to improve the online experience. Along the way we've learned a lot about web performance and developed an in-depth understanding of what goes into making sites and apps fast.

Creating high-performance web applications is essential for every web developer, whether we're talking about websites that run on a standards-based web browser or apps for the Windows Store. The goal of the developer is to improve web performance by reducing the following factors:

Display time

The most important objective is what we refer to as 'Display Time'. This has many names across the industry including 'time to glass' and 'primary paint'. Display Time measures the time from when the user performs an action until the user sees the result of that action on the screen. This is the duration from when the user navigates to the site until when the site is visually complete loading.

Elapsed time

Most sites continue to perform work in response to the user action after the content has been displayed to the screen. This may include downloading user data (such as email messaging) or sending analytics back to a provider. From users' perspective, the site may appear loaded. However, significant work is often occurring in the background, which impacts responsiveness.

CPU time

Web browsers are almost exclusively limited by the CPU; the work a browser performs on the CPU and how efficiently that work occurs will make the single largest impact on performance. That's why offloading work to the GPU has made such a significant impact to IE9 and IE10 performance. The amount of CPU time required to perform the action and the CPU efficiency are critical.

Resource utilization

Building a fast browser means ensuring resources across the entire PC work well together. This includes network utilization, memory usage patterns, GPU processing, graphics, memory and hundreds of other dimensions. Since customers run several applications at the same time on their PC, it's important for browsers to responsibly share these resources with other applications.

Power consumption

When utilizing underlying PC hardware, it's important to take power consumption into consideration. The more efficiently a browser uses power, the longer batteries will last in mobile scenarios, the lower the electricity costs for operating the device, and the smaller the environmental impact. Power and performance are complementary goals.

I want to share with you some of the things that I've learned about developing faster websites and apps, and the changes you can make today to improve performance.

There are seven key principles that developers can take into consideration:

  • Respond quickly to network requests
  • Minimize bytes to download
  • Efficiently structure markup
  • Optimize media usage
  • Write fast JavaScript
  • Render in standards mode
  • Monitor what your app is doing

Within these seven principles I've included a number of helpful performance tips below that will make your HTML5 websites and apps run faster (watch a video of a presentation I made for even more HTML5 tips).

Respond quickly to network requests

01. Prevent 3xx redirections

When a user clicks on a link, they expect to receive content as quickly as possible. 3xx redirections can create a 250 millisecond delay in an application. This may seem like a short delay, but it's roughly 10 per cent of page load time. Around 63 per cent of the world's top websites contain 3xx redirections.

02. Use content distribution networks (CDNs)

CDN

With CDNs you can easily geographically locate your data closer to your user. Services such as Azure help with this and can help to reduce the amount of time that content spends travelling between locations. On today's networks, that can save up to as much as 300 milliseconds.

03. Maximize concurrent connections

When people think about a network, they often think of a single pipeline conducting content back and forth. In fact, the browser can make six concurrent connections at a given time, enabling it to download six resources at once. This is possible across multiple domains: by distributing your content you can further increase the number of resources you can download simultaneously. If your website holds images across six or seven domains, you can significantly reduce the amount of time the page takes to load.

04. Understand your network timing

Understand the breakdown of your network timing – navigation timing, resource timing and user timing – and use standards-based APIs available in modern browsers such as IE10, and in Windows Store apps. You can get higher resolution timing information on the navigation of your document. Navigation timing allows you to understand the amount of time your application spends in various phases.

timing

Minimize bytes downloaded

05. Download fewer resources and bytes

The fewer resources you can download the better. Look at the resources you are downloading and work out where you can cut down. The average website today downloads 777kB of data. The vast majority of these bytes are taken up by images, followed by JavaScript and Flash content.

06. Gzip: compress network traffic

The best way to download fewer bytes is to Gzip your content. Most people will get this service for free because of the servers they are using, but many unintentionally turn off this decoding technique.

07. Standard file capitalization

This often catches people by surprise, but the server will pick up on variations in upper/ lower case. The following download requests are two variations of the same request:

Lower case

  1. <img src="icon.png"/>

Title case

  1. <img src="Icon.png"/>

From the web platform perspective, these are two different files. Therefore, two different network requests will be processed as a result.

Efficiently structure markup

08. Avoid quirks mode

Always use a standards-based doctype to avoid quirks mode. Start with <!DOCTYPE html>. The modern web has no place for quirks mode, which was designed so that mid-90s web pages would be usable in turn-of-the-century 'modern' browsers like IE6 and Firefox 2.

Most web pages today end up in quirks mode accidentally, as a consequence of an invalid doctype or extraneous text before the doctype. This can cause strange layout issues that are hard to debug.

09. Avoid inline JavaScript events

Take care that you don't use inline JavaScript events inside your HTML markup. One example of this would be <button onclick="validate()">Validate</button>. This is a practice that breaks the clean separation that should exist between the markup, presentation and the behaviour.

Also, if your scripts are loading at the bottom of the file, it's possible for a user to interact with the page, trigger an event and attempt to call a script that hasn't loaded yet, which will cause an error.

10. Link style sheets at the top of the page

By placing the CSS at the top of the page, the browser will issue that request first and simultaneously block painting until CSS is complete. By placing CSS in the head, images, JavaScript and other time intensive resources can be downloaded later. To that end, we would also advise against linking your CSS at the bottom of the page.

11. Only include the necessary styles

It may seem like a good idea to hold one very large CSS file that's shared across the entirety of your website. Some of the top news sites in the world use this approach today. One news site, for example, has one style sheet with 4,000 rules in it, of which only 5-10 per cent are used in a single page. The browser has to download all these styles, pass them and create internal data structures, most of which will never be used.

12. Link your JavaScript at the bottom of the page

This is common best practice. You should always make sure that the styles and the visuals are downloaded first. That way, the JavaScript can follow later and manipulate the page how it likes. However, if you absolutely have to link the JavaScript in the header, as determined by the CRM system or hosting service you are using, then be sure to use the defer tag.

13. Understand the limits of HTML5 tags

New HTML5 tags like <section>, <header>, and <footer> improve the semantics of markup, but require a special shiv script to run in Internet Explorer 6, 7, and 8, or they won't be recognised. Pages that need to work with these legacy browsers, even when scripts are disabled, cannot use the new HTML5 tags. Using plain <div> elements and classes is often a safer course of action for those cases.

14. Standardize on a single framework

There are a lot of JavaScript frameworks out there and a many of them do the same things in terms of functionality. The browser has to download the JavaScript, pass it and create internal data structures without knowing whether or not you will execute them. By sticking to one, single framework, you will significantly improve your performance.

15. Don't include scripts to be cool

There are a lot of 'cool' scripts out there. It's really easy to include a script file that does something 'cool' in the background, but that script file is competing with resources in your page load. Are you sure it's necessary?

Optimize media usage

16. Minimize the number of images

media

The majority of bytes downloaded are for images. The average number of images on the top 100,000 websites is 58. When you go over 20-30 images, you'll start to see a performance impact. Take a look at all your resources and ask yourself whether you need them or not.

17. Use image sprites

Image sprites can significantly reduce the amount of data that needs to be downloaded. Where possible create image sprites by hand.

sprites

18. Consider which image formats you use

PNG offers the most efficient balance between compatibility, coding size, CPU decoding time and the bytes required for CPU decoding. It also has one of the best compression rates. However, for photographs, JPEG tends to be the better format.

19. Avoid complex SVG paths

Complex SVG paths take the browser a long time to download, pass and construct internal data structures. Where possible, try to generate the most concise SVG path you can. It may take a while, but generating the SVG path by hand really can help.

svg

20. Specify an image preview for HTML5 video

If you don't specify an image preview, the browser has to download the video, figure out what the first frame is, and then display that to the user. By specifying an image preview the browser doesn't need to download the video, only the image. It only starts downloading the video when the user requests it.

preview

21. Minimize media plug-in usage

Despite perceptions, most media plugins aren't as fast as HTML5 at running video. Furthermore, they'll compete with the application for resources.

Write fast JavaScript

22. Minify your JavaScript

Many people will be familiar with this technique. Essentially, you take your JavaScript, remove some characters, and then simplify the variables. As demonstrated below:

Initial (66 characters)

  1. function Sum(number1, number2) {
  2. return (number1 + number 2);
  3. }

Characters removed (54 characters)

  1. function Sum(number1,number2){return
  2. number1+number2;}

Compacted (30 characters)

  1. function Sum(a,b){return a+b;}

You're left with less JS to download, and there's a hidden benefit: runtime performance. With tighter code and smaller variable names, the runtime can look up variables faster, improving download as well as runtime performance.

js

In addition, initialize JavaScript on demand. Don't load your entire JavaScript library during the page load – you can do it dynamically when you need to.

Render in standards mode

23. Use web standards

Not only does using web standards help reduce the cost of development and the complexity of testing across browsers and devices, but it can also achieve some noticeable performance benefits. In fact, we found that sites in IE10 got an average of 30 per cent better page load time when they switched to standards mode. The benefit is similar in IE9 too.

standardsmode

Standards mode is the default rendering mode of all browsers and offers the best implementation of web standards that work the same in all browsers. In addition to standards mode, IE provides compatibility modes to keep websites designed for old versions of IE working. This was particularly helpful in the past when it was a common practice to first detect a browser and then serve code meant only for it. But this is no longer necessary in many cases because the web standards code is displayed similarly in modern browsers, including IE10 and 9, through standards mode. This new practice is commonly known as feature detection and was made popular by Modernizr).

Monitor what your application is doing

24. Combine application timers

Most applications on the web today have 5-10 timers that are running at all times. About half of those timers are dormant. Make sure timers or sequences aren't running unnecessarily. When possible, combine your application timers and, if you can, create one timer that manages all your sequences.

25. Check your app's visibility status

Without knowing the visibility status of your application, you're forced to design it as always visible. Page visibility is a new standards-based API supported in IE10 and Windows Store apps, as well as most modern browsers.

Page visibility allows you to determine the visibility status of the application, so you can reduce activity when you know the application is not visible and, in doing so, save CPU time and increase battery life.

Conclusion

Better site performance can have a big impact on your users. Slow web page loading is a major factor in users abandoning a site, can reduce perceived site credibility and affect product sales. The time savings of some of the tips above may seem relatively small, but, when implemented together, they can create considerable time savings. What's more, they will encourage you to think efficiency first and create apps and sites with the user experience at the forefront.

Performance Matters

I wrote this column on A List Apart on why performance matters and how we can accomplish a lot when working together with industry and community leaders in the W3C.

While a decade ago it may have been okay to go brew a pot of coffee while the computer was loading, today we all expect our software and devices to be fast and responsive. The same holds true for web performance. The load speed and responsiveness of a web application play a critical role in our choice of which applications we want to use. At the same time, how efficiently software runs plays a critical role in device battery life. No one wants to be carrying around a brick.

To build fast web applications, web developers need to be able to qualitatively measure application performance, effectively use their hardware, and most importantly, know which patterns to optimize and which to avoid.

We found that web developers haven’t had the right tools. Left on their own to guess, web developers in many cases only emphasized JavaScript performance and equated JavaScript performance with web performance.

However, web performance is truly a multi-dimensional problem. How quickly the network can download resources, how efficiently the CPU can perform web runtime operations, and the amount of available GPU memory all impact web performance. Let’s take a closer look at an example I gave in a Build presentation involving five real world travel booking web applications: Kayak, Expedia, Priceline, Travelocity, and Orbitz. All of these sites have very similar elements on the page. They all have logos, banner ads, interactive real time flight data, etc. We would expect them all to have very similar performance characteristics.

Figure 1 shows the metrics for each of these sites, anonymized here to avoid poking at anyone. We can see that even though these travel sites have very similar functions, they have all been designed very differently. Some have more bytes coming down the wire, others have more JavaScript.

  Total Size (k) Number of Elements CSS Rules Image Files Script Lines
Site #1 3,697 1,504 1,392 41 77,768
Site #2 2,278 1,100 5,325 29 39,183
Site #3 1,061 2,673 1,105 66 12,643
Site #4 1,812 4,252 1,672 12 10,284
Site #5 1,372 900 3,902 6 38,269

Many developers would assume that the fastest site would be the one with the least number of formatted lines of JavaScript, like Site #4, or the one with the least bytes downloaded, like Site #3. However, that’s not the case. Site #5 is actually the fastest, even though it has more JavaScript and bytes downloaded. Figure 2 breaks down the amount of time each of these sites spent in the different browser subsystems, subsystems all standards-based web browsers have.


Bar chart showing browser subsystem load times in milliseconds for five sites.

Time spent in browser subsystems to load top five travel sites.

There are really two points here. It’s not just about how to most efficiently execute JavaScript, it’s about how all of the browser subsystems can most effectively work together. Without the right tools, developers don’t really know why their applications are slow and what to optimize.

Most web experts realized that there was a need for better interfaces to help developers measure their applications and write faster code. More importantly, there was a need for a forum to talk about performance problems that web developers faced and how to solve them.

At tech conferences, folks like Steve Souders and Arvind Jain from Google, Jason Weber from Microsoft, Jason Sobel from Facebook, and others started discussing performance-related issues, such as an accurate and interoperable way to measure page navigation time. These conversations soon led to the establishment of a working group in the W3C focused solely on performance. It was in the summer of 2010 that Jason Weber from Microsoft and Arvind Jain from Google were selected to co-chair the newly minted W3C Web Performance Working Group, chartered to identify and solve performance issues that developers encountered on the web.

Not everyone was certain all these competitors could work together. The first comment on the IE Blog post announcing this working group, referring to the Google and Microsoft co-chairing of the working group, was “I bet the Richter scale could pick up the tension in that room.” Fast forward to 2013 (and ignoring the fact that the Richter scale isn’t an actual instrument), and the Web Performance Working Group can really be seen as an example of the ideal working group. In just three years, Microsoft, Google, Mozilla, Intel, DynaTrace, and others in this group have designed APIs to help developers measure their web applications more accurately and precisely than ever before (Navigation Timing, Resource Timing, User Timing, Performance Timeline, High Resolution Time), and more efficiently schedule activities in web applications (requestAnimationFrame, Page Visibility, Efficient Script Yielding). Based on the conversations from the W3C Workshop on Web Performance, which this working group put together to hear from 45 performance experts and web developers from 21 organizations, the working group is now actively working on six new specifications to solve other performance problems: efficiently prioritizing the download of resources (Resource Priorities), asynchronously transferring data without blocking page unloads (Beacon), getting detailed network error and availability information (Navigation Error Logging, Resource Error Logging), marking links to be instantly loaded (Prerender), standardizing on a performance metric format (HAR—HttpArchive), and improving existing interfaces (Navigation Timing L2, High Resolution Time L2). The High Resolution Time specification went from an idea to fully implemented by all major browser vendors and fully standardized in just 11 months. That must be some sort of a record!

Looking back, the W3C was the perfect forum to bring together the top browser vendors, web properties, and performance experts to discuss and try to solve major performance issues. As web developers design more efficient web applications and browser vendors design runtimes that can execute those web applications more efficiently, we will all benefit from a faster and more responsive web browsing experience.

Beyond the Web Performance Working Group, the W3C has plans to work with the community to continue to enhance browsers, develop tools, improve education, and promote performance in W3C specifications.

Let’s keep innovating. Follow the performance conversation in the Web Performance Working Group!

Using Hardware to Decode and Load JPG Images up to 45% faster in Internet Explorer 11

In this IE Blog article, I talk about how IE11 uses hardware more efficiently to decode and load JPG images up to 45% faster!

Internet Explorer 11 and Windows Store Apps on Windows 8.1 offload parts of the image decoding pipeline to the graphics hardware, resulting in up to 45% faster image load, up to 40% lower memory consumption, and improved battery life. Images on average account for the most bytes downloaded on the Web today. To improve the performance of loading JPG images, IE11 has streamlined its JPG decoding by moving some steps of the decoding process directly to the GPU where it can be done significantly faster and in parallel.

JPG Image Format

Images account for 61% of bytes downloaded on the Web today, and 47% of image requests are for JPG images. By using hardware more efficiently to decode JPG images, IE11 now loads JPG images up to 45% faster and uses up to 40% less memory over previous IE versions.

To understand these performance improvements, let’s start by looking at how JPG images are typically encoded. The first step of JPG encoding is to convert the bitmap from the RGB color space to YCbCr color space.

JPG encoding process.

RGB defines color in three color components: red, green, and blue. If we break down this image of the Grand Tetons in to its RGB color channels, you can see that there is a high degree of detail in each of the channels. The amount of memory required by IE for an image in the RGB color model can be calculated by multiplying image width x image height x 32 bits.

RBC Channels of a JPG image

Image of the Grand Tetons with its Red, Blue, and Green components

YCbCr (commonly referred to as YUV color space) defines a color space in terms of one luma (Y) and two chroma (CbCr) components. The luma component represents the brightness information of a color. The chroma components contain the color differences, with Cb containing the blue difference and Cr containing the red difference. The figure below shows the same image of the Grand Tetons broken down into its Y, Cb, and Cr channels.

YCbCr channels of JPG image

Image of the Grand Tetons with its Y, Cb, Cr channels

The next step in the encoding process is to compress the image size using a lossy compression known as chroma subsampling. The chroma channels can be significantly compressed because the human eye is more sensitive to the brightness of an image and less sensitive to color or hue. For example, looking at the Cb and Cr channels, the last two on the right, you can see that there is very little information contained in these channels relative to the Y channel. The level of subsampling is typically expressed using a three part ratio, e.g., 4:2:0. The first part of the ratio represents a horizontal sampling reference, and the second two parts represent the number of chrominance samples in the first and second rows of that sampling reference. Chroma is typically subsampled at one of three levels:

  • 4:4:4, where the chroma data isn’t subsampled
  • 4:2:2, where the chroma data is downscaled horizontally by 2,
  • 4:2:0, where the chroma data is downscaled horizontally and vertically by 2.

A 4:2:0 subsampled YCbCr JPG image can use up to 62.5% less memory than the original RGB bitmap! Most JPG images used on the Web today are already in 4:2:2 or 4:2:0 chroma subsampling modes. Most image processing tools will automatically subsample to 4:2:2 or 4:2:0 when using the ‘Save for Web’ option.

Once the image has been subsampled, the discrete cosine transformation, quantization, and Huffman encoding processes are also applied to get the final encoded JPG image.

More Efficiently Using Hardware in IE11

To decode a JPG image, you run the same encoding steps in reverse. Traditionally, IE had always decoded JPG images into RGB bitmaps by running all of the below decoding steps directly on the CPU. At render time, IE would copy the RGB bitmap to the GPU for rendering.

2727.Decode

To more efficiently use the hardware, IE11 now splits the JPG decoding work between the CPU and GPU. IE11 decodes the JPG image into the chroma subsampled YCbCr color space on the CPU, but then does the chroma upsampling and YCbCr to RGB color conversion steps on the GPU at draw time, where it can happen much faster and in parallel. This process frees CPU time to perform other operations, as the CPU is a common bottleneck in modern sites and apps. In addition to the decode time improvements, copying the much smaller YCbCr image to the GPU reduces the amount of memory that is copied and stored on the GPU (a limited resource). Using less CPU and memory also reduces power consumption and increases data locality.

The below CPU chart shows the amount of time it takes to decode and draw a 4:2:0 subsampled YCbCr JPG version of the Grand Tetons image on IE10 on Windows 8. Image decoding took 31.9ms, and total time to draw the image was 81.5ms.

4477.uhtdjiiie-image5_760x169

If we load the same JPG image on IE11 on Windows 8.1, you can see that the decode time is now only 17.9ms, which is an improvement of 44%! The total time to draw the image is now 57.5ms, which is 30% faster.

6507.uhtdjiiie-image6_760x170

Because most JPG images on the Web today are in YCbCr format, IE11 on Windows 8.1 users will experience these improvements automatically. We recommend that developers ensure their JPG images are compressed in 4:2:2 or 4:2:0 chroma subsampling modes to maximize the performance benefits of hardware accelerating the JPG decoding pipeline.

Summary

By using the hardware more efficiently to decode JPG images, IE11 improves the performance of nearly every page you browse, while also improving the power consumption and battery life of your device. Please install the Windows 8.1 Preview from the Windows Store and try IE11. As always, we welcome your feedback, either through the IE11 Send Feedback tool or on Connect.

Jatinder Mann, Internet Explorer Program Manager

Web Performance APIs Rapidly Become W3C Recommendations

In this IE Blog post article, I discuss how the W3C Web Performance APIS are rapidly becoming W3C Recommendations with interoperable support for all major web browsers.

The W3C Web Performance Working Group recently published three specifications as W3C Recommendations with full implementations from all major browser vendors, advancing developers’ ability to accurately measure the performance of Web applications and make the Web faster. Over the last three years, companies including Microsoft, Google, Mozilla, Intel, Facebook, and others have been working towards standardizing the Navigation Timing, High Resolution Time, and Page Visibility interfaces in the Working Group. Rapid adoption of these recommendations demonstrates what’s possible when the industry and community come together through the W3C.

To make the Web faster, developers need the ability to accurately measure the performance characteristics of Web applications and the ability to effectively use the underlying hardware to improve the performance of their applications. To solve these problems, the Web Performance Working Group worked on 15 different specifications that address those issues. The table below shows the maturity level of all the specifications currently edited by the Working Group.

Web Performance Spec Status 5_22_2013

The Navigation Timing, Resource Timing, User Timing, and Performance Timeline specifications help developers accurately measure the timing of the navigation of the document, fetching of resources on the page, and developer script execution. Prior to these APIs, this data wasn’t easily obtainable. Navigation Timing was published as a W3C Recommendation, and all major browser vendors support it. The other three interfaces are currently at the Candidate Recommendation stage awaiting two full implementations from browser vendors. IE10 is currently the only browser that implements all of these interfaces, however, other vendors are working on implementations.

To ensure these performance metrics are measured in the most accurate way possible, the High Resolution Time specification allows developers to measure operations with sub-millisecond accuracy. This interface not only benefits accurate measurements of performance metrics, but also allows better frame rate calculations and synchronization of animations or audio cues. This interface has been published as a W3C Recommendation, with all major browser vendors implementing the performance.now() method defined in the specification.

The Page Visibility API allows for programmatically determining the current visibility state of the page. Developers can use this data to make better CPU- and power-efficiency decisions, e.g., throttling down activity when the page is in the background tab. This specification has also been published as a W3C Recommendation, with all major browser vendors implementing it.

The Timing Control for Script-Based Animations, and Efficient Script Yielding specifications help developers write more CPU- and power-efficient Web applications. The requestAnimationFrame API, from the Timing Control for Script-Based Animations specification, allows for creating more efficient JavaScript animations. All browser vendors fully support this interface, with the Working Group actively working on publishing this specification as a Candidate Recommendation. The setImmediate API, from the Efficient Script Yielding specification, allows developers to efficiently yield control flow to the user agent and receive an immediate callback, efficiently leveraging the CPU. IE10 is the first browser to implement this interface.

This year the Working Group also started to look at new ideas, with editor’s drafts of those ideas currently being discussed in the Working Group. The Beacon API is intended to help scripts asynchronously transfer data to a Web server without blocking the unload event, which can negatively impact the perceived performance of the next navigation. The Resource Priorities API defines a means for Web developers to give the browser hints on the download priority of resources to help improve the page load time. As a corollary to the Timing specs, the Navigation Error Logging and Resource Error Loggingspecifications help developers understand the errors and availability of their applications. The Navigation Timing Level L2 specification adds High Resolution Time and Performance Timeline support to Navigation Timing, and High Resolution Time L2 specification adds Web Worker support. These are just some of the drafts the Working Group is currently defining, with more specification drafts on Prerender and other diagnostics areas forthcoming.

The W3C Web Performance Working Group is a great example of how quickly new ideas can become interoperable standards that developers can depend on in modern HTML5-enabled browsers. Together with industry and community leaders who participate in the Working Group, we hope to continue to make rapid progress on interoperable standards that will help developers make the Web faster.

Thanks,
Jatinder Mann
Internet Explorer Program Manager

W3C Web Performance: Continuing Performance Investments

I discuss the W3C Workshop on Performance in this IE Blog post article.

The W3C Web Performance working group recently held the W3C Workshop on Performance on Thursday, November 8, 2012. The goal was to hear current challenges and proposals for new performance ideas for the working group to consider. There were 45 attendees from 21 organizations, including most browser manufactures (Microsoft, Google, and Mozilla), hardware organizations (Intel, Qualcomm, Nokia, Motorola), network organizations (Cisco, Akamai, F5), and top Web properties (GMail, Google Search, Bing, NetFlix, LinkedIn, Zynga, and more). Details on the presentations and discussions from the workshop can be found in this report.

Providing the ability to accurately measure the performance characteristics of Web applications and create power- and CPU-efficient applications is critical to Web performance. The W3C Web Performance working group worked on achieving those goals in its recently completed second chartered period. In under two years, the working group rapidly standardized and modern HTML5-enabled Web browsers implemented these eight interfaces: Navigation Timing, Resource Timing, User Timing, Performance Timeline, Page Visibility, Timing control for script-based animations, High Resolution Timeand Efficient Script Yielding. Internet Explorer 10 is the first browser to support all eight of these new APIs.

The working group has since been focused on gathering data to understand which areas to focus on in its third chartered period. In addition to the Workshop on Performance, the working group has invited performance experts to its weekly conference calls and has broadly surveyed the performance community on ideas.

Based on all the data gathered these past few months, the Web Performance working group has decided to focus on the following areas in the third chartered period:

  • Timing Metrics The working group will continue to improve the Timing interfaces, Navigation Timing, Resource Timing, User Timing and Performance Timeline. For example, we will consider providing Web workers support in the Timing interfaces and including information on video byte ranges in Resource Timing.
  • Efficient Script Yielding The working group will continue to improve on power- and CPU-efficient APIs, like the setImmediate API defined in the Efficient Script Yielding specification.
  • Prerender The working group will standardized the prerender feature which allows navigations to appear almost “instantly” in cases where the browser has high confidence that a user will visit an URL. The way this feature would work is that the browser will proactively navigate to a Web page in a hidden tab, when it sees the “prerender” link type or has high confidence that user will visit that link. When the user does visit that link, the browser will make the hidden tab visible, giving the perception of instant navigation.
  • Resource Priorities Today, browsers download resources in the priority order that they believe are most efficient in helping the page load occur quickly. However, developers may want to prioritize some resources over others. For example, downloading images above the fold may be of higher priority than those below the fold. Though, developers can give some hints to the browser on download priority, like using the “defer” and “async” attributes in markup, these concepts do not include most resources. To help the browser prioritize downloading resources, the working group is expanding the charter to include interoperable means for developers to give the browser hints on the download priority of resources.
  • Diagnostics Interfaces Developers are interested in learning how to make their Web applications faster and less error prone. The working group is expanding the charter to include interoperable means for developers to get browser diagnostics information on their Web applications. For example, using these interfaces a developer could understand where memory is leaking or what errors users are encountering on their Web applications.
  • Beacon Today, analytics scripts will block the current page from unloading by running in a loop in order to confirm that analytics data has been sent to a Web server. This behavior will delay the navigation to the next page, resulting in user perception of poor performance. To help developers avoid that pattern, the working group is expanding the charter to include an interoperable means for developers to asynchronously transfer data from the browser to a Web server, with a guarantee from the browser that the data will eventually be sent.
  • Display Performance Developers are interested in understanding the performance of their games and animations.

    The working group is expanding the charter to include interoperable means for developers to get frame rate and throughput of the display type of information.

This working group is a great example of how quickly new ideas can become interoperable standards that developers can depend on in modern HTML5-enabled browsers. Together with industry and community leaders who participate in the working group, we hope to continue to make rapid progress on interoperable standards that will benefit developers and everyone who uses the Web.

Jatinder Mann, Internet Explorer, Program Manager

Build 2012: 50 Performance Tricks to make your HTML5 apps and sites faster

Creating high performance web applications is crucial for every web developer, be it a web site that runs on a standards based web browser or a Windows Store App. Microsoft recently hosted the BUILD 2012 conference at the Microsoft campus in Redmond, WA. At this conference, I had the opportunity to share the Internet Explorer team’s favorite 50 performance tips to make HTML5 apps and sites faster. If you weren’t able to attend the conference, I recommend you check out the video.

Video link to the Build 2012 session: 50 Performance Tricks to make your HTML5 app and sites faster.

These performance tips and tricks apply equally to web sites that run on standards based web browsers, and Windows Store Apps, which are also just the web. There are six principals detailed in the talk that will help you improve the performance of your apps and sites today:

  1. Quickly response to network requests
  2. Minimize bytes downloaded
  3. Efficiently structure markup
  4. Optimize media usage
  5. Write fast JavaScript
  6. Know what your application is doing

I hope you enjoy the talk.

Thanks,
Jatinder Mann

Web Performance: When millisecond resolution just isn’t enough

I discuss when milliseconds just aren’t enough in this IE Blog article:

Sometimes measuring time in millisecond resolution just isn’t accurate enough. Together with industry and community leaders, the W3C Web Performance working group has worked to solve this problem by standardizing the High Resolution Time specification. As of this week, this specification has been published as a Proposed Recommendation (PR) and is widely adopted in modern browsers. Take a look at the What Time is it? test drive demo to see how this API works.

This specification has gone from just an idea to PR in eight short months. The PR stage of standardization is the final step before a Web standard becomes an official W3C Recommendation. Additionally, this interface has been broadly adopted in browsers, including full support in Internet Explorer 10 and Firefox 15, and supported with a prefix in Chrome 22. This is a great example of what’s possible when the industry and community come together through the W3C.

So why aren’t milliseconds good enough? Time has long been measured in the Web platform using some form of the JavaScript Date object, whether it is through the Date.now() method or the DOMTimeStamp type. The Date object represents a time value as time in milliseconds since 01 January, 1970 UTC. For most practical purposes, this definition of time has been sufficient to represent any instant to within 285,616 years from 01 January, 1970 UTC.

For example, at the time of writing this blog, my Date.now() time value from my IE10 Developer Tools Console was 1350509874902. This thirteen digit number represents the number of milliseconds from the origin of this time base, 01 January, 1970. That time corresponds to 17 Oct 2012 21:37:54 UTC.

Though this definition will continue to be genuinely useful for determining the current calendar time, there are some cases where this definition is not sufficient. For example, it is useful for developers to determine if their animation is running smoothly at 60 frames per second (one frame painted every 16.667 milliseconds). Using the simple method of calculating the instantaneous FPS by measuring when the frame drawing callback was last made, one can only determine FPS to 58.8 FPS (1/17) or 62.5 FPS (1/16).

Similarly, sub-millisecond resolution is also desirable when accurately measuring elapsed time (e.g., using the Navigation Timing, Resource Timing and User Timing APIs to instrument your network and script timing) or when attempting to synchronize animation scenes or audio with animation.

To solve this issue, the High Resolution Time specification defines a new time base with at least microsecond resolution (one thousandth of a millisecond). To reduce the number of bits used to represent this number and to increase readability, instead of measuring time from 01 January, 1970 UTC, this new time base measures time from the beginning of navigation of the document, performance.timing.navigationStart.

The specification defines performance.now() as the analogous method to Date.now() for determining the current time in high resolution. The DOMHighResTimeStamp is the analogous type to DOMTimeStamp that defines the high resolution time value.

For example, looking at the current time using performance.now() and Date.now() in the IE10 Developer Tools Console at the time of writing this blog, I see the following two values:

performance.now():           196.304879519774
Date.now():        1350509874902

Even though both of these time values represent the same instance in time, they are being measured from a different origin. The performance.now() time value definitely feels more readable.

As High Resolution Time is measured from the start of a document’s navigation, performance.now() in a sub-document will be measured from the start of navigation of the sub-document, not the root document. For example, suppose a document has same-origin iframe A and cross-origin iframe B, where a navigation occurred in iframe A about 5 milliseconds after the start of navigation of the root document and in iframe B about 10 milliseconds after the start of navigation of the root document. If we measure the exact moment of time 15 milliseconds after the start of navigation of the root document, we would get the following values for performance.now() calls in the different contexts:

performance.now() in iframe B:                                5.123 ms

performance.now() in iframe A:                               10.123 ms

performance.now() in root document:                    15.123 ms

Date.now() in any context:                         134639846051 ms

Figure: Date.now() is time measured since 01 January 1970, whereas performance.now() is time measured since the start of the document navigation

This design not only ensures there is no data leakage on the time of creation of the parent in cross-origin iframes, it also allows you to measure time relative to your start. For example, if you were using the Resource Timing interface (which uses High Resolution Time) to determine how long it takes for a server to respond to a resource request in a sub-document, you wouldn’t need to make adjustments to take the time of adding the sub-document to the root document into account.

If you wish to do cross frame time comparisons, you would just need to request top.performance.now() to get a time value relative to the root document’s start of navigation, which would return the same value in all same-origin iframes.

Another significant benefit of this API over Date.now() is that performance.now() is monotonically increasing and not subject to clock skew or adjustment. The difference between subsequent calls to performance.now() will never be negative. Date.now() doesn’t have such a guarantee and in practice we have heard of reported cases where negatives time have been seen in analytics data.

High Resolution Time is another great example of how quickly new ideas can become interoperable standards that developers can depend on in modern HTML5-enabled browsers. Thanks to everyone in the W3C Web Performance Working Group for helping design this API and to other browser vendors for rapidly implementing it with an eye towards interoperability.

Thanks,
Jatinder Mann
Internet Explorer Program Manager

W3C Web Performance Workshop

In this IE Blog article, I discuss the W3C Web Performance Workshop:

The W3C Web Performance Working Group is looking for new use cases and performance issues to solve in its next chartered period. To that effect, the Working Group is holding a public workshop on November 8, 2012, in Mountain View, CA where performance experts and Web developers are invited to present ideas and discuss current challenges. Statements of interest will be the basis of the discussion at the Workshop and must be submitted by October 29th to this mailing list.

Fast HTML5 Web applications benefit consumers who browse the Web and developers building innovative new experiences. Just over two years ago, the W3C announced the formation of the Web Performance Working Group chartered with two goals: making it easier to measure and understand the performance characteristics of Web applications and defining interoperable methods to write more CPU- and power-efficient applications.

Over the course of these two years, together with Google, Mozilla, Facebook, and other industry and community leaders who participate in the W3C Web Performance Working Group, the working group has designed and standardized eight interfaces that are now widely adopted in modern HTML5-enabled browsers: Navigation Timing, Resource Timing, User Timing, Performance Timeline, Page Visibility, Timing control for script-based animations, High Resolution Time and Efficient Script Yielding. These APIs are supported in Internet Explorer, Firefox, and Chrome, and are great examples of how quickly new ideas can become interoperable standards that developers can depend.

If you are interested in sharing feedback to the Working Group and cannot attend the workshop, you can do so by completing this survey. The deadline for the survey is November 2nd.

Thanks, — Jatinder Mann, Program Manager, Internet Explorer

IEBlog: Web Performance APIs Rapidly Become W3C Candidate Recommendations

In this IE Blog article, I discuss how the Web Performance APIs are rapidly standaridizing.

We’re pleased to share that three new W3C Web Performance Working Group specifications moved to W3C Candidate Recommendations. Accurately measuring the performance characteristics of Web applications is critical to making the Web faster. In addition, developers need the ability to effectively use the underlying hardware to improve the performance of their applications. Over the last two years, companies including Microsoft, Google, Mozilla, Intel, and Facebook have been working toward these goals through the working group. This is a great example of what’s possible when the industry and community come together through the W3C.

The Navigation Timing, Resource Timing, User Timing and Performance Timeline specifications help developers accurately measure Web application performance. The first three specifications provide developers with information related to the navigation of the document, resources on the page, and developer scripts, respectively. The Performance Timeline specification defines a unifying interface to retrieve this timing data. Prior to these API’s it was not possible for developers to accurately measure their site performance.

To ensure these performance metrics are measured in the most accurate way possibly, the High Resolution Time specification defines a sub-millisecond clock resolution. This interface not only benefits accurate measurements of performance metrics, but also allows better frame rate calculations and synchronization of animations or audio cues. For the first time developers can measure operations with sub-millisecond accuracy.

The Page Visibility, Timing control for script-based animations, and Efficient Script Yielding specifications help developers write more CPU- and power-efficient Web applications. The Page Visibility API allows for programmatically determining the current visibility state of the page. Developers can use this data to make better CPU- and power-efficiency decisions, e.g., throttling down activity when the page is in the background tab. The requestAnimationFrame API, from the Timing control for script-based animations specification, allows for creating more efficient JavaScript animations. Finally, the setImmediate API, from the Efficient Script Yielding specification, allows developers to efficiently yield control flow to the user agent and receive an immediate callback, efficiently leveraging the CPU.

In order to ensure Web developers only have to write code once and have it work interoperably in all browsers, the Working Group has worked diligently these last two years to standardize these APIs. The table below shows the maturity level of all the specifications currently being edited in the Working Group.

Web Performance Spec Status 7_27_2012

Table showing the status of W3C Web Performance Specifications

As of this month, the Navigation Timing specification has been published as a Proposed Recommendation (PR). This stage of standardization is the final step before a Web standard becomes an official W3C Recommendation. Additionally, this interface has been broadly adopted in browsers, including support since Internet Explorer 9, Chrome 6 and Firefox 7. The working group recently started incorporating feedback and working on Navigation Timing 2, the next version of the specification.

As of this month, User Timing, Performance Timeline and Page Visibility specifications have been published as a Candidate Recommendation (CR). This stage of standardization is prior to the PR stage and reflects that the W3C believes this specification has been widely reviewed and satisfies the Working Group’s technical requirements. Resource Timing was published as CR just two months ago, along with High Resolution Time, which went from an Editor’s Draft to CR in just three months.

These APIs are a great example of how quickly new ideas can become interoperable standards that developers can depend on in modern HTML5-enabled browsers. Thanks to everyone in the W3C Web Performance Working Group for helping design these APIs and to other browser vendors for starting to implement these APIs with an eye towards interoperability.

—Jatinder Mann, Program Manager, IE Performance

Improving Web Performance: It’s All In The (Navigation) Timing

I gave the following interview to the Benchmark: The Mobile & Internet Performance Review.

An interview with Microsoft IE Program Manager Jatinder Mann

Users’ thirst for speed seems increasingly unquenchable. Even as they (barely) tolerate the sluggish performance of mobile devices, they demand more and more of their PCs. Make them wait one blink of an eye too long, and they are gone, taking the revenue they would generate with them. In 2010, The World Wide Web Consortium chartered a Web Performance Working Group to give developers client-side tools, in the browsers, to gain greater visibility into the timing of each aspect of page loading and help them see how they can make their pages faster. The first product of the working group is the Navigation Timing API, which Keynote is already leveraging to provide more granular site performance reporting and to provide operations managers and developers a common language to address site improvements. Microsoft IE Program Manager Jatinder Mann lives and breathes performance, both on the Internet Explorer team and on the W3C Web Performance Working Group, and is an expert on the Navigation Timing API. Benchmark recently caught up with him to get an overview of the Navigation Timing API and other initiatives and what they offer the Web community.

Benchmark: You work on the Internet Explorer team and focus on performance, and you’ve also been involved with the Navigation Timing API. Tell us what’s going on in your performance world.

Jatinder Mann: The Web Performance Working Group was chartered about a year and a half ago with two main goals – making it easier to measure and understand the performance characteristics of Web applications, and also to define interoperable methods to write more CPU-efficient applications.

So together with Google, Mozilla, Facebook, and other industry leaders that participate in this working group, we designed a few APIs – Navigation Timing, Resource Timing, User Timing, and Performance Timeline. And these APIs help developers accurately measure Web performance.

The first three specifications – Navigation Timing, Resource Timing, and User Timing – define interfaces for Web apps to access the timing information related to the navigation of the document; also resources on the page and developer scripts. Performance Timeline defines the unifying interface to retrieve the data.

Benchmark: The Navigation Timing API is actually available now. What is it, and why is it important for website owners to know about it?

Jatinder Mann: The short answer is, Navigation Timing allows site owners to accurately measure how fast their site is loading for their customers.

And this is not like an instrumented build where you are looking at the server side. With Navigation Timing what you’re getting is data from actual client machines.

So let’s say you are trying to improve page performance for a site which varies from location to location. You can look at machines in various locales and figure out what their actual load times are and understand the different causes of the performance problems.

Before this API there really wasn’t a way to accurately measure that without instrumenting builds and whatnot. So this is natively done within the browser and that makes it much more powerful.

Benchmark: You mentioned some other specs, Resource Timing, et cetera. Can you give us a quick overview of those?

Jatinder Mann: Navigation Timing was getting the time for the document navigation, but if you look at the timeline of activities that happen for a page to load, that’s just the first portion. There’s also all the resources within a page. Resource Timing tackles that problem.

None of the browser vendors have actually implemented anything yet, mainly because the spec isn’t yet at the stage of standardization. But both IE and Chrome are very interested in implementing it as soon as the spec is in a good state which I believe will happen soon.

Then there’s User Timing which focuses on developer scripts within a page. Again, if we look at the timeline of how a web page is loaded and the point at which the user can start interacting with the page, there’s a lot of script execution happening. User Timing is designed for developers to measure the time it takes to execute the scripts on the page.

And all of this information is included in what we call the Performance Timeline. It’s a unifying interface where you can obtain timing data for any of these different measurements – you can get your Navigation Timing, your Resource Timing and your Script Timing through one interface.

Benchmark: So what does this mean in practice? How will it change what developers do?

Jatinder Mann: We’re giving performance information on things that developers can change. You can look at the time spent in these various phases, for example, if your page is very slow to load, you can look at Navigation Timing to understand the culprit. If you see that the redirects are taking too long, you may choose to reduce the number of redirects you’re doing for a page. If the DNS look-up phase is very slow, you may want to use DNS proxies or caching. And if connection times are slow, you can determine which locales are slow and build CDNs closer to those customers.

So a lot of the timing information here is actionable information – you get timing information about why your page is slow and there’re actions you can take to improve that.

Benchmark: A web page goes through a long series of steps and there are performance implications in every one of them.

Jatinder Mann: Yes. The advantage here is that now you can see the breakdown of where the time is being spent. As I mentioned earlier, if your redirects are taking too long, as an owner of a website I may choose to reduce the number of redirects, or be careful of what server I’m redirecting to. Maybe that server’s slow.

Benchmark: So the goal of all this is to make the experience faster for the users. That’s obviously a focus of the IE team and of the working group. What is Microsoft’s perspective on speed? Have you identified any critical thresholds that guide what you do?

Jatinder Mann: For many releases now we’ve made significant product changes to IE to make sure it’s fast and fluid for end users. For example, hardware-accelerated HTML5 implementation, and reworking our layout and JavaScript engines from scratch. I don’t know if it’s easy to characterize a threshold, because we see that the Web consists of many different patterns.

Within our IE performance lab we test many different pages, representative of patterns on the Web, and try to improve those patterns.

For example you can have five different news sites and they may all be using different patterns. So it’s not as easy as saying ‘improve JavaScript and all Web pages will be faster’ because some sites may be JavaScript heavy, some may be layout heavy.

Benchmark: What is your sense of the how the developer community and the site community are embracing this API? Or is it still too early to tell?

Jatinder Mann: Keynote, for example, has already started performance monitoring using this API. Facebook had a lot of requests for this API while we were building it. So I’m certain at least for Resource Timing, once that’s available, they may be interested as well. And there are many Microsoft teams that are looking at this API.

So there’s definitely some interest already through people that are in the business of performance improvement and I would love to see more site owners start using these tools to improve their site performance.

Benchmark: What’s next for the working group?

Jatinder Mann: My expectation is that Resource Timing, User Timing and Performance Timeline will probably be the next ones that the browser vendors will want to implement, mainly because it would complete the story with Navigation Timing and you would get the complete Performance Timeline view.

That’s something I think would be very interesting, coming up very soon.

Benchmark: Very soon as in 2012?

Jatinder Mann: It’s hard to say. But I would say, yes, very soon. Chrome, and IE, and Mozilla are all active in this working group and I think we all appreciate that Navigation Timing is out there and it’d be nice to get the next set of things out there too.

Benchmark: So to close, what is the most important thing to communicate about the new Navigation Timing API and its use?

Jatinder Mann: The Navigation Timing API is a new way to get information about what’s happening in the browser that wasn’t available before or was difficult to get. This API makes it easier for developers to identify ways for improving Web performance.

Internet Explorer Performance Lab: reliably measuring browser performance

Matt, Jason and I wrote this article on the Building Windows 8 engineer blog on how the Internet Explorer team measures web browing performance. PC Magazine discusses the article as well. Enjoy!

A big part of this blog is going behind the scenes to show you all the work that goes into the engineering of Windows 8.  In this post we take a look at something we all care very deeply about–as engineers and as end-users–real world web performance. We do a huge amount of work to get beyond the basics of anecdotes and feel as we work to build high performance web browsing.  This post is authored by Matt Kotsenas, Jatinder Mann, and Jason Weber on the IE team, though performance is something that every single member of the team works on.
–Steven Sinofsky, President of Windows and Windows Live.

Web performance matters to everyone, and one engineering objective for Internet Explorer is to be the world’s fastest browser. To achieve this goal we need to reliably measure browser performance against the real world scenarios that matter to our customers.

Over the last five years we designed and built the Internet Explorer Performance Lab, one of the world’s most sophisticated web performance measurement systems. The IE Performance Lab collects reliable, accurate, and actionable data to inform decisions throughout the development cycle. We measure the performance of Internet Explorer 200 times daily, collecting over 5.7 million measurements and 480GB of runtime data each day. We understand the impact of every change to the product and ensure that Internet Explorer only gets faster. This blog post takes a deep look at how the IE Performance Lab is designed and how we use the lab to ensure we’re continually making the web faster.

In this post, we present:

  • Overview of the IE Performance Lab
  • Lab infrastructure
  • What (and how) we measure
  • Testing a scenario
  • Results investigation
  • Testing third-party software
  • Building a fast browser for users

Overview of the IE Performance Lab

In order to reliably measure web performance over time, a system needs to be able to reproducibly simulate real world user scenarios. In essence, our system needs to create a “mini version of the Internet.”

The IE Performance Lab is a private network completely sealed from both the public Internet and the Microsoft intranet network, and contains over 140 machines. The lab contains the key pieces of the real Internet, including web servers, DNS servers, routers, and network emulators, which simulate different customer connectivity scenarios.

Although this may appear complex at first glance, this approach allows all sources of variance to be removed. By controlling every aspect of the network, down to individual packet hops and latencies, our tests become deterministic and repeatable, which is critical to making the results actionable. In the IE Performance Lab, activity is measured with 100 nanosecond resolution.

Diagram shows content servers connected to Network emulators, connected to DNS servers, connected to Test clients, connected to Raw data storage, connected to Data analysis, connected to SQL database.

This type of network configuration also provides a great amount of flexibility. Because we’re simulating a real world setup, our lab can accommodate nearly any type of test machine or website content. The IE Performance Lab supports desktops, laptops, netbooks, and tablets with x86, x64, and ARM processors, all simultaneously.

Similarly, because the lab uses the Windows Performance Tools (WPT), we can run the same tests using different web browsers, toolbars, anti-virus products, or other third-party software and directly compare the results. WPT provides deep insight into the underlying hardware. Using WPT, we can capture everything from high-level CPU and GPU activity, to low-level information such as cache efficiency, networking statistics, memory usage patterns, and more. WPT allows us to measure and optimize performance across the stack to ensure that the hardware, device drivers, Windows operating system, and Internet Explorer are all efficiently optimized together.

A single test run takes 6 hours to complete and generates over 22GB of data during that time. This highly automated system is staffed by a small team that monitors operations, analyzes results, and develops new infrastructure features.

Lab infrastructure

The Performance Lab infrastructure can be broken into three main categories: Network and Server, Test Clients, and Analysis and Reporting. Each category is designed to minimize interaction across components, both to improve scalability of testing and to reduce the possibility of introducing noise into the lab environment.

A large room full of computers

Here’s a view of the IE Performance Lab, including a number of test and analysis machines on our private network.

Network and server infrastructure

Let’s start by discussing the DNS servers, network emulators, and content servers; all the components that together create the mini Internet. Over the next three sections we’ll work our way from right to left in the architectural diagram.

Content servers

Content servers are web servers that stand in for the millions of web hosts on the Internet. Each content server hosts real world web pages that have been captured locally. The captured pages go through a process we refer to as sanitization, where we tweak portions of the web content to ensure reproducible determinism. For example, JavaScript Date functions or Math.Random() calls will be replaced with a static value. Additionally, the dynamic URLs created by ad frameworks are locked to the URL that was first used by the framework.

After sanitization, content is served similarly to static content through an ISAPI filter that maps a hash of the URL to the content, allowing instantaneous lookup. Each web server is a 16-core machine with 16GB of RAM to minimize variability and ensure that content is in memory (no disk access required).

Content servers can also host dynamic web apps like Outlook Web Access or Office Web Apps. In these cases, the application server and any multi-tier dependencies are hosted on dedicated servers in the lab, just like real world environments.

Network emulators

Since many sources of variability have been removed, network speeds no longer reflect the experiences of many users with slower connections. To simulate real world customer environments, a test can take advantage of network emulation to understand the performance across the wide range of networks in use today. The lab supports emulating several DSL configurations, cable modems, 56k modems, as well as high-bandwidth, high-latency environments like WAN and 4G environments. As HTTP requests are passed to the emulator, it simulates network characteristics like packet delay and reordering, then forwards the request on to the web hosts. Upon receiving a response, emulation is again applied and then passed back to the test client.

Using dedicated hardware for network emulation provides the most realistic testing environment possible, and significantly reduces the observer effect. Although dedicated hardware adds cost and complexity compared to proxy or software-based solutions, it’s the only way to accurately measure performance. Browsers limit the number of simultaneous proxy connections to prevent proxy saturation, so using a proxy for network emulation has the unintended effect of sidestepping domain sharding and other optimizations made by the webpage. Additionally, local network emulation will compete with the browser for local machine resources, especially on low power machines.

DNS servers

Like real world DNS servers, the lab’s DNS servers link the content servers to the test clients. The lab also uses a different DNS server for each network emulator, meaning that changing from one network speed to another is as simple as changing the DNS server. In these cases, instead of resolving domain names to the web hosts, the DNS server resolves all domain names to the associated network emulator.

Test client configurations

We want to ensure that Internet Explorer consistently runs fast across all classes of computer hardware. The lab contains over 120 computers used to measure Internet Explorer performance. We refer to these as test clients; they range from high-end x64 desktops, to low-powered netbooks, to touch-first tablet devices, and everything in between. Because repeatability of measurements is paramount, all test clients are physical machines.

A long desk and two shelves, each containing 12 or more computers

Internet Explorer Performance Lab change comparison machine pool

Different machine classes contain both discrete and integrated graphics platforms to ensure that Internet Explorer continues to take full advantage of hardware acceleration across the ecosystem of devices. Above is our main machine pool. These PCs approximate the average consumer experience over the lifetime of a Windows 7 or Windows 8 PC.

Machines are ordered from the OEM to be identical; they all come from the same manufacturing lot and their performance characteristics are verified prior to use. Since the lab runs 24/7, hardware failures are inevitable. Replacing failed components with identical parts from a different manufacturing lot almost always results in the repaired computer running faster than the other machines in the pool. While this difference would be unnoticeable in the real world, when you’re measuring down to 100 nanoseconds, even a few cycles can impact the results! If after a repair a machine no longer runs identically to the rest of the pool, it is removed from the lab and the pool’s size permanently shrinks. In response, the lab’s purchases include extra “buffer” machines, so that when a failed machine is removed from the pool, the excess capacity provides a cushion, and the lab’s operations are not affected.

To add hardware breadth, we have additional machine pools that run the spectrum of consumer scenarios. Good performance on these machines ensures that IE uses the underlying hardware effectively across the PC ecosystem.


Assortment of laptop and desktop PCs on two shelves

Low-powered test machines. Each one is in a different state of testing.

If even more diversity is needed, the IE Performance Lab can also make use of the Windows Graphics Lab. The Windows Graphics Lab stocks nearly every graphics chipset manufactured. PCs can be configured into nearly any permutation imaginable and then used for performance testing. The Windows Graphics Lab is invaluable for diagnosing graphics problems across chipsets and driver revisions.

Analysis and reporting servers

Collection and analysis of test results are divided into two separate steps. By offloading analysis to dedicated machines, the test clients can begin another performance run earlier, and more powerful server class machines can be used to perform the analysis more rapidly. The sooner the analysis completes, the more efficiently we can identify performance changes.

For analysis, we use 11 server class machines, each of which has 16 cores and 16GB of RAM. During analysis, each trace file is inspected and thousands of metrics are extracted and inserted into a SQL server. Over the course of 24 hours these analysis machines will inspect over 15,000 traces that will be used for trend analysis.

Two server racks

Pictured are two of several server racks which contain file servers, a SQL server, and several analysis and content servers.

The SQL Server used to store the nearly 6 million measurements we collect each day is a 24 logical core machine with 64GB of RAM. Reports can be generated directly from SQL, or results can be inspected using either an HTML-based comparison application or WCF service that provides results in XML or JSON formats.

What (and how) we measure

With the infrastructure in place, let’s review the different types of scenarios measured in the Performance Lab, and the tools we use to gather metrics.

Scenarios measured daily

The Performance Lab focuses on real world scenarios that matter to users. As a result, we run over 20,000 different tests daily. These tests fall into four, sometimes overlapping, categories:

4 overlapping circles: Loading Content, Interactive Web Apps, IE "The Application", Synthetic Platform Benchmarks

Loading content – Navigating from one page to another is still the

most common activity inside a web browser. Loading web content is also the only

category that touches most of the browser’s eleven subsystems. Loading web content is a prerequisite

for the other categories of scenarios.

Interactive web apps – This category covers what is sometimes referred

to as content creation, AJAX applications, or Web 2.0 sites. It includes interacting

with popular news and social networking sites as well as interacting with mail and document applications like Outlook Web Access and Office Web Apps.

IE “the application” – Important but often forgotten are scenarios that interact with the browser itself. Common interactions include opening or closing the browser, switching tabs, using browser features like History and Favorites, and panning and zooming with both keyboard and mouse, and touch inputs.

Synthetic benchmarks – Rarely forgotten but often overstated are synthetic benchmarks like WebKit SunSpider. Benchmarks can be a useful engineering tool as they are designed to stress individual browser subsystems and accentuate differences between browsers. However, in order to maximize those differences, benchmarks often resort to atypical usage patterns or edge cases.

Real world patterns

When measuring performance it is important to ensure that the tests reflect real world usage patterns. Most Software Engineering textbooks refer to this process as workload

modeling, or application usage modeling. To ensure that the Performance Lab measures real world patterns, the Performance Lab uses real Web pages that represent real world patterns and exercise different browser subsystems.

In order to determine which sites to use for testing, we regularly crawl millions of sites and compile a list of site attributes and coding patterns. We use 68 different data points to determine commonalities across sites – things like the depth and width of the resulting DOM, CSS layout patterns, common frameworks used, international features, and more. From the results we chose sites that best represent the common patterns and diversity of the broader Web.

Engineering metrics

Performance is a multi-dimensional problem. The only way to get an accurate view of performance is to understand the scenario you’re testing, and how the hardware and OS interact with the browser. Here’s a closer look at five important performance metrics in the context of loading a major sports site for the first time.

Chart comparing Display time, elapsed time, CPU time, resource uitilization, and power consumption

Display Time – Display Time measures the time from when the user performs an action until the user sees the result of that action on the screen.

Elapsed Time – Most sites continue to perform background work after content has been displayed to the screen. Examples might include downloading the next email in a web mail application or sending analytics back to a provider. From the user’s perspective, the site might appear finished; however, significant work is often occurring which can impact overall responsiveness.

CPU Time – Modern web browsers are almost exclusively limited by the speed of the CPU. Offloading work to the GPU and making the CPU more efficient makes a large impact on performance.

Resource Utilization – Building a fast browser means ensuring resources across the entire PC work well together, including network utilization, memory usage patterns, GPU processing, graphics, memory, and hundreds of other dimensions. Since users run several applications at the same time on their PCs, it’s important for browsers to responsibly share these resources with other applications.

Power Consumption – Increasing power efficiency leads to longer the battery life in mobile scenarios, lower electricity costs for the device, and a smaller environmental impact.

Concentrating only on a single metric creates an overly simplistic view of performance. By focusing on a single metric, humans naturally tend to optimize for that metric, often at the expense of other equally important metrics. The only way to combat that tendency is to measure all aspects of performance, and then make the tradeoffs consciously, rather than implicitly.

In total, the Performance Lab measures over 850 different metrics. Each one provides part of the picture of browser performance. To give a feel for what we measure, here’s a (non-exhaustive) list of key metrics: private working set, total working set, HTTP request count, TCP bytes received, number of binaries loaded, number of context switches, DWM video memory usage, percent GPU utilization, number of paints, CPU time in JavaScript garbage collection, CPU time in JavaScript parsing, average DWM update interval, peak total working set, number of heap allocations, size of heap allocations, number of outstanding heap allocations, size of outstanding heap allocations, CPU time in layout subsystem, CPU time in formatting subsystem, CPU time in rendering subsystem, CPU time in HTML parser subsystem, idle CPU time, number of threads.

Windows event tracing infrastructure

Metrics are gathered using Windows Event Tracing Infrastructure (ETW) and VMMap. ETW is the Windows-wide event logging system that is used by many Windows components and third-party applications, including the Windows Event Log. ETW logging APIs are extremely low level and low overhead, which is critical for performance testing.

The view shows 6 graphs stacked vertically. Graphs are named CPU Usage by Process, Generic Events, WinINet End-to-End Downloads, IE CPU Breakdown, WinInet Transfer Setups, and IE Repaint.

The trace viewer included in WPT, xperfview.exe, is a powerful visualizer that allows correlation and overlaying kernel, CPU, GPU, I/O, networking, and other events. WPT also supports stack walking. Stack walking takes a snapshot of the program’s callstack at regular intervals and saves the stack as part of the trace. By correlating ETW events with stacks, WPT will display not only what work was being done, but the callstack associated with that work and the amount of time spent doing that work, with 10 microsecond resolution. Stack walking can be enabled on any process, even one that does not use ETW events. The drawback to stack walking is that it requires debugging symbols to decode the stacks, and is susceptible to aliasing.

Testing a scenario

The final piece of the puzzle is the actual test process. Testing can be broken into 3 phases: setup, testing, and errors and cleanup. Here’s a flowchart of the entire process to follow along.

A complex flow chart, starting with "User requests run" and ending with "Run is marked finished"

Setup

The process starts when a user requests a run through the lab website or automation framework. The run is placed into a priority queue with other pending runs. When a test client becomes available, it checks the queue and starts the highest priority job that it can. First, the test client installs the Test OS specified. The IE Performance Lab supports testing on Vista, Windows 7, and Windows 8. The test client installs a fresh copy of the Test OS for every run so the machine always starts in a known

good state.

Once the Test OS is installed, the client configures WPT, VMMap, and the test harness. The run also specifies a number of IE settings such as the homepage, use of Suggested Sites, InPrivate browsing, and others. Any third-party software is also installed and configured at this point.

The final step before testing is ensuring that the test client is idle to minimize test interference. Windows defines a concept of idle tasks. Idle tasks are a way for Windows and other developers to schedule non-critical work to happen at a later time when the user is not competing for resources. OS idle tasks include prefetching or SuperFetching, disk defragmentation, updating search indexes, and others, depending on OS version and configured services. To ensure that no idle work is done during the tests, the idle task queue is flushed.

Additionally, Windows Defender is paused and the log location for the test harness is marked as excluded from the Windows Indexing Service to prevent log and trace files from causing the indexer to start during a test run. Testing is done in multiple passes to minimize the number of providers needed, since additional providers increase the observer effect. The first pass is always a warm-up pass. Warm-up ensures that the browser binaries are “warm” and that the maximum amount of cachable page content is available in the WinINET cache. Subsequent passes each focus on a specific type of instrumentation, like stack walking, memory tracing, and I/O and registry tracing.

Errors and cleanup

If at any time during the test the browser crashes, the test pass is considered failed and the run moves on to the next test pass. If at any time during the tests Windows crashes, the computer reboots and the OS is reinstalled, since its state cannot be guaranteed. If the number of retries exceeds the threshold, the whole run is considered failed and the machine moves on to another run to prevent endlessly trying to test an unstable build.

When all the test cases are complete, the test client uploads the logs and traces for analysis. The test client then returns to an idle state and begins polling for a new run.

Results investigation

Each metric is tracked change-over-change. We run each test case a minimum of ten times, and duplicate runs on at least two different machines to create the sample population. Using statistical tools, uncharacteristic results can be automatically flagged for investigation. A variance change is also considered a regression. Users interact with IE under a wide range of circumstances and on a wide range of hardware, and one of our goals is to ensure a smooth and predictable experience every time.

In addition to automated analysis, a triage team investigates the daily results to watch for trends and other interesting behaviors. Manual investigation cannot be eliminated because many statistical tools assume both a normal distribution and that all samples are independent.

Neither assumption may be strictly true for our measurements. Some activities in IE are driven by a timer from the OS, meaning results are also dependent on when (along the timer’s cycle) the page load begins. A page load that starts right before or after a timer interrupt may do more or less work because IE must service the interrupt at different points in the page-load process. This interruption can have a rippling affect that leads to a bimodal distribution. Also, because we use repeated trials (and we don’t wipe the machine between iterations) the next trial is influenced by previous trials. Here’s a sample Elapsed Time graph for Bing Maps for change-over-change comparison.

A bar chart with a red line superimposed. A mouse pointer hovers over one point in the chart, and next to this is a tooltip listing max, median, min, and other info.

The red series shows the median value of each test run, and grey bars show the range. Hovering over a test run will show the iterations for the metric (in blue) as well as a tooltip that provides the exact values for minimum, median, max values, as well as the absolute and relative difference with the previous test run. The tooltip shown in this image also provides additional context like the build being tested, and a quick link to our source control system to view the changes in the build.

The combination of automated analysis and manual investigation provides the IE team with reliable and actionable data for performance tuning.

Testing third-party software

Many third-party applications depend on Trident, the network stack, and other IE components. Extensions like BHOs and toolbars load within the IE context. Other applications, like security software, can inject themselves between IE components. These applications become part of the IE stack, and can lead to poor performance. The Performance Lab is capable of measuring the impact of third-party software on browsing real world content in a controlled environment. These studies are important to IE and the ecosystem because users generally cannot quantify the impact of popular software on their browsing experience.

When testing third-party software impact, we compare a run with third-party software installed, with a clean run with only IE installed, to determine the impact of the software. In particular, we are interested in measuring two metrics: startup time and navigation time. Startup time measures the time it takes to launch the browser and navigate to an URL, whereas navigation time measures the time it takes to navigate to an URL when the browser has already been launched. Startup will also include the time that third-party applications take to load their IE extensions.

Using cached content allows repeatability in our measurements. Further, by measuring a cached site, we can definitively know that a performance regression is caused by the third-party software and not by differences in the site. Whenever measuring the impact of third-party software, we also validate our findings by testing startup and navigation on a direct connection to the Internet, to verify that the testing environment is not responsible for any deltas.

Many third-party applications offload work during a page navigation to cloud services. While parallelization of work and use of cloud services are excellent techniques to improve performance, some applications wait synchronously for the results from the network, blocking the navigation in the process. There are many real world scenarios, like strict firewalls, WAN connections, and offline scenarios, where such patterns can lead to poor performance for users. Third-party software should never process synchronously in response to an IE or user action, and should batch UI and DOM updates to minimize disruption.

Building a fast browser for users

Real world browser performance matters. Measuring performance at scale is a significant investment and a full-time job, but the results are well worth the effort. The data gathered by the Internet Explorer Performance Lab is instrumental in our understanding of browser performance and of the underlying PC hardware, and in developing a fast, fluid, and responsive web experience for users.

—Matt Kotsenas, Jatinder Mann, and Jason Weber for the Internet Explorer Performance Team

Nuclear Fission Powered Web Browsing

For the last 8 years, every Windows machine has shipped with a Graphics Proccessing Unit (GPU). A GPU is a specialized circuit designed to accelerate creating images to output to the computer display. Whether it’s a super powerful gaming rig, or a small compact netbook, there’s a GPU in there. And for the last decade, GPUs have gotten exponentially faster, at a rate much higher than CPUs. Today, much of that processing power is wasted because applications don’t use it.

When the Internet Explorer team was reading the HTML5 spec, we saw the potential of using the GPU to improve performance and we built our implementation of Internet Explorer 9 with the GPU in mind. That means that we offload work from the CPU to the GPU whenever we can, to make use of the graphics card.

To help demonstrate the power of hardware accelerated HTML5, I wrote the following Particle Acceleration demo on the IE Test Drive site. I like to think of it as nuclear fission powered web browsing. Check it out!

If you own a touch monitor, doing the swipe gesture will accelerate the molecules in that direction until inertia slows it back down. Pinching will let you expand the particles and double tapping will make the entire setup explode. With a mouse, clicking and dragging has the same effect as the swipe gesture and clicking in the center does the explode.

This entire demo was written with HTML5 Canvas and JavaScript – no WebGL was used. I have used the Rotation Matrix to multiple my 3D coordinate space to get the motion here. Karlito Bonnevie and I have documented the model on MSDN.

Particle Acceleration IE Test Drive Demo

New to HTML5 Canvas? Try out Canvas Pad

If you are new to HTML5 Canvas, I recommend you try out a tool I created on the IE Test Drive site called Canvas Pad.

If you’ve seen the Test Drive demos for hardware accelerated Canvas graphics, you are probably excited by the potential of this technology and want to learn more about it. With all major browsers supporting HTML5 Canvas, a scriptable 2D drawing context, Canvas is quickly becoming the natural choice for graphics on the web.

Even though the Canvas API, as defined in the HTML5 2D Context spec, has less than a hundred methods, attributes and interfaces, sometimes it’s easier to learn a technology by looking at sample code and simple demos.

Canvas Pad attempts to fulfill these needs. As the Internet Explorer Program Manager for Canvas, I find myself answering a lot of questions on how to do things in Canvas – I have seen great results by just pointing people to this tool.

As you can see from the image below, this site contains both a view of the actual Canvas and a script console containing the code generating the Canvas context. You can update the code and make real-time updates to the Canvas.

Canvas Pad

The Canvas Pad demo shows both the Canvas and sample code that you can manipulate.

Canvas Pad contains 22 samples on shapes, color/styles, line styles, shadows, text, images/videos, transformations, animations and mouse movement, which you can click through on the left hand pane. Further, below each sample, there is a reference to the API signature and the 2D Context spec definition of the API you are looking at.

I have found quite a few of my friends finding this tool useful to learn Canvas.  If I want to tinker with a new idea, sometimes I just open this up and drop some quick code. Try it out yourself!

Jatinder Mann

IEBlog: The Year in Review: W3C Web Performance Working Group

In this IEBlog article, I look back at a year in the W3C Web Performance Working Group:

17 Aug 2011 9:36 AM

Fast HTML5 Web applications benefit consumers who browse the Web and developers building innovative new experiences. Measuring performance characteristics of Web applications and writing efficient applications are two important aspects of making Web sites fast. Browser manufacturers can rapidly address developers’ needs through interoperable APIs when collaboratively partnering through the W3C.

One year ago today, the W3C announced the formation of a Web Performance Working Group chartered with two goals: making it easier to measure and understand the performance characteristics of Web applications and defining interoperable methods to write more CPU- and power-efficient applications.

Together with Google, Mozilla, Facebook, and other industry and community leaders who participate in the W3C Web Performance Working Group, we designed the Navigation Timing, Resource Timing, User Timing and Performance Timeline specifications to help developers accurately measure Web application performance. The first three specifications, Navigation Timing, Resource Timing, and User Timing, define interfaces for Web applications to access timing information related to the navigation of the document, resources on the page, and developer scripts, respectively. The Performance Timeline specification defines a unifying interface to retrieve this timing data.

Resource Timing, User Timing, and Performance Timeline specifications are all in the Last Call phase of specification. Last Call is a signal that the working group believes the spec is functionally complete and is ready for broad review from both other working groups and the public at large. This Last Call period extends until September 15, 2011. The Navigation Timing specification is already in the Candidate Recommendation phase and has two interoperable implementations, starting with Internet Explorer 9 and Chrome 6. Together these APIs help Web developers create faster and more efficient applications by providing insights into the performance characteristics of their applications that just weren’t possible before.

Over the last four months, the Web Performance Working Group defined interoperable methods to write more CPU- and power-efficient applications by producing the Page Visibility, Timing control for script-based animations, and Efficient Script Yielding specifications. The Page Visibility specification is in the Last Call phase until September 8th and has two implementations starting with the second IE10 Platform Preview and Chrome 13. The requestAnimationFrame API, from the Timing control for script-based animations specification, has three implementations starting with the second IE10 Platform Preview, Firefox 4 and Chrome 10. This specification is very close to entering Last Call. For more information on these two APIs, see the blog posts on using PC Hardware more efficiently with these APIs (link and link). IE10 is the first browser to implement the emerging setImmediate API from the Efficient Script Yielding specification.

It’s encouraging to see how much progress we’ve collectively made in just one year. These APIs are a great example of how quickly new ideas can become interoperable standards that developers can depend on in modern HTML5-enabled browsers. Thanks to everyone in the W3C Web Performance Working Group for helping design these APIs and to other browser vendors for starting to implement these APIs with an eye towards interoperability.

—Jatinder Mann, Program Manager, IE Performance

IEBlog: Using PC Hardware more efficiently in HTML5, Part 2

This IEBlog article is the second part of a two part series I wrote on using PC hardware more efficiently with new Web Performance APIs:

8 Jul 2011 12:56 PM

Web developers need API’s to efficiently take advantage of modern PC hardware through HTML5, improving the performance of Web applications, the power efficiency of the Web platform, and the resulting customer experience. The second IE10 Platform Preview supports emerging API’s from the W3C Web Performance Working Group which enable developers to make the most of the underlying hardware and use battery power more efficiently. This post details how to use Page Visibility, one of the emerging API’s, for better performance and power efficiency.

Page Visibility: adjust work depending on if the user is looking

Knowing whether a page is visible makes it possible for a developer to make better decisions about what the page does, especially around power usage and background tasks. Take a look at the Page Visibility Test Drive to see how a Web application can be aware of whether the page is visible or not to the user.

 
The Page Visibility API is now available through vendor prefixed implementations in IE10 and Chrome 13.

The developer can adjust or scale back what work the page does based on visibility. For example, if a Web based email client is visible, it may check the server for new mail every few seconds. When hidden it might scale checking email to every few minutes. Other examples include a puzzle application that can be paused when the user no longer has the game visible or showing ads only if the page is visible to the user.

The Page Visibility specification enables developers to determine the current visibility of a document and be notified of visibility changes. It consists of two properties and an event:

  • document.hidden: A boolean that describes whether the page is visible or not.
  • document.visibilityState: An attribute that returns the detailed page visibility state, e.g., PAGE_VISIBLE, PAGE_PREVIEW, etc.
  • visibilitychange: An event that gets fired any time the visibility state of the page changes.

IE10 has prefixed these attributes and event with the ‘ms’ vendor prefix.

With this interface, Web applications may choose to alter behavior based on whether they are visible to the user or not. For example, the following JavaScript shows a theoretical Web based email client checking for new emails every second without knowledge of the Page Visibility:


<!DOCTYPE html>
<html>
<head>
<title>Typical setInterval Pattern</title>
<script>
var timer = 0;
var PERIOD = 1000; // check for mail every second

function onLoad() 
{
   timer = setInterval(checkEmail, PERIOD);
}

function checkEmail() 
{
   debugMessage("Checking email at " + new Date().toTimeString());
}

function debugMessage(s) 
{
   var p = document.createElement("p");
   p.appendChild(document.createTextNode(s));
   document.body.appendChild(p);
}
</script>
</head>
<body onload="onLoad()">
</body>
</html>

Using Page Visibility, the same page can throttle back how often it checks email when the page is not visible:


<!DOCTYPE html>
<html>
<head>
<title>Visibility API Example</title>
<script>
var timer = 0;
var PERIOD_VISIBLE = 1000; // 1 second
var PERIOD_NOT_VISIBLE = 10000; // 10 seconds
var vendorHidden, vendorVisibilitychange;

function detectHiddenFeature() 
{
   // draft standard implementation
   if (typeof document.hidden != "undefined") 
   {
      vendorHidden = "hidden";
      vendorVisibilitychange = "visibilitychange";
      return true;
   }
   // IE10 prefixed implementation
   if (typeof document.msHidden != "undefined") 
   {
      vendorHidden = "msHidden";
      vendorVisibilitychange = "msvisibilitychange";
      return true;
   }
   // Chrome 13 prefixed implementation
   if (typeof document.webkitHidden != "undefined") 
   {
      vendorHidden = "webkitHidden";
      vendorVisibilitychange = "webkitvisibilitychange";
      return true;
   }
   // feature is not supported
   return false;
}

function onLoad() 
{
   // if the document.hidden feature is supported, vary interval based on visibility.
   // otherwise, just use setInterval with a fixed time.
   if (detectHiddenFeature()) 
   {
      timer = setInterval(checkEmail, document[vendorHidden] ? PERIOD_NOT_VISIBLE : PERIOD_VISIBLE);
      document.addEventListener(vendorVisibilitychange, visibilityChanged);
   }
   else 
   {
      timer = setInterval(checkEmail, PERIOD_VISIBLE);
   }
}

function checkEmail() 
{
   debugMessage("Checking email at " + new Date().toTimeString());
}

function visibilityChanged() 
{
   clearTimeout(timer);
   timer = setInterval(checkEmail, document[vendorHidden] ? PERIOD_NOT_VISIBLE : PERIOD_VISIBLE);
   debugMessage("Going " + (document[vendorHidden] ? "not " : "") + "visible at " + new Date().toTimeString());
}

function debugMessage(s) 
{
   var p = document.createElement("p");
   p.appendChild(document.createTextNode(s));
   document.body.appendChild(p);
}
</script>
</head>
<body onload="onLoad()">
</body>
</html>

With the Page Visibility API, Web developers can create more power conscious Web applications. To learn about other emerging API from the W3C Web Performance Working Group supported in the second IE10 Platform Preview, read my post on the requestAnimationFrame API (link).

—Jatinder Mann, Internet Explorer Program Manager

IEBlog: Using PC Hardware more efficiently in HTML5, Part 1

In this two part IEBlog article, I discuss how to use PC hardware more efficiently with the new Web Performance APIs.

5 Jul 2011 4:43 PM

Browsers need amazing performance to deliver on the promise of HTML5 applications. Web developers need API’s to efficiently take advantage of modern PC hardware through HTML5 and build high performance Web applications with efficient power use for great, all around customer experiences. The second IE10 Platform Preview supports three emerging API’s from the W3C Web Performance Working Group which enable developers to make the most of the underlying hardware and use battery power more efficiently: requestAnimationFrame, Page Visibility and setImmediate.

Together with Google, Mozilla, and others on the W3C Web Performance Working Group and the community, we designed these new API’s over the last three months. These three API’s are a great example of how quickly new ideas can become interoperable standards developers can depend on in modern HTML5 enabled browsers.

This post details how to use the requestAnimationFrame API for better performance and power efficiency.

requestAnimationFrame API: like setTimeout, but with less wasted effort

Web developers can now schedule animations to reduce power consumption and choppiness. For example, animations today generally occur even when a Web site is in a background tab, minimized, or otherwise not visible, wasting precious battery life. Animations today are not generally aligned with the display’s refresh rate, causing choppiness in the animation. Until the requestAnimationFrame API, the Web platform did not provide an efficient means for Web developers to schedule graphics timers for animations.

Let’s take a look at an example. Most animations use a JavaScript timer resolution of less than 16.7ms to draw animations, even though most monitors can only display at 16.7ms periods (at 60Hz frequency). The first graph below represents the 16.7ms display monitor frequency. The second graph represents a typical setTimout or setInterval of 10ms. In this case, the consumer will never see every third draw because another draw will occur before the display refreshes. This overdrawing results in choppy animations as every third frame is being lost. Reducing the timer resolution can also negatively impact battery life by up to 25%.

Figure: At top, each arrow represents a monitor refresh at 16.7ms periods. Below is a 10ms page redraw cycle. Every third redraw is wasted because the monitor will never show it. The result is choppy animations and wasted battery.

The requestAnimationFrame API is similar to the setTimeout and setInterval API’s developers use today. The key difference is that it notifies the application when the browser needs to update the screen, and only when the browser needs to update the screen. It keeps Web applications perfectly aligned with the browser’s painting, and uses only the necessary amount of resources.

If you take a look at this requestAnimationFrame example, you will notice that even though the animations look identical, the clock drawn with requestAnimationFrame is always more efficient in power consumption, background interference and CPU efficiency than the setTimeout clock.


The requestAnimationFrame animation (right) is more efficient in power consumption, background interference and CPU efficiency than the setTimeout animation (left).

It is a simple change to upgrade your current animations to use this new API. If you are using setTimeout(draw, PERIOD); in your code, you can replace it with requestAnimationFrame. The requestAnimationFrame API is the furthest along of these three API’s with vendor prefixed interoperable implementations available starting with Firefox 4, Chrome 13, and IE10.

Remember, requestAnimationFrame only schedules a single callback, like setTimout, and if subsequent animation frames are needed, then requestAnimationFrame will need to be called again from within the callback. In this Platform Preview, IE implements the API with a vendor prefix like other browsers. Here is an example of how to write cross-browser mark-up that is future proof:


// Future-proof: when feature is fully standardized
if (window.requestAnimationFrame) window.requestAnimationFrame(draw);
// IE implementation
else if (window.msRequestAnimationFrame) window.msRequestAnimationFrame(draw);
// Firefox implementation
else if (window.mozRequestAnimationFrame) window.mozRequestAnimationFrame(draw);
// Chrome implementation
else if (window.webkitRequestAnimationFrame) window.webkitRequestAnimationFrame(draw);
// Other browsers that do not yet support feature
else setTimeout(draw, PERIOD);

Thank you, to everyone in the W3C Web Performance Working Group for helping design these APIs, and thanks to other browsers, for starting to implement these APIs with an eye towards interoperability. With these APIs, Web developers can create a faster and more power conscious Web.

—Jatinder Mann, Internet Explorer Program Manager

IEBlog: Debugging Common Canvas Issues

I wrote this article on the IEBlog discussing common Canvas debugging issues:

8 Sep 2010 3:12 PM

As we’ve previously discussed, IE9 includes support for HTML5 canvas. You can test it out right now by downloading the latest platform preview. In our testing of sites that use the latest web standards, we are pleased to see that many canvas sites just work in IE9. For those of you using <canvas> on your site, we have two tips to make sure it works properly across browsers and in IE9: use feature detection instead of browser detection, and use <!DOCTYPE html>.

Be sure to use feature detection instead of browser detection

If you are using browser detection, such as automatically falling back to a non-canvas implementation if you detect that the user is using an IE User Agent string, you may be blocking HTML5 content from rendering in IE9. Instead of doing browser detection, you should
do feature detection
to check if the user’s browser has a certain capability. For instance, you can use this code to check if your user’s browser supports canvas:

 
var canvas = document.createElement("canvas"); 
if (canvas.getContext && canvas.getContext("2d")) 
{ 
   // Code requiring canvas support 
} 
else 
{ 
   // Canvas isn't available. Put non-canvas fallback here 
} 

This eliminates the need for you to make assumptions about current browser feature support and ensures your site will continue to work as browsers evolve. We explain more about feature detection in this post.

How to check if the user’s browser supports Canvas:

  • DO: Canvas feature detection code
  • DON’T: Browser detection using User Agent string
  • DON’T: Conditional comments

Make sure your site is in IE9 mode

By default, if your site is following web standards, such as using a standards DOCTYPE, IE9 will render it in standards mode. You can check if your site is in this mode by bringing up the Developer Tools (press F12) and checking to see if your site is in IE9 standards Document Mode.

Canvas is a new feature only supported in IE9 standards mode – a design decision we took to ensure that legacy document modes remain fully backward compatible. If you see a Document Mode for your site other than IE9 standards, HTML5 elements like canvas won’t be displayed. For example, if you don’t have a DOCTYPE in your page, IE9 will display the site in Quirks Mode. To ensure your page works as expected in IE9, we recommend that you add a strict DOCTYPE to your webpages. For example, you could add the W3C HTML5 DOCTYPE:

<!DOCTYPE html>

Or you can use a common strict DOCTYPE such as this:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">

You can read more about how IE determines the document mode here.

Interoperability and Canvas

Interoperability is a high priority for IE9, to the point where we recommend sending IE9 the same standards-based markup your site sends to other browsers. Most canvas sites should just work on IE9 if the site was originally developed for another browser. That being said, there are a few behavior differences between browsers. For instance, consider the shadow demo from the Canvas Pad test drive site.

This is one example of a canvas feature that is rendered a little differently in each browser. We are making IE9 interoperable whenever possible, but for some canvas features, other browsers do not have a complete or correct implementation. In these cases, we follow the W3C spec. We submit test cases to the W3C as a way to help ensure everyone agrees on how the spec should be interpreted and implemented. To learn more about our shadow implementation, check out our canvas tests from the IE Test Center.

The purpose of the W3C spec is to define a standard that all browsers should follow. If we find examples where browsers uniformly behave differently from the spec, we feel that spec should be updated to reflect the interoperable behavior, if it makes sense for web developers. For instance, HTMLFrameElement did not contain the contentWindow attribute in the W3C spec; however IE8, Firefox, and Chrome all support this attribute. We filed a bug with a proposed change, and the HTML5 editor updated the latest revision of the spec.

If something looks unexpected in IE9 and you believe it is an interoperability issue or an area where we deviate from the spec, please let us know by filing a bug with Microsoft Connect. One of our goals around the platform previews and the beta of IE9 is to give our users a chance to give us as much feedback as possible, so don’t hesitate to let us know if you think you see a bug!

Thanks,

Elizabeth Ford and Jatinder Mann

Program Managers, Internet Explorer

IEBlog: IE9 Includes Hardware Accelerated Canvas

In this IEBlog post, Paul and I announce Internet Explorer 9 support for HTML5 Canvas:

1 Jul 2010 6:24 PM

With the recent release of the latest IE9 platform preview, we talked about how we’re rebuilding the browser to use the power of your whole PC to browse the web, and to unlock a new class of HTML5 applications. One area that developers are especially excited about is the potential of HTML5 canvas. Like all of the graphics in IE9, canvas is hardware accelerated through Windows and the GPU. In this blog post we discuss some of the details behind canvas and the kinds of things developers can build.

Canvas enables everything from basic shapes to fully interactive graphics

Canvas is a way to program graphics on the web. The <canvas> tag is an immediate mode 2d drawing surface that web developers can use to deliver things like real time graphs, animations or interactive games without requiring any extra downloads.

At the most basic level, canvas enables you to draw primitives like lines, rectangles, arcs, Bezier curves, quadratic curves, images and video like the following:

This image is a simulation of what you’d see in a canvas enabled browser.

Please use the IE9 preview to see these examples running in canvas.

The Canvas Pad demo on the IE test drive site goes into detail on the canvas syntax and enables you to easily experiment with a wide range of examples. Feel free to make changes to any of the samples that are there to see how it works — for example, try changing colors or sizes of things.

Taking things a step further, you can use JavaScript to animate canvas drawings or make interactive experiences. The next example draws lines as you move your mouse (or as you move your finger on touch enabled devices) over the black box. You could also choose to have your canvas experience react to keyboard input, mouse clicks or any browser event.

This image is a simulation of what you’d see in a canvas enabled browser.

With canvas support in IE9, you can move your mouse over the black box and draw lines.

By utilizing the full power of the PC with hardware acceleration for graphics and fast JavaScript for animation, web developers can use IE9 to build deep, graphically rich experiences. Since canvas is an element like other elements in HTML, it participates in the page layout and its API is exposed to JavaScript so it can be fully incorporated into a web page’s design. This makes it possible for sites to include things like live data visualizations, games, splash pages and ads without the need for any extra downloads or plugins.

The IE testdrive site includes several examples that demonstrate the kinds of things that sites are now able to do in an interoperable way.

Shopping

The Amazon Shelf shows what shopping for books could look like when the web site designer is able to use the kind of graphics, animations and smooth transitions that canvas enables.

Immersive game experiences:

The following demos showcase some gaming concepts like physics, character animation, collision detection and mouse interaction coupled with hardware accelerated graphics. In these demos, you’ll notice that not all browsers can update the screen with the same frequency (FPS or frames per second). IE is able to maintain a high FPS by using Windows technologies to make use of your GPU – your computer’s hardware that’s optimized for rendering graphics.

FishIE Tank

This demo makes use of sprites to animate the fish and basic collision logic to redirect the fish when they hit the edges of the tank. It’s also good for measuring graphics performance because you can change the number of fish to increase or decrease the graphics load.

Asteroid Belt

The asteroid in the demo follows your mouse, scales and rotates. It’s an example of direct interactivity that you might find in a game.

Mr. Potato Gun

A physics engine in this demo defines how the different parts of Mr. Potato head are launched from the gun and then how they react when they bounce off the ground. Many games use some form of physics engine like this to manage particle movement and their response.

Canvas Zoom

This demo enables you to start with a very wide angle on an image like this mountain range and then zoom in very close image like people at a picnic. For games, it’s an interesting example of scaling and smooth transitions.

Demos from around the web:

There are some pretty amazing demos floating around the web and I’d like to share a couple of our favorites — there are many more. An important part of implementing canvas is that we do it in an interoperable way so that developers can use the same markup. To help achieve this goal, we’re always looking for examples that work and those that don’t. A future canvas blog post will go into detail about how we work to be interoperable and what we do when there’s an issue reported.

I hope you enjoy some of these canvas examples from people around the web.

Cloth Simulation

This demo is interactive and the cloth is responsive to movement and gravity.

Zwibbler

The shapes in this drawing app are preserved so you can select and then move, resize, or change their styling.

Liquid Particles

The particles in this demo are drawn to or repelled from the mouse.

Kaleidoscope

This one does a nice job of drawing you in – it’s engaging and interesting to watch the patterns as they evolve.

Nebula Visualization

The alpha blending used by this demo are really well done. The result is a cloudy atmospheric look. It’s graphics intensive and it’s still very fast and smooth in IE9.

Animated Reflection

The author of this demo says, “The script is currently using 80% of my cpu so it’s not really practical. Hopefully we will be getting JIT’d javascript sometime soon.” Well, now JavaScript is compiled in IE9. It generally uses about 1% of my CPU.

Asteroids in Canvas

This is a full game with nice graphics, collision detection, keyboard interactivity, score keeping and… green lasers.

Particle Animation

See your name in lights. This is another demo that includes a particle system. You can run this with 300 or 1500 sprites. Go ahead and bump it up to 1500.

We’re looking forward to seeing the kinds visual experiences web developers will be able to build with a fully hardware accelerated browser.

Give it a try yourself. Watch the videos, get the latest platfrom preview, try out the canvas demos and build some examples of your own. If you find a bug in how canvas works or where the same markup behaves differently, please report bugs on Connect.

– Thanks, Paul Cutsinger and Jatinder Mann

Deep Dive into HTML5 Canvas

My MIX11 talk, Deep Dive into HTML5 <canvas>, is now available online for viewing on the Channel 9 MSDN site.

The topics covered in this talk are:

  • What is Canvas and when to use it?
  • Drawing Model
  • Security Model
  • Accessibility features
  • Interoperability differences between browsers
  • Best practices for coding and performance

MIX11: Deep Dive into HTML5 <canvas> session

MIX 2011

I will be giving a talk about HTML5 Canvas at this year’s MIX11 conference – tune into the webcast. I will share details after the talk.

Deep Dive Into HTML5 <canvas>

Breakers H
Mandalay Bay, Las Vegas
Wed, Apr 13 11:30 AM12:30 PM

If you’ve seen the demos for Internet Explorer 9’s hardware accelerated graphics, you are probably excited to learn the details of HTML5 Canvas. With all major browsers supporting HTML5 Canvas, a scriptable 2D drawing context, Canvas is quickly becoming the natural choice for graphics on the web.  In this session, you will learn advanced Canvas concepts (including the origin-clean security and the Canvas Drawing Model), understand when to use Canvas versus SVG and get a deeper look at how the Internet Explorer team solved interoperability issues as we implemented the specification. You will learn to build HTML5 Canvas websites through best practices and lots of code samples.

Welcome

Welcome to my webpage. I am hoping to use this site to both convey technological ideas and share my experiences. That is all for my first post. Hope you have a Merry Christmas and Happy New Year!

ps. Considering it’s raining in Vancouver, I thought I would share a view of the spring time Pacific North West. I took this picture on Burnaby Mountain.