Canvas

Nuclear Fission Powered Web Browsing

For the last 8 years, every Windows machine has shipped with a Graphics Proccessing Unit (GPU). A GPU is a specialized circuit designed to accelerate creating images to output to the computer display. Whether it’s a super powerful gaming rig, or a small compact netbook, there’s a GPU in there. And for the last decade, GPUs have gotten exponentially faster, at a rate much higher than CPUs. Today, much of that processing power is wasted because applications don’t use it.

When the Internet Explorer team was reading the HTML5 spec, we saw the potential of using the GPU to improve performance and we built our implementation of Internet Explorer 9 with the GPU in mind. That means that we offload work from the CPU to the GPU whenever we can, to make use of the graphics card.

To help demonstrate the power of hardware accelerated HTML5, I wrote the following Particle Acceleration demo on the IE Test Drive site. I like to think of it as nuclear fission powered web browsing. Check it out!

If you own a touch monitor, doing the swipe gesture will accelerate the molecules in that direction until inertia slows it back down. Pinching will let you expand the particles and double tapping will make the entire setup explode. With a mouse, clicking and dragging has the same effect as the swipe gesture and clicking in the center does the explode.

This entire demo was written with HTML5 Canvas and JavaScript – no WebGL was used. I have used the Rotation Matrix to multiple my 3D coordinate space to get the motion here. Karlito Bonnevie and I have documented the model on MSDN.

Particle Acceleration IE Test Drive Demo

New to HTML5 Canvas? Try out Canvas Pad

If you are new to HTML5 Canvas, I recommend you try out a tool I created on the IE Test Drive site called Canvas Pad.

If you’ve seen the Test Drive demos for hardware accelerated Canvas graphics, you are probably excited by the potential of this technology and want to learn more about it. With all major browsers supporting HTML5 Canvas, a scriptable 2D drawing context, Canvas is quickly becoming the natural choice for graphics on the web.

Even though the Canvas API, as defined in the HTML5 2D Context spec, has less than a hundred methods, attributes and interfaces, sometimes it’s easier to learn a technology by looking at sample code and simple demos.

Canvas Pad attempts to fulfill these needs. As the Internet Explorer Program Manager for Canvas, I find myself answering a lot of questions on how to do things in Canvas – I have seen great results by just pointing people to this tool.

As you can see from the image below, this site contains both a view of the actual Canvas and a script console containing the code generating the Canvas context. You can update the code and make real-time updates to the Canvas.

Canvas Pad

The Canvas Pad demo shows both the Canvas and sample code that you can manipulate.

Canvas Pad contains 22 samples on shapes, color/styles, line styles, shadows, text, images/videos, transformations, animations and mouse movement, which you can click through on the left hand pane. Further, below each sample, there is a reference to the API signature and the 2D Context spec definition of the API you are looking at.

I have found quite a few of my friends finding this tool useful to learn Canvas.  If I want to tinker with a new idea, sometimes I just open this up and drop some quick code. Try it out yourself!

Jatinder Mann

IEBlog: Debugging Common Canvas Issues

I wrote this article on the IEBlog discussing common Canvas debugging issues:

8 Sep 2010 3:12 PM

As we’ve previously discussed, IE9 includes support for HTML5 canvas. You can test it out right now by downloading the latest platform preview. In our testing of sites that use the latest web standards, we are pleased to see that many canvas sites just work in IE9. For those of you using <canvas> on your site, we have two tips to make sure it works properly across browsers and in IE9: use feature detection instead of browser detection, and use <!DOCTYPE html>.

Be sure to use feature detection instead of browser detection

If you are using browser detection, such as automatically falling back to a non-canvas implementation if you detect that the user is using an IE User Agent string, you may be blocking HTML5 content from rendering in IE9. Instead of doing browser detection, you should
do feature detection to check if the user’s browser has a certain capability. For instance, you can use this code to check if your user’s browser supports canvas:

 
var canvas = document.createElement("canvas"); 
if (canvas.getContext && canvas.getContext("2d")) 
{ 
   // Code requiring canvas support 
} 
else 
{ 
   // Canvas isn't available. Put non-canvas fallback here 
}

This eliminates the need for you to make assumptions about current browser feature support and ensures your site will continue to work as browsers evolve. We explain more about feature detection in this post.

How to check if the user’s browser supports Canvas:

  • DO: Canvas feature detection code
  • DON’T: Browser detection using User Agent string
  • DON’T: Conditional comments

Make sure your site is in IE9 mode

By default, if your site is following web standards, such as using a standards DOCTYPE, IE9 will render it in standards mode. You can check if your site is in this mode by bringing up the Developer Tools (press F12) and checking to see if your site is in IE9 standards Document Mode.

Canvas is a new feature only supported in IE9 standards mode – a design decision we took to ensure that legacy document modes remain fully backward compatible. If you see a Document Mode for your site other than IE9 standards, HTML5 elements like canvas won’t be displayed. For example, if you don’t have a DOCTYPE in your page, IE9 will display the site in Quirks Mode. To ensure your page works as expected in IE9, we recommend that you add a strict DOCTYPE to your webpages. For example, you could add the W3C HTML5 DOCTYPE:

<!DOCTYPE html>

Or you can use a common strict DOCTYPE such as this:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">

You can read more about how IE determines the document mode here.

Interoperability and Canvas

Interoperability is a high priority for IE9, to the point where we recommend sending IE9 the same standards-based markup your site sends to other browsers. Most canvas sites should just work on IE9 if the site was originally developed for another browser. That being said, there are a few behavior differences between browsers. For instance, consider the shadow demo from the Canvas Pad test drive site.

This is one example of a canvas feature that is rendered a little differently in each browser. We are making IE9 interoperable whenever possible, but for some canvas features, other browsers do not have a complete or correct implementation. In these cases, we follow the W3C spec. We submit test cases to the W3C as a way to help ensure everyone agrees on how the spec should be interpreted and implemented. To learn more about our shadow implementation, check out our canvas tests from the IE Test Center.

The purpose of the W3C spec is to define a standard that all browsers should follow. If we find examples where browsers uniformly behave differently from the spec, we feel that spec should be updated to reflect the interoperable behavior, if it makes sense for web developers. For instance, HTMLFrameElement did not contain the contentWindow attribute in the W3C spec; however IE8, Firefox, and Chrome all support this attribute. We filed a bug with a proposed change, and the HTML5 editor updated the latest revision of the spec.

If something looks unexpected in IE9 and you believe it is an interoperability issue or an area where we deviate from the spec, please let us know by filing a bug with Microsoft Connect. One of our goals around the platform previews and the beta of IE9 is to give our users a chance to give us as much feedback as possible, so don’t hesitate to let us know if you think you see a bug!

Thanks,

Elizabeth Ford and Jatinder Mann

Program Managers, Internet Explorer

IEBlog: IE9 Includes Hardware Accelerated Canvas

In this IEBlog post, Paul and I announce Internet Explorer 9 support for HTML5 Canvas:

1 Jul 2010 6:24 PM

With the recent release of the latest IE9 platform preview, we talked about how we’re rebuilding the browser to use the power of your whole PC to browse the web, and to unlock a new class of HTML5 applications. One area that developers are especially excited about is the potential of HTML5 canvas. Like all of the graphics in IE9, canvas is hardware accelerated through Windows and the GPU. In this blog post we discuss some of the details behind canvas and the kinds of things developers can build.

Canvas enables everything from basic shapes to fully interactive graphics

Canvas is a way to program graphics on the web. The <canvas> tag is an immediate mode 2d drawing surface that web developers can use to deliver things like real time graphs, animations or interactive games without requiring any extra downloads.

At the most basic level, canvas enables you to draw primitives like lines, rectangles, arcs, Bezier curves, quadratic curves, images and video like the following:

This image is a simulation of what you’d see in a canvas enabled browser.

Please use the IE9 preview to see these examples running in canvas.

The Canvas Pad demo on the IE test drive site goes into detail on the canvas syntax and enables you to easily experiment with a wide range of examples. Feel free to make changes to any of the samples that are there to see how it works — for example, try changing colors or sizes of things.

Taking things a step further, you can use JavaScript to animate canvas drawings or make interactive experiences. The next example draws lines as you move your mouse (or as you move your finger on touch enabled devices) over the black box. You could also choose to have your canvas experience react to keyboard input, mouse clicks or any browser event.

This image is a simulation of what you’d see in a canvas enabled browser.

With canvas support in IE9, you can move your mouse over the black box and draw lines.

By utilizing the full power of the PC with hardware acceleration for graphics and fast JavaScript for animation, web developers can use IE9 to build deep, graphically rich experiences. Since canvas is an element like other elements in HTML, it participates in the page layout and its API is exposed to JavaScript so it can be fully incorporated into a web page’s design. This makes it possible for sites to include things like live data visualizations, games, splash pages and ads without the need for any extra downloads or plugins.

The IE testdrive site includes several examples that demonstrate the kinds of things that sites are now able to do in an interoperable way.

Shopping

The Amazon Shelf shows what shopping for books could look like when the web site designer is able to use the kind of graphics, animations and smooth transitions that canvas enables.

Immersive game experiences:

The following demos showcase some gaming concepts like physics, character animation, collision detection and mouse interaction coupled with hardware accelerated graphics. In these demos, you’ll notice that not all browsers can update the screen with the same frequency (FPS or frames per second). IE is able to maintain a high FPS by using Windows technologies to make use of your GPU – your computer’s hardware that’s optimized for rendering graphics.

FishIE Tank

This demo makes use of sprites to animate the fish and basic collision logic to redirect the fish when they hit the edges of the tank. It’s also good for measuring graphics performance because you can change the number of fish to increase or decrease the graphics load.

Asteroid Belt

The asteroid in the demo follows your mouse, scales and rotates. It’s an example of direct interactivity that you might find in a game.

Mr. Potato Gun

A physics engine in this demo defines how the different parts of Mr. Potato head are launched from the gun and then how they react when they bounce off the ground. Many games use some form of physics engine like this to manage particle movement and their response.

Canvas Zoom

This demo enables you to start with a very wide angle on an image like this mountain range and then zoom in very close image like people at a picnic. For games, it’s an interesting example of scaling and smooth transitions.

Demos from around the web:

There are some pretty amazing demos floating around the web and I’d like to share a couple of our favorites — there are many more. An important part of implementing canvas is that we do it in an interoperable way so that developers can use the same markup. To help achieve this goal, we’re always looking for examples that work and those that don’t. A future canvas blog post will go into detail about how we work to be interoperable and what we do when there’s an issue reported.

I hope you enjoy some of these canvas examples from people around the web.

Cloth Simulation

This demo is interactive and the cloth is responsive to movement and gravity.

Zwibbler

The shapes in this drawing app are preserved so you can select and then move, resize, or change their styling.

Liquid Particles

The particles in this demo are drawn to or repelled from the mouse.

Kaleidoscope

This one does a nice job of drawing you in – it’s engaging and interesting to watch the patterns as they evolve.

Nebula Visualization

The alpha blending used by this demo are really well done. The result is a cloudy atmospheric look. It’s graphics intensive and it’s still very fast and smooth in IE9.

Animated Reflection

The author of this demo says, “The script is currently using 80% of my cpu so it’s not really practical. Hopefully we will be getting JIT’d javascript sometime soon.” Well, now JavaScript is compiled in IE9. It generally uses about 1% of my CPU.

Asteroids in Canvas

This is a full game with nice graphics, collision detection, keyboard interactivity, score keeping and… green lasers.

Particle Animation

See your name in lights. This is another demo that includes a particle system. You can run this with 300 or 1500 sprites. Go ahead and bump it up to 1500.

We’re looking forward to seeing the kinds visual experiences web developers will be able to build with a fully hardware accelerated browser.

Give it a try yourself. Watch the videos, get the latest platfrom preview, try out the canvas demos and build some examples of your own. If you find a bug in how canvas works or where the same markup behaves differently, please report bugs on Connect.

– Thanks, Paul Cutsinger and Jatinder Mann

Deep Dive into HTML5 Canvas

My MIX11 talk, Deep Dive into HTML5 <canvas>, is now available online for viewing on the Channel 9 MSDN site.

The topics covered in this talk are:

  • What is Canvas and when to use it?
  • Drawing Model
  • Security Model
  • Accessibility features
  • Interoperability differences between browsers
  • Best practices for coding and performance

MIX11: Deep Dive into HTML5 <canvas> session

MIX 2011

I will be giving a talk about HTML5 Canvas at this year’s MIX11 conference – tune into the webcast. I will share details after the talk.

Deep Dive Into HTML5 <canvas>

Breakers H
Mandalay Bay, Las Vegas
Wed, Apr 13 11:30 AM – 12:30 PM

If you’ve seen the demos for Internet Explorer 9’s hardware accelerated graphics, you are probably excited to learn the details of HTML5 Canvas. With all major browsers supporting HTML5 Canvas, a scriptable 2D drawing context, Canvas is quickly becoming the natural choice for graphics on the web.  In this session, you will learn advanced Canvas concepts (including the origin-clean security and the Canvas Drawing Model), understand when to use Canvas versus SVG and get a deeper look at how the Internet Explorer team solved interoperability issues as we implemented the specification. You will learn to build HTML5 Canvas websites through best practices and lots of code samples.