State charts

Thanks to Bill Budge for pointing me to State Charts.

I've been carefully sussing out the logic of Chrome's Fullscreen controller into a finite state machine, mainly so that we can get a group to agree on exactly what we think it should be doing. Chrome can go fullscreen in numerous ways: users entering and exiting via a menu or button, web pages making the transition (e.g. when you make a video fullscreen by clicking on it in a page), extensions, a special mode on Mac, and Windows 8 Metro Snap. Most of these include asynchronous transitions, and all API initiated transitions must be serviced (even if we're 'in the middle' of another transition).

Initially the diagram resembled a flying spaghetti monster, I've been winnowing it down discarding states we can design out of the naive total possibility space. But, it's also much simpler to notate using clustering and history offered in state charts.

BTW, state charts are also included in UML state machines.

Pointer Lock (Mouse Lock) shipped in Chrome

Pointer lock has shipped with Chrome 22, it's out of Beta and releasing on time! Fancy demo first, check out Mozilla's First Person Shooter Demo.

Pointer lock offers the ability to hide the mouse cursor and use movement for controlling the camera without bumping into the edge of the screen or moving off the browser window. I've been working on it for some time, including writing the Pointer Lock specification.

It's been a long path. Things started with early prototypes and discussions of mouse lock in standards lists (and more, as it was renamed to pointer lock). Not everyone in the browser community was ready to just jump on board. It took some explaining, and still there were the those who think "Browsers are for looking at static documents, not web applications! This API is too powerful for the web." Also, people who wanted more, to the point of impracticality.

One challenge of specification was that I limited the resolution to be the same as mouse movement events without pointer lock. A direct connection to a high resolution mouse offers higher precision unaltered by an operating systems acceleration, or 'ballistic' algorithms. Sounds great, but how can this be specified, calibrated, and offered across browser vendors and operating systems? I stuck with keeping it simple and producing the exact same mouse movements received today, but without the limits of screen boarders or moving out of an application's focus area. More details in the spec FAQ.

I was pleased when David Humphrey jumped onto the Mozilla Issue. He is a professor at Seneca College and implemented Pointer Lock in Mozilla as a class project. They did great work, and eventually shipped to Firefox 14 a bit before Chrome.

One of the differences is that Firefox only allows pointer lock when pages have made an element full screen. This provides good security shelter, as Fullscreen had already been rigorously designed, implemented, and tested due to a real security threat (tricking a user to enter data into a website pretending to be another application). It was important to me to ship Chrome with non-fullscreen support on day 1, as I believe it to be a real need and wanted to discover security issues up front. The security policy in Chrome is split between the Chromium and WebKit open source projects, adding to that complexity.

On the topic of Fullscreen, there was a significant Pointer Lock API re-write in order to be as consistent as possible with the Fullscreen API. They do have many similarities, and are likely to be used at the same time by developers. However, I do lament using the Fullscreen style pointerlockchange and pointerlockerror events to notify a developer of success or failure. In my original specification I used callbacks. The advantage of callbacks is a guaranteed 1 to 1 correspondence of request and response. The events allow for cross talk between different bits code using the API, and are more fragile. On every event, code must validate if the event was due to a related request it had made (which it must keep track of).

With the help of Yuzhu Shen and others, we shipped an early version of Mouse Lock to Pepper plugins much earlier, including Native Client games. There security concerns were somewhat more relaxed as content using the API had to be published in the Chrome Web Store, as opposed to the open web. Via the store, we could have more control to remove content if it were misbehaving.

I'm looking forward to seeing new classes of applications implemented on the web using this API. ;)

What!? I had kids?

I started a new blog, What!? I had kids?, because the world needed another blog. And though I love my job I've had less ideas and/or motivation to write them up on Beautiful Pixels recently. I've been spending more time planning on how to have fun with the kids.

The current article, Strollers on Steps? is proper geekery of a topic at the levels you've come to expect from Beautiful Pixels. If you're more into color, try Colored Rice, or Food Colors in the Bathtub.

Perhaps some day there will be a magical post about both children and pixels.

Game Developers Conference 2012 presentation: The Bleeding Edge of Open Web Tech

I presented at GDC last week, and it went well. Here is a pre-recorded version of the talk. You can also check out the notes for the presentation.

JavaScript Pointer Lock (Mouse Lock) in Chrome Developer Preview

Mouse Lock is the canonical term for when a game (or other application) removes the mouse cursor from view and interprets mouse motion for something else, e.g. looking around in a 3D world. To date this has not been possible for web applications without a plugin, which makes many game genres and other applications just terrible to use. (e.g. forcing users to drag the mouse button to pan the view, while a native application would use the mouse button for something else, e.g. interacting with the world.).

Good news: I've been working on a w3c specification to address this. The feature is available for developers to experiment with now in Chrome (details below). And, a FireFox implementation is on it's way too, thanks primarily to David Humphrey using it as a class project.

Today the first Chrome Canary build is available to try the feature out:

• Get a build.
• Chrome Canary builds auto-update daily and install without interfering with your normal installation of Chrome, they're the way to go since they're so easy. Unless your on Linux, where you need to just get a recent Linux Build.
• Enable pointer lock.
• Navigate to about:flags, find Enable Pointer Lock, and restart (there's a button below).
• Try a demo.
• http://media.tojicode.com/q3bsp/ is a nice one from Brandon Jones and what you see in the screenshot above. It's a quake 3 BSP viewer.
• Others are sure to show up soon, minutes after landing the keystone WebKit patch I received a few IMs and emails from excited developers. 
• Go Fullscreen. (more on that later)
• In Brandon's demo find the fullscreen button in the bottom right corner. (It has to be JavaScript initiated fullscreen, not chrome's "presentation mode".)
• Allow the site to disable your mouse cursor.
• This setting is remembered for each site, and can be forgotten in Chrome's preferences / content settings.
• Read some doc https://developer.mozilla.org/en/API/Mouse_Lock_API.

There's a lot that will be changing with this. First, I have several issues still to deal with in the WebKit implementation, a security review will certainly find more, and oh yeah we're going to overhaul the spec to be as identical as possible to the Fullscreen spec.

Some have asked if pointer lock will be forever restricted to fullscreen mode. No, in fact by the time the JavaScript bindings ship without developer flags I'm hoping to have removed the restriction, and if not then soon after. The only real requirement is that the user not suffer from a poorly behaving website. At the moment that means if a site spams lock requests that we require a user-gesture (e.g. clicking on the web page content, or pressing a key). Fullscreen currently requires the same, so we hide in it's shadow, but I'll implement the check even in windowed mode.

Chrome 19 is the earliest JavaScript pointer lock could ship without a developer flag, but it's predicated on many things coming into place. However, Native Client apps have been able to use it since Chrome 16.

Developers interested in the nitty gritty implementation details can see the chromium bug http://code.google.com/p/chromium/issues/detail?id=72754. To follow along, please only star the issue (top left). If  you've made awesome demos and add this feature, drop a comment here. ;)

Happy locking.

Automatic Screen Window Titles in Bash and Vim

I finally configured screen to display the list of windows open with useful titles. Here are the few bits I needed

First, if you don't know what I'm talking about:
I occasionally SSH into work, and when I do I use a program called screen that can host multiple terminal shells. It can persist those shells and allow me to reconnect if I get disconnected or want to change computers I'm working on.

BTW, I always start screen with screen -RaAd -S x

• -a   include all capabilities ...
• -A   Adapt  the sizes of all windows ...
• -d -R   Reattach a session and if necessary detach or even create it first.
• -S Name of the session

The default configuration makes it hard to keep track of how many terminals you have running, and what they are doing. I made three changes that help my personal workflow:

Add a persistent display at the bottom of screen
In .screenrc I appended this line:
hardstatus alwayslastline "%{=b}%{G} Screen(s): %{b}%w %=%{kG}%C%A  %D, %M/%d/%Y "


Source: Julien Chaffraix, a coworker.

Set the current directory name as the window title from bash
In .bashrc I appended these lines:

if [ "$TERM" = "screen" ]; then
  screen_set_window_title () {
    local HPWD="$PWD"
    case $HPWD in
      $HOME) HPWD="~";;


      ## long name option:
      # $HOME/*) HPWD="~${HPWD#$HOME}";;


      ## short name option:
      *) HPWD=`basename "$HPWD"`;;


    esac
    printf '\ek%s\e\\' "$HPWD"
  }
  PROMPT_COMMAND="screen_set_window_title; $PROMPT_COMMAND"
fi


You can see that I'm using a short name for each directory, e.g. "chromium" instead of the full path or suffix path after my home directory, e.g. "~/projects/chromium". You can toggle the commented lines to try alternates.

Source: http://unix.stackexchange.com/questions/6065/gnu-screen-new-window-name-change.

Set the name of the buffer I'm editing in vim
In .vimrc I appended these lines ("To create ^[, which is escape, you need to enter CTRL+V < Esc"):

if &term == "screen"
  let &titlestring=expand("%:t")
  set t_ts=^[k
  set t_fs=^[\
  set title
endif


Source: http://vim.wikia.com/wiki/Automatically_set_screen_title.