Endquote is Josh Santangelo, an interface developer and former man-about-town in Seattle. Lately, he talks a lot about Silverlight, Surface, and Stimulant.
A couple of months ago we finished the Microsoft Local Impact Map, a Silverlight app highlighting all of the progressive and wonderful things Microsoft does around the world. Today we launched a new look at the same data, presented on Microsoft Surface.
Aside from being a cool multi-touch, multi-user, 360-degree application, this one is technically interesting because it shares so much with the Silverlight app. It uses the same data classes and hits the same web services as the RIA, but presents the data in a completely different way.
Whereas the web app is pretty much a single-user experience, we hope that the Surface version allows people to literally sit around the table and share an experience and a discussion about the stories and data in front of them.
It will be installed at Microsoft facilities around the world shortly. Look for it next time you’re on campus!
Microsoft Research and Stimulant just launched the first version of Project Tuva, a platform for the delivery and study of classic lecture material. At launch, it features the legendary “Messenger Lectures on Physics” which were given by Prof. Richard Feynman at Cornell University in 1964, recorded by the BBC, and later purchased by Bill Gates.
The project was led by Curtis Wong at MSR who supplied the general vision, constant feedback, and amazing content. More about the MSR team is available on the contributors page. Design, development, and QA took over the whole team at Stimulant. I was the developer of the Silverlight client piece.
The most obvious (and so far, most criticized) thing about Tuva is that it’s a Silverlight 2 application. This means that users will need to have the plugin installed, and if you’re using Google Chrome, Safari on Windows, or any Linux browser, you won’t be able to use the application. That said, I really don’t think it could have been built any other way. As a long-time Flash/Flex developer, I’m very aware of the capabilities of that platform, and can imagine the pain involved in building the app in Flex. After a few Silverlight and WPF projects, I’m a .NET convert.
Special Features
Tuva is, basically, yet another Silverlight video player. However, this one is purpose-built to facilitate learning. Here are the features that make it different from other players.
Smooth Streaming video: Video is played back using IIS Smooth Streaming technology. The basic idea of Smooth Streaming is that a video file is broken up into chunks and served up over regular HTTP. This allows for easier caching by browsers and CDNs, and allows the client to switch to different quality streams on the fly depending on available bandwidth and playback performance. For example when a video begins playing, the player will use a lower bandwidth stream to minimize buffering time. Once the buffer is full, it will switch to a higher bitrate stream to improve video quality. Seeking from one point to another in a video is also much faster than with other technologies.
Full transcripts: Each lecture comes with a complete text transcript which is shown as a real-time caption beneath the video. The entire transcript can be viewed in paragraph form as well, and clicking on words in the transcript will seek to that point in the video.
User notes: Viewers can take notes as they watch, and can include hyperlinks. Notes are keyed to the specific point in the video where the note was entered.
Commentary tracks: Commentaries are like notes left by experts. These play along with the video like the captions to provide more insight into the subject. (Only lecture number one has commentaries at launch.)
Extras: Tuva supports “bonus content” that can be tied to any given point in the video. These can be HTML text, images, links to external websites, links to other Tuva content, equations, and other Silverlight applications. The most interesting type of extras are WorldWide Telescope and WWT Tour extras. For these, Tuva loads up the entire WWT web client from worldwidetelescope.org, instantiates it in a modal dialog, and you can explore the universe straight from there. I was surprised that this was even possible (which is not to say that it was easy…). (Only lecture number one has extras at launch.)
Multitrack timeline: The timeline beneath the video includes three tracks — one shows your basic playback position, but also includes chapter markers for quick navigation between sections of the lecture. There is another track which shows the position of any notes you’ve taken, and a third shows the position of the extras. Clicking on a note or extra takes you right to that spot in the video.
Live searching: There is a search box at the top which lets you find any term in a video transcript or metadata. Clicking a result in a transcript takes you right to that point in the video.
Persistence: Many things about your user session are stored in Silverlight’s isolated storage. The UI for extras, notes, and the timeline all have expanded and minimized states, and return to those states when you come back to the site. If you change the volume, it will stay where you left it. Your notes will be there when you come back. And if you have watching a video before it’s over, it will start where you left off when you return.
Bookmarking and history: You can bookmark a URL to any video, and use the browser’s back and next buttons to navigate the application like a normal web site.
Development Notes
This was my third major Silverlight project, and the first where I really felt like I knew what I was doing. We wanted to build the project in Silverlight 3, but since we had a hard launch date and we weren’t sure if Silverlight 3 would slip, we had to stick with the shipping version. In the end, Silverlight 3 shipped about a week before Tuva. We’re already planning some changes to take advantage of some of the features in the new version.
The overall architecture of the project used the MVC and command patterns as implemented in the Silverlight Cairngorm project. I got hooked on the Cairngorm style of development while working on Flex project and have carried those habits with me to Silverlight. We evaluated Prism but found it to be overkill on a single-developer project, but we may use it in the future.
All UI controls were implemented as custom Controls, not UserControls. This decision was made both because custom controls require a little more structure and rigor to develop, and also because we had a project requirement to support reskinning in the future, and this is much easier to do with custom controls. Because Silverlight 2 doesn’t support merged resource dictionaries, this meant for one 7,500 line Page.xaml file. This worked pretty well, but performance problems with Blend made visual edits very tedious. We’ll work on addressing this in future versions by reorganizing resources.
The process of getting from design to final product was fairly smooth. The interaction design was implemented by Nathan Moody in the form of a 60pg PDF which was used to create consensus among all the team members. Jules König, our visual designer, translated that into detailed comps in Illustrator. Once those were approved, they were converted to Expression Design files. I took those files and implemented the project in Visual Studio and Expression Blend. As components were completed, Jules would edit the project in Blend to refine design elements (and fix the things I didn’t quite finesse properly). The fact that my designer could edit the project himself really freed me to work on real features and issues without burning time on small visual tweaks.
The application has no backend, which presented some challenges. Content is managed through a standalone WPF app (developed by Darren David) which shares the data model classes from the Silverlight app. These classes implement IXmlSerializable, which allows the WPF app to save and load the data in XML form. When the Silverlight app starts, it loads a zip file containing the XML for all the videos (currently 1.2MB compressed) and deserializes them into data objects on a BackgroundWorker thread using lots and lots of LINQ. With all of the data for the app in memory, features like search were easier to implement. This approach clearly wouldn’t scale for hundreds or thousands of videos, but it’s good enough for seven.
During the project there were frequent code reviews, mainly between Darren and I. We worked together on much of the data model code, since he was writing data with the WPF app and I was reading it from Silverlight. Later on the project there was a formal code review by a developer at Microsoft Research. This was a little nerve-wracking because technical folks at MSR are extremely talented, experienced, and educated compared to a lowly art-school RIA-developer. In the end I survived, learned a few new best practices, and got hooked on Visual Studio’s “Code Analysis” feature for keeping things clean.
At Stimulant we care a lot about QA. Put simply, bugs make us a look bad, and we hate that. However, since we don’t have large development or QA teams, nor do we have long timelines, we don’t practice test-driven development. My process is to develop a feature and test it as thoroughly as I can myself before passing it off to QA, who tests it ten times more thoroughly and finds all the issues. QA and development run in parallel until final delivery, not just for a week at the end of the project.
In the end it was an exciting and extremely gratifying project, but it was not without its long nights (and weeks), dev challenges, and arguments. All the effort paid off though — the project has been well-received and is not one of those that I’m left wanting to totally rewrite when I’m done. A port to Silverlight 3 with additional features is coming, stay tuned.
Aside from MSR and Stimulant, some additional credit is due:
The RichTextBox component in ComponentOne’s Studio for Silverlight product is used for the display of HTML text in extras and notes.
My friends at Plexipixel created the intro video which is viewable by clicking on the right side of Tuva’s intro page.
Thanks to the Expression team for help in integrating Smooth Streaming video.
Thanks to Jonathan Fay from the WorldWide Telescope team for help in integrating the WWT client, including making changes to the client to support Tuva’s needs.
I worked on an image viewer for MSN that displays an arbitrary set of photos and lets you filter them based on attributes of each. LINQ is used heavily for the filtering logic. You can also drag, flip, and throw images with an experience that borrows heavily from Surface’s ScatterView component. When you select an image, Deep Zoom lets you get up close to the details. The first app to use it displays a set of celebrity tattoo photos.
The viewer has been reskinned six times with six different sets of photos:
We were asked to make an cool image viewer for a bunch of celebrity photos, and it sounded really uninteresting at first, but we went pretty next-level on it and made it fun. That’s Stimulant’s style.
Microsoft produced a promo video for the Local Impact Map project. The video makes the app look like it’s so exciting and fun! It is exciting and fun, but it’s interesting to see my work repurposed into such a marketingish context.
A project that’s been in the works since late last year at Stimulant has just launched on Microsoft.com. The Local Impact Map presents over 500 stories about the positive impact of local programs supported by Microsoft around the world.
Some marketing will be following the launch soon, but I’m more interested in the technical side. Some cool features:
The map in the background is a Deep Zoom image, with a XAML version of the map overlayed on top for rollovers and data visualization.
Intense support for bookmarking and the browser back button. Pretty much any state of the app is bookmarkable and shareable.
The bookmarking support is extended by the “snapshots” feature, which lets you save a series of app states and recall them later, for a sort of presentation.
Live as-you-type search throughout all the data in the app, thanks to LINQ.
Lots of great stories, images, and even video.
This has definitely been the most complex Silverlight project I’ve done to date, maybe even the most complex RIA so far. That is if you don’t count the one I’m working on now for MSR, of course.
This is the Stimulant Surface demo reel that Nathan put together for MIX09 (and was a third-place winner in the show-off contest). It covers all of our Surface projects to date. I’m actually working on a new Surface project right now, which is a reimagining of a Silverlight app that we’re just finishing up. No rest for the wicked.
This is about a year old, but I’ve been referring to it recently, as the current Silverlight project is also a big world map. The Silverlight Around the World app is used to visualize data from the Silverlight Showcase (which I also built). I can’t prove it, but I think these are some of the most complex apps that were made in the Silverlight 1.0 timeframe. Keep in mind that this is JavaScript, not C#, and Canvas, no Grids. Both use the ASP.NET AJAX component model, and a Cairngorm-style architecture to bring a little sanity and reusability to the code.
The next map/datavis project is all SL 2, Deep Zoom, and other madness, and will launch on microsoft.com… when it’s ready.
The “XRay” project I’ve been working on has been posted to the Stimulant site, and talked about here, here, and here. The implementation is pretty simple — two data-bound ScatterView instances, some funky bitmap manipulation, and a WCF service to send the images to the device over HTTP. This approach means that not only did I not have to figure out how to write iPhone apps (yet), but that it also works on Android (really well) and on Windows Mobile (sort of).
A couple of freelance projects I worked on a while ago have gone live. Actually they’ve probably been up a while but I’ve just now noticed. Both are for the International Quilt Study Center and Museum in Nebraska.
The first is the quilt collection, which presents 600 quilts in a pan-and-zoom grid, along with details about each and UI for searching the various collections.
The other half is the quilt maker, which lets you make and save your own quilt.
I believe these are also hosted as kiosk apps in the museum itself.
The project was designed and managed by the geniuses at Second Story in Portland. In both cases, I took the apps to a late beta stage, and final production and delivery was handled by Second Story. Both apps were implemented in Flex 3.
