Jim Gray on Channel 9

<blockquote>
<em>I came to Microsoft ... to build the biggest computer you can imagine, by just using components you can find at Radio Shack.</em>
</blockquote><p>
There are two hours of video interview with <a href="http://research.microsoft.com/~gray/">Jim Gray</a> online at <a href="http://channel9.msdn.com">Channel 9</a> that have captured Jim at his best (<a href="http://channel9.msdn.com/ShowPost.aspx?PostID=49891" id="49891">part1</a>, <a href="http://channel9.msdn.com/ShowPost.aspx?PostID=50428" id="50428">part 2</a>, via <a href="http://pluralsight.com/blogs/dbox/archive/2005/04/02/7170.aspx">dbox</a>). Reviewing the past and future of scalable distributed computing, operating systems and databases, Jim shines as the formidable story teller he truly is.
</p><p>
About ten years ago I hooked up with Jim in working on the <em>Scalable Servers</em> project. At that moment the debate was still raging what would be better, to <em>scale up</em> or to <em>scale out.</em> SGI and Sun where still heavily investing in building very advanced multi-processor boxes, but theses machines were expensive and tools for failure isolation in multi-processors were still minimal. Scale-out appeared the way to overcome these obstacles and we were advocates that the basis for this should be true commodity components. Together with Thorsten von Eicken (now chief architect at Citrix) I started working on the intra-node communication obstacles. At that time we believed that to make distributed systems work on clusters of scalable low-end components you needed to cut-away as much of the intra-node communication latency as possible. This meant removing the biggest latency introducer: the operating system. Big commercial clusters such as the IBM SP2 already had direct user-level communication, but in a fashion that didn't allow more than one application to use the interconnect directly. Our work resulted in the <a href="http://www.cs.cornell.edu/tve/u%2Dnet/">U-Net Architecture</a> which allowed for delivery of data directly into application memory. It was <a href="http://www.cs.cornell.edu/tve/u%2Dnet/papers/sosp.pdf">published </a>in the 1995 SOSP.
</p><p>
What set this work apart from earlier approaches was that it was available for standard workstations and operating systems and that it provided full isolation such that multiple applications could access the interconnect concurrently. At that time the approach was considered a major breakthrough to enable the scaling of commodity clusters. The U-Net approach as well as technologies such as <a href="http://www.cs.princeton.edu/shrimp/html/communication.html">VMMC</a> by Kai Li at Princeton were the precursors to what later became the Virtual Interface Architecture standard. VIA has been largely abandoned as source of hardware design in favor of Infiniband which also centers around direct user-level access. In "<a href="http://www.ece.purdue.edu/~arch/seminar/schedules/spring99-pdf/via.pdf">The Evolution of the Virtual Interface Architecture</a>" Thorsten and I reviewed the transition of the research prototypes into industry standard. The U-Net work was made very popular by <a href="http://www.eecs.harvard.edu/~mdw/">Matt Welsh</a>, who added dynamic memory management, ported it to Linux and added support for Fast-Ethernet (Matt's <a href="http://www.eecs.harvard.edu/~mdw/proj/old/unet/">old u-net project pages</a>).
</p><p>
Thorsten and I built the first prototype on the SunOS platform, but in 1995 Sun pulled the plug on the operating system in favor of Solaris. Access to the OS source code was necessary to understand the complex workings of device and user-level address mappings, and the new Solaris license was a nightmare for Universities. We didn't consider Linux as an option, because it was an old-fashioned monolithic operating system that didn't sport any of the advantages of the new operating systems world. E.g. the network buffer management was even worse than in SunOS. Microsoft saw the potential of working with Universities on these issues and produced a source code license that allowed faculty and students to work with the Windows OS source code, without signing their lives away. We were happy to work on Windows as it meant that our research could now potentially reach millions of real computer users.
</p><p>
Jim Gray, visionary as always, had this figured out long before anybody else and was actively promoting the use of Windows source by academics. Thorsten and I were guinea pigs as we showed up at Jim's office in downtown San Francisco to work on porting U-Net to Windows, as the first access to Windows source code was only on-site at Microsoft offices. The port took us 4 days and we returned homed with sufficient material to make U-Net work on the Windows platform. In those 4 days Jim made a deep impression on me with his insight into computing, industry and lives as a technologists. After that week I would return frequently to Jim for guidance in research and have never been disappointed. The video interview give a good sense of his breadth and depth.
</p><p>
The other passion of Jim we became familiar with that first week was sailing. Since that time I have made the after-work trip on Jim's boat to Sausalito for dinner and good stories several times. I like to believe that Jim's story telling talents are rooted in his sailors heart, and there is never a dull moment with him around.
</p><p>
Jim got recognized for his many contributions to modern computing with a Turing Award in 1998.
</p>