View source for Cray-3

{{Short description|Supercomputer by Cray research}}
{{good article}}
[[File:Seymore Cray poses with Cray 3.jpg|thumb|right| Seymour Cray stands behind a Cray-3 processor tank. The CPU occupies only the top of the tank, the rest contains memory and power supplies.]]

The '''Cray-3''' was a [[Vector processor|vector]] [[supercomputer]], [[Seymour Cray]]'s designated successor to the [[Cray-2]]. The system was one of the first major applications of [[gallium arsenide]] (GaAs) semiconductors in computing, using hundreds of custom built [[integrated circuit|ICs]] packed into a {{convert|1|cuft}} [[central processing unit|CPU]]. The design goal was performance around 16 [[FLOPS|GFLOPS]], about 12 times that of the Cray-2.

Work started on the Cray-3 in 1988 at [[Cray|Cray Research]]'s (CRI) development labs in [[Chippewa Falls, Wisconsin]]. Other teams at the lab were working on designs with similar performance. To focus the teams, the Cray-3 effort was moved to a new lab in [[Colorado Springs, Colorado]] later that year. Shortly thereafter, the corporate headquarters in [[Minneapolis]] decided to end work on the Cray-3 in favor of another design, the [[Cray C90]]. In 1989 the Cray-3 effort was spun off to a newly formed company, [[Seymour Cray#Cray Computer Corporation|Cray Computer Corporation]] (CCC).

The launch customer, [[Lawrence Livermore National Laboratory]], cancelled their order in 1991 and a number of company executives left shortly thereafter. The first machine was finally ready in 1993, but with no launch customer, it was instead loaned as a demonstration unit to the nearby [[National Center for Atmospheric Research]] in [[Boulder, Colorado|Boulder]]. The company went bankrupt in May 1995, and the machine was officially decommissioned.

With the delivery of the first Cray-3, [[Seymour Cray]] immediately moved on to the similar-but-improved [[Cray-4]] design, but the company went bankrupt before it was completely tested.<ref>{{cite web |url=http://www.techagreements.com/agreement-preview.aspx?num=121632 |archive-url=https://archive.today/20120919123656/http://www.techagreements.com/agreement-preview.aspx?num=121632 |url-status=dead |archive-date=September 19, 2012 |title=CCC 1994 Annual Report }}</ref> The Cray-3 was Cray's last completed design; with CCC's bankruptcy, he formed SRC Computers to concentrate on parallel designs, but died in a car accident in 1996 before this work was delivered.<ref name=Lazou>{{cite web |url=http://www.hoise.com/primeur/96/pr-96-oct/CL-PR-10-96-3.html |title=Obituary – Seymour Cray, Father of supercomputing |url-status=dead |archive-url=https://web.archive.org/web/20080507080148/http://www.hoise.com/primeur/96/pr-96-oct/CL-PR-10-96-3.html |archive-date=2008-05-07 }}</ref>

== History ==

===Background===
Seymour Cray began the design of the Cray-3 in 1985, as soon as the [[Cray-2]] reached production.{{sfn|Trew|2012|p=245}} Cray generally set himself the goal of producing new machines with ten times the performance of the previous models. Although the machines did not always meet this goal, this was a useful technique in defining the project and clarifying what sort of process improvements would be needed to meet it.{{sfn|MacKenzie|1998|p=141}} For the Cray-3, he decided to set an even higher performance improvement goal, an increase of 12x over the Cray-2.{{sfn|MacKenzie|1998|p=153}}

Cray had always attacked the problem of increased speed with three simultaneous advances; more [[execution unit]]s to give the system higher [[Parallel computing|parallelism]], tighter packaging to decrease signal delays, and faster components to allow for a higher clock speed. Of the three, Cray was normally least aggressive on the last; his designs tended to use components that were already in widespread use, as opposed to leading-edge designs.{{sfn|MacKenzie|1998|p=141}}

For the Cray-2, he introduced a novel 3D-packaging system for its [[integrated circuit]]s to allow higher densities,{{sfn|Readings|2000|p=10}} and it appeared that there was some room for improvement in this process. For the new design, he stated that all wires would be limited to a maximum length of {{convert|1|foot}}. This would demand the processor be able to fit into a {{convert|1|cuft}} block, about {{frac|3}} that of the Cray-2 CPU. This would not only increase performance but make the system 27 times smaller.{{sfn|Trew|2012|p=246}}

For a 12x performance increase, the packaging alone would not be enough, the circuits on the chips themselves would also have to speed up. The Cray-2 appeared to be pushing the limits of the speed of [[silicon]]-based [[transistor]]s at 4.1&nbsp;ns (244&nbsp;MHz), and it did not appear that anything more than another 2x would be possible. If the goal of 12x was to be met, more radical changes would be needed, and a "high tech" approach would have to be used.{{sfn|MacKenzie|1998|pp=153–154}}

Cray had intended to use [[gallium arsenide]] circuitry in the Cray-2, which would not only offer much higher switching speeds but also used less energy and thus ran cooler as well. At the time the Cray-2 was being designed, the state of GaAs manufacturing simply was not up to the task of supplying a supercomputer.{{sfn|Readings|2000|p=9}} By the mid-1980s, things had changed and Cray decided it was the only way forward.{{sfn|MacKenzie|1998|p=154}} Given a lack of investment on the part of large chip makers, Cray decided to invest in a GaAs chipmaking startup, GigaBit Logic, and use them as an internal supplier.<ref>{{cite news |first=James |last=Peltz |url=https://www.latimes.com/archives/la-xpm-1990-01-23-fi-494-story.html |title=GigaBit Logic Negotiating Sale With Cray Computers |newspaper=Los Angeles Times |date=23 January 1990}}</ref>

Describing the system in November 1988, Cray stated that the 12 times performance increase would be made up of a three times increase due to GaAs circuits, and four times due to the use of more processors. One of the problems with the Cray-2 had been poor multiprocessing performance due to limited [[Bandwidth (computing)|bandwidth]] between the processors, and to address this the Cray-3 would adopt the much faster architecture used in the [[Cray Y-MP]]. This would provide a design performance of 8000 [[Million instructions per second|MIPS]], or 16 [[FLOPS|GFLOPS]].{{sfn|Trew|2012|p=246}}

===Development===
The Cray-3 was originally slated for delivery in 1991.<ref name="CRAY-COMPUTER-CORPORATION-Mar-1996-8-K">{{cite web|url=http://edgar.secdatabase.com/2189/92735696000121/filing-main.htm |title=CRAY COMPUTER CORPORATION, Form 8-K, Current Report, Filing Date Mar 26, 1996 |publisher=secdatabase.com |access-date =May 14, 2018}}</ref> This was during a time when the supercomputer market was rapidly shrinking from 50% annual growth in 1980, to 10% in 1988.{{sfn|MacKenzie|1998|p=154}} At the same time, Cray Research was also working on the Y-MP, a faster multi-processor version of the system architecture tracing its ancestry to the original [[Cray-1]]. In order to focus the Y-MP and Cray-3 groups, and with Cray's personal support,{{sfn|Murray|1997|p=195}} the Cray-3 project moved to a new research center in [[Colorado Springs, Colorado|Colorado Springs]].{{sfn|Trew|2012|p=245}}

By 1989, the Y-MP was starting deliveries, and the main CRI lab in [[Chippewa Falls, Wisconsin]], moved on to the C90, a further improvement in the Y-MP series.{{sfn|Trew|2012|p=243}}{{sfn|MacKenzie|1998|pp=154–155}} With only 25 Cray-2s sold, management decided that the Cray-3 should be put on "low priority" development. In November 1988, the Colorado Springs lab was spun off as [[Cray Computer|Cray Computer Corporation]] (CCC), with CRI retaining 10% of the new company's stock and providing an $85 million promissory note to fund development.{{sfn|Trew|2012|p=245}} Cray himself was not a shareholder in the new company, and worked under contract.<ref name="www.nytimes.com business-people-chief-executive-quits-at-cray-computer">{{cite news |url=https://www.nytimes.com/1992/04/17/business/business-people-chief-executive-quits-at-cray-computer.html?src=pm |title=Chief Executive Quits At Cray Computer |newspaper=The New York Times |date=17 April 1992}}</ref>{{sfn|Murray|1997|p=190}} As CRI retained the lease on the original building, the new company had to move once again, introducing further delays.{{sfn|Trew|2012|p=245}}{{sfn|Readings|2000|p=10}}

By 1991, development was behind schedule.<ref>{{cite news |url=https://www.nytimes.com/1991/12/17/business/company-news-cray-computer-is-behind-schedule.html?src=pm |title=Cray Computer Is Behind Schedule |newspaper=The New York Times |date=17 December 1991}}</ref> Development slowed even more when [[Lawrence Livermore National Laboratory]] cancelled its order for the first machine,<ref>{{cite news |title=Cray Loses Only Order For Product |work=The New York Times |date=24 December 1991 |url=https://www.nytimes.com/1991/12/24/business/company-news-cray-loses-only-order-for-product.html?src=pm }}</ref> in favor of the C90. Several executives, including the CEO, left the company.<ref name="www.nytimes.com business-people-chief-executive-quits-at-cray-computer"/> The company then announced they would be looking for a customer that needed a smaller version of the machine, with four to eight processors.<ref name=cold/>

The first (and only) production model (serial number S5, named ''Graywolf'') was loaned to [[NCAR]] as a demonstration system in May 1993. NCAR's version was configured with 4 processors and a 128&nbsp;MWord (64-bit words, 1&nbsp;GB) common memory.{{sfn|Lester|1993}} In service, the [[static RAM]] proved to be problematic. It was also discovered that the [[square root]] code contained a bug that resulted in 1 in 60 million calculations being wrong. Additionally, one of the four CPUs was not running reliably.<ref>{{cite magazine |first=Gary |last=Anthes |title=Research lab sizes up slew of supercomputers |magazine=Computerworld |date=1 August 1994 |page=55 |url=https://books.google.com/books?id=mEU4Qex5594C&pg=PA55}}</ref>

CCC declared bankruptcy in March 1995, after spending about $300 million of financing. NCAR's machine was officially decommissioned the next day.<ref>{{cite web |title=CRAY-3 (graywolf): 1993–1995 |website=SCD Supercomputer Gallery |url=https://www.cisl.ucar.edu/computers/gallery/cray/graywolf.jsp |archive-url=https://web.archive.org/web/20161010004607/https://www.cisl.ucar.edu/computers/gallery/cray/graywolf.jsp |archive-date=2016-10-10 |access-date=2016-10-20 |url-status=dead }}</ref> Seven system cabinets, or "tanks", serial numbers S1 to S7, were built for Cray-3 machines. Most were for smaller two-CPU machines. Three of the smaller tanks were used on the [[Cray-4]] project,<ref>{{cite magazine |title=Seymore R. Cray |magazine=Computerworld |date=18 July 1994 |page=20 |url=https://books.google.com/books?id=atdXb3xr3vgC&pg=PA20}}</ref> essentially a Cray-3 with 64 faster CPUs running at 1&nbsp;ns (1&nbsp;GHz) and packed into an even smaller space.<ref name= Stedman>{{cite magazine |first=Craig |last=Stedman |title=Cray Computer ends its quest |magazine=Computerworld |date=3 April 1995 |page=32 |url=https://books.google.com/books?id=J-JEQiC7c3oC&pg=PA32}}</ref> Another was used for the [[Cray-3/SSS]] project.<ref>{{cite book |first= Reinhard |last=Wobst |title= Cryptology Unlocked |publisher= John Wiley & Sons |date= 2007 |page=[https://archive.org/details/Cryptology_Unlocked/page/n166 150] |isbn=9780470060643 |url=https://archive.org/details/Cryptology_Unlocked }}</ref>

The failure of the Cray-3 was in large part due to the changing political and technical climate. The machine was being designed during the collapse of the [[Warsaw Pact]] and ending of the [[Cold War]], which led to a massive downsizing in supercomputer purchases.<ref name=cold>{{cite news |url=https://www.nytimes.com/1992/02/21/business/cold-war-s-end-hits-cray-computer.html?src=pm |title=Cold War's End Hits Cray Computer |newspaper=The New York Times |date=21 February 1992}}</ref><ref name=wsj/> At the same time, the market was increasingly investing in [[massively parallel]] (MP or MPP) designs. Cray was critical of this approach, and was quoted by ''[[The Wall Street Journal]]'' as saying that MPP systems had not yet proven their supremacy over vector computers, noting the difficulty many users have had programming for large parallel machines. "I don't think they'll ever be universally successful, at least not in my lifetime".<ref name=wsj>{{cite news |first=Michael |last=Allen |title=Pushing Big Iron: Seymour Cray's Woes Reflect Tough Times for Supercomputers |newspaper=Wall Street Journal |date= 1998 |url=http://www.isip.piconepress.com/publications/courses/msstate/ece_8463/projects/1998_spring/data/lm_training/wsj93_042.text |archive-url=https://web.archive.org/web/20120317081044/http://www.isip.piconepress.com/publications/courses/msstate/ece_8463/projects/1998_spring/data/lm_training/wsj93_042.text |archive-date=17 March 2012}}</ref>

==Architecture==

===Logical design===
The Cray-3 system architecture comprised a ''foreground processing system'', up to 16 ''background processors'' and up to 2 gigawords (16 GB) of ''common memory''. The foreground system was dedicated to [[input/output]] and system management. It included a 32-bit processor and four synchronous data channels for [[mass storage]] and network devices, primarily via [[HiPPI]] channels.{{sfn|Brochure|1993|p=6}}

Each background processor consisted of a ''computation section'', a ''control section'' and ''local memory''. The computation section performed [[64-bit]] scalar, [[floating point]] and [[Vector processor|vector arithmetic]]. The control section provided instruction buffers, memory management functions, and a [[real-time clock]]. 16 kilowords (128 kbytes) of high-speed local memory was incorporated into each background processor for use as temporary scratch memory.{{sfn|Brochure|1993|p=4}}

Common memory consisted of silicon [[CMOS]] [[Static Random Access Memory|SRAM]], organized into ''octants'' of 64 banks each, with up to eight octants possible. The [[word size]] was 64-bits plus eight [[error-correction]] bits, and total memory bandwidth was rated at 128 gigabytes per second.{{sfn|Brochure|1993|p=5}}

===CPU design===
[[File:Cray-3 complete module.jpg|right|288px|thumb|Typical module layout, with a 4x4 arrangement of "submodules", stacked 4-deep. The metal connectors on the bottom are power connections.]]

As with previous designs, the core of the Cray-3 consisted of a number of [[Computer module|modules]], each containing several circuit boards packed with parts. In order to increase density, the individual [[GaAs]] chips were not [[Integrated circuit packaging|packaged]], and instead several were mounted directly with ultrasonic gold bonding to a board approximately {{convert|1|inch}} square. The boards were then turned over and mated to a second board carrying the electrical wiring, with wires on this card running through holes to the "bottom" (opposite the chips) side of the chip carrier where they were bonded, hence sandwiching the chip between the two layers of board.  These ''submodules'' were then stacked four-deep and, as in the Cray-2, wired to each other to make a 3D circuit.{{sfn|Lester|1993}}

Unlike the Cray-2, the Cray-3 modules also included [[edge connector]]s. Sixteen such submodules were connected together in a 4×4 array to make a single module measuring {{convert|121 × 107 × 7|mm|in}}. Even with this advanced packaging the circuit density was low even by 1990s standards, at about 96,000 gates per cubic inch.{{sfn|Brochure|1993|p=8}} Modern CPUs offer gate counts of millions per square inch, and the move to 3D circuits was still just being considered {{asof|2017|lc=yes}}.<ref>{{cite magazine |first=Jared |last=Newman |url=http://www.pcworld.com/article/227260/intels_3d_transistor_why_it_matters.html |title=Intel's 3D Transistor: Why It Matters |magazine=PCWorld |date=5 May 2011}}</ref>

Thirty-two such modules were then stacked and wired together with a mass of twisted-pair wires into a single processor. The basic cycle time was 2.11&nbsp;ns, or 474&nbsp;MHz, allowing each processor to reach about 0.948 [[GFLOPS]], and a 16 processor machine a theoretical 15.17 GFLOP. Key to the high performance was the high-speed access to main memory, which allowed each process to burst up to 8 GB/s.<ref>{{cite web |first=Aad |last=van der Steen |url=http://netlib2.cs.utk.edu/benchmark/top500/reports/report94/Architec/node6.html |title=Short Description of Architectures in the TOP500: The Cray Computer Corporation Cray-3 |website= TOP500 |date=14 November 1995 |archive-url=https://web.archive.org/web/20120328110827/http://netlib2.cs.utk.edu/benchmark/top500/reports/report94/Architec/node6.html |archive-date=28 March 2012}}</ref>

===Mechanical design===
[[File:Cray-3 CPU section, 1995, Computer History Museum.jpg|right|288px|thumb|Complete processor "brick". The modules are visible inside, mounted vertically.]]

The modules were held together in an aluminum chassis known as a "brick". The bricks were immersed in liquid [[fluorinert]] for cooling, as in the Cray-2. A four-processor system with 64 memory modules dissipated about 88&nbsp;kW of power.{{sfn|Lester|1993}} The entire four-processor system was about {{convert|20|in|mm}} tall and front-to-back, and a little over {{convert|2|feet}} wide.{{sfn|Brochure|1993|p=15}}

For systems with up to four processors, the processor assembly sat under a translucent bronzed acrylic cover at the top of a cabinet {{convert|42|in|m}} wide, {{convert|28|in|m}} deep and {{convert|50|in|m}} high,{{sfn|Brochure|1993|p=15}} with the memory below it, and then the power supplies and cooling systems on the bottom. Eight and 16-processors system would have been housed in a larger octagonal cabinet. All in all, the Cray-3 was considerably smaller than the Cray-2, itself relatively small compared to other supercomputers.{{sfn|Brochure|1993|p=15}}

In addition to the system cabinet, a Cray-3 system also needed one or two (depending on number of processors) ''system control pods'' (or "C-Pods"), {{convert|52.5|in|m}} square and {{convert|55.3|in|m}} high, containing power and cooling control equipment.{{sfn|Brochure|1993|p=15}}

===System configurations===
The following possible Cray-3 configurations were officially specified:{{sfn|Brochure|1993|p=10}}

{|class="wikitable"
!scope="col"|Name
!scope="col"|CPUs
!scope="col"|Memory (Mwords)
!scope="col"|I/O Modules
|-
!Cray-3/1-256
|1
|256
|1
|-
!Cray-3/2-256
|2
|256
|1
|-
!Cray-3/4-512
|4
|512
|3
|-
!Cray-3/4-1024
|4
|1024
|3
|-
!Cray-3/4-2048
|4
|2048
|3
|-
!Cray-3/8-1024
|8
|1024
|7
|-
!Cray-3/8-2048
|8
|2048
|7
|-
!Cray-3/16-2048
|16
|2048
|15
|-
|}

===Software===

The Cray-3 ran the Colorado Springs Operating System (''CSOS'') which was based upon Cray Research's [[UNICOS]] [[operating system]] version 5.0.
A major difference between CSOS and UNICOS was that CSOS was ported to standard C with all [[Portable C Compiler|PCC]] extensions that were used in UNICOS removed.<ref name=cray-3-soft-intro-manual>{{cite book |title=CRAY-3 Software Introduction Manual |url=http://bitsavers.org/pdf/cray/CRAY-3/3102_CRAY-3_Software_Introduction_Nov91.pdf |publisher=Cray Computer Corporation |date=1991}}</ref>

Much of the software available under the Cray-3 was derived from Cray Research and included for instance the [[X Window System]], vectorizing [[FORTRAN]] and [[C (programming language)|C]] compilers, [[Network File System|NFS]] and a [[TCP/IP]] stack.{{sfn|Brochure|1993|p=14}}<ref name=cray-3-soft-intro-manual/>

== References ==

=== Citations ===
{{reflist|30em}}

=== Bibliography ===
{{refbegin}}
* {{cite book |title=CRAY-3 Supercomputer Systems|publisher=Cray Computer Corporation |type=brochure |date=1993 |url=http://archive.computerhistory.org/resources/access/text/2010/02/102685990-05-01-acc.pdf |ref=CITEREFBrochure1993}}
* {{cite book |first=Mark Donald |last=Hill |display-authors=etal |title= Readings in computer architecture |publisher= Gulf Professional Publishing |date= 2000 |isbn=9781558605398 |url=https://books.google.com/books?id=I7o8teBhz5wC |ref=CITEREFReadings2000}}
* {{cite magazine |first=Lynda |last=Lester |title=The making of a CRAY-3 | magazine= Ninth SCD User Conference |date= June 1993  |url=http://www.cisl.ucar.edu/docs/SCD_Newsletter/News_summer93/04e.cray3.html |archive-url=https://web.archive.org/web/20070630055048/http://www.cisl.ucar.edu/docs/SCD_Newsletter/News_summer93/04e.cray3.html |archive-date=30 June 2007 }}
* {{cite book |first=Donald |last=MacKenzie |title=Knowing machines: essays on technical change |publisher=MIT Press |date=1998 |isbn=9780262631884 |url= https://books.google.com/books?id=c5YaHkcP6DEC }}
* {{cite book |first= Charles |last=Murray |title=The Supermen: the story of Seymour Cray and the technical wizards behind the supercomputer |publisher=John Wiley |date=1997 |isbn=9780471048855 |url=https://archive.org/details/supermenstory00murr |url-access= registration }}
* {{cite encyclopedia |editor1-first= Arthur |editor1-last=Trew |editor2-first=Greg |editor2-last=Wilson  |first= Arthur |last=Trew |title=Vector Supercomputers: It's Never too Late to Parallelise |encyclopedia= Past, Present, Parallel: A Survey of Available Parallel Computer Systems |publisher= Springer Science & Business |date= 2012 |page=245 |isbn=9781447118428 |url=https://books.google.com/books?id=HZzeBwAAQBAJ }}

{{refend}}

== External links ==
* [http://www.digibarn.com/collections/systems/crays/cray3/index.html Digibarn's Cray-3 Modules]
* [http://www.0x07bell.net/WWWMASTER/CrayWWWStuff/Cfaqp2.html#TOC1 Cray Research and Cray computers FAQ Part 2]
* [http://klausler.com/cray2.txt Cray-2 and −3 instruction sets] ([https://web.archive.org/web/20070912041704/http://klausler.com/cray2.txt archived)]

{{Cray computers}}

[[Category:Cray products|3]]
[[Category:Vector supercomputers]]
[[Category:64-bit computers]]