Tornado Watchers

How the British Royal Air Force is using VR to train its Tornado jet mechanics

By Karen Moltenbrey

As an avionics technician in the British Royal Air Force (RAF) assigned to maintain and repair Tornado F3 military fighter-planes, you are required to know all the nuts and bolts of keeping the country's main air-defense fleet operational. So how do you learn to identify as well as disassemble and reassemble the plethora of pieces involved in this massive jigsaw puzzle of electronics and machinery? Virtually, of course.


To train its Tornado avionics technicians more cost-efficiently and effectively, the RAF recently restructured its avionics maintenance course content, replacing the lecture-based program with one that immerses the students in a simulated Tornado environment. "Students now get hands-on experience almost from the word go, as opposed to sitting in a classroom chalk-and-talk environment for weeks before coming into contact with the machinery," says Sam Southwell, a RAF avionics technician and coordinator of the revised course. The simulation familiarizes the students firsthand with the equipment from the onset of the course. Therefore, the users need less time training on an actual Tornado, which means less downtime for the multimillion-dollar aircraft used in the exercise.

Using the high-resolution, real-time simulation on the desktop, the students first familiarize themselves with the hundreds of parts and assemblies, known as line replaceable units (LRUs) that make up the crux of the Tornado's avionics. Then, they learn to perform simulated tasks set up by the course instructors. "In the real world, if an aircraft experienced a radar fault, we would debrief the air crew, fix the fault, and perform the functional checks before getting the aircraft back into service," explains Southwell. "In the virtual environment, the students debrief the instructors, and use the training system as if they were working on an actual aircraft. Everything they need is in front of them on the computer screens. They can perform almost any repair task, then use the test equipment, and even perform the functional checks afterward-just like in the real world-to make sure everything is operating correctly."

Situation Room

Located in a newly constructed facility at Marham air base near Norfolk, England, the simulation setup by Muse Virtual Presence of London includes two sets of five networked special-purpose, multi-pipeline Intergraph (Huntsville, AL) TDZ workstations. Each workstation contains three 3Dlabs' Wildcat graphics boards, each one delivering the output from a Muse Virtual Presence rendering engine to the workstation's three screens that are placed side by side. "We wanted to create a desktop environment that gives the trainee the feeling of immersion in and around the Tornado," says Bob Stone, scientific director at Muse Technologies, the parent company of Muse Virtual Presence.

"Squeezing the aircraft images onto a single desktop screen so the students could have some convincing visual detail when they opened an aircraft panel simply wouldn't work," Stone explains. "Therefore, we spread the aircraft images over three screens to achieve acceptable resolution for each scenario. For example, if the trainee is testing the behavior of a certain subsystem, he or she might need to see specific instruments in the pilot's cockpit, together with external aircraft views of the relevant avionics bay and a close-up of one or two of the 50 items of the virtual test equipment."

The centerpiece of this setup is the graphics, which had to be extremely accurate and realistic in order for the simulated experience to work as it was intended. Because of the aircraft's sophistication, its test and maintenance crews must meet extremely high standards of familiarity with its complex systems. To model the more than 450 parts and assemblies, a team from Muse Virtual Presence, along with Southwell and other military personnel, stripped down a Tornado at an RAF base during a two-week photography session. "The RAF also had enough catalogs of the surface skin of the aircraft and the individual components to sink a battleship," says Stone. To ensure accuracy, a team from Muse Virtual Presence also used rulers and other devices to "crawl all over the aircraft" to gather exact measurements. This enabled the group to control the quality of the final model since CAD data was unavailable.

"In the UK, a lot of the military systems that are currently operational do not have any CAD data associated with them. In this case, it led to a much longer project-implementation time scale than we're used to, simply because we had to build a Tornado from scratch," Stone adds. "Graphically, we constructed the entire plane from the cockpit to all the replaceable units."

Using a number of modeling packages, mainly Discreet's (Montreal) 3D Studio Max, two Muse Virtual Presence modelers spent more than a year creating the high-fidelity images, which were then archived in VRML 1. The group then used digital photography and Adobe Systems' (San Jose, CA) Photoshop to fully texture all the surfaces for each of the hundreds of units. "The aircraft is fully and realistically textured, although the texturing doesn't come out in some of the images until you look inside the bays, because the outside of a Tornado is painted military gray, with very sparse markings," says Stone. "This is the closest I believe anyone has ever come to building a completely faithful representation of a military aircraft in 3D, outside of the manufacturer."

The image processing is accomplished by a real-time event manager created by Muse Virtual Presence, a software system writ ten under OpenGL that links the virtual-reality models and the Alenia Marconi Systems (Frimley, Surrey, England) simulator into the Intergraph workstation for interaction with the human operator. "We had to guarantee software availability for at least 10 years, and we didn't feel confident to do that with a third-party product. Hence we used an open systems approach, VRML, with our own OpenGL-based rendering system," explains Stone.

Virtual Performance

Just like in the real world, a student can access any of the more than 450 LRUs that typically must be repaired or replaced on the Tornado, only this time it is done with a mouse click (and representations of the pilot and navigator joysticks for certain tasks) instead of a wrench or screwdriver. The trainee can remove a part from its location inside the cockpit or avionics bay, and the system presents the part to the user, allowing the person to manipulate the 3D image in real time.

Using the information presented on the screen, the trainee must decide if the system is indeed serviceable and whether or not it requires LRU replacement for repair. After deciding on the course of action necessary, the student then performs the tasks as he or she would in the real world. All the while, a simulation clock is ticking away to give the student a measuring tool to compare his or her virtual performance with an identical actual performance.


Because the students' workstations are linked to the instructors' computers, the performances are closely monitored. The images are also linked to a mathematical counterpart within the simulation system and deployed to a performance logger, so the instructors can review performances with the students. As a result, the instructors are able to identify a student's weaknesses much earlier in the training program. Prior to using the system, problem areas wouldn't surface until there were only a few days left in the course, "and then there wouldn't be enough time to bring them up to speed," says Southwell.

The simulation also enabled the instructors to set up more diversified and realistic problems for the students to solve. The 400-plus scenarios include unit problems that commonly occur in the field. Previously, the group was limited to troubleshooting cable-connection breaks between the units, a limitation resulting from practicing on an Avionics Ground Training Rig (AGTR) with actual (and expensive) equipment that was not easily replaced. Constant in-service demand for the aircraft meant trainers constantly struggled to acquire a complete training system. "Now we can also show students certain pitfalls-'watch this plug, it doesn't always go on correctly and you can easily break the wires,' that sort of thing. And we can do that with the 450 LRUs," says Southwell. "We couldn't have hoped to do that before, even if we had a school full of LRUs, because of the space limitations alone. If we lined up enough parts for eight students, we'd likely need a football field."

Simulation Ups and Downs

With eight workstations, the RAF can simultaneously train eight students, or even more if they work in pairs or threes, which is often required in real-world situations. Using the two training rigs, the instructors could only teach two students at a time. "So in an 11-week course, they'd get only about a week or so on an actual AGTR," Southwell explains. "Using the simulation, we've been able to expand the instructions while cutting the training program to nine weeks, and the simulators are in use the whole time."


Although the virtual training has replaced the classroom work, the program still includes a week of training on an actual Tornado. "There are still things you have to experience on a real aircraft, such as tactile feedback," explains Southwell. "When you bring the wings forward on a real Tornado, it will lurch up and down as you vary the geometry. To see people's reactions when they do that for the first time is quite a picture-they think the aircraft is going to sit on its tail." In this sense, the simulation training masks actual hazards. "The aircraft will bite you if you're not careful. People can, without due care, become injured while working on real aircraft, and you just can't get that sense from using a virtual environment," he adds.