Inside Lockheed Martin's Human Immersive Lab
VR and Motion Tracking Solve F-35 Engineering Challenges
A version of this article appeared in SME's publication, Aerospace & Defense Manufacturing 2010All images courtesy of Lockheed Martin Aeronautics
One of the major innovations in engineering in recent years is immersive engineering—the fusion of motion tracking and analysis, simulation, and virtual reality(VR) with CAD. Immersive engineering tackles engineering challenges by letting problem solvers work inside lifelike, room-sized graphical displays of their data.
The Human Immersive Laboratory (HIL) at Lockheed Martin Aeronautics Co.’s military aircraft facility in Fort Worth, TX, is a good example. Immersive engineering’s development is being spurred by savings—described as cost avoidance—of more than $100 million from HIL (formerly known as SAIL for Ship/Air Integration Lab). The company credits immersive engineering and digital mock-ups.
Inside the "lighthouse" at Suffolk, VA, which is used by HIL for remote immersive events.
Those savings accrued in one program, albeit a big one—Lockheed Martin’s F-35 Lightning II, also known as the Joint Strike Fighter or JSF. The savings represent a 15-fold return on the Aeronautics business unit’s total investment in applying immersive engineering to the detailed design of the aircraft—roughly equal to the cost of one F-35.
Within Lockheed Martin, HIL is extending the fundamentals of immersive engineering:
- Organizationally, as other business units put HIL to work for programs as diverse as the US Navy’s amphibious assault ships and improving the loading process for the KC-130J military transport. HIL is also being used for the F-16 and F-22 aircraft and life-cycle management for all these programs.
- Electronically, with real-time interoperability through video conferencing between Fort Worth and the corporation’s Center for Innovation (Suffolk, Va).
- Physically, as new systems are being installed at Suffolk and at Lockheed Martin Space Systems in Denver, a missiles and spacecraft factory.
These systems are built around motion tracking with Eagle digital cameras and Cortex analytical software from Motion Analysis Corp. (Santa Rosa, CA). Motion tracking puts users into fully interactive VR environments.
Along with motion tracking, The HIL integrates several VR technologies to allow the user to interact with realistic 3-D models in a simulated environment. Michael R. Yokell, a Lockheed Martin senior manager involved with SAIL from its inception, observes that “the value of this technology became clear throughout our product life cycle—through all the stages from proposal, conception, design, production, sustainment, and even operations, and then beyond in facilities and safety.”
Yokell led the F-35 Basing and Ship Suitability (BASS) effort, which ensures that the aircraft is compatible with and operationally supportable from all of its basing options.
In particular, immersive engineering “greatly reduces the need to create physical mock-ups,” he adds. “HIL and SAIL also reduce the risk of damaging aircraft components in production, or the need to ground an aircraft during testing or training.”
“When we find issues, even in the late stages of the design process, we’re still finding them well before they would have been found through other means,” Yokell notes. “HIL gives us the opportunity to fix them sooner. The virtual approach lets us rapid-prototype the changes in engineering, without tying up an actual aircraft to make them.”
When F-35 detailed design was completed, “SAIL’s mission broadened to include safety analyses and facility reviews,” he says. “The name-change from SAIL to HIL more accurately reflects its specific functions and its broader applications to other Lockheed Martin programs, to maintenance analysis, sustainment (in the field), and modernizations, and upgrades.”
Schematic of the Lockheed Martin Aeronautics HIL in Fort Worth, TX.
A big part of cost avoidance relates to the sophisticated visualization capabilities of immersive engineering and motion-tracking technology. Examples from the F-35 product life cycle follow.
Aircraft conception: Immersive engineering can quickly validate design concepts. Early in the design of an F-35 drag chute, the team leader saw the need to evaluate reachability for two different assembly pins that permit removing the drag chute door. In just four hours, CAD models and an avatar were loaded to validate the type and the position of two pins.
“The designer put on a head-mounted display [HMD] in HIL’s motion tracking area, and in an hour made a critical decision on his design,” says Pascale Rondot, an ergonomics expert who is HIL’s lead engineer. “The position and type of one pin was confirmed, but the other pin needed to be modified. As the designer reached out to remove the pin, avoiding collision with the aircraft structure, he told us, ‘It’s amazing how fast your brain lets you believe you are inside the model!’”
Preliminary design: As part of preliminary design, every component is integrated with the rest of the aircraft, both for production and in the contexts of operations or maintenance. Immersive engineering encourages team review and highlights potential problems and solutions cost-effectively.
HIL lets designers bring in different perspectives on maintainability and accessibility. “A one-hour work session can save a retrofit that might cost millions of dollars if caught too late,” says Ray Harbor, F-35 carrier integration lead for BASS. In designing the F-35’s wing low-point drain, HIL was used to ensure access when the aircraft is parked tail-over-water on an aircraft carrier.
“The capability within HIL to scale a motion-tracked user to any size of mannequin validated a redesign suitable for the full maintainer population,” Harbor says. The baseline design also did not allow a sufficient amount of residual fuel to drain from the tank. A short study validated a redesign that reduces residual fuel by a factor of three.” Harbor worked closely with Yokell in developing SAIL.
The analysis was presented to the Navy using the HIL theater, the Cave Automated Virtual Environment, known simply as “The Cave.” The Navy concurred.
Detailed design: Two of the three F-35 variants are intended for shipboard operations on Navy aircraft carriers and amphibious assault ships. These aircraft are adapted for tie-down chains, jacks, and fittings beneath the aircraft. Conventional “aircraft-centric” CAD failed to disclose collisions with aircraft doors for the nose and main landing gear, Harbor notes. “It’s not until you model the flight deck operations with the chains, the tie-down options, and the aircraft moving around the deck that you see it clearly.”
Early HIL immersive engineering analyses caught tiedown collisions in the Marine Corps and Navy variants. HIL also revealed a collision with the location of the jack adapter on the Navy’s F-35C. HIL helped relocate the tie-down points, and then confirmed that the revised design accommodated landing gear retraction and extension on jacks aboard ship.
Manufacturing: The rear engine nozzle of the Marine Corp’s aircraft, the F-35B, swings down sharply for short takeoffs and vertical landings. In flight, a pair of swivel doors protects the nozzle. When a delayed door delivery threatened the pace of production, assembly team members used HIL to analyze installing the engine before rather than after the doors. The question was whether assemblers would have all the necessary access for door installation after the engine was in place.
“Putting the assembler in the motion-tracking system validated the change while the assembly team leader observed from the Cave,” says Rondot. “The team concluded that the task could be done without risk, thereby reducing production delays. The simulation was put together in two days.”
Operational safety: The Navy’s F-35C is launched from aircraft carriers with steam-powered catapults. HIL was used to evaluate launch safety procedures for personnel workingnear the aircraft’s jet engine intakes.
HIL simulation ensuring that maintainers on the flight deck can properly drain excess fuel from beneath the F-35’s wing.
Harbor explains that a critical part of every launch is verifying the proper interface between the catapult, the aircraft launch bar, and the holdback bar: “This task falls to the carriers’ TSPOs [topside petty officers], who are responsible for prelaunch verification of the interface.” On the F-35, “the close proximity of the jet intakes to the equipment required for catapult interface would have placed the TSPO within the personal hazard zone [PHZ], Harbor notes.
This multifaceted simulation started with CAD—developing new PHZs with engines at full power for carrier deck takeoff, at maximum power, and at idle. “The new PHZs were fed into HIL as CAD geometry,” Harbor explains. “We used the system’s built-in postures library of anthropometric sizes to see if we could keep personnel clear of the PHZs.”
Lockheed Martin systems engineers noted safety concerns that negatively affected launching the F-35C, “that pertained to all catapult-launched aircraft,” Harbor said. “We shared our concerns with the Navy through visual means in the HIL.
“The engineers went on to modify catapult-launch procedures that mitigated these safety concerns,” Harbor said. “The revised procedures were demonstrated in the HIL, which displayed the modified procedures in the environment familiar to Navy leadership.”
The Navy ran the new procedures through its Operational Risk Management process and adopted the modified launch procedures “as written,” he says.