A late change, and fatal flaws, in a Boeing jet design

The fatal flaws with Boeing’s 737 Max can be traced to a breakdown late in the plane’s development, when test pilots, engineers and regulators were left in the dark about a fundamental overhaul to an automated system that would ultimately play a role in two deadly crashes.

A year before the plane was finished, Boeing made the system more aggressive and riskier. While the original version relied on data from at least two types of sensors, the ultimate used just one, leaving the system without a critical safeguard. In both doomed flights, pilots struggled as a single damaged sensor sent the planes into irrecoverable nose-dives within minutes.

But many people involved in building, testing and approving the system, known as MCAS, said they hadn’t fully understood the changes. Current and former employees at Boeing and the Federal Aviation Administration who spoke with The New York Times said they had assumed the system relied on more sensors and would rarely, if ever, activate. Based on those misguided assumptions, many made critical decisions, affecting design, certification and training.

“It doesn’t make any sense,” said a former test pilot who worked on the Max. “I wish I had the full story.”

While prosecutors and lawmakers try to piece together what went wrong, the current and former employees point to the single, fateful decision to change the system, which led to a series of design mistakes and regulatory oversights. As Boeing rushed to get the plane done, many of the employees described a compartmentalised approach, each focusing on a small part of the plane. The process left them without a complete view of a critical and ultimately dangerous system.

The company also played down the scope of the system to regulators. Boeing never disclosed the revamp of MCAS to FAA officials involved in determining pilot training needs, according to three agency officials. As a result, most Max pilots did not know about the software until after the first crash, in October.

“Boeing has no higher priority than the safety of the flying public,” a company spokesman, Gordon Johndroe, said in a statement. “The FAA considered the final configuration and operating parameters of MCAS during Max certification, and concluded that it met all certification and regulatory requirements.”

At first, MCAS — Maneuvering Characteristics Augmentation System — wasn’t a very risky piece of software. The system would trigger only in rare conditions, nudging down the nose of the plane to make the Max handle more smoothly during high-speed moves. And it relied on data from multiple sensors measuring the plane’s acceleration and its angle to the wind, helping to ensure that the software didn’t activate erroneously.

Then Boeing engineers reconceived the system, expanding its role to avoid stalls in all types of situations. They allowed the software to operate throughout much more of the flight. They enabled it to aggressively push down the nose of the plane. And they used only data about the plane’s angle, removing some of the safeguards.

A test pilot who originally advocated for the expansion of the system didn’t understand how the changes affected its safety. Safety analysts said they would have acted differently if they had known it used just one sensor. Regulators didn’t conduct a formal safety assessment of the new version of MCAS.

The current and former employees, many of whom spoke on the condition of anonymity, said that after the first crash, they were stunned to discover MCAS relied on a single sensor.

“It seems like somebody didn’t understand what they were doing,” said an engineer who assessed the system’s sensors.

MCAS IS BORN

In 2012, the chief test pilot for the Max had a problem. During the early development of the 737 Max, Ray Craig, a retired Navy airman, was trying out high-speed situations on a flight simulator.

But the plane wasn’t flying smoothly, partly because of the Max’s bigger engines. To fix the issue, Boeing decided to use a piece of software. The system was meant to work in the background, so pilots effectively wouldn’t know it was there.

To ensure it didn’t misfire, engineers initially designed MCAS to trigger when the plane exceeded at least two separate thresholds, according to three people who worked on the 737 Max. One involved the plane’s angle to the wind, and the other involved G-force, or the force on the plane that typically comes from accelerating.

The Max would need to hit an exceedingly high G-force that passenger planes would probably never experience. For the jet’s angle, the system took data from the angle-of-attack sensor. The sensor, several inches long, is essentially a small wind vane affixed to the jet’s fuselage.

ADDING MORE POWER

In late January 2016, the first Max lifted off for its maiden test flight.

“The 737 Max just felt right in flight, giving us complete confidence that this airplane will meet our customers’ expectations,” Ed Wilson, the new chief test pilot for the Max, said in a news release at the time. Wilson had replaced Craig the previous year.

But a few weeks later, Wilson and his co-pilot began noticing that something was off, according to a person with direct knowledge of the flights. The Max wasn’t handling well when nearing stalls at low speeds.

Wilson told engineers that the issue would need to be fixed. He and his co-pilot proposed MCAS, the person said.

The change didn’t elicit much debate. It was considered “a run-of-the-mill adjustment,” according to the person.

The change proved pivotal. Expanding the use of MCAS to lower-speed situations required removing the G-force threshold. MCAS now needed to work at low speeds so G-force didn’t apply.

The change meant that a single angle-of-attack sensor was the lone guard against a misfire. Although modern 737 jets have two angle-of-attack sensors, the final version of MCAS took data from just one.

Using MCAS at lower speeds also required increasing the power of the system.

The FAA had already approved the previous version of MCAS. And the agency’s rules didn’t require it to take a second look because the changes didn’t affect how the plane operated in extreme situations.

‘EXTERNAL EVENTS’

After engineers installed the second version of MCAS, Wilson and his co-pilot took the 737 Max for a spin.

They tested two potential failures of MCAS: a high-speed maneuver in which the system doesn’t trigger, and a low-speed stall when it activates but then freezes. In both cases, the pilots were able to easily fly the jet, according to a person with knowledge of the flights.

In those flights, they did not test what would happen if MCAS activated as a result of a faulty angle-of-attack sensor — a problem in the two crashes.

Boeing engineers did consider such a possibility in their safety analysis of the original MCAS. They classified the event as “hazardous,” one rung below the most serious designation of catastrophic, according to two people. In regulatory-speak, it meant that MCAS could trigger erroneously less often than once in 10 million flight hours.

That probability may have underestimated the risk of external events that have damaged sensors in the past, such as collisions with birds, bumps from ramp stairs or mechanics’ stepping on them. While part of the assessment considers such incidents, they are not included in the probability.

A Times review of two FAA databases found hundreds of reports of bent, cracked, sheared-off, poorly installed or otherwise malfunctioning angle-of-attack sensors on commercial aircraft over three decades.

A SIMPLE REQUEST

On March 30, 2016, Mark Forkner, the Max’s chief technical pilot, sent an email to senior FAA officials with a seemingly innocuous request: Would it be OK to remove MCAS from the pilot’s manual?

The officials, who helped determine pilot training needs, had been briefed on the original version of MCAS months earlier. Forkner and Boeing never mentioned to them that MCAS was in the midst of an overhaul, according to the three FAA officials.

Under the impression that the system was relatively benign and rarely used, the FAA eventually approved Forkner’s request, the three officials said.

Boeing wanted to limit changes to the Max from previous versions of the 737. Anything major could have required airlines to spend millions of dollars on additional training. Boeing, facing competitive pressure from Airbus, tried to avoid that.

Forkner, a former FAA employee, was at the front lines of this effort. As the chief technical pilot, he was the primary liaison with the FAA on training and worked on the pilot’s manual.

“The pressure on us,” said Rick Ludtke, a cockpit designer on the Max, “was huge.”

Forkner’s attorney, David Gerger, said his client did not mislead the FAA.

“In thousands of tests, nothing like this had ever happened,” he said. “Based on what he was told and what he knew, he never dreamed that it could.”

The FAA group that worked with Forkner made some decisions based on an incomplete view of the system. It never tested a malfunctioning sensor, according to the three officials. It didn’t require additional training.

William Schubbe, a senior FAA official, told pilots and airlines in an April meeting in Washington, D.C., that Boeing had underplayed MCAS, according to a recording reviewed by The Times.

“The way the system was presented to the FAA,” Schubbe said, “the Boeing Corporation said this thing is so transparent to the pilot that there’s no need to demonstrate any kind of failing.”

Boeing continued to defend MCAS and its reliance on a single sensor after the first crash, involving Indonesia’s Lion Air.

Four months later, a second 737 Max crashed in Ethiopia. Within days, the Max was grounded around the world.

As part of the fix, Boeing has reworked MCAS to more closely resemble the first version. It will be less aggressive, and it will rely on two sensors.