Former top Gartner telecommunications infrastructure analyst turned independent technologist, advisor, and developer, Akshay Sharma describes his work in the Avionics industry, and specifically the incorrect usage of the ARINC 629 Databus he advised Boeing on for the 777, in the early ‘90s.
As seen in the news lately, Boeing has had it’s 737 MAX airplanes grounded due to two planes that crashed last year.
Issues persist with Boeing’s usage of $9/hour engineers from HCL’s India teams performing only the unit testing and not the rigorous systems testing to their usage of simulation testing, which only tests for sequences of events that are known about, not unknown scenarios.
Sharma advised Boeing on the 777, and when the Boeing 737 MAX crashes occurred last year, he was shocked to find out that Boeing was using the protocol he advised on decades ago.
“Back in ’89 and in the early ‘90s as a freshly minted engineer, I worked for Nortel on ISDN Q.931 standards where I became an expert on protocol design, conformance testing, performance testing and reachability analysis to detect real-time issues like deadlocks, queuing delays, and pre-emption/prioritization issues,” Sharma described in a recent interview.
“I then joined Telelogic which became part of SAAB Avionics Combitech division, and was assigned deep dive discussions with Boeing on a new plane called the 777 using a new system called ‘fly-by-wire’, which I advised them on, and helped train the FAA’s standards body called ARINC leading to the ARINC 629 Databus, used in the Boeing 777 and a few Airbus aircraft.” Sharma continued.
Initially, Boeing engineers were looking at using IBM’s Token-ring protocol, but due to latency delays in the seconds, which could be a problem for an airplane flying at 500mph or higher, Sharma recommended they look at ISDN Q.931 for flight control protocols, with millisecond latencies.
“I viewed the cockpit control systems as a big dial-pad to a phone with a bunch of buttons and numbers being displayed, where the pilot was in total control.” He quipped.
“ISDN Q.931 has spun many subsequent applications including DSL applications, PBX applications, VoIP, and it’s decoupling of signalling from the “bearer” plane is the basis of advances in SDN: software defined networking.” He reflected.
“Telelogic’s SDT tools, which are now part of IBM’s Rational group, provided unit-level, integration-level, and systems level conformance testing, simulation functions, but also real-time analysis using what’s known as State-Space Exploration to determine unknown issues. These are the hairy bugs that you can’t detect from unit-level testing.” He continued.
But regrettably, due to recent reports, Boeing cut corners on the testing process by just doing the unit-level testing and put an “AI” Controller on top of the ARINC 629 Databus, called the MCAS system: Maneuvering Characteristics Augmentation System.
The “AI” MCAS system essentially took over from pilots and downed the aircraft.
Unfortunately, from the latest analysis, it can be fooled by erroneous telemetry data. It can be flooded with data preventing other systems from operating, including the pilot’s cockpit systems, and can, therefore, prevent the human pilot from controlling things.
These ARINC Databus systems are 2Mb/s for the 629 protocol and 100 kb/s for the 429 protocol which are orders of magnitude slower than the WiFi speeds offered in the planes. With newer 5G/IoT services like streaming mixed reality video, 5G/IoT services with ultra low latencies (sub 5ms) along with 1 Gb/s, and 5G Edge Computing with on-site Private LTE services, it’s time we re-engineer the Databus systems with newer innovative thinking, moving from the technologies of the ‘90s to modern-day 2019/2020 thinking.
Sharma recommended we get back to rigorous testing with processes, methods, tools, and education.
He organizes this perspective on testing:
- Unit-Level Conformance Testing: Unit-level and Component level testing, which addresses functional compliance at the unit or component level.
- Integration Testing: Addresses interoperability between the components that are integrated.
- Systems Testing: Addresses conformance and interoperability of the entire system.
- Protocol Testing: Bandwidth, conformance, performance, real-time aspects (latency, queuing delays, deadlock detection, prioritization/pre-emption issues, to resiliency testing.
- User-centric testing: Acceptance testing, UI testing, etc.
- Security testing: Addresses vulnerabilities in the system.
Additionally, Test Management systems are needed to document the test cases exercised and defects discovered, which can then be tracked, monitored, and managed.
Tools from established vendors like IBM and start-ups like Kovair are needed with complete Project Lifecycle Management, with task-based process engines allowing users to automate and execute test runs, retrieve test results, and capture defects automatically. Full end-to-end traceability is required between multiple tool’s data, with test coverage reports and test results reporting that helps in continuous improvement of product quality.
When asked how the industry will get there, Sharma said, “The telecom industry solved the protocol issues, and we can solve this again. We are solving cloud-based solutions in telecom, and the aerospace industry is definitely cloud-based in more ways than one. We need to get back to basics. We can solve this!”
While Sharma makes it sound obvious and even easy, he says there are significant questions to be answered, including collectively as an increasingly interdependent software networking ecosystem is occurring in avionics, with flight controls, WiFi in airplanes which he also advised the FAA on in 2008, and newer infotainment systems with video, games, in planes today, to more modern AR/VR and Mixed Reality systems in the future.
In a future feature, we will share Sharma’s recommendations for CSPs, MSOs, and CDN providers, including the requirement to employ AI and automation to support what he calls “turbo-experiences” without taking huge capital expenditure risks.
Arti Loftus is an experienced Information Technology specialist with a demonstrated history of working in the research, writing, and editing industry with many published articles under her belt.
Edited by Ken Briodagh