If you are in some way involved in flight train­ing, you may have noticed that recent updates to the Airman Testing page (www.faa.gov/training_testing/testing/) on the FAA’s website provide some impor­tant information. Among other things, it states that beginning February 9, 2015, types of questions elimi­nated from the private pilot airplane knowledge test include “aircraft performance and weather questions that involve multiple interpolations across multiple charts.” I’ll come back to this specific topic shortly.

Regular readers of FAA Safety Briefing may remember reading already about the agency’s col­laboration with aviation community experts on development of, and tran­sition to, the integrated and holistic Airman Cer­tification Standards (ACS) approach to airman cer­tification. I won’t repeat the details here — you can now find extensive ACS-related information on the Airman Testing web page — but the announced improvements are only the first of many ongoing updates and enhancements to airman knowledge testing.

The FAA is now applying industry’s ACS-related tools and procedures to the review, revision and, even­tually, the development of knowledge test questions.

The “Boarding” Process

While most people do tend to equate test-taking with waterboard-style unpleasantries, that’s not what we mean when we talk nowadays about “boarding” questions. In keeping with both industry recommen­dations and best practices, the FAA has established a formal exam review board to improve its knowledge test question data bank. The board is comprised of subject matter experts from a range of FAA Flight Stan­dards policy divisions, such as the Air Transportation Division, the Flight Technologies and Procedures Divi­sion, the Regulatory Support Division, and the General Aviation and Commercial Division. Consistent with industry best practices, the exam board also includes an “outside stakeholder” who has extensive qualifica­tions in aviation, aviation training, and the overall test development and test management process.

The task is monumental, but the board mem­bers are ferociously dedicated to ensuring that we make knowledge test questions accurate, up-to-date, educationally sound, and relevant to real-world operations in today’s National Airspace System (NAS). We are using a range of tools and processes to achieve this objective. First, there is considerable real-world expertise on the board itself. Our members include air carrier, corporate, and general aviation pilots whose industry experi­ence is recent. Second, the board is using a newly-developed and formally documented process for evaluating questions for educational and opera­tional relevance. Third, we are using the ACS coding system (explained and illustrated on the Airman Testing web page) to ensure that each question links to a specific ACS Area of Operation/Task.

Performance-Related Results

On both the FAA and the industry sides, there are enormous amounts of “invisible” progress and activities underway in all aspects of the airman certification system. The announced changes to the private pilot airplane knowledge test are important not only on their own terms, but also as a visible marker of our collective commitment to improv­ing the airman certification system. The deletion of performance questions “that involve multiple interpolations across multiple charts” is an excel­lent example of how the new exam review board has approached its task. Working together, board mem­bers quickly concluded something most of us have known for years: overly complicated performance calculations that yield impossible “precise” answers are, ironically, grossly inaccurate — not only in terms of real-world operations, but also in terms of safety. Decimal-point precision makes little sense on per­formance charts developed in carefully controlled conditions and by a professional test pilot’s simula­tion of “average” pilot skills.

As I said, it’s a monumental task, with thou­sands of questions still to be “boarded.” But just as the journey of a thousand miles begins with a single step, the boarding of many thousand questions is now solidly underway. (FAA Safety Briefing – MayJun 2015)