If you are in some way involved in flight training, you may have noticed that recent updates to the Airman Testing page (www.faa.gov/training_testing/testing/) on the FAA’s website provide some important information. Among other things, it states that beginning February 9, 2015, types of questions eliminated from the private pilot airplane knowledge test include “aircraft performance and weather questions that involve multiple interpolations across multiple charts.” I’ll come back to this specific topic shortly.
Regular readers of FAA Safety Briefing may remember reading already about the agency’s collaboration with aviation community experts on development of, and transition to, the integrated and holistic Airman Certification Standards (ACS) approach to airman certification. I won’t repeat the details here — you can now find extensive ACS-related information on the Airman Testing web page — but the announced improvements are only the first of many ongoing updates and enhancements to airman knowledge testing.
The FAA is now applying industry’s ACS-related tools and procedures to the review, revision and, eventually, the development of knowledge test questions.
The “Boarding” Process
While most people do tend to equate test-taking with waterboard-style unpleasantries, that’s not what we mean when we talk nowadays about “boarding” questions. In keeping with both industry recommendations and best practices, the FAA has established a formal exam review board to improve its knowledge test question data bank. The board is comprised of subject matter experts from a range of FAA Flight Standards policy divisions, such as the Air Transportation Division, the Flight Technologies and Procedures Division, the Regulatory Support Division, and the General Aviation and Commercial Division. Consistent with industry best practices, the exam board also includes an “outside stakeholder” who has extensive qualifications in aviation, aviation training, and the overall test development and test management process.
The task is monumental, but the board members are ferociously dedicated to ensuring that we make knowledge test questions accurate, up-to-date, educationally sound, and relevant to real-world operations in today’s National Airspace System (NAS). We are using a range of tools and processes to achieve this objective. First, there is considerable real-world expertise on the board itself. Our members include air carrier, corporate, and general aviation pilots whose industry experience is recent. Second, the board is using a newly-developed and formally documented process for evaluating questions for educational and operational relevance. Third, we are using the ACS coding system (explained and illustrated on the Airman Testing web page) to ensure that each question links to a specific ACS Area of Operation/Task.
On both the FAA and the industry sides, there are enormous amounts of “invisible” progress and activities underway in all aspects of the airman certification system. The announced changes to the private pilot airplane knowledge test are important not only on their own terms, but also as a visible marker of our collective commitment to improving the airman certification system. The deletion of performance questions “that involve multiple interpolations across multiple charts” is an excellent example of how the new exam review board has approached its task. Working together, board members quickly concluded something most of us have known for years: overly complicated performance calculations that yield impossible “precise” answers are, ironically, grossly inaccurate — not only in terms of real-world operations, but also in terms of safety. Decimal-point precision makes little sense on performance charts developed in carefully controlled conditions and by a professional test pilot’s simulation of “average” pilot skills.
As I said, it’s a monumental task, with thousands of questions still to be “boarded.” But just as the journey of a thousand miles begins with a single step, the boarding of many thousand questions is now solidly underway. (FAA Safety Briefing – MayJun 2015)