Human Factors can truly be a win-win for manufacturers, engineers and consumers, but you must PLAN. Human Factors is NOT a “wait and see what happens” activity solely at the end of medical device development.
Virginia A. Lang, PhD
Medical Device Human Factors by HirLan, Inc part of HirLan Institute of Human Factors
You plan for weddings, graduations, holidays and other significant events. Why would you not plan for the successful market launch of a medical device that is going to impact the health and well-being of patients? You plan so that you have a clear picture of costs, timing, profit, etc. You bring in experts from different functions. But, are you including Human Factors/Usability experts at the early planning stages? It has been consistently shown that integrating Human Factors expertise and processes into the early phases of development decreases overall costs and time to market.
Be a Planner. Don’t “wait and see”.
The FDA requires human factors/usability testing of medical devices and in all their guidances they discuss the importance of early integration of human factors/usability. By the way, so do all the ISO/IEC standards for granting the CE mark. They do this because it is known that Human Factors/usability processes and testing is not only a win-win for medical device manufacturers, but also for end users.Human Factors: FDA and CE MarkHuman Factors Engineering is grounded in scientific research methodology, statistics, human physiology, and cognitive information processing. It is the intersection of engineering, human physiology, behavioral performance, and cognitive science. Human factors professionals use this foundational scientific knowledge to explain how humans interact with devices, products, and/or systems. Human factors engineers approach both hardware and software design with the USER as the focal point. Doing so assures devices, products and systems are Safe, Effective, and Usable by the intended users in the intended environment for the intended use.
The FDA and the notified bodies for the CE mark require medical device manufacturers to create a user-based risk analysis for products and related software. By creating the Human Factors Risk Analysis, medical device manufacturers complete a thorough examination of product(s) from the point of view of the user. This Human Factors Risk Analysis is the basis of all human factors/usability testing. One important component of the risk analysis is the proposed mitigations for high risk/high frequency tasks. According to the FDA, all proposed mitigations must be related to the device design since Instructions for Use (IFU), User Training, and Product Information Leaflet (PIL) are not considered acceptable methods for risk mitigation. Once potential risk mitigations are identified, the trick is to deter- mine which potential mitigations, if any, will actually deliver the expected result of risk mitigation. This is where a Human Factors Formative Test can save the day.
Human Factors Activities: When to Start
Human Factors methods can be used to place the device in front of users early in the product development cycle to evaluate risk mitigation ideas – when it is relatively inexpensive to make design changes. Early formative tests evaluate ideas, designs, and prototypes. Getting a product in front of end users early in the design phase will quickly tell if proposed risk mitigations are effective. In addition, it gives the opportunity to make required changes to product designs early in the product development cycle. As a result, the cost of design change is minimized.The first step in human factors testing is to identify all user groups, realizing that a user and a customer are often quite different. Customers are people who will buy a product and are the focus of marketing departments. Users are people who will personally use the product. There is often more than one user group for a product. A ventilator is a perfect example of a device, which has multiple different user groups: the technician who calibrates the ventilator, the respiratory therapist, the nurse, the caregiver, and the physician. Consequently, all user groups and how each user group will use the product must be identified. The FDA is interested in all user groups. However, they have a particular interest in the groups that are most at risk (e.g., children, untrained caregivers, home health aides). It is also advisable to include a mix of experienced/inexperienced users. Remember, usability testing is placing a product in the hands of actual end users to determine if the product design is safe, effective, and usable.
Formative Tests are evaluative in nature and place product concepts, and/or 2-D/3-D prototypes in the hands of users. When conducting a Formative Test, the goal is to use five to eight users from the high-risk user groups focusing on the highest-risk/highest frequency task(s). For a Formative Test, you do not need five to eight of each user group – you only need five to eight users from each user group that performs the highest risk/highest frequency tasks. Five to eight participants maximize the return on investment during formative testing as using additional participants typically don’t yield substantially more data (Nielsen, 2012). A Formative Test can be designed to focus on a specific product design and/or specific primary operating task(s) to gather data on how a user group interacts with the device. Consequently, a Formative Test could use a few as 5 participants or as many as five times the number of user groups that could ultimately use the product/device. Manufacturers often failto include the Caregiver User Group. However, the FDA considers caregivers to be one of the highest risk user groups, as they have no professional training. For this reason, Caregivers need to be considered, even if they just monitor the device output.Formative Testing usually takes two weeks of preparation, two or more days of testing, and two weeks to obtain the data analysis and final report. That amounts to about five weeks (or more). Cost is a little more difficult to estimate as it depends on a number of different variables. If your user groups are medical professionals, it will cost substantially more for participant recruiting and incentives than if your user groups are caregivers or patients. There are also costs for the Human Factors Lab Facilities, creation of the test protocol, facilitation of the test, and the data analysis/report writing. Most Formative Tests cost between $30,000 and $75,000+ depending on the number and type of user groups.
The FDA has changed its position on Formative Human Factors testing in all their recent guidances. In addition, they want a Human Factors Summary Report at submission that includes a summary of Formative Human Factors input and results. Although it is not explicitly required, it is a red flag when the Human Factors Summary Report has no formative testing results reported. Remember, the goal is to plan to release a product that is safe, effective, usable, and approved by the FDA in the shortest time possible. Finding out about serious design problems with the device early in the development cycle will cost substantially less to mitigate, than finding the same design problems at the end of the product development cycle.
Integrating Human Factors into the Product Lifecycle. The earlier manufacturers integrate human factors research into the design process, the more cost effective it will be.
Typically, a product development cycle will include two or three rounds of Formative Human Factors Testing before the final Summative/Validation Test. Follow this practice and there should be no surprises in the Summative Test.
Summative Testing is the final validation that a product design is safe, effective, and usable in the hands of ALL intended user groups. It is the final test that renders a pass/fail judgment on a device, product or system risk mitigations. Again, the basis for the Summative Test is the Human Factors Risk Analysis created by the Quality Assurance Team. All medium- and high-risk tasks identified in the test will be handled by various personnel. For this reason, Caregivers need to be considered, even if they just monitor the device output. The risk analysis will be tested in a simulated real-world environment/ situation. Test participants for the Summative Test, will be at least 15 users for each user group that could typically use the product. Consequently, the number of participants will range from 15 to many. It is quite clear the cost of a Summative Human Factors Test will be considerably more than a Formative Human Factors/Usability Test. Unfortunately, it is impossible to provide an estimated cost as the cost depends on many factors. For example, the number of user groups tested, the complexity of the device, whether or not training required, the number of days in the lab, the complexity of the final analysis, etc. With that said, the minimum cost for a Summative Test is about roughly $70,000, but more likely in the $90,000 to $150,000+ range.
Because you did early planning, early iterative testing (formative tests) followed by iterative device design changes, you have set yourself up for a successful Summative Test. However, if you decided to take the “wait and see approach”, the risk is extremely high that you will have to not only repeat your Summative Test, BUT make changes in a device that was supposed to be production ready. These costs do not only apply to the device design, but also to all the support materials, e.g., Information for Use, Training Protocol, marketing materials, etc. So, Proper Prior Planning Prevents Poor Performance. Be a PLANNER! You will then be creating safe, effective, usable and PROFITABLE products.
Author Bio: Virginia has extensive experience in the areas of product design, Customer Centered Participatory Design processes, and product lifecycle management. She is also known in the Human Computer Interaction and Usability Communities for her innovative techniques for collecting and incorporating user/customer feedback throughout the product lifecycle. In July 2014, Virginia founded and established the HirLan Institute of Human Factors with labs in Carlsbad, California and a Consortium arrangement in London, UK. In February 2013, Virginia founded HirLan International SA, a Geneva, Switzerland based company. Both HirLan and HirLan International have been supporting Top Ten pharma/medical device companies, as well as, start-ups with consulting and testing services. www.hirlan.comwww.medicaldevicehumanfactors.comCaptions