Have you ever been involved in FAT testing?  I don’t mean the kind of test where you determine how big the bulge around your waist is, I mean Factory Acceptance Testing (FAT) of an automated system.  This test is a major project milestone in a laboratory automation project where an equipment and/or system integrator demonstrates that the system design and manufacturing meets the specifications originally set (in a Functional Requirements Document) by the system owner or project manager. A successful FAT is a green light to ship the system to the “users” facility and often triggers a milestone payment to the vendor.  Therefore, the process should be conducted in a formal and signed, documented manner.  The LabMan talked about this process to several experts on both the provider and customer side of the equation - Kevin F. Keras, the Business Unit Manager for the  Caliper Life Sciences Automation, Consulting, Engineering & Services group (A.C.E.S), Bryan Greenway from Treis Technologies, and Daniel Cinacola from the Research and Automation Technologies group at Amgen Inc. 

How is FAT a unique part of the project process?  The FAT is unique because it is frequently the first time that the eventual system owners have the chance to see and use the instrument/system in its completed and operational state.  Automated systems are by nature unique, and rarely are any two systems identical.  Technology providers use the FAT process as a tool to ensure that the product created is functionally identical to the concept proposed and quoted to the user.

Who should be present at FAT?  From the system purchasing side the team should consist at least of the project lead (the “automation champion” engineer or an automation-savvy scientist), an expert on the process the system is to perform (the “end user” or lead scientist), and perhaps an IT specialist if integration of the system into the corporate net will be involved.  Typically present from the provider side will be the sales lead, the project management lead, and the technical lead, with other technical resources on-call if needed. 

Is there a single critical factor that leads to a successful FAT?  Everyone agrees that a formal, mutually agreed upon FAT document developed well in advance of the FAT and based on honest and thorough communication is essential.  Management of expectations on both sides is key. 

What specific things should be evaluated during FAT?  First of all, it’s much easier to fix a problem on the vendor integration floor than it is in the customer’s lab, so look at functional, operational, logistical, environmental, and safety requirements.  Evaluate the physical configuration of the system, the throughput of specified assays, error recovery modes, system documentation and aesthetics (such as the condition of third party equipment, table tops…etc).   Evaluate the FAT procedure results carefully and confirm that the procedure that was agreed upon in the FAT is actually performed by the system.  Not everything in the system specification should be addressed at the FAT.  Many categories should have been addressed well before the FAT, such as electrical power requirements, required floor space, facility connections (air, water, gases, etc.), human factors, environmental factors, and shipping method.  Others may not be checked until installation, such as IT integration…although this should be thoroughly discussed early in the project, it may not be demonstrated until delivery. 

What should be the product of the FAT?  At the end of the FAT, there should be a document signed by both parties indicating either: a) the system has successfully passed all the agreed upon tests of the FAT; or b) some items failed the FAT, specification of the failure mode, and specification of the remediation plan for each failure/deficiency.  If the remediation plan cannot be fully addressed at the FAT, then a deadline for presenting this plan to the customer should be set.  If another FAT is required, this should be part of the remediation plan.  In the worst case, the remediation plan may include how the system will be corrected on site, after installation at the customer facility.

A successful FAT is a green light to ship the system to the customer’s site.  It may also trigger milestone payments.  The ultimate product of FAT is confidence that the system to be delivered meets expectations.  This confidence is built from a long process consisting of several major milestones, beginning with users developing a Functional Requirements Document (FRD), working with a technology provider to translate that FRD into agreed upon systems specifications and a proposed FAT protocol,  and finally the FAT. 

How much of the FAT should be repeated once the system is on-site (Site Acceptance Testing: SAT)?  The two vendors the LabMan consulted both agree the full FAT should be repeated at the user’s site.  Vendors should bring proper technical resources to deal with unexpected problems, but this is also the time to begin to let the users run the system.  This often uncovers some last remaining issues.  Users may want to see the system perform well over the course of several days, not just one.  It is also often the first time that actual chemistry or biology testing (CAT or BAT) can be run on the system and this must meet expectations if such specifications were placed in the original requirements. 

To read much more details about the FAT process, go to the LabAutopedia collection of FAT articles. 

Until Next Time,

Domo Arigato, Mr, Roboto!