Automation and High-Throughput Technologies
Open Source Automation
There is an increasing interest in discoveries from images acquired by high-throughput and high content microscopy imaging of multi-well plates with biological specimens under a variety of conditions. As multi-dimensional automated imaging increases its throughput to thousands of images per hour, the computational infrastructure for handling the images has become a major bottleneck. The bottleneck associated challenges arise due to big image data, complex phenomena to model, non-trivial computational scalability that leverages advanced hardware and cutting-edge algorithms, and incompatible software tools that vary in the language they were written in, platform they were written for, and capabilities they were designed to execute.
To address the above challenges, groups have developed software solutions based on client-server systems with modern web technologies on the client side and a spectrum of databases, computational workflow engines, and communication protocols on the server side to hide the infrastructure complexity. However, these solutions have not focused on inter-operability of imaging specific computational plugins and visual exploratory capabilities of such plugins over very large image collections.
To address these inter-operability and visual exploration challenges, the National Institute of Standards and Technology (NIST) and the National Institutes of Health (NIH) - National Center for Advancing Translational Science (NCATS) have formed a close collaboration to develop an open source platform for executing web-based image processing pipelines over very large image collections with interoperable plugins. The plugins developed by both institutes are based on software containers as standardized units for server-side deployment, as well as on dynamically created web user interfaces (UI) to enter parameters needed for the software execution and for advanced visual data explorations on the client side. Each container packages code, with all its dependencies, and has an entry point for running the computation in any computing environment. Each UI description file contains metadata about the plugin container and the computation parameters.