New paradigm for macromolecular crystallography experiments at SSRL: automated crystal screening and remote data collection.

Complete automation of the macromolecular crystallography experiment has been achieved at SSRL through the combination of robust mechanized experimental hardware and a flexible control system with an intuitive user interface. These highly reliable systems have enabled crystallography experiments to be carried out from the researchers' home institutions and other remote locations while retaining complete control over even the most challenging systems. A breakthrough component of the system, the Stanford Auto-Mounter (SAM), has enabled the efficient mounting of cryocooled samples without human intervention. Taking advantage of this automation, researchers have successfully screened more than 200 000 samples to select the crystals with the best diffraction quality for data collection as well as to determine optimal crystallization and cryocooling conditions. These systems, which have been deployed on all SSRL macromolecular crystallography beamlines and several beamlines worldwide, are used by more than 80 research groups in remote locations, establishing a new paradigm for macromolecular crystallography experimentation.

Introduction

The Stanford Radiation Laboratory (SSRL) has a long history in the use of synchrotron radiation for macromolecular crystallography research. The first experiments demonstrating the utility of synchrotron radiation to examine diffraction from protein crystals (Phillips et al., 1979 ▶) as well as some of the early experiments showing the effectiveness of the multi-wavelength anomalous dispersion (MAD) technique for phasing, were performed at SSRL BL1-5 (Hendrickson et al., 1988 ▶). The field has continually evolved since these early days, taking advantage of technological advancements in electronics and computing. This trend is clearly evident when examining the progression of area-detector technology over the last 20 y. In the early 1990s, image-plate detectors replaced traditional film with digital images of the X-ray diffraction pattern. These detectors grew in size and increased in acquisition speed, evolving into the advanced CCD area detectors of today that are capable of producing a million pixel images in seconds. Other important advancements have been made in the areas of accelerator hardware and X-ray optics. Now third-generation sources such as the SPEAR3 lattice at SSRL produce intense high-brightness X-rays that are well suited to high-speed data collection from poorly diffracting samples. The intense X-ray beams at such sources, combined with fast detectors, allow high-quality diffraction images to be collected in seconds and complete data sets consisting of high-quality diffraction images to be collected in a few minutes (Walsh et al., 1999 ▶).

Despite these time savings, the demand for protein crystallography beam time on high-performance beamlines exceeded capacity as crystallographers attempted more challenging experiments, at times having to screen hundreds of samples to obtain a single crystal useful for data collection. Moreover, structural genomics consortia began to emerge. In particular, the Joint Center for Structural Genomics (JCSG) group required automated systems for their proposed ‘high-throughput structure-determination pipeline’ (Lesley et al., 2002 ▶). To meet these demands, in addition to building new experimental stations, the SSRL Macromolecular Crystallo­graphy Group concentrated on developments to increase experimental throughput and reliability. These include the mechanization of all experimental equipment, the powerful DCS control system and the user-friendly interface Blu-Ice (McPhillips et al., 2002 ▶; http://smb.slac.stanford.edu/public/research/developments/blu-ice/). In addition to reliable operation of motorized devices, the DCS control system enabled the automation of complex tasks (for example, the collection and processing of fluorescence data from heavy atoms).

With the time savings achieved from the additional automation provided by Blu-Ice/DCS, the manual mounting of samples on the goniometer became a significant bottleneck. The process was tedious, difficult for some researchers and prone to sample loss. Apprehensions of sample loss often led to the collection of data that were not necessarily from the best diffracting samples. To address these issues, the Macromolecular Crystallography Group and the Structure Determination Core of JCSG developed the Stanford Auto-Mounter (SAM; Cohen et al., 2002 ▶), a robotic system used to remotely mount pre-frozen samples on the goniometer. Different approaches to robotic development have been undertaken by numerous groups worldwide for this purpose (Muchmore et al., 2000 ▶; Karain et al., 2002 ▶; Shu et al., 2002 ▶; Snell et al., 2004 ▶; Pohl et al., 2004 ▶; Ohana et al., 2004 ▶; Ueno et al., 2004 ▶; Cipriani et al., 2006 ▶) and a current listing of robotic installations for macromolecular crystallography experiments at synchrotron sources can be found on the RoboSync website (http://smb.slac.stanford.edu/facilities/hardware/SAM/robosync/). At SSRL, the SAM system significantly increased the efficiency and reliability of sample screening and data collection.

As the number of macromolecular crystallography groups that came to depend on synchrotron radiation for their research continued to grow, many scientists found that the time and money required to travel to the synchrotron to conduct the experiment had become a major expense. To address this, many synchrotrons began offering researchers an option for conducting the experiment using ‘Service’ crystallo­graphy, also known as ‘FedEx and ‘Mail-In’ crystallo­graphy (Robinson et al., 2006 ▶). In this case, samples are shipped to the synchrotron and diffraction data are collected by the local synchrotron staff with minimal input from the researchers. In 2004, SSRL began to offer ‘Remote Access’, an alternative approach to the ‘Service’ model. Remote Access is a mode of data collection whereby researchers can take advantage of the extensive automation of the SSRL facilities and conduct experiments from remote locations using the standard beamline interface Blu-Ice and the newer browser-based interface Web-Ice (González et al., 2008 ▶). In this mode, decision making, strategy determination, control of the experiment and monitoring of the beamline is carried out by the researcher and his or her collaborators, all in potentially different locations. This paper describes the advancements in hardware automation and control software which have made Remote Access possible and the impact that these developments have had on macromolecular crystallography experiments and structural biology at large.

The macromolecular crystallography facilities

SSRL is a National User Facility which provides extremely brilliant X-ray photon beams for use in materials science, environmental science and structural biology research. Several structural biology beamlines (BL1-5, BL7-1, BL9-1, BL9-2, BL11-1 and BL12-2) are dedicated to macromolecular crystallography experiments. The SSRL Macromolecular Crystallography Group provides operational, scientific and technical support for monochromatic, multi-wavelength, anomalous diffraction and ultrahigh-resolution experiments (http://smb.slac.stanford.edu/). These state-of-the-art-facilities employ large active-area (>300 mm) rapid-readout X-ray detectors (∼1 s) and cover the energy range 6–20 keV. The end stations (BL1-5, BL9-2 and BL12-2) employ double-crystal monochromators with an energy resolution of 0.02% and the side stations (BL7-1, BL9-1, BL11-1) employ side-scattering monochromators with an effective energy resolution of ∼0.04% near the Se absorption edge (12.6 keV).

Computer-system infrastructure

Instrument control, datafile management and data processing are handled using a powerful and highly organized computational infrastructure. A centralized high-performance storage and computational system is comprised of individual machines connected through a high-speed network. Multiple RAID systems, currently totalling 100 TB, are part of a Storage Area Network (SAN) that utilizes a clustered file system, allowing post-processing up to six months after the experiment has been completed. Dedicated high-performance computers are assigned different tasks, isolating each critical computational function from the loading effects of the others. Individual tasks include beamline control, equipment automation, data transfer, real-time data processing, web applications and remote terminal access. The centralized computational system allows an individual account and single-password architecture, providing simultaneous file protection and ease of use.

Automated hardware

While the X-ray optical arrangement varies, the hardware in the experimental hutches and the control software at the macromolecular crystallography beamlines are standardized. Standardization yields a number of advantages: the training of scientific and technical support staff is simplified, a common pool of spare equipment is affordable and easily maintained, and new equipment and automation routines developed on one beamline can be implemented immediately on all other beamlines. Standardization has also made it straightforward to train new scientists and research groups are able to easily operate several beamlines simultaneously.

All hardware components that require movement at the beamline (i.e. optics, apertures, positioners, goniometer etc.) have been mechanized and all motors and other devices (i.e. video cameras, lighting, cryocooler, detectors, ion chambers etc.) are controlled or monitored remotely either directly or indirectly through the control software package DCS/Blu-Ice. The standard equipment in the experimental hutch (Fig. 1 ▶) includes the following: an adjustable experimental table that incorporates a pitch and yaw motion, a beam-conditioning system, a Huber kappa goniostat employing a sample xyz positioner, a fluorescence detector on a translation stage, a cryogenic cold stream incorporating an automated sample-annealing system, the Stanford Auto-Mounter (SAM) for mounting pre-cooled samples onto the goniometer, a large-area CCD X-ray detector [either an ADSC Q315R (Area Detector Systems Corp.; http://www.adsc-xray.com/) or a Rayonix MAR Mosaic325 (http://www.mar-usa.com/)] mounted on a custom xyz positioner, a video-monitoring system and sample lights with adjustable brightness. Detailed descriptions of automated hardware specifically developed in-house are described below.

Figure 1

(a) Schematic of the standard experimental hardware in the experimental hutches. (b) An expanded view of the vicinity around the sample position. (c) An expanded view of the robot dispensing Dewar. All critical components are motorized and remotely controlled.

The beam-conditioning system

The beam-conditioning system is used to define the size and intensity of the X-ray beam. The system includes the following: ionization chambers, adjustable collimating slits, a high-speed shutter, a scatter-guard shield and a motorized beam stop with an embedded X-ray diode sensor (Ellis et al., 2003 ▶). The variable beam size enables matching of the incident beam and sample size, maximizing the signal-to-noise of the diffraction data. A small beam can also be used to expose a portion of the crystal that might be of higher quality. Attenuation is used to prevent overexposure of the samples or to protect the fluorescence detector from saturation during an absorption-scan experiment. The beam stop protects the detector from the incident beam and the embedded sensor is used to accurately align the beam stop to the X-ray beam. The sensor can also be used to verify that the beam stop is intercepting the direct beam, adding additional protection for the detector.

The Stanford Auto-Mounter (SAM)

Automated mounting and dismounting of cryocooled crystals on the beamline goniometer has been a standard feature on the SSRL beamlines since 2003 and was a key step in the development of automated sample screening and remote data collection. The SAM system was developed jointly by the SSRL Macromolecular Crystallography group and the Structure Determination Core of JCSG and is based on a small industrial robot and high-capacity compact cylindrical cassettes, each holding up to 96 crystals mounted on Hampton Research-style sample pins. A cassette toolkit was developed for loading protein samples into cassettes at the researcher’s remote laboratories. Designed for easy shipping and storage, the cassettes fit inside several commercial dry-shipping and long-term storage Dewars. A dispensing Dewar adjacent to the beamline goniometer holds up to three cassettes sub­merged in liquid nitrogen (see Fig. 1 ▶). This enables up to 288 frozen samples to be mounted and screened without opening the experimental hutch door. The SAM system is also compatible with the uni-puck sample container employed by many synchrotron auto-mounting systems. The robot uses a permanent magnet tool to extract samples from and insert samples into the cassette or puck and a cryo-tong tool is used to transfer frozen samples to and from the beamline gonio­meter. The cryo-tong is dried between each mount and dismount in a specialized heating unit that surrounds the cryo-tong with dry warm air. The SAM system has also been installed or is in the process of being installed at several synchrotron beamlines worldwide, including the Advanced Light Source (12.3.1), Australian Synchrotron (BL1), Canadian Light Source (CMCF-1), National Synchrotron Radiation Research Center (BL13B1 and BL13C1) and the Photon Factory (BL-5A, BL-17A and AR-NW12A).

Video and lighting

A sample camera comprised of an Optronics color CCD and a Navitar motorized zoom lens provides an overview and a zoomed-in high-resolution view of the sample mounted on the goniometer. To illuminate the sample, a custom backlight comprised of a 9 × 9 array of ultrabright LEDs can be remotely switched on or off and the intensity of a Fostec optical fiber side light can be adjusted remotely. The side light is beneficial for viewing the details of the crystal mounted inside a cryo-loop. The backlight is optimal for automated centering of the cryo-loop, which requires high contrast between the cryo-loop and the surrounding background.

Two pan–tilt–zoom cameras are used to monitor the experimental equipment. A camera inside the experimental hutch provides a view of all the hardware and a second camera outside the hutch provides views of the electronics racks and beamline-control consoles. These cameras provide researchers and support staff with the means to remotely view the complete experimental environment though the Blu-Ice interface or from a standard web browser using the newly developed Web-Ice interface. For example, the SAM robot can be monitored while a sample is being mounted on the gonio­meter.

A dedicated Axis video server at each beamline encodes the video-camera feeds and generates a motion JPEG stream for each video channel. In order to prevent overloading of the Axis server with video-stream requests, an SSRL-developed video-server application acts as a proxy and collects the Axis video streams from each beamline and fans out the JPEGS to each of the Blu-Ice and Web-Ice clients as needed. The video server can also digitally filter the JPEG streams to improve the visualization of the sample. Typical frame rates in Blu-Ice range from one to five images per second.

Video streams from all of the available cameras are displayed simultaneously on the same page in Web-Ice. Single video streams may be viewed in a separate window at higher resolution. Video streams and snapshots may also be saved from this window. Users present at the beamline have the option to block the video signals from being displayed in the GUIs.

The Distributed Control System (DCS)

DCS is an instrument-control and data-acquisition package that provides unified control over the hardware resources at a macromolecular crystallography beamline. DCS controls all the SSRL macromolecular crystallography beamlines and is used on beamlines BL9-3, BL4-2 and BL11-3 to support single-crystal X-ray absorption spectroscopy, small-angle scattering and material scattering experiments, respectively. The Blu-Ice/DCS software is open source, free for download and can be customized readily by other synchrotrons (McPhillips et al., 2002 ▶).

The DCS architecture distributes the functions of the control software into three main tiers that communicate over a network using a lightweight asynchronous message protocol (Fig. 2 ▶). The three tiers provide users with a robust, secure and standard interface to each beamline. The first tier of DCS consists of multiple clients (the Blu-Ice GUI or the web-based interface Web-Ice) that provide a simple and intuitive interface to configure, initiate and monitor crystallography experiments. The Blu-Ice or Web-Ice clients connect through the network to the second tier, the Distributed Control System Server (DCSS). DCSS is responsible for executing and managing the automation of the experiment, keeping all of the clients up to date on the status of the experiment and routing commands to the appropriate hardware components in the third tier. The third tier provides DCSS with a consistent interface to the various hardware components of the beamline and allows DCSS to control the crystallography experiment regardless of the hardware implementation. Control over each hardware component is provided by individually tailored programs known as Distributed Hardware Servers (DHS). DHS programs, free to run on any network-enabled computer, are typically developed on the operating system required by the hardware’s API. The DHS programs are not limited to direct control of hardware, but can also act as a gateway to different control systems, such as ICS (another SSRL control system) or EPICS. Recently, the EPICS gateway was deployed as part of the Blu-Ice installation at the Australian Synchrotron (BL1), which uses EPICS for all beamline-motion control.

Figure 2

The Distributed Control System (DCS) three-tier message-passing architecture. The DCS server (DCSS) communicates with the GUI and the Hardware layers via TCP/IP on a gigabit network. This architecture enables multiple GUI connections to DCSS and allows DCSS to run data collection or crystal screening decoupled from the Blu-Ice user interface, increasing the uptime and efficiency of the beamline. Hardware running on potentially different computing platforms and control systems ‘plug in’ at the Hardware layer. These services are typically protected on a private network.

The three-tier architecture also increases the reliability and security of the remote-access experiment. Data collection and other automated processes are not dependent on the stability of the interface layer, but are instead managed by the DCSS program, which executes continuously on a dedicated machine. This architecture allows Blu-Ice or Web-Ice to be closed or disconnected from the network without interrupting the experiment. DCSS also stores the state of the beamline and authenticates user access. All requests for hardware control first passes through the DCSS program, which runs on a special multi-homed machine with access to both the public and private network.

The Blu-Ice experimental interface

The Blu-Ice user interface provides beamline experimenters and support staff with unified control over all hardware and instrumentation at a particular beamline. The interface remains fully synchronized with the current positions of the beamline motors, the state of the experimental equipment and the latest readings from the relevant detectors.

Each Blu-Ice instance is an independent client of the DCSS control system and a user can open several Blu-Ice windows at any one time on the same desktop or at multiple locations as needed. Once a task is initiated from Blu-Ice, the program may be closed and DCSS will continue the task until it is finished. This is an important feature for remote access, as the continuation of long-lasting experiments must not depend heavily on the reliability of network connections outside of SSRL.

Although the experiment may be monitored by all running Blu-Ice processes, only one instance of Blu-Ice has full control of the beamline at any given time and the interface is partially disabled until control is acquired with a simple click on the status bar. When control of the beamline changes hands, all Blu-Ice clients are informed. This system works extremely well in a collaborative environment.

Blu-Ice divides the layout of its tools with a tabbed-notebook interface (Fig. 3 ▶). A status bar along the bottom of the Blu-Ice window remains visible for all tab selections and indicates the energy of the X-ray beam, the synchrotron ring current, whether the shutter is open or closed and the status of any active experiment. The first three tabs, Hutch, Sample and Collect, guide the researcher from left to right in an order that closely matches the manual steps required to perform a diffraction experiment. The Hutch tab (shown in Fig. 3 ▶) orients the remote user to the physical layout of the hutch equipment with tools for motor control overlaid on a graphical representation of the critical instrumentation. The user can move the experimental hardware (such as the X-­ray energy, beam attenuation, beam size, detector distance etc.) within this tab and can view the physical motion of the equipment with live streaming video provided by the cameras in the hutch. A video stream of the sample is also available and as an alternative to automated loop centering, the user can align the crystal to the beam center by clicking directly on the video image. The Sample tab provides a larger view of the sample and displays additional controls for the SAM robot. Samples can be mounted (or dismounted) in this tab by selecting a port from a two-dimensional representation of each SAM cassette or uni-puck followed by a click on a button for mounting or dismounting. From the Collect tab (shown in Fig. 4 ▶), multiple monochromatic or MAD data-collection runs can be set up and executed. The diffraction images are displayed as they are collected and can be magnified to observe individual diffraction spots.

Figure 3

The Hutch tab in the tab-based experimental interface Blu-Ice. Researchers can set experimental parameters and align samples using this intuitive interface. The diffraction resolution of the experimental equipment is updated as the parameters are entered. Several video streams of views inside and outside the experimental hutch are available for real-time monitoring. The bottom status bar is displayed on all tabs and includes system messages, the accelerator current, control status, shutter status and a digital clock.

Figure 4

The Collect tab in the Blu-Ice interface. Multiple monochromatic or MAD data-collection runs are set up and executed in this tab. The image file names that will be generated based on the input parameters are displayed in a list located in the center of the window. Dose control provides a constant X-ray flux on the sample compensating for the SPEAR current decay. Data collection can be interrupted by clicking on the ‘pause’ button and the diffraction images are displayed as they are collected.

Additional tabs provide control for automated crystal screening, absorption-edge scanning and experimental monitoring. The Screen tab takes advantage of the SAM system for the automated screening of a large number of samples. An Excel spreadsheet con­taining information about the samples can be uploaded and displayed in this tab. The Scan tab (shown in Fig. 5 ▶) is used to collect fluorescence data from heavy-atom scatterers that may be present in the protein sample. The User tab lists the names of the user accounts that are connected to the beamline and, if known, their physical locations. It also displays the current user in control of the beamline and a verbose log file recording every step of the experiment.

Figure 5

The Blu-Ice Scan tab. Users select an absorption edge to scan using the periodic table graphic. A complete absorption scan is recorded and analyzed automatically, identifying optimized energies for a multi-wavelength anomalous dispersion (MAD) experiment.

The Setup tab is accessible only to staff for configuring, aligning and maintaining the beamline. It incorporates graphical representations of the optics and control widgets for all low-level components. Numerous controls are available for configuring and monitoring various hardware devices such as the cryogenic cooler, ion-chamber amplifier, annealer, motors etc. The tab includes a general diagnostic ‘scan’ tool which will step the position of one or two motorized devices along a defined path and plot multiple signals as a function of position.

The style of the Blu-Ice interface has been adopted at several macromolecular crystallography beamlines worldwide, including the Advanced Light Source (4.2.2, 8.3.1 and 12.3.1), Australian Synchrotron (BL1), Canadian Light Source (CMCF-1), Advanced Photon Source (GM/CA-CAT and NE-CAT), Brazilian National Light Source (MX1) and the National Synchrotron Radiation Research Center (BL13B1 and BL13C1).

The Crystal Analysis Server

The Crystal Analysis Server is a standalone web service responsible for performing diffraction data analysis for other applications. It can route indexing and integration jobs to high-performance Linux machines, monitor these jobs, store the resulting raw files in the user’s directory and summarize the results in the user’s screening spreadsheet. DCSS uses the Crystal Analysis Server during automated pro­cesses and Web-Ice provides an intuitive interface to the Crystal Analysis Server as an alternative to dealing with the com­plexities of the various underlying crystallography pro­grams.

The Crystal Analysis Server provides a standard interface to a number of data-analysis packages: LABELIT (Sauter et al., 2004 ▶), which incorporates a special version of MOSFLM, is used to index the diffraction images, DISTL (Zhang et al., 2006 ▶) assesses features of the diffraction images such as spot shape, number of ice rings and diffraction resolution, BEST (Popov & Bourenkov, 2003 ▶) calculates optimal data-acquisition strategies and RADDOSE (Murray et al., 2004 ▶) estimates radiation damage to the sample.

Diffraction analysis of crystal screening results is carried out in real time and requires no manual input by the user. The software obtains experimental information from DCSS and from the diffraction image-header files. For each sample successfully auto-indexed, the results are written to the Screen tab spreadsheet.

The Web-Ice browser interface

Web-Ice is an application developed at SSRL that capitalizes on the web browser’s inherent ability to display complex data in great detail, without the installation of any additional software, from practically any computer in the world (González et al., 2008 ▶). Straightforward navigation of web pages simplifies the organization and management of the data-analysis results. Web-Ice does not depend upon a database backend, but instead interacts with the SSRL file system. Execution of data-analysis pro­grams occurs within the user’s directory and intermediate output and log files can be viewed and analyzed manually from the shell prompt of SSRL computers.

Unlike Blu-Ice, which is closely coupled to control of a beamline, Web-Ice is accessible whether or not a user currently has beam time. Without beam time, the user is able to view screening results, analyze diffraction images, rank crystal samples according to diffraction quality and determine optimal data-collection strategies.

With beam time, additional features become available. Web-Ice can index samples, collect MAD scans for a complete strategy determination and initiate data collection after the strategy has been calculated. Data-collection parameters generated by Web-Ice appear in Blu-Ice, allowing researchers to freely switch between Web-Ice and Blu-Ice as needed.

Automated experimental procedures

The combination of reliable computer-controlled hardware and a robust and flexible control system has enabled the automation of complex experimental processes. Several standard automated processes available in the control software are described below.

MAD scans

A complex process can be made easy with beamline automation, as demonstrated by the simplicity of the anomalous experiment at SSRL. Firstly, a user defines a MAD or SAD experiment by selecting the absorption edge from a periodic table on the Blu-Ice Scan tab (Fig. 5 ▶). The user starts the scan with the click of a button and DCSS (i.e. the control software) optimizes the position of the fluorescence detector and searches for an appropriate beam attenuation to avoid detector saturation. The software then initiates a scan of the fluorescence data around the absorption edge, displaying the data in the Scan tab as it is collected. Once complete, DCSS analyzes the data using AUTOCHOOCH (Evans & Pettifer, 2001 ▶) a derivative of the CHOOCH program. AUTOCHOOCH determines the maximum f′′ (‘peak’) and minimum f′ energies in the scan and DCSS, taking into account the beamline energy range (González, 2003 ▶), determines a suitable remote energy for the experiment. Finally, the control software calls the RADDOSE program to estimate the dose to be received by the crystal at the energies determined by AUTOCHOOCH. If the predicted dose exceeds a predefined limit, the software will default to the use of only two wavelengths. Once presented with the recommended energies, the user may override the values or transfer them to the Collect tab with the click of a button.

Changes of energy, as required by the MAD experiment, occur automatically during data collection. At side-station beamlines equipped with single-crystal side-scattering monochromators, the beam is deflected horizontally with changes in energy. The experimental table tracks this motion, moving over 1000 kg of equipment (including the final beam-conditioning system, goniometer and detector) a metre across the experimental hutch. After large energy moves and before data collection, a short scan of the table position is performed to fine tune the beam in the vertical direction. These automated changes of energy make it feasible to collect MAD data in energy wedges, which reduces the effects of radiation damage.

Excitation scans

From the Scan tab in Blu-Ice, the user can perform an excitation scan to identify the heavy elements that may be present in a mounted sample. Following a procedure similar to the MAD scan, the user first selects an excitation energy by clicking on an element in the periodic table. The system moves to the selected energy range, optimizes the fluorescence signal to avoid saturation and acquires the resulting spectrum. Blu-Ice displays the spectrum and searches the peaks for the emission energies of stable elements that have absorption edges below the excitation energy. The emission energies of probable elements are overlaid on the spectrum and the user can add pointers to visually verify additional emission energies for elements that they expect to find in the sample.

Screening samples

With the Screening tab in Blu-Ice, the user can set up, initiate and monitor an automated search for their best diffracting sample. To perform this experiment, the user first uploads an Excel spreadsheet describing the contents of the users’ cassette (or uni-puck). Once the cassette contents are available to Blu-Ice, samples can be selected for screening and SAM automatically mounts the selected samples in succession. After each sample has been mounted, the software controls the sample lighting, analyzes the sample video and automatically aligns the loop to the X-ray beam (Miller et al., 2004 ▶). If the user selects the default screening parameters, the system will record video images and collect diffraction patterns of the sample 90° apart. These diffraction images are then automatically indexed and analyzed using the Crystal Analysis Server. For each sample, the software records a diffraction quality score in the user’s spreadsheet that is based on the mosaicity, resolution and r.m.s. residual of the analyzed images. After screening, the user can sort and rank the screened samples based on these results. The sample-to-sample cycle time for screening crystals is currently 3.5 min and a 96-sample cassette can be screened in less than 6 h.

Data collection

The Collect tab provides full control over the collection of diffraction data from the sample currently mounted on the goniometer (Fig. 4 ▶). Multiple runs can be defined for a single sample and run definitions can easily be copied and modified from previous runs. If the sample has been screened automatically, a strategy determined by the Crystal Analysis Server may be imported through Web-Ice. As the data-collection parameters are entered into the fields, the names of image files to be created are listed in order for review along with their associated energy and ϕ values. Additionally, this list can be used to select a starting point for data collection, a useful feature for manually recollecting data that may have been interrupted mid-exposure. Once data collection has been initiated, all defined runs are executed in succession and files and directories are automatically created based on the root names defined.

The Collect tab displays the diffraction images as they are collected. In order to prevent each full image from being transferred to every monitoring Blu-Ice client, an independent program known as the Diffraction Image Server loads each diffraction image from disk only once and sends JPEG snapshots to each of the Blu-Ice clients as needed. Using this system, each Blu-Ice client can be used to pan and zoom the view of the diffraction images without loading the full image into memory. Alternatively, Web-Ice can be used to view the images or the graphics program ADXV (Arvai, 2008 ▶) can be launched to use additional visualization tools, such as a three-dimensional rendering of the diffraction peaks.

DCSS oversees all of the required beamline components required for data collection, including (but not limited to) the experimental table, monochromator, X-ray shutter, gonio­meter and X-ray detector. Data collection involves many subtasks, which are either controlled by DCSS directly or delegated to a DHS program responsible for the task. These tasks require no human intervention and greatly simplify and standardize the experiment for users. Optimizing the beam intensity, exposing the sample with several passes of ϕ and changing energy and tracking all related beam motion are standard features on all beamlines. Table optimizations are automatically performed at predefined intervals (typically every hour) by DCSS between exposures. For exposure times exceeding ∼60 s, multiple ϕ oscillations are performed to average instabilities of the incident beam or detector and if dose mode is selected by the user the incident beam intensity is monitored and the exposure time is normalized to provide a constant X-ray flux. Data collection is paused if the beam is lost in the hutch for any reason (e.g. a refill of the storage ring) and automatically resumes when the beam is restored. If the beam loss occurs during an exposure, the image is automatically recollected.

Diffraction-based crystal alignment

Although samples can be automatically screened for diffraction by simply centering on the loop and using a relatively large beam size, a conventional data-collection experiment requires accurate alignment of the crystal to the X-ray beam. An automated crystal-alignment procedure based on diffraction is available in the Collect tab (Song et al., 2007 ▶). The procedure begins with the automated loop-centering routine (described above) to determine the dimensions of the loop and sample volume, followed by the collection of diffraction images generated with low-flux X-rays in a grid pattern over the edge and face planes of the loop. A modified version of Spotfinder (Zhang et al., 2006 ▶) running on the Crystal Analysis Server outputs the number of diffraction spots in the image. A weighted average of the number of spots is used to determine the ‘center’ of the crystal. The calculated center of the crystal is then aligned to the X-ray beam. Typical samples can be aligned in ∼2–3 min. Because the procedure is based on maximizing the number of ‘good’ spots as determined by the program Spotfinder, the best diffracting part of the crystal is normally aligned to the X-ray beam.

Crystal washing

Ice can sometimes accumulate on the exterior of the sample during the freezing stage or during shipment, giving rise to unwanted ice diffraction. A routine is available on the Sample tab for washing the external ice from the sample. For this operation, the SAM system is used to remove the sample and return it to inside the SAM dispensing Dewar. The sample is placed on the magnetic post and the robot is used to move the sample through the liquid nitrogen in a predefined ‘washing’ motion. For the majority of cases where external ice is present, washing the sample in this manner removes all ice from the sample and ice diffraction from the images.

Crystal annealing

Annealing or temperature cycling can improve the diffraction quality of some crystal systems. Significant improvements in mosaicity and/or diffraction resolution have been reported by several SSRL user groups. There are two methods to anneal the crystal from the Sample tab in Blu-Ice. The first method is based on software developed for Blu-Ice at beamline 8.3.1 at the Advanced Light Source (Holton, 2006 ▶). The software controls the nitrogen cold-stream flow of the cryocooling unit and turns it off for a brief period as specified by the user. During the entire process, the shield stream of dry nitrogen continues to flow around the sample, protecting it from water condensation. The second option physically blocks the cold stream so that rapid annealing of the sample can be accomplished.

Robot-component calibration

A multi-axis force sensor attached above the robot’s cryo-tongs is used by the robot-control software to automatically calibrate the hardware and perform run-time calibration checks. This automated calibration capability significantly reduces the staff time required to support the robot and enables error-free operation. Forces are measured by contacting the critical components (magnet tool post, cassettes and goniometer) with the magnet tool held in the cryo-tongs. The positions of these components are measured to within 15 µm. Several thousand measurements are made for each calibration point and outliers are excluded to achieve this resolution. The calibration of the magnet tool post takes 15 min, calibration of each cassette location requires 10 min and calibration of the goniometer takes 5 min to complete. These calibration procedures can be run individually or the entire process may be completed in 50 min with a single click in Blu-Ice. The calibration routine is performed on all SAM systems every two weeks. During normal robot operation, the forces on the cryo-tong are also monitored to ensure that the system remains within normal calibration tolerances.

Sample-pin probing

A staff-operated feature automatically probes a cassette with the force sensor (described above) prior to sample screening to detect pins that may be tilted, icy or otherwise loaded improperly into the cassette or uni-puck. Pins that are associated with a high force measurement are color-coded in the GUI and if the force exceeds a predetermined threshold the option to mount those particular pins is disabled. Furthermore, the robot software can detect and remember whenever the dispensing Dewar lid has been opened by hand and upon resuming normal operation will first determine whether cassettes or pucks are present and seated correctly within the Dewar.

Sample sorting

The SAM system has been programmed to sort samples between cassettes and/or uni-pucks using an intuitive interface. This option allows researchers to consolidate and arrange crystals that have been screened and ranked into a single container or to interchange samples between cassettes and uni-pucks in preparation for a future synchrotron run.

The remote-access interface

Once the experimental procedures had been fully automated, remote control of the beamline became possible. Since June 2005, researchers have had access to all of the tools described above to conduct experiments from their home institutions and other remote locations with the full capability to mount, center and screen samples and to collect and analyze diffraction data. The GUIs and computational resources at SSRL are accessed through a remote X11 session using the NX application provided by NoMachine (http://www.nomachine.com/). The NX protocol addresses the latency and bandwidth problems associated with remote X sessions by reducing round trips and using differential compression of the core X protocol. The result is a remote session that has a typical response close to that obtained at the beamline when a standard broadband connection is used. Underlying applications (such as DCSS, Blu-Ice and MOSFLM) run locally on SSRL machines. The server, which runs on a dedicated Linux machine at SSRL, is accessible to remote users through NX Client, a free application which can be easily and quickly installed on a laboratory or home computer. NX Client is available for Windows, Linux, Macintosh and Solaris operating systems. Once installed, NX Client has access to a complete Linux desktop that mimics the local beamline desktop environment. Fig. 6 ▶ shows the NX Client window running on the Windows operating system. This system enables the user to run all command-line and X-­window-based applications available at the beamline, including the Blu-Ice control software, ADXV image display and all the standard data-processing programs using minimal CPU and bandwidth resources on the user’s computer.

Figure 6

View of NX Client running on a Windows operating system. The user is presented with a beamline Linux desktop within a standard window. Blu-Ice and other applications (such as MOSFLM) are executed remotely through this interface exactly as if the user was at the beamline. NX Client also runs on the Mac and Linux operating systems.

Data-backup and archiving service

A data-backup service utilizing robotic DVD burners is available to researchers through a web application. A web interface is used to conveniently drag and drop files or directories for archiving to DVDs. Once the DVDs have been generated, they are shipped via FedEx to the requester.

Impact on macromolecular crystallography experiments

Beamline automation has improved the efficiency of macromolecular crystallography facilities, generating more experimental beam time for researchers. The beamlines are straightforward to operate and experience fewer failures. The SAM system in particular has had a significant impact on the screening process and has enabled a remote-access mode of experimentation.

The increasing usage of the SAM robotic system is represented in Fig. 7 ▶(a). More than 200 000 crystals have been screened by researchers with only ∼15 samples lost owing to a SAM system failure. The automation has lead to a new paradigm for crystallography experiments: researchers are efficiently and safely screening all their samples before choosing the best quality crystal for data collection. This has had the effect of simultaneously increasing both the throughput and the quality of the data that is being collected on the SSRL beamlines. The SAM system is used routinely by more than 85% of the SSRL macromolecular crystallography community, including both academic and industrial researchers.

Figure 7

(a) Total number of samples screened each year with the Stanford Auto-Mounter (SAM) system since its release to general users in 2003. To date, over 200 000 samples have been screened by more than 80 research groups. (b) The percentage of user groups that collect data remotely each year since remote access was first offered to general users in 2005. To date, more than 75% of the SSRL user community collects data remotely.

Several remote-access workshops have been held locally and remotely where researchers have been taught how to properly prepare samples for shipping and how to conduct remote-screening and data-collection experiments. The remote host locations included the Hauptmann–Woodward Medical Institute in Buffalo, New York (August 2006) and the University of Melbourne in Australia (February 2007). During the workshop held in Australia, one of the participants screened crystals that had been previously shipped to SSRL, collected MAD data from a sample that had been optimized via the screening software and solved a novel protein structure using SSRL computational resources (Schmidberger et al., 2008 ▶). The NX Client interface has proven to be responsive and reliable and is used by staff to remotely support the users’ experiments. Today, more than 75% of the SSRL macromolecular crystallography user community are carrying out experiments remotely (see Fig. 7 ▶ b). The remote-access capability has led directly to cost savings, reduced travel time and convenience for researchers. Moreover, young scientists who would not normally have the opportunity to travel to the synchrotron are being trained in their home laboratories and are participating in real-time experiments.

The remote-access capability has also enabled efficient use of unscheduled beam time. Select groups have the option to ship samples to SSRL at any time and their samples are entered into a ‘queue’ until beam time becomes available owing to a cancellation or when a research group finishes an experiment early. Researchers are contacted via telephone or e-mail and the remote experiment is under way once the ‘queued’ sample cassettes have been mounted on the beamline and access has been granted in the control software.

Current developments include the implementation of remote training tools such as on-line screen-capture video demonstrations (http://smb.slac.stanford.edu/users_guide/tutorials/) live remote-beamline training (Warren et al., 2008 ▶), utilization of NoMachine’s desktop-sharing tools (http://www.nomachine.com/) and Blu-Ice beamline ‘simulation’ software that will enable scientists to learn how to successfully use the SSRL facilities without having to physically travel to the SSRL site. Once implemented, all geographical barriers to using the facility will be completely removed.

Challenging structures

Many cutting-edge structural studies, such as membrane proteins and multi-protein complexes, require screening hundreds or even thousands of samples. These projects traditionally require many trips to the synchrotron and the physical endurance and dexterity to manually screen throughout the full shift of beam time. The automated experimental facilities have had a significant impact on projects that require such intense screening.

One such example is the multi-protein complex RNA polymerase II. Roger Kornberg and his research group (Stanford University School of Medicine) have been investigating this molecular machine for more than 20 y. RNA polymerase is responsible for transcribing the DNA sequences that comprise genes into a message (m-RNA) that is then read by the ribosome to produce proteins. Transcription is the first step and a key control point in gene expression, underlying all aspects of cellular metabolism.

The structure determination of RNA polymerase II and its complexes with various transcription factors were significantly hampered owing in part to the intensive screening effort required to refine the crystallization and cryocooling conditions and furthermore to find sufficiently diffracting crystals (Cramer et al., 2000 ▶). Since 2001, the SAM system has enabled the efficient screening of thousands of polymerase samples and in one case was used to find a single crystal that diffracted to 2.3 Å resolution, 0.5 Å better than any polymerase crystal previously screened (Westover et al., 2004a ▶). The beamline automation has expedited the scientific results for visualizing the transcription complex and detailing the transcription mechanism (Bushnell et al., 2002 ▶, 2004 ▶; Bushnell & Kornberg, 2003 ▶; Westover et al., 2004a ▶,b ▶; Wang et al., 2006 ▶), demonstrating that these types of traditionally difficult studies have become more practical and can be successfully carried out using the SSRL automated facilities.

Pharmaceutical industries and structural genomics

The automation is also utilized by several industrial groups. In particular, pharmaceutical companies are using SSRL resources in their effort to discover and develop new drugs and therapeutics. Several pharmaceutical groups have recently screened more than 11 000 crystals using the SAM system and have solved more than 4 000 structures combined, directly aiding in their drug-discovery efforts.

Members of the Structure Determination Core for the Joint Center for Structural Genomics (JCSG) have used the SAM system to screen over 80 000 crystals and have solved more than 750 novel structures to date. A plethora of structural information has been made available to the scientific community through these structural results, highlighting 45 new protein folds and 50 novel structural features (http://www.jcsg.org/). JCSG is currently determining structures at a rate of 200 structures per year, which requires screening of more than ∼20 000 crystals each year. With minimal personnel, this rate can only be sustained and further increased by using beamline automation. Other high-throughput programs such as the Medical Structural Genomics of Pathogenic Protozoa (http://www.msgpp.org/index.shtml), the newly formed Center for High Throughput Structural Biology (CHTSB; http://www.chtsb.org/) and the High-Throughput Crystallization Service at the Hauptman–Woodward Medical Research Institute (HWI; http://www.hwi.buffalo.edu/High_Through/High_Through.html) have also benefited by using the automation, primarily in the remote-access mode. In 2006 and 2007, HWI researchers screened 2300 crystals, collected more than 200 data sets and solved 44 new structures.

Conclusions

At SSRL, we have automated the macromolecular crystallo­graphy beamlines and have demonstrated that researchers can conduct the experiment remotely. Currently, 75% of our researchers screen samples and collect data from offsite locations, trusting the hardware, software and robot, for their most challenging crystal systems. Additionally, the software architecture is ready to meet the needs of the evolving scientific community. In the coming years, we plan to develop new sample-queuing capabilities, deploy a new pixel-array detector and venture into combined methods, such as UV–Vis and X-ray absorption spectroscopy. These features, and more, will be developed within our current framework and will be made available to onsite and remote users.