Very-high-resolution time-lapse photography for plant and ecosystems research.

PREMISE OF THE STUDY: Traditional photography is a compromise between image detail and area covered. We report a new method for creating time-lapse sequences of very-high-resolution photographs to produce zoomable images that facilitate observation across a range of spatial and temporal scales. • METHODS AND
RESULTS: A robotic camera mount and software were used to capture images of the growth and movement in Brassica rapa every 15 s in the laboratory. The resultant time-lapse sequence (http://timemachine.gigapan.org/wiki/Plant_Growth) captures growth detail such as circumnutation. A modified, solar-powered system was deployed at a remote field site in southern Arizona. Images were collected every 2 h over a 3-mo period to capture the response of vegetation to monsoon season rainfall (http://timemachine.gigapan.org/wiki/Arizona_Grasslands). •
CONCLUSIONS: A technique for observing time sequences of both individual plant and ecosystem response at a range of spatial scales is available for use in the laboratory and in the field.

The information contained in traditional single-frame photographs is a compromise between image detail and the amount of area covered. Panoramic tripod mounts can assist with taking sequences of images that can be stitched to produce composite images. A variety of manual and robotic mounts are commercially available. Ongoing advances in technologies for capturing and viewing very-high-resolution images have greatly expanded the capacity to study a broad range of biotic and abiotic ecosystem processes across spatial scales (Sargent et al., 2010a; Brown et al., 2012). The GigaPan system (http://gigapansystems.com) consists of hardware to capture multiple images on a grid through controlled positioning, software to stitch the images and create very-high-resolution panoramas, and a website to facilitate viewing, searching, exploring, and annotating, and to encourage discussion. Very-high-resolution photography has proven valuable for the study of processes in extreme detail over space in a number of scientific disciplines (Sargent et al., 2010a) including geology, archaeology, biodiversity, glaciology, and rangeland ecosystem research (Nichols et al., 2009).

A logical extension of the ability to capture high-resolution spatial detail is to capture repeat images over time to develop time-lapse sequences. Repeat photography plays an important role in quantitatively assessing changes and processes across many areas of ecosystem research. Examples include landscape change detection and analyses (Turner et al., 2003; Webb et al., 2007; Villarreal et al., 2013) and phenological responses to abiotic drivers (Crimmins and Crimmins, 2008; Richardson et al., 2009; Kurc and Benton, 2010; Sonnentag et al., 2012). Time-lapse imagery is also an important tool in studies of plant behavior (Trewavas, 2009). Time-lapse at the level of a single plant has revealed searching behavior by parasitic plants (Runyon et al., 2006), solar tracking (Hangarter, 2000), and other plant responses to the environment. Typically, only one or a few plants can be captured with a single time-lapse camera at the desired level of detail, limiting sample size and typically confining observations to a laboratory setting. The advantage of imagery captured with the GigaPan system over traditional time-lapse photography is the large increase in resolution over a broader spatial scale, which enables the viewer to make observations both at the level of the individual plant and at the ecosystem level. This increased resolution allows a researcher to capture both variation within a population (Fig. 1) and interactions between the environment and the population within a single sequence, a technique not possible with traditional time-lapse photography.

Fig. 1.

Frames from a GigaPan time-lapse sequence of Wisconsin Fast Plants showing (A) the entire group of plants and (B) flowers on three plants. The user can zoom from one level of detail to another while the sequence is running. The full sequence is at http://timemachine.gigapan.org/wiki/Plant_Growth.

The goals of our project were to produce a very-high-resolution, zoomable, time-lapse video in the laboratory and subsequently to build a reliable, weatherized, solar-powered system for capturing images in the field. Here we describe example applications showing individual plant growth in a laboratory and landscape-scale vegetation change at a remote rangeland site. Very-high-resolution panoramic photography provides a low-cost (<$2000) method for collecting large amounts of data across a range of spatial and temporal scales.

METHODS AND RESULTS

Laboratory application

A GigaPan EPIC Pro robotic camera mount (GigaPan Systems, Portland, Oregon, USA) and a Canon PowerShot G10 camera (Canon USA, Melville, New York, USA) with a 58-mm telephoto lens were used to capture the growth and movement of Wisconsin Fast Plants, a quick-growing variety of Brassica rapa L. developed by Paul Williams at the University of Wisconsin–Madison. Although a wide range of cameras are compatible with the robotic mount, the Canon G10 was selected because it is a moderately priced camera that allows manual control of white balance, shutter aperture and speed, and focus. The general procedure for setting up the hardware was to level the robotic mount and set the field of view following the onscreen instructions that guide the user through aligning the top and the bottom of the LCD screen with a point in the field of view. The camera was then set to full zoom and the top left and bottom right corners of the overall image were selected by rotating the robotic mount. Overall image geometry, including the upper left and lower right corners of the scene to be captured and the position of a visually identifiable reference point, was stored in memory to support capturing repeat sets of the image sequences as the Wisconsin Fast Plants grew. A generalized set of procedures for setting up and capturing time-lapse panoramas can be found in Appendix 1 with complete details at http://wiki.gigapan.org. The plants were grown under 24 h of fluorescent light, and cabbage white butterfly (Pieris rapae) eggs were introduced on day 11.

Appendix 1.

Generalized procedure for setting up and capturing time-lapse panoramas; see http://wiki.gigapan.org/home for details.

Parts

GigaPan EPIC Pro (http://www.gigapan.com/cms/shop/store) with firmware installed

Digital camera

Camera power supply

Large memory card (e.g., 32 GB)

Tripod or other secure mount

Uninterruptible power supply

Camera settings

Turn off digital zoom, IS mode, and auto power down

Select aperture priority and F6.3 or less

Set resolution to superfine and image size to largest

Set focus. Focus can require experimentation. If there are near and far items, try auto focus; if the entire panorama is far away, setting manual focus to infinity may be preferable.

GigaPan settings

Perform a setup to establish the field of view

Determine the image extents and enter the time between the start of each panorama

Turn on “Motors Rigid” and “Last Pic Hold”

Save the panorama setting to memory

Capture pictures

The GigaPan and camera will capture pictures according to the hardware and timing settings.

Create video

Create time-lapse video with Time Machine Creator Software (http://wiki.gigapan.org/creating-time-machines)

The EPIC Pro is battery operated, but includes an external power adapter for using AC power. Both the camera and the GigaPan were continuously powered using commercially available AC adapters, and a 32-GB card was used to store images. The card was changed every other day. To test and demonstrate the system, a grid of images seven photos wide by three photos high was taken at 15-min intervals over the course of 32 d to create single panoramic images. The capture of 21 images at the 15-min interval allowed us to capture enough change to make the final time-lapse video fluid. This combination also meant that the memory card filled in about 48 h. The appropriate number of images to capture and the time step will vary depending on the scientific application and choice of camera and component parts.

The resolution achievable within a time lapse is determined by camera equipment, image storage, and image acquisition and processing time, and the resolution desired by the researcher is determined by the subject matter and questions asked. The cost of the system is therefore partially determined by the camera equipment used. Any camera that can be connected to a shutter release cable can be used with the GigaPan mount. The interval between card changes is determined by the size and frequency of panoramas, and by the size of the memory card used in the camera. There are 256-GB cards available, but not all camera models are able to read them. A 25-image panorama taken once an hour of images that are 14 MB each will take about 91 h to fill a 32-GB card. A camera with a telephoto lens will take pictures with a smaller field of view, which will increase resolution in the overall image. However, the number of images acquired to cover the same area will increase, as will both the time it takes for the GigaPan to acquire the images and the time it takes to stitch an image. Cameras that take pictures with more megapixels will also provide higher resolution. The time interval between images affects the speed at which the card fills as well as the amount of change captured by the time lapse. The time interval should be determined by the rate at which the subject of interest is changing and the level of detail within that change the researcher is interested in capturing. Ultimately, the spatial and temporal resolution within a panorama is a balance between the area the researcher wishes to cover, the detail needed within the larger image, the camera equipment available, the frequency with which the cards can be changed out of the camera, and the image storage space available.

Individual sets of images were aligned and stitched to create panoramas using software developed at the Community Robotics, Education and Technology Empowerment (CREATE) Laboratory within the Robotics Institute at Carnegie Mellon University. The software for stitching single panoramic images is included with the GigaPan hardware. Alignment through time is accomplished based on overlap in the images and the geometry of the GigaPan robot (see Sargent at al., 2010b for additional detail). Directions and software for stitching panoramas into a time-lapse sequence can be found at http://wiki.gigapan.org/creating-time-machines. The plant growth sequence was played back at 24 frames/s, accelerating plant growth and movement to 21,600 times its natural speed. The resultant time-lapse sequence (http://timemachine.gigapan.org/wiki/Plant_Growth) captures details such as circumnutation and caterpillar feeding and growth. The very high resolution of the images allows the viewer to focus on individual leaves and observe the caterpillars while they are very small. The size of the panorama also allows for comparison among plants and captures behavioral changes as the plants grow and come into contact with one another.

Time-lapse sequences can be viewed at http://timemachine.gigapan.org through browsers supporting HTML5 video such as Google Chrome. The viewer allows for seamless exploration through both space and time. Images can be zoomed to realize spatial detail as the image sequence is stepped forward or backward. The website supports the creation of time warps, which allow a user to save and replay specific areas within a larger panorama. This is useful for highlighting a particular feature of interest. Currently, extracting quantitative data from time-lapse sequences involves user interpretation and manipulation of individual frames using software such as ImageJ (http://rsbweb.nih.gov/ij/) or KineRoot (Basu et al., 2007), and there is a distinct need for tools to automate quantitative data extraction from high-resolution images (Brown et al., 2012).

Field application

Following the demonstrated success in the laboratory, the system was modified for use in the absence of AC power. We designed a prototype solar charging system and electronics to provide regulated voltage to the camera and the robotic mount. A 20-W solar panel was connected to a controller and 12-V deep-cycle marine battery. Power from the 12-V battery was supplied to both the camera and the GigaPan camera mount by connecting in to a 12-V DC isolated power supply and out through two 7.4-V regulators (one for the camera and one for the robotic mount). In addition, a small fan was powered by the solar panel to provide airflow through the support base to facilitate cooling. A list of parts and costs is included in Appendix 2. The prototype was set up in a weatherproof housing behind an acrylic dome. The solar panel was mounted above the acrylic dome and oriented to shade the robotic mount and camera. The system was subsequently modified to include a Pelican case for the housing and a tripod base to facilitate moving the system among field sites. Additional details can be found at (http://wiki.gigapan.org/outdoor-time-machine-capture-v1-0).

Appendix 2.

Parts list, costs, and example sources for components used to build solar-powered very-high-resolution photo system.

Part	Cost (US$)	Example source and notes
GigaPan EPIC Pro	$895	http://gigapan.org
Canon PowerShot G10	$500	A list of GigaPan-compatible cameras can be found at: http://gigapan.org/cms/support/camera-compatibility
20-W 12-V solar panel	$100–150	http://www.ecodirect.com/Small-Solar-Panels-5-Watt-s/24.htm
Regulatora,b	$40	Morningstar SunSaver SS-10L-12 V, PWM charge controller, 10 A, 12 V; power from the battery is fused (10 A) to the isolated power supplies.
		http://www.ecodirect.com/Morningstar-SunSaver-SS-6-12V-6-Amp-12-Volt-p/morningstar-sunsaver-ss-6-12v.htm
12-V deep-cycle 120-Ah battery	$60
12-V DC isolated power supplyc	$70	http://search.digikey.com/us/en/cat/power-supplies-external-internal-off-board/dc-dc-converters/590047?k=vcd30-d12-s12
Two 7.4-V regulators (one for the camera and one for the GigaPan)d		LM317T; http://search.digikey.com/us/en/products/LM317T%2FNOPB/LM317TNS-ND/6178
Fan		PC fan, connected with bypass diode from solar panel to regulator
16″ Acrylic dome	$100	http://www.edmundoptics.com/products/displayproduct.cfm?productid=1395
Aluminum channel stock for holding up solar panel
Aluminum base plate to hold GigaPan
12″-diameter Sonotube painted with epoxy paint	$12	A large PVC pipe would also work well. The tube was just for convenience; it may also be possible to use a Pelican case with a cut out for the dome.
Sheet metal collar for dome		In-house fabrication
Concrete base plate		This can be leveled in the field; the battery sits on top of the plate, inside the tube.
Miscellaneous screws, hardware, silicone sealer, foam

Note: Ah = ampere-hour.

The regulator is located between the solar panel and battery.

Cables and fuse from regulator to battery were made in house.

This isolates the power from the camera and GigaPan. The camera and the GigaPan should be wired through an isolated power supply.

Because both the camera and GigaPan use 7.2–7.4-V power, it is possible to build an adjustable power supply to reduce the 12 V to 7.2 V DC.

The solar-powered time-lapse system was set up in an instrumented subwatershed within the Walnut Gulch Experimental Watershed (WGEW) in southern Arizona (Moran et al., 2008). The system was set up in the Kendall grassland subwatershed, and 28 pictures were taken every 2 h with a Canon PowerShot G10 to create panoramas from a grid seven photos wide by four photos high. Images were stored on the camera’s 32-GB Secure Digital High-Capacity (SDHC) flash card. Although the 32-GB card can hold approximately 5000 images taken at the highest resolution with the G10, we visited the site once per week to verify that the hardware was working and we downloaded the images during these visits. The solar-powered GigaPan system was successfully used to capture summer monsoon season precipitation-induced vegetation changes on the WGEW. In outdoor settings, diurnal lighting variations and changing weather conditions create panoramas with substantial differences in exposure. These differences can affect algorithms for quantifying ecological processes, such as those that rely on red, green, and blue intensity values for assessing phenological changes (Kurc and Benton, 2010). Within an individual GigaPan image, information identifying shutter speed, aperture, and ISO stored in exchangeable image file format (Exif) tags associated with each image is used to adjust exposures to the average exposure for the overall image. Lighting variations are compensated for algorithmically during the stitching process by aligning panoramas with those with the most similar lighting. The resultant time-lapse video, which runs from 11 July 2011 through 8 August 2011, can be seen at http://timemachine.gigapan.org/wiki/Arizona_Grasslands. Rainfall and runoff data collected at the site provide supplemental information that can be used to interpret the qualitative information in the video. A total of 208 mm of rain was recorded during 31 precipitation events from June through September. There was one runoff event on 29 July 2011 during the video, and there was an additional runoff event on 9 July 2011, two days prior to the start of image capture. The temporal dynamics of changing vegetation as the landscape becomes green in response to rainfall events that occur between 20 July and 30 July and reduction in bare soil can be observed within the video. In addition, the dynamism and range of scales accommodated within the time-lapse video is exemplified by viewing a cholla cactus (Cylindropuntia) as it hydrates through the monsoon season (Fig. 2 and http://timemachine.gigapan.org/wiki/Arizona_Grasslands).

Fig. 2.

Frames from a GigaPan time-lapse sequence of summer monsoon season precipitation–induced vegetation change. A, B. Landscape views on (A) 13 July 2011 and (B) 8 August 2011. C–E. The level of image detail capture is illustrated by the sequence of photos showing the response of a cholla cactus (Cylindropuntia) (detail from white box in parts A and B) to precipitation. The full sequence is at http://timemachine.gigapan.org/wiki/Arizona_Grasslands.

During the monsoon season, clouds typically formed midday and were persistent during days with rainfall. On such days, the 20-W solar panel was not sufficient to run the system without draining the battery so a 60-W panel was installed. The system ran through the night (and images were deleted) because we have not yet incorporated a timer to cycle the power, so our power draw was greater than necessary. Additional work to improve the efficiency of solar charging is needed and is ongoing. This improvement will also allow for a greater number of images to be taken during daylight hours. Each of the stitched panoramic images taken every 2 h is 0.20 gigapixels. The overall resolution (and thus detail captured) can be increased by capturing a large number of individual images at a higher zoom. The capture of the 28 images that make up a single panorama took between 2 and 4 min, indicating that a much larger set of images could be captured on the 2-h time step.

CONCLUSIONS

Very-high-resolution photography can be employed to collect very large amounts of qualitative data across a range of spatial and temporal scales at low cost. We have described and demonstrated a system for acquiring very-high-resolution time-lapse imagery using commercially available hardware and off-the-shelf digital cameras with modification to accommodate use at remote field sites. Both the laboratory and remote field site examples provided clear images at multiple scales that contain data on size, color, growth, and movement for a large number of plants. Time-lapse photography can be used to document conditions and provide baseline data against which to compare future conditions. The current system can be employed to capture processes and responses that occur at time scales ranging from minutes and hours (e.g., flash floods), to months (e.g., seasonal phenologic responses), to years and decades (e.g., postfire recovery). Very-high-resolution time-lapse photography is being used in research to study habitat and ecosystem dynamics (Smith, 2010), and field research is ongoing to study plant response to grazing and precipitation patterns using the GigaPan system. Coupling data extracted from the images with environmental data are likely to be a powerful technique for collecting large data sets over time with a minimum of manpower. There is an opportunity to build upon current research ongoing in the remote sensing, phenology, visualization, and engineering communities to extract data from time-lapse and repeat photographic series of very-high-resolution ground-based images.

The GigaPan robotic camera mount and stitching software are commercially available. Development of value-added software and analysis tools is ongoing. Example time-lapse images and information on the current status of the viewer are available at http://timemachine.gigapan.org, a website and wiki for developing collaborative content.