Marsh Explorer

Dec 21, 2020

Citizen scientists,

Thank you for your interest in helping advance marsh ecology research by volunteering at this site. The web application that we had previously posted here is showing signs of age, so we have decommissioned it for the moment. But the overall project is making progress. In particular:

  1. Jayant Parashar received an MS degree from the University of Georgia in 2020 for his work using computer deep learning to identify the plant species present in each photograph. Mr. Parashar worked on one year of images, but the techniques he developed should allow us ultimately to rapidly identify the plants present in all the photographs from all seven years-a total of over 70,000 images. Manual identifications of the photographs were important in creating a "training set" of data to teach the computer how to identify each species. Mr. Parashar's work was supported by the Georgia Coastal Ecosystems Long-Term Ecological Research program, which also supported acquisition of the photographic images.
  2. A paper explaining Mr. Parashar's work is in press. J. Parashar, S. M. Bhandarkar, J. Simon, B. M. Hopkinson, S. C. Pennings, Estimation of abundance and distribution of salt marsh plants from images using deep learning. Proceedings, 25th International Conference on Pattern Recognition, September 13-18, 2020, Milano, Italy.
  3. We have received a grant from the National Science Foundation to continue working on automating image analysis. About half the work will focus on salt marshes; the other half on coral reefs. Hopkinson, Bhandarkar, Porter and Pennings, IIBR Informatics: Automatic generation of multi-layered, information-rich 3D maps of ecosystems from images, 2020-2022.

Our next steps:

  1. We plan to develop a new version of Marsh Explorer in 2021 using up to date software that we will re-post at this address.
  2. We are working on a manuscript led by a new PhD student, Jacob Simon, which will examine ecological patterns extracted from the dataset that Mr. Parashar analyzed. We hope to submit this manuscript by the end of 2021.
  3. Our ultimate goal is to be able to automatically align all the photographs in each mosaic (>10,000 per year) and to automatically identify all the species present in each. We will need your help in developing training datasets to do this, so please check back in at this address periodically.

More about our goals and methods

We are interested in better understanding the spatial relationships among the different salt marsh species. For example, snails may associate closely with one plant species and avoid another. To examine these relationships, we need a map that shows the locations of all the species.

Think of Google Earth - it has transformed how scientists in many disciplines work, because it allows them to see spatial relationships over a large landscape. But Google Earth images do not have enough resolution to see individual plants or snails, and so are not useful for community ecologists. That is where we come in.
We built a carriage that suspends a camera between two bicycles. The camera points down, about 45 inches above the ground, and takes photographs with enough resolution to see individual crabs, snails and plants.

We pull the carriage back and forth across the marsh using ropes. This avoids disturbing the area very much because we do not have to walk through the area that we are photographing.

The camera is set to take a photograph every second. Because we are moving the carriage slowly, the images overlap. Because the area that we are photographing is large (about 3,200 m2), we end up with 10-15 thousand photographs from one session. We did this once a year, seven years in a row.

Citizen Science volunteers help us align the photographs and tell us what species are in the photographs. Using these data, we create a training set for machine learning, and then the computer can then rapidly tell us what species are present in every single photograph.

The final goal is to create maps of abundance for each species in each year that are derived from the photo-mosaics. These maps are what we will use for our research.

In addition to understanding the spatial relationships of the species at a given time, we can look at change in these relationships over time, because we have seven years of photographs. During this time, we had substantial year-to-year variation in the weather at the site that caused changes in the plant distributions from one year to the next. We will be able to document these by seeing how the species distribution maps change from one year to the next.