This brings together ideas and questions around consciousness, the visual and the haptic, eyes and skin. Or, neurons, retinas, lenses, or ocelli as they are termed in the botanical science - a kind of „skin-eye“ on leaves that suggest plant vision. The idea that plants may have “eyes” is nothing new. In 1907 Francis Darwin, Charles's son, hypothesized that leaves have organs that are a combination of lens-like cells and light-sensitive cells. Experiments in the early 20th century seemed to confirm that such structures, now called ocelli, exist. František Baluška, a plant cell biologist at the University of Bonn in Germany, and Stefano Mancuso, a plant physiologist at the University of Florence in Italy, recently in 2016 provided some evidence for visually aware vegetation. Referring to acyanobacteria, Synechocystis, single-celled organisms capable of photosynthesis, that act like ocelli. I quote from a brief article in the ScientificAmerican from January 2017 „Do Plants See the World Around Them?“: “These cyanobacteria use the entire cell body as a lens to focus an image of the light source at the cell membrane, as in the retina of an animal eye,” says University of London microbiologist Conrad Mullineaux, who helped to make the discovery.

the-observation-of-the-pea

We asked the question “what could seeing as for us humans mean for a plant / pea?”. Humans perceive the visible part of the electromagnetic wave spectrum using their eyes. Photons hit the light-receptive cells inside the retina and trigger electrical signals that are processed by the brain. In plant cells photons are absorbed and provide the energy for photosynthesis. Blue and red light is most relevant for the chemical process. So on a high and abstract level this process of absorption of photons for photosynthesis could be seen as a way plants “see”. This raises the question of what they “see”. For the human eye the image of the surroundings is projected through the lens onto the retina, forming something like a 2D image. For a plant the “retina” would be its whole surface area, with its topology being more complex than a basic 2D image. A process called UV unwrapping can be used to flatten any topology into a 2D image, which can introduce distortions or unconnected regions. However, in this way it is possible to generate some form of image-based representation of what the plant “sees”. In order to produce such an image, it would be necessary to know how much photons (light) is incident on the surface of the plant. For this one could either use an infrared camera or simulate the illumination using some form of geometric representation (e.g. mesh) of the plant. To generate such a mesh one could either scan the plant or use some form of procedural model. In this project we experimented with both. We tried to scan some peas using structured light projected using patterns developed on diaframes. And we put together a procedural pea model in Blender using animation nodes. In addition, we want to reenact an experiment by Darwin (reference TODO) where he tracked the movement and growth of plants. The idea is to use several scans taken at different times to track the motion in space of selected parts of the plant.

structured-dia

One way to measure or see depth is to have two different views (images) of the same scene. By finding corresponding points in the two images it is possible to estimate the depth based on their disparity. A common way to reconstruct the surfaces of a real object or scene is to take many photos from different angles and use structure-from motion to estimate the geometry. This approach belongs to the passive methods. An active stereo reconstruction method is called structured light. Here one of the two views (stereo cameras) is replaced by an active projector that projects pattern images into the scene. By taking pictures of the scene illuminated by those projected patterns from the other stereo view it is possible to reconstruct the depth based on the distortion of those patterns. This method is especially suited for fast and accurate scanning and reconstruction of objects. In this project we were interested in scanning peas by using patterns developed onto dias. We used an existing software for the reconstruction and a DSLR for image capture. Unfortunately, the quality of the scans is quite low, so the light simulation or 3D tracking of the plant is likely to fail.

the-generation-of-the-pea

By watching peas grow we saw patterns emerge. In particular we saw that the pea usually has a section of stem and at the beginning leaves grow and the stem sometimes branches. So we developed a procedural plant model based on a random concatenation of such basic structures, i.e. stem segment and leaves. We programmed this in Blender using the Animation Nodes plugin.