An interview with Pierre Bonnet and Alexis Joly: AI, plant recognition, and biodiversity

As part of our AI in Your Community series, I spoke with Pierre Bonnet, a tropical botanist, and Alexis Joly, a computer scientists who have been working on a project called Pl@ntNet for the past ten years. Pierre and Alexis work together to develop tools that teach people about biodiversity and plant identification while also building a collaborative data set that spans continents.

Pierre Bonnet, Alexis Joly and Jean-François Molino, winners of 2016 La Recherche prize | © Pl@ntNet / Rémi Knaff

Tara Chklovski: Let’s start by having you introduce yourselves and tell me a little bit about your work and the problems that you’re trying to solve.

Pierre Bonnet: I’m Pierre Bonnet, I’m a scientist, mainly working in tropical botany. I work at the CIRAD Institute – we conduct research in tropical regions, which are hotspots for biodiversity. I’ve been working in the field of biodiversity informatics for 12 years now. From my point of view, my purpose is to collaborate with computer scientists to design a new approach to solve problems, like the problem presented by identifying hard-to-identify plants at a large scale.

I have worked with developers on tools for plant identification in tropical Africa and southeast Asia, and for the last ten years or so, I’ve been working with Alexis on the Pl@ntNet project. With Pl@ntNet we’re dedicated to trying to solve the problem of identifying plants at a large scale using images. My field is mainly botany so I collaborate with engineers and computer scientists like Alexis – Alexis has been my main collaborator for ten years now. Alexis?

Alexis Joly: My name is Alexis Joly. I’m a computer scientist and part of a research organization in France, called Inria. I’m a specialist of machine learning and computer vision technologies, and I’ve been applying this research to biodiversity and informatics for more than ten years. As for the Pl@ntNet project, at the beginning it was really a research project, with the idea of building and evaluating the technology, and so we have spent many years improving all these technologies and evaluating them at a large scale with researchers.

For three years we have been funded by an educational initiative called Floris’tic, and we have collaborated with similar associations all over the world to do a lot of activities related to education.

TC: The whole project is very very interesting. Why did you start Pl@ntNet? What was your first goal?

AJ: I think the first motivation was to understand the impact of humanity on biodiversity. We wanted a better knowledge of plants around the world, but there are very few people that can identify plants like Pierre can – there aren’t that many botanists. This is often called the “taxonomy gap” or “taxonomy impediment”. Our idea was to find ways to engage non-specialists in a Citizen Science Initiative to document plants all over the world.

Our motivation was to have non-specialists work together with specialists and have this principle of citizen science, but assisted by machine learning through automated tools.

TC: So what was the process like, 10 years ago when you started, and how does it work now?

PB: We initiated the project with a strong collaboration with a French NGO called Tela Botanica. At first everything was based on leaf recognition. We asked people who had trees in their gardens (and knew the name of that tree) to scan some of its leaves and share them with us. Through this we started to build a collection of different images of different trees (mainly from the South of France). As the project progressed, we increased the number of species, the number and diversity of images, and started to create this rich collaborative data set. Then a research team used that data to start working on the automated identification tool.

AJ: The primary principle of this was that we needed to have a positive feedback loop, so the more data we had, the better the machine learning tools would perform, and the more people we engaged, the more the tool improved. And then with more people, we’d also have more specialists in that network, and they could validate the hardest-to-identify species of plants. We started with maybe a few dozen species and now there are 16,000 species in the application.

TC: How did you get people to contribute?

AJ: We were lucky, and didn’t have to do anything special. There was a buzz on the web and everywhere, which was what made us successful. At the beginning the Tela Botanica Network put out a few communications to their 20,000 members.

PB: The iPhone application launched in 2013, and at that time it was something that was really new, but also original because there weren’t many applications using image recognition. So that was a really interesting challenge and probably something that was really attractive for the media to write about. Back then it wasn’t efficient smartphones couldn’t take the kinds of high quality photos they can now, but it was the beginning of people seeing the potential of smartphones, and we were lucky that we had the support of media that was interested in highlighting the originality of the technology. Plus the media was also interested in promoting the Citizen Science Initiative’s original project of involving non-scientists in biodiversity and conservation projects. And so on and so on.

AJ: Social networks also played a big role. People were talking about the “Shazam for plants.” There was a big buzz on Twitter, and on other social media networks.

TC: How many people are using the tools now, and where are they from?

AJ: We have seven million downloads, and in the last year there were three million users of the application. Of those, there are maybe 500,000 users who actively contribute and upload pictures. But we have people who contribute just one image or a few images and then people who contribute very very actively.

Until recently, the most active users were located in France, but in the last 12 months it seems there are more users in the US then in France. But there are still more users in Europe than in the US.

PB: The project was initially well-adapted to the European flora, European plants, and from there we’ve developed collaborations with people and organizations in South America, and North America. We’ve seen a strong correlation with the number of people in a country and the accessibility to mobile technology in that country. So North America is well-connected, so is Europe, and Southeast Asia  like in India, we have some bigger views in that region.

We also have a lot of users in Australia and unfortunately for them and us the application is not yet adapted to their flora, and so people can only identify common tropical plants that can be found outside the region at the moment. But we invest in each regional adaptation and work with specific groups who have the expertise of the native flora and who can give us a recognized national checklist (which is the first step to deploy an adaptive project in a new country). The growth of the project is also tied to the support of local actors who will invest time and expertise in that app to initiate the project and to create this virtual circle mentioned by Alexis.

AJ: For instance, for educational use of the application, one of the most active countries is Slovakia. There is an association there and they have a program with 50 schools all over the country where they really use Pl@ntNet as a social tool for an environmental education activity within schools.

TC: And so what’s the next step for Pl@ntNet? What questions do you think users might be able to use this database for?

AJ: That’s a good question. The data is starting to be studied by researchers, so there are ecological studies modeling species distribution over time happening. There is one focused on invasive species, and the application is good for that because invasive species often live where people live because there is often a correlation between the presence of humans and the presence of these plants.

We would really like a tool like Pl@ntNet to provide information about what biodiversity is. It’s not only plants you can buy in a shop, but it’s really a very large diversity of plants. It’s complex. A single common name can be used for a large number of species. Most people don’t know that.

We also want to start investing in the compatibility of this system to be able to recognize plants at a larger taxonomic scale. Of the 16,000 plants we cover, a lot of them are European or North American plants. There are several common tropical plant species, but the idea is that this system would be the most appropriate in a region where there is a lack of information on specific flora that are really important to the area’s biodiversity.

PB: We hope to develop some content about local biodiversity to promote local biodiversity, which is often lost in gardens because gardens are often designed to feature foreign plants. Those foreign plants can be really attractive, but they can do a lot of damage, to the soil quality, or the biodiversity, and so on. The idea is to highlight the local biodiversity of the user environment, which can be really accessible, but which is not well seen or known by people because most of their environment is populated by exotic plants in the end.

And now, a lot of plants used in landscape management are from really far away countries, so in that regard we have lost a lot of native plants, and plants used by our grandparents have disappeared because we don’t allow them to grow. It’s that kind of information we want to promote to users eventually, but right now we only ask them to collect data and not act on their local environment, because we want to avoid damaging the local environment.

AJ: Yes. The educational aspect and the heritage aspect are really important for us. I mean, there are a lot of places where we already know that biodiversity is in danger, and often what we need is political action, and to have political action you need people to vote for the right people or you need people to know that this is important. Which is why the educational point is very important.

But we still need some data. For us scientists, what we are thinking about currently is having some personalized recommendations for where to collect data, but for areas where biodiversity is really endangered, you want to avoid having a mass of people out there, so for those areas we’d recommend highly skilled experts survey those places, whereas if you wanted to educate young children, you could go to places with good biodiversity. We would like to have recommendation mechanisms like that, which could be done through badges in the application. And so in the further versions we will work to have more personalized recommendation of what people could observe and also to have them observing complimentary things. We don’t want to have everybody looking at the same species they all know…so that’s what we’re looking to do with this tool in the future.

TC: Awesome. Thank you so much for doing this.