Harnessing big data
for agricultural excellence
Part 1: Understanding big data in agriculture
Simon Fraser University’s Big Data Hub &
BC Centre for Agritech Innovation
November 20, 2024
Simon Fraser University
SFU hosts the Cedar supercomputer—a cluster of 100,400 CPUs and 1,352 GPUs soon to be replaced by an even larger computer cluster
Simon Fraser University
SFU also works with the Digital Research Alliance of Canada to offer researchers large amounts of computing power to solve challenging data and technology problems, as well as training to optimize their solutions
SFU’s Big Data Hub
Since 2016, Simon Fraser University’s Big Data Hub has been offering workshops, events, and consulting services to researchers and industry partners helping them remain at the top of the fast evolving data landscape
BC Centre for Agritech Innovation
Since 2022, SFU BCCAI has been helping small and medium enterprises in the farming industry to embrace technology driven solutions
BC Centre for Agritech Innovation
Session 1
Today
A (hopefully) friendly lecture to:
- Demystify big data
- Demonstrate the critical importance of big data in agriculture and farming
Session 2
Tomorrow at 11am in the Mount Baker Room
An interactive workshop to:
- Brainstorm on how big data can benefit your operation
- Help you make the transition to smart farming
What is big data?
The 3 “V”: Volume
Before
Farmers were taking measurements (e.g. on soil moisture) manually creating low volumes of data
The 3 “V”: Volume
Before
Farmers were taking measurements (e.g. on soil moisture) manually creating low volumes of data
Now
Internet of Things (IoT) (e.g. hundreds of soil moisture sensors) collects large volumes of data
The 3 “V”: Variety
Before
There was a limited set of data a producer could collect
The 3 “V”: Variety
Before
There was a limited set of data a producer could collect
Now
There are so many different types of data (e.g. satellite images, market data gathered from internet browsing…)
The 3 “V”: Velocity
Before
A farmer could only gather so much data, even with a lot of employees
The 3 “V”: Velocity
Before
A farmer could only gather so much data, even with a lot of employees
Now
Data is generated in real time and accumulates at high speed
Why has big data become so essential?
All this data is key to the development of artificial intelligence (AI)
so…
What is AI?
AI
Very loosely, you can think of neural networks (the most powerful form of AI) as an attempt to create a computer model that mimics the brain
AI
In traditional computing, a programmer writes code that gives a computer detailed instructions of what to do
These instructions are called a program
Some action
AI
This learning is nothing magical: some numbers in the model get tweaked a tiny bit, with each new piece of data, to make the model a little bit better
AI
Basically, we start with a model, train it with data, and we end up with a trained model that can be used as a traditional computer program
That trained model can be used to get predictions, generate art or speech, identify objects in images or spams in emails…
The only difference from traditional computing is that we don’t write the program ourselves. Instead, we write a starting point (the untrained model), then train it with A LOT of data and let it get better by itself
AI
To get a very good model at the end—one that can write human language like ChatGPT or voice assistants for instance—you really need A LOT OF DATA
AI: an example
Imagine that you want a program able to detect tomatoes in pictures
This could be very useful to get real time data on your upcoming crop so that you can plan adequately (personal to harvest, price, markets)
As human, this is straightforward
Yet, for a traditional programming, this is truly impossible because there are too many factors (location of the tomatoes in the image, quality of the picture, colour of the tomatoes…)
AI: an example
However, by feeding a very large number of images with and without tomatoes along with labels that give the number of tomatoes for each image to a neural network, we can train it to recognize tomatoes in images that it has never seen
With each pair of image/label (e.g. “Picture 34, label: 5 tomatoes”), the model gets better
AI: an example
We don’t write the program to do this. We write the starting model, then let it adjust by itself based on the data
It is a form of learning by experience, which is exactly what happens to us as we grow up. It is a form of programming that is much closer to how brain works than traditional programming
Why now?
The idea is not new, but it is only recently that we have had enough computing power, internet connectivity, and storage capacity to implement it
Smart farming
We already talked about data collection thanks to the Internet of Things (e.g. moisture sensors)
Smart farming
But this goes much further as AI algorithms can be used for “precision agriculture”
Smart farming
Many AI tools are involved in improving all domains of agriculture, from irrigation management to supply chain and demand forecasting
The benefits are huge for both farmers (increased yields, reduced costs, better planning) and the environment (optimization of resource and pesticide use)
Decision making
Before
Farmers had to make decisions as best they could based on their experience and their limited data
Decision making
Before
Farmers had to take decisions as best they could based on their experience and their limited data
Now
Farmers can use powerful models to make informed decision in real time. This can be followed by the automation of some action (e.g. watering)
Livestock monitoring
A case study
Livestock successfully monitored remotely via sound sensors and algorithms for background noise filtering
Animal welfare and efficiency improvements
Market analysis and supply chain optimization
A review
Systematic literature review of peer-reviewed articles and conference papers published between 2014 and 20241 showed large improvements of demand forecasting accuracy and supply chain optimization
Real time data analysis helped with predictive maintenance, market volatility, resource constraints, and climate variability
Challenges
There are challenges to the implementation of such transformative methods
- Infrastructure development
- Skill gaps among agricultural professionals
We are here to help!
Come to session 2 tomorrow!
Acknowledgements
Feedback
Please give us feedback by scanning the QR code:
Questions?