You’ve heard a lot about machine learning. You know it’s related to artificial intelligence. People talk about the machine learning revolution. They say machine learning will transform the world. Businesses that don’t use machine learning will be out-competed by those that do. Machine learning will empower you to reach your goals. It will save money and time, even lives.
But you haven’t seen any concrete examples and you can’t picture exactly how machine learning can help you achieve your goals. You suspect it’s only really useful for big tech companies. If that sounds familiar, this post is written for you.
Related post: artificial intelligence: an introduction
You are on a mission
Imagine you are looking for a new passion project.
You do some soul-searching and uncover a hidden fascination with insects. The more you think about it, the more it shocks you that your knowledge of these creatures is so sparse. Your new mission in life is to learn all about the creepy crawlies that live in your neighbourhood.
You do some reading and decide that learning how to identify different species is the best place to start.
So, each morning, armed with your camera phone and feeling motivated, you get up early to go out bug-hunting. You take pictures of every bug you find. You turn over stones and prod the ground, you start to empathise with blackbirds.
You record as many observations of the insects and their environment as you can think of in case they are relevant to your mission. Checking the outdoor temperature, noting the cloud cover, measuring rainfall and chasing insects with a ruler become part of your morning routine.
Then every evening, after work, you examine your findings and identify each species of insect, with the help of Google and a pile of books you found in the library.
You are meticulous and name all your image files by species. If there is an animal you are not sure about, you call a friend who happens to be an entomologist. They are ecstatic that you’ve shown an interest in their work and are glad to give you some pointers.
Eventually, you become an expert at bug classification yourself and you can quickly identify and label them correctly. Your folder of insect images is brimming with a thousand files per species.
Each image sparks joy so you keep them all.
With your daily excursions, you’ve also inspired a whole community of bug hunters that go out and take their own pictures.
They send them to you to classify.
At first, you are thrilled, but it soon becomes time-consuming and your excitement ebbs away under the flood of new images.
Data overload
You can classify maybe one hundred images per evening, but you are getting thousands per day from your friendly neighbours.
The word is spreading and the data coming in will only increase. You hear rumours that the local entomological society is interested in sending you archive images.
You feel overwhelmed.
Still, you are determined to classify every single bug. You want to learn more and there is a treasure trove of hidden knowledge lurking in all these images. If only you had the capacity to focus on extracting it.
You try to persuade others to help you with labelling but they lack your expertise and don’t have the time.
One of your friends tells you that this is a perfect task for machine learning.
“Machines can learn in seconds what can take a lifetime for a human: it will save you months of work!”
You find this hard to believe but you are willing to give it a try.
Your mechanical trainee
A few days later your friend sends you some code and tells you to run it. She says it will take the images you have already labelled and train an algorithm to classify the insects. Once trained, she says, this algorithm will be able to classify the unlabelled images better than you can.
You scoff a little at this before clicking run.
The program begins training. Numbers labelled ‘accuracy’ start trundling down your screen. The values are low but increasing. Your friend tells you that this is a good sign: the algorithm is learning.
A few hours later you return to your computer and see that the program has finished training. It reports an accuracy of 97%. That sounds good, although you are pretty sure you can do better.
When you open up the images that the trained algorithm got wrong, you see that many of them are insects that you yourself struggled to classify. You even find one that you may have incorrectly labelled. You send these to your entomologist friend who confirms this.
Now you are impressed.
You run the trained algorithm on your unlabelled images and cross your fingers. There are thousands of files. It would take you a month to go through them all.
In under a minute you have the results.
You spend the evening checking a hundred of the algorithm’s classifications on the unlabelled images. It got ninety-five of them correct. Four of the ones it got incorrect were unclear images. One was a species that you hadn’t seen before and wasn’t in any of the photos that you’d labelled. You make a note to find more of this new species.
You stare out the window at the night sky. Stars twinkle at you. They look like fireflies.
Your excitement for this project returns.
The path to discovery
Now you can focus on taking it to the next level. The algorithm you’ve trained can take over the task of classifying images so that you can work on deeper questions.
You can finally progress with your life’s mission.
Machine learning will empower you to reach new heights of entomology.
Your friend tells you that you can use machine learning to discover other patterns in your data.
Since you’ve been so careful with recording your observations, you could train a machine learning algorithm to predict where each species will be on any given day in the year, and how many there will be at each location. This would give you the top times and locations for finding different insects, making your bug hunts much more efficient and opening up new questions for discovery: why do the insects favour these locations? If you could find that out then you could make other locations more bug-friendly – allowing you to collect even more data.
Your heart skips a beat as you realise that you will never have to feel overwhelmed by too much data again, now that you have the power of machine learning at your fingertips!
Achieving your goals
This gets you thinking.
What if you set up cameras in the top bug locations and train an algorithm to detect and classify insects in real-time? Then you could collect and label data 24 hours a day. This would give you a much better understanding of what kinds of creepy crawlies you have in the neighbourhood.
You could even set up alerts so that you are notified when your favourite species is around. Or when there is a species that the algorithm doesn’t recognise – you could discover a new species!
Taking it to another level entirely, you realise that you could monitor the local insect populations for any unusual spikes or dips by comparing the actual numbers to your algorithm’s predictions.
The possibilities suddenly seem endless. You wonder if your life’s mission is too narrow.
Perhaps machine learning can also help you with other projects…
You add machine learning to the top of your list of ways to achieve your goals.
