Intro Machine learning, in layman terms, is the concept of teaching a computer to do something without explicitly programming it by utilizing complicated algorithms to make predictions. To make things easier, AWS has built a set of cloud based tools that allow any developer to leverage machine learning in their app. Amazon has a tutorial you can follow here.
I recently had the opportunity to experiment with machine learning in a project I had been working on for a while. The project, in summary, is a supply chain management tool for organizations that donate drugs to the fighting of NTDs (Neglected Tropical Diseases). In this project, there ended up being two areas I wanted to see if machine learning could be used. The first was incoming email. Part of our system relies on existing email chains to get important information into the system. Since we are inserting ourselves into a system that already existed, we wanted to avoid trying to change their process. However, the main problem with these emails is that there is very little consist in the formatting, so we currently attempt to parse out all the important information we can, but the rest must be entered manually. Our hope is to automate as much of our system as possible so there is minimal manual data entry, freeing up our users to spend time on more important tasks. The second area I wanted to try using machine learning for was in predicting the amount of drugs a country might be requesting/needing. As I mentioned before, we would like as much of the project to be automated as possible and the forecasting component was something that could benefit from prediction. Currently, forecast numbers are entered/done by one person and we do not know what/if there are any guidelines for making these forecasts, so naturally machine learning seemed like a good option at first.
My first attempt at using the AWS machine learning tools involved trying to parsing data out of an email subject line. Currently, our system can parse an email subject by using regex, which is great when what you want to find fits a pattern, however, we had some issue with specific data so we had hoped machine learning could help. Unfortunately, I soon discovered that AWS ML wouldn't be able to help, and looking back on this I see two problems with what I wanted/tried. 1) Regex In hindsight, the correct solution is more robust regex code. As AWS puts it, you don't need ML if you can determine a target value by using simple rules, computations, or predetermined steps that can be programmed without needing any data-driven learning . 2) Data - Simply put, there was no data to really feed the machine learning model. All we had was a subject line that we knew had data we wanted, but machine learning needs numerous amounts of data points to produce any results.
One of the main uses of our app is keeping track of forecasted demands, which currently is done by one person. In my eyes, this was a better use case for using machine learning. The idea was to gather historical data about a specific country that could be used to predict the demand for the next year. The downfall was the lack of data and how the data related. For one thing, there wasn't much historical data and the data we did have was segmented into 3 categories. In our case we had maybe 4 or so good columns of data, to be effective at prediction I would say we needed no less than 10 columns of good data. When talking about how much data to use in machine learning, more is better. Since all the data was related to a country and the NTDs in that country, most of the data was only connected by country. The result were predictions that seemed to have no relation to the numbers we wanted machine learning to use for prediction. What now? At this point I had spent a few days on machine learning, which included reading articles, documentation and experimenting with new datasets, without much to show. On top of that, even though I failed to leverage it in our app, I still needed to write up what I learned (the thing you are reading now). However, I really wanted to have a working example for this, without using something I knew would already work, so I started looking at open sourced datasets that I could use with machine learning. After stumbling onto Kaggle, an open source website for sharing machine learning data, I settled on using a dataset containing credit card transaction data. The file contained over 200,000 records, so I didn't have to worry about not having enough data, but I still wasn't sure I would see positive results plugging it into AWS. On my first attempt I got some somewhat promising results. I was able to correctly predict the results of 2 out of 4 records. I decided to change the datatype of the column I wanted AWS to predict. By switching the predicted column from a numeric value (0 or 1) to a string value (yes or no) I was able to correctly predict 9 out of 10 records. To test the prediction I removed 10 rows of data from the dataset since I could not provide any new data points. Going off these results, I would say that the AWS machine learning tool was able to predict whether a credit card transaction was fraudulent with pretty good accuracy. Conclusion I think that machine learning is a very powerful tool, however knowing when it's usable for a solution can be tricky unless you are familiar with the concepts and use cases. Although I did not find a place for it in my project, I learned a lot about machine learning and its potential. I think AI and machine learning are going to become a bigger part of the internet and things connected to the internet. I just hope no one creates Skynet.
151 Ted Turner Drive, Atlanta, GA
Let's talk: info@standardco.de
