Overview-

It would be a lie to say that I wasn’t very excited about this Unit. In many ways, the entire reason I enrolled in Lambda was to cover the material found in this section. Neural Networks have been a fascination of mine ever since I realized their applications in evolutionary processes or enabling us to approach the Theory of Mind from a new angle. They also represent some of the harder concepts found in machine learning. They’re the corner stone for future AI and therefore, this unit was precious to me.

Sprint 1- Natural Language Processing

This was the hardest sprint for me. I had some personal hardship this week with my son being sick that caused me to miss some of the lectures and after-lecture work. I muscled through, but it was hard. NLP is all about sentiment analysis, understanding text prediction and pointing out tokenized vectors. Part of it feels like modern-day magic, and part of it feels sort of like a guessing game. TFIDF was invaluable here, both for tokenizing and predicting. My biggest take away here was the concept of stop words. You don’t really understand how little of our language actually means anything until you see how many stop words you have to pull out of a thing to make it functional in a model.

Sprint 2- Neural Network Foundations

This sprint was fire for me. I ended up having to move to Google Colab and get a pro account to keep up with the GPUs. Not because my videocard is bad, but I had more difficulties than I wanted to overcome getting my machine to get keras and tensorflow via VS-code to work with my GPU. The difference in runtime between not having a GPU and having one when training neural networks is wild. The big note here are understanding the structure of a neural network. Input nodes, hidden nodes, output nodes, structured in layers, and with values highly relevant to the thing you want the network to work with. While these libraries are very good, there is still a great deal of fine tuning necessary to get your networks to perform well. What I found to be most surprising here, was how quickly they could process data once they were trained though. Models could be light weight enough to deploy to websites or phone apps, and that is also wild. Some of the stuff coming out of YOLO is truly mind bending.

Sprint 3- Major Neural Network Architectures

Did I mention how many hyperparameters you need to tweak in a well functioning model? Sometimes millions. We had a whole sprint about how you fiddle with these different networks. Recurrent Neural Networks related closely to NLP and Convolutional networks work well with pictures. All of them take many, many parameters to tune. This sprint was all about exploring the different tests you can do to train models, and dial in their precision. Also some very cool stuff. As this was the last unit in the data science track, we also spent some time exploring the ethics of AI, where we think the field is going, where we want the field to be going, and what we will do when we get there. All very dear to my heart conversations.

Closing Thoughts

I loved this unit. Start to finish, this was my favorite material to cover in Lambda. While I’m not sure if I’ll be able to apply neural networks when I get into the ‘real’ world, I adored having the chance to really dig into them, even briefly. I’ve since used a few in Kaggle challenges to mixed success. Sometimes the simple models are better, but its hard to trump the ‘cool’ factor of a Neural Network.