To dive into “What is Neural Network” we must at least step back a little bit further to discuss the discovery of Artificial Intelligence and why it is not only interesting but helpful in the modern world. These should be topics of when, what, and their applications to the real cases. It is necessary to see some historical view to grasp the creation of neural networks under the umbrella of Artificial Intelligence as a whole and its subfields “Machine Learning”. Though, I want to pinpoint that we will not get into too many details caused by different terminologies.
Figure 1
Venn Diagram
Note: This image was created by the author which presents Machine Learning and Neural networks under the umbrella of Artificial Intelligence as a whole.
In 1956 John McCarthy held the first academic conference about AI. Interestingly a few years later, Alan Turing publish a paper about the notion of machines being able to simulate human beings and be intelligent on their own (University of Washington, 2006). Both bring about prompt such question of how can a machine think like humans, although it is unquestionably that machines are capable of processing vast amount of logic efficiently. As seemingly complementation to Alan Turing’s research, it turns out, machines can actually learn through the help of a mathematical algorithm that iterates experience (observations) based on the gathered data to perform a task.
Just view that the goal of AI is to bring the capability for machines to figure out patterns based on correlations, and relationships between variables (Data). Thus can be performed in conjunction with integrating statistical methods in crunching relevant data for validating hypotheses to make decisions. With that being said, this is what makes the machine can really mimic the human brain based on the experience of different observations that enables them to become more intelligent.
For example, imagine a game of chess being played without human interaction, and all the decisions for the next move are based on the previous opponent’s observations in making the best move.
Types of Machine Learning
The “Supervised Learning”
As the name implies the supervised machine learning is a discipline of evaluating model (Model is constructed of different variables output) based on pre-defined values. This is a kind of lazy machine learning technique as we need to gather, feed, and train the model with relevant data(Data set). This is widely used in classification and clustering, for example, if we were given a scenario to predict the vehicle transmission type (as output) based on the Input Engine-Weight, MPG, Cylinder, and horsepower as feature/predictor. The latter scenario is what is called classification in machine learning.
Now, what if you have a new type of engine that does not exist ever before? Well, you can perform clustering with K-Nearest Neighbors (KNN, a non-parametric classification method). Do not be confused about what the latter is, but you basically still gather previous groups of data to make a vote(based on the KNN parameter) on where the new engine type may belong (Classify). If it’s a new engine type, maybe including the manufacturing company as one of the groups can be useful. Now, if the latter produces satisfying or tied results with other classified groups, then it’s up to you to accept otherwise decline (Unclassified).
The “Unsupervised Learning”
Unsupervised machine learning is in which the algorithm never (or has very minimal) provided constructed pre-defined labeled data as its baseline ahead of time. This is one of its advantages as it will discover recurring events patterns (self discover). Let’s say you want to implement a strategic system that detects anomaly patterns based on sessions (Unsupervised anomaly detection). Since it already knows what are the usual (Normal) sessions pattern during self-discovery, the algorithm can detect any unusual pattern against it and trigger an event e.g notifications &, etc… You got the idea…
The example above is what we called an anomaly detection system based on an unsupervised training model. Remember when mentioned “or has very minimal” predefined labeled data prior to training, that complements to semi-supervised machine learning so I can bring the connection between supervised and unsupervised.
The “Reinforcement Learning”
The reinforcement takes us into another level of machine learning as this contains what we called an “Agent” to do the job. Just put this agent as almost a 1-year-old baby that wants to outrace a puppy (as a reward) at a certain distance. Now since the baby is not able to stand and walk but able to crawl, of course, it needs to learn how to maximize its chance against the puppy in that environment. So in this sense, the baby needs to learn how to get through the obstacles step-by-step to be able to stand, walk, and /or run to outrace the puppy.
So basically reinforcement learning is determining the best strategy to maximize rewards through the experience of failure and pass (rewards) via iterations of different steps. A real-world use case of this is in robotics, self-driving cars for avoiding collisions & choosing the best routes (dynamic path), natural language processing(NLP) e.g word corrections & recommendations, and etc…
So, What is Neural Network in AI?
In this case, since we are talking about AI we have artificial neurons that are inspired by the human brain. These artificial neurons are basically connected like circuits to create a set of interconnected artificial neurons (combined) since a single neuron is useless. Connected neurons (A.K.A layers) have to accept and measure the weights and biases of inputs parameters to make adjustments to produce the accurate output as much as possible. It is important to mention that, there are multiple layers, the output of the previous layer becomes the input of the next layer.
For example, to classify an image as cat or dog, what are the possible features we can extract to determine if it’s really cats or dogs? Of course, we want to get the attributes such as length, ear or nose shapes, height, &, etc then weighted this in different groups via artificial neuron network layers to give a distinct output. So basically Neural Network in AI refers to the structure of interconnected artificial neurons that are ready to be trained, this all falls under the umbrella of Deep Learning.
Figure 2
Visualizing Neural Network Sizes
Note. This video is borrowed from the website (https://nnfs.io/ntr/)
As you can see in the video, there are input(s), hidden layers, and the output layer. Each neuron in the next layers has a connection from each preceding neuron. Each neuron has an association of distributed weights that affect the output.
The layer size refers to the number of neurons in each layer. In Convolutional Neural Network (CNN, Class of Deep Network), the weight is calculated with the formula of (( #input) * ((#filters) * (#filter size)), the bias is equal to the number of filters (Purpose to offset/shift the result in the negative or positive to support the best result). The params are just the summed of the results from weights and the biases.
As mentioned we are not going to dig into other confusing terminologies about “how” but only focus on “what” to see what is neural network is in the umbrella of artificial intelligence as a whole.
Conclusion
In this post, we tackled the different machine learning disciplines under the AI umbrella in a conjunction with the topic “What is Neural Network in AI”. As you can see in the image (See figure 1), AI encompasses all the processes for the machine to mimic the human capability to think and make decisions. AI works through machine learning with the application of the different statistical methods in the algorithms. As result, artificial neurons are created to perform tasks then produce the best output (result) based on desired assignment accountability.
Hope that this post helps you understand how neural networks (interconnected neurons) work in AI, the applications, and how it’s significantly impacted the real world.
Have suggestions? Please put in the comment…
References
University of Washington (2006). The History of Artificial Intelligence. Retrieved from https://courses.cs.washington.edu/courses/csep590/06au/projects/history-ai.pdf
Kentaro Katahira (2015). The relation between reinforcement learning parameters and the influence of reinforcement history on choice behavior. Retrieved from https://www.sciencedirect.com/science/article/pii/S0022249615000218
