In recent years, we have seen the success of machine learning in various fields and with it, we have witnessed the development of machine learning algorithms. Sometimes these algorithms are based in whole or in part on mathematical operations and logic. Similarly, Boolean algebra is also a piece of mathematics and mathematical logic that can be used in machine learning. In this article, we will discuss the application of Boolean algebra in machine learning. The key points to be discussed in part one of the article are listed below.
Table of Contents
- Why is Boolean algebra used in machine learning?
- Related Works: Boolean Algebra in Machine Learning
- Important applications in machine learning
- Real life applications
Let’s start the discussion by understanding why Boolean algebra is used in machine learning.
Why is Boolean algebra used in machine learning?
Boolean algebra is introduced in machine learning to overcome some of the disadvantages of this field. One of the biggest disadvantages is that machine learning algorithms are a kind of black box technique. To understand it better, we can take an example of a multi-layer perceptual network or support vector machines. With these techniques we can get good accuracy when modeling, but when it comes to understanding the inner workings of the model, we don’t get as much detail. On the other hand, algorithms like random forest and decision trees can describe how it works, but often we don’t get good results. This disadvantage of the black box can be solved with Boolean algebra.
Also, this introduction of Boolean algebra to machine learning used Boolean algebra to create sets of understandable rules that can achieve very good performance. The example above is just a basic application of Boolean algebra in machine learning. multilayer perceptrons. Perceptron is a supervised learning algorithm for binary classifiers.
As we know, Boolean algebra works on the logic and conditions of rules. A basic model, which may contain Boolean algebra, can operate on the rules. For example, a basic Boolean algebra model starts with the data with a target variable and input or learning variables and, using the rule set, produces an output value by considering a particular configuration of input samples. A simple rule can be written as follows:
When premise then consequences
In the rule above, the premise contains one or more conditions on the input, and the consequence contains an output value. The condition of the premises can take different forms depending on the type of input:
- If variables are categorical, the input value must be in a subset.
- If variables are ordered, the condition is written as inequality or interval,
Therefore, the possible rule can be written as follows:
In the expression above, we discussed a basic intuition why we need Boolean algebra in machine learning. In the next section of the article we will discuss the work that has been done based on this intuition.
Related Works: Boolean Algebra in Machine Learning
In this section of the article, we mainly focus on the works where we can find Boolean algebra in machine learning. Some of the related work is listed below:
- Switching Neural Networks: A New Connectionist Model for Classification: In this work we can see an example of a model using Boolean algebra with the layers of connectionist neural networks. In the architecture of this work, we notice that the first layer of the model contains an A/D converter that converts the input samples into binary strings, and then the next two layers of the network use a positive Boolean function that resolves to an A /D converter area the original classification problem. The function used by the neural network in this work can be written in the form of understandable rules. A suitable method for reconstructing the positive Boolean function can be adapted to train the model. You called the model Switching Neural Network. The image below is a representation of the scheme of Switching Neural Networks.
We can consider this work as a neural network with three layers feedforward, where the first layer is used for the binary mapping and the next two layers are used to express the positive Boolean function. Each port in the second layer is only connected to some of the outputs leaving the latticizers.
- Learning algorithms via neural logic networks: This work is based on the creation of a neural network paradigm for learning using the Boolean neural network. Basic differential operators from the Boolean system such as conjunction, disjunction and exclusive OR are used. These basic differential operators can be combined with deep neural networks like MLP. This work can be a testament to overcoming some of the disadvantages of MLP in learning discrete-algorithmic tasks. The model of this work is known as a logic neural network, in which neural logic layers based on an arbitrary boolean function have been introduced. Types of these neural logic layers are as follows:
- Neural Conjunction Layer: Keep the conjunction function from Boolean algebra.
- Neural Disjunction Layer: Contains the disjunction function from Boolean algebra.
- Neural XOR Layer: Hold the XOR or exclusive OR function from Boolean algebra.
The image below is a comparison of MLP and NLN for learning boolean functions.
The approaches given above are two fundamental works that have been updated after the introduction and used in various real-world applications. In the next section of the article, we will discuss the real-world application of machine learning algorithms using Boolean algebra.
Major Applications of Boolean Algebra in Machine Learning
Some of the most important applications of Boolean algebra in the field of machine learning are listed below:
- Demonstration of classification by a perceptron: To demonstrate how perceptrons can classify the linearly separable patterns, the truth tables of Boolean AND or OR operations can be used. The results of the operations specify the class labels, while the input patterns represent the data points in 2D space.
- XOR problem and multilayer perceptron: As discussed above, the perceptrons can classify the input patterns of Boolean AND or OR operations with a single layer architecture. But they cannot classify the patterns of an XOR operation. To properly classify them, perform the development of the multilayer perceptrons.
- Various gates used in the LSTM Recurrent Neural Network: We can see the use of gates in the LSTM networks, especially in gates. We can take an example of a gate to forgetting where the result of the sigmoid function (i.e. forgetting state) is an indicator of pointwise multiplication by cell state that causes the cell state to “forget all information” or “attach all information remind”. “. This can be completed using Gates’ Boolean algebra based concept.
Real life applications
We see the applications of this approach, i.e. machine learning with Boolean algebra, in various fields such as medicine, financial services and supply chain management. In this section of the article, we will discuss some of the important and famous real-life applications that are listed below.
- The work of the switching neural network has been applied in medical science, where it is used to classify the new multiple osteochondromas, which are a type of bone tumor. More details about this application can be found here here.
- This approach is applied to create a prognostic classifier for neuroblastoma patients. Neuroblastoma is a type of cancer that is mainly detected in the small gland. Basically, this classifier consisted of 9 rules using mainly two conditions of relative expression of 11 sets of probes and algorithms applied to microarray data and patient classification. We can find more details about the work here.
- This approach has been applied to the diagnosis of pleural mesothelioma. To do this, they applied the logical learning machine to a data set of 169 patients in northern Italy. They also compared the result of the algorithm with the results of other algorithms such as decision trees, ANN and artificial neural networks and found out the performance of switching neural networks. We can find more details about this work here.
In this article, we discussed why we need Boolean algebra in machine learning with an intuition on how to apply it in machine learning. Alongside this, we have also discussed some of the major related works with their application in real life.