Which tasks can be performed by AI?
Generally, the probably most prominent AI applications include image processing (i.e. computer vision) and natural language processing (“NLP”, i.e. language comprehension).
Typical tasks performed by a deep learning method include information prediction (“predictive AI”) and content generation (“generative AI”). Sub-tasks of information prediction (“predictive AI”) include regression and classification. While classification predicts a categorical value (e.g. “cat” or “dog”), regression predicts a continuous value (e.g. 0.0001-0.9999). Particular embodiments of regression include image segmentation, text recognition or medical diagnosis. Content generation includes e.g. text generation (using e.g. GPT-4) or image generation (using e.g. a generative adversarial network or a Dall-E).
Simplified, predictive neural networks are trained to perform pattern recognition. Anyway, also generative networks can be quite strong in pattern recognition based on which they generate new content (e.g. GPT-4 in understanding and summarizing a text). Similarly, text translation may be a type of predictive AI or generative AI. In my personal view, since in particular generative neural networks quickly evolve, also my differentiation made above will change.
Which neural network types exist?
There exist many different types of neural networks (i.e. network architectures) which however all follow the substantial principles described in the previous blog posts about AI basics. Some prominent examples (beside many others) include:
- convolutional neural networks (“CNN”; mainly used for feature extraction in images), and
- transformer models (mainly used in Natural Language Processing for text comprehension, e.g. in BERT or in more recent LLMs (large language models) like GPT-4).
What is unsupervised learning?
Beside “supervised learning” there also exists “unsupervised learning” where the samples do not contain labels, i.e. no human supervision. Unsupervised learning can be used to train a model to identify e.g. patterns or clusters within the data. For example, the trained model may than be able to assign any unknown input data sample to one of these clusters. However, most machine learning applications use supervised learning, as the models are intended to learn tasks which have formerly been carried out by humans (e.g. translations).
Since when does deep learning (neural networks) exist?
The basics for deep learning exist since long time. For example convolutional neural network (CNN), a specific neural network type, was already developed in 1989 by LeCunn among others. LSTM (Long short-term memory) another specific neural network type has already been invented in 1997 by Hofreiter et.al. However, I would say the actual breakthrough was only in 2017, when a CNN significantly outperformed all other technologies at the ImageNet Challenge. Why so late? As only at that time both sufficient computational resources and large datasets were publicly available.
Who is actually working in AI and how do they call their activity?
Since machine learning is mainly about data modeling (or in other words: pattern recognition in data clouds), it usually requires less coding work and more math (and by the way, lots of very unfancy data cleaning). For this reason, the guys working in this field are mainly data scientists having a strong background in statistics. Their activity is thus called data science.
However, there are various further tasks, e.g. pre- or post-processing data, or building an operational software program. These tasks require specialized software engineers, full-stack developers and other kind of coders.
Why is the terminology in AI so confusing?
If you have this feeling you are not alone. Indeed, the terminology used in AI often seems vague and arbitrary to me.
As explained in Part 1, “AI”, “machine learning” and “deep learning” are used pretty interchangeably. The same is true for the terms “neural network” and “model”.
You have heard about “SOTA models” and “vanilla models” and wonder, whether this is about AI in the food industry? Nope, it means “State Of The Art model” (the currently most performant “best in class” neural network) and “regular neural network without fancy stuff” (= vanilla). So, when you are asked, you should better say that you are always using the SOTA model for you specific task.
Furthermore, the field of “deep learning” (i.e. referring to “deep” neural networks) evolved in some works to “very deep neural networks” (e.g. very deep CNNs, pointing probably to the circumstance that further layers have been added). The question remains: How “deep” is “very deep”?
Recently, the term “LLM” (= large language model, not “master of laws”) has become popular in the field of NLP. But when does a language model actually become “large”? A data scientist I have been working with considers BERT (published in 2018 and having 110 Million parameters) as a LLM. However, personally I cannot remember having heard this term until the rise of generative AI/foundational models (= Chat-GPT) in late 2022. The underlying models, such as GPT-3.5 or GPT-4 have 175B to 1 Trillion parameters, i.e. 1.500 to 10.000 times more than BERT (correct me if my calc is wrong). GPT-3.5 or GPT-4 are thus comparatively “large” in view of BERT.
You probably have to be a data scientist who works all day with quantifiable data to come up with such quantitatively meaningless terms. Anyway, the trend seems to be going from “deep” towards “large”. Let’s see what comes next, maybe “high”?.
How can you learn more about AI?
You want to understand how a neural network actually works in a nice and gentle way? Then I strongly recommend that you read the book “Make Your Own Neural Network” by Tariq Rashid, 2017. The book guides you of step by step through the math going on in a neural network. This nice thing: You don’t have to be a math pro, basically addition and multiplication are enough (that’s principally all what’s happening in a neural network).
You want to learn more about the internal mechanisms of neural network?
Then have a look to https://ayearofai.com/ . The blogs are indeed also some years old, but they give easy-to-understand insights into different topics, such as CNNs, back propagation, or the logistic regression. If you really want to understand AI (SOTA technology like GPT 4o included), you should know and understand these things.
You want to dive deeper into the theories behind machine learning and deep learning?
Then I recommend the book “Deep Learning” of Ian Goodfellow et.al. from 2016, which can be accessed for free here: https://www.deeplearningbook.org/ . I anyway suggest buying the book. It has a nice cover and looks impressive in any bookshelf.
You are into NLP, want to understand Transformer models and even get some hands-on experience?
Then I recommend “Natural Language Processing with Transformers: Building Language Applications with Hugging Face” of Tunstall et.al. of 2022.
I must admit though that I have not found a good book about SOTA LLMs yet. Suggestions are very welcome!
author: Christoph Hewel
email: hewel@paustian.de
(photo: Hanging out in Calanque de Sormiou, Marseille, PACA, France. If AI could ever be so fancy)