AI training is about acquiring knowledge and inference is applying that knowledge to make predictions, generate answers and create original content. However, although a lot goes into both stages, inference is "where the magic takes place."
A machine learning (ML) model is the foundation of chatbots such as ChatGPT and Gemini. After its neural network architecture is designed and programmed, it is fed samples of data, which can include a highly selected amount of data or nearly all the information in the world.
The training and fine-tuning stages analyze the patterns in the data. This can take from days to months in data centers with tens to hundreds of thousands of servers operating together. For example, the GPT-4 model took several months and was trained on trillions of words. See
AI training passes,
AI training and
AI model.
Inference Processing
The AI application that people use (ChatGPT, Gemini, Grok, etc.), is called "inference," a term that dates back to the first AI programs (see
expert system). Once programmed, the inference software may be able to use different models for various purposes.
To generate output from the questions people enter, the inference software relies on the model for its intelligence. To get a single answer for the user takes a minuscule amount of time compared to the training phases. However, consider that millions of people are simultaneously using the same AI chatbot; therefore, if the model is very successful, inference processing can take much more data center time than the model's original training. See
neural network and
AI data center.
REGULAR APPLICATION DEVELOPMENT:
1. design the logic
2. code the logic
3. test application
4. run application
--------------------------------------------
--------------------------------------------
CHATBOT DEVELOPMENT:
(see AI secret sauce).
Model Development (Create the intelligence)
1. INITIAL DESIGN
1a. choose model design: CNN, RNN, etc.
1b. code the model
1c. choose hyperparameters
(number of layers, neurons, passes)
1d. parameters learned by software:
(weights and biases)
2. PRE-TRAIN with data samples
(sometimes the world's information)
2a. adjust hyperparameters
2b. parameters adjusted by software
3. FINE-TUNE with correct answers
3a. hyperparameters adjusted by programmer
3b. parameters adjusted by software
3c. programmer grades results (see RLHF)
See AI hyperparameter.
Inference Development (Execute the model)
1. design
2. code
3. optimize (see AI quantization)
Chatbot Development (Create the interaction)
1. design
2. code
3.
Execute Chatbot
1. run inference with built-in model
or
2. run inference and select model