ChatGPT was formed by OpenAI using a machine learning model called GPT (Generative Pretrained Transformer). Its initial training involved learning from an extensive amount of internet text data. However, it’s important to note that ChatGPT doesn’t know the specifics about which documents were in its training set and it was not directly trained to understand specific books, authors, sources, etc. After this preliminary training, the model was fine-tuned with reinforcement learning from human feedback. This included both supervised fine-tuning, with human AI trainers providing conversations and playing both user and AI roles, and reinforcement learning from human feedback, where AI trainers ranked different model responses by quality to train a reward model. These methods combine to form ChatGPT.
