Train Your First Small Language Model — TRI AI Saturdays, Cohort 10
The hands-on lab you used in class: tokenize text, build the transformer, and train your own tiny SLM.
Open the SLM Lab notebook in ColabThe original notebook this session is adapted from — handy for comparing and extra experiments.
Open the original Colab notebookWhat you explored today:
• Tokenization (BPE): how text becomes tokens.
• Embeddings: each word becomes a vector — the heart of the model.
• Attention & transformer blocks: how context flows. --> [We will be covering these in the next classes]
• Training: learning better embeddings to predict the next token.
• Scale: the same recipe at larger size reaches Larger Number of parameters.
All session materials live in the course repository.
Repo: github.com/AlexKalll/tri-AI-Teaching
Instructor: Kaletsidik Ayalew · github.com/AlexKalll · LinkedIn
Try these interactive tools to build intuition before/after the lab.
Tokenizer Playground Word Embedding Playground Embedding VisualizerIn partnership with Google DeepMind · TRI AI Saturdays, Cohort 10