Pioneering AI: Navigating the 2025 AI Learning Journey

Watch Neural Networks: Zero to Hero

Watch the “Neural Networks: Zero to Hero” video series by 3Blue1Brown to gain a visual and intuitive understanding of how neural networks work.

It starts with explaining and coding backpropagation from scratch and ends with writing GPT from scratch.

Neural Networks: Zero To Hero by Andrej Karpathy

He just released a new video → Let’s build the GPT Tokenizer

You can also look at GPT in 60 Lines of NumPy | Jay Mody while you’re at it.

Free LLM boot camp

Enroll in a free LLM boot camp like the one offered by Anthropic to learn about the fundamentals of large language models and how to build applications using them.

A paid LLM Bootcamp released for free by Full Stack Deep Learning.

It teaches prompt engineering, LLMOps, UX for LLMs, and how to launch an LLM app in an hour.

Now that you’re itching to build after this boot camp,

Build with LLMs

Build applications using pre-trained language models like GPT-3, BERT, and T5. Fine-tune these models on your own datasets to solve specific problems.

Want to build apps with LLMs?

Watch Application Development using Large Language Models
by Andrew Ng

Read Building LLM applications for production by Huyen Chip

As well as Patterns for Building LLM-based Systems & Products by Eugene Yan

Refer to the OpenAI Cookbook for recipes.

Use Vercel AI templates to get started.

Participate in hackathons

Participate in hackathons and challenges focused on building applications using large language models. This will help you learn from others and gain practical experience.

lablab.ai has new AI hackathons every week. Let me know if you want to team up!

If you want to go deeper into the theory and understand how everything works:

Read papers

Read research papers on large language models published in conferences like ACL, EMNLP, and NAACL. Focus on understanding the key ideas and how they can be applied in practice.

A great article by Sebastian Raschka on Understanding Large Language Models, where he lists some papers, you should read.

He also recently published another article with papers you should read in January 2024, covering mistral models.

Follow his substack Ahead of AI.

Write Transformers from scratch

Implement Transformer models from scratch using PyTorch or TensorFlow. This will give you a deeper understanding of how they work and how to customize them for your specific needs.

Read The Transformer Family Version 2.0 | Lil’Log for an overview.

Choose whichever format suits you best and implement it from scratch.

Paper

• Attention Is All You Need
• The Illustrated Transformer
• The Annotated Transformer by Harvard
• Thinking like Transformer

Some good blogs

Follow blogs like The Gradient, Lil’Log, and Distill.pub to stay up-to-date with the latest research and trends in AI and machine learning.

• Creating a Transformer From Scratch — Part One: The Attention Mechanism (part 2) (code)
• Understanding and Coding the Self-Attention Mechanism of Large Language Models From Scratch by Sebastian Raschka, PhD
• Transformers from scratch

Videos

• Coding a Transformer from scratch on PyTorch, with full explanation, training and inference
• NLP: Implementing BERT and Transformers from Scratch

You can code transformers from scratch now. But there’s still more.

Watch these Stanford CS25 — Transformers United videos.

Some good blogs

• Gradient Descent into Madness — Building an LLM from scratch
• The Illustrated Transformer — Jay Alammar
• Some Intuition on Attention and the Transformer by Eugene Yan
• Speeding up the GPT — KV cache | Becoming The Unbeatable
• Beyond Self-Attention: How a Small Language Model Predicts the Next Token
• Llama from scratch (or how to implement a paper without crying) | Brian Kitano
• Improving LoRA: Implementing Weight-Decomposed Low-Rank Adaptation (DoRA) from Scratch

Watch Umar Jamil

Watch Umar Jamil’s video series on YouTube to learn about advanced topics in deep learning and large language models.

Learn how to run open-source models

Learn how to run open-source models like GPT-2, GPT-3, BERT, and T5 using tools like Hugging Face Transformers. This will give you a better understanding of how these models work and how to use them in practice.

He has fantastic in-depth videos explaining papers. He also shows you the code.

• LoRA: Low-Rank Adaptation of Large Language Models — Explained visually + PyTorch code from scratch
• Mistral / Mixtral Explained: Sliding Window Attention, Sparse Mixture of Experts, Rolling Buffer
• Attention is all you need (Transformer) — Model explanation (including math), Inference and Training
• LLaMA explained: KV-Cache, Rotary Positional Embedding, RMS Norm, Grouped Query Attention, SwiGLU
• Retrieval Augmented Generation (RAG) Explained: Embedding, Sentence BERT, Vector Database (HNSW)

Some more links related to LLMs that are not exhaustive. Look at LLM Syllabus for a more comprehensive syllabus for LLMs.

Prompt Engineering

Learn about prompt engineering, which is the art of crafting effective prompts to get the desired output from large language models. This is a crucial skill for building applications using LLMs.

Read Prompt Engineering | Lil’Log

ChatGPT Prompt Engineering for Developers by Ise Fulford (OpenAI) and Andrew Ng

DeepLearning.ai also has other short courses you can enroll in for free.

Fine-tuning LLMs

Learn how to fine-tune large language models on your own datasets to solve specific problems. This involves training the model on your data and adapting it to your specific use case.

Read the Hugging Face fine-tuning guide.

A good guidebook: Fine-Tuning — The GenAI Guidebook

Check out axolotl.

This is a good article: Fine-tune a Mistral-7b model with Direct Preference Optimization | by Maxime Labonne

RAG

Learn about Retrieval Augmented Generation (RAG), which is a technique for combining large language models with information retrieval systems to improve their performance on specific tasks.

A great article by Anyscale: Building RAG-based LLM Applications for Production

A comprehensive overview of Retrieval Augmented Generation by Aman Chadha

Remember, learning AI is a journey, not a destination. Stay curious, experiment, and don’t be afraid to make mistakes. By following this guide and continuously learning, you’ll be well on your way to becoming an AI trailblazer in 2024 and beyond.