Generative AI

Generative AI Fundamentals

• Lesson 1: Introduction to Generative AI foundations
  Employ the abilities of Generative AI with a deep dive into fundamentals. This course examines how various models are developed, how they work and how to use them to their full potential.
• Lesson 2: Generative AI Overview
  Explore the fundamentals of generative AI, its key modalities, advanced capabilities and essential ethical considerations shaping responsible AI development.
• Lesson 3: Applications of Generative AI
  Explore real-world applications of Generative AI, including LLM-assisted coding and learn to prompt, validate and improve AI-generated code and tests.
• Lesson 4: Introduction to Foundation Models
  Discover foundation models: large, versatile AI systems trained on massive datasets that generalise across tasks, surpassing traditional models in scalability and adaptability.
• Lesson 5: Building Applications using Foundation Models
  Learn to build text classifiers with foundation models, using zero-shot and few-shot prompt engineering for tasks like sentiment and spam detection and evaluate classifier accuracy.
• Lesson 6: How Generative AI works
  Learn how generative AI creates new data with architectures like Transformers and diffusion models and how training enables creativity, reasoning and task-specific abilities.
• Lesson 7: Evaluating Generative AI Models
  Learn how to assess generative AI using human evaluation, exact metrics, AI judges and benchmarks, ensuring robust performance for open-ended, probabilistic model outputs.
• Lesson 8: Implementing Evaluations for Generative AI Models
  Learn practical techniques to evaluate generative AI models, from Exact Match to ROUGE, semantic similarity, code correctness, Pass@k and LLM-as-a-Judge scoring.
• Lesson 9: Neural Networks and Multilayer Perceptrons
  Explore neural networks from perceptrons to multilayer perceptrons, learning how they adapt via training, gradient descent and backpropagation to solve complex AI tasks.
• Lesson 10: Implementing Neural Networks using PyTorch
  Learn to implement neural networks in PyTorch by mastering tensors, model building, loss functions, optimisers, data loading and complete training loops for practical machine learning.
• Lesson 11: Model Interpretability and Ethics
  Explore AI model interpretability and ethics, including bias, misinformation, environmental impact and fairness for responsible development and deployment of AI technologies.
• Lesson 12: Generating Text using LLMs
  Discover how LLMs generate text token by token using Hugging Face's Transformers, from tokenisation to model use and explore hands-on demos with efficient generation methods.
• Lesson 13: Role-Based Prompting
  Explains the theory of using roles or personas to control the tone, style and expertise of an LLM's output.
• Lesson 14: Implementing Role-Based Prompting with Python
  Provides hands-on practice in iteratively developing a role-based prompt to create a believable historical figure persona.
• Lesson 15: Adapting Foundation Models
  Learn to adapt foundation models for specialised tasks using prompt engineering, RAG, fine-tuning, model compression and agentic AI tools for efficient, tailored AI solutions.
• Lesson 16: Applying PEFT on Foundation Models
  Learn to efficiently customise foundation models with PEFT and SFT, using LoRA to teach LLMs new skills like spelling via hands-on data preparation and fine-tuning.
• Lesson 17: Post-Training Foundation Models
  Explore post-training for foundation models, including supervised and preference fine-tuning, to align AI with human values, improve usability and ensure responsible interactions.
• Lesson 18: Reinforcement Fine-tuning on Foundation Models
  Learn to fine-tune LLMs for structured tasks like counting and spelling using GRPO and LoRA, applying reinforcement-based reward functions for targeted skill improvements.

Project: Teaching an LLM to count the number of letters in a word using GRPO.

Large Language Models (LLMs) and Retrieval Augmented Generation (RAG)

• Lesson 1: Introduction to LLMs and Retrieval Augmented Generation (RAG)
  Introduces Large Language Models (LLMs), their core concepts and the course structure. Covers prerequisites, environment setup and defines Retrieval-Augmented Generation (RAG).
• Lesson 2: The Large Language Model Landscape
  Explore the four core capabilities of LLMs: generation, summarisation, classification and reasoning. Real-world applications and the importance of RAG for building trust.
• Lesson 3: Implementing a Chatbot with an LLM
  Learn to build a stateful chatbot using an LLM. Covers managing conversation history, using system prompts to define behaviour and understanding message roles (system, user, assistant).
• Lesson 4: LLM Prompting and Inference
  Defines prompt engineering and its components. Explains how to control LLM outputs using inference parameters like temperature, top-p, max tokens and stop sequences.
• Lesson 5: Applied Prompting and Inference
  Apply prompting techniques hands-on. Implement Chain of Thought (CoT) prompting to improve reasoning and test how different inference parameters change model behaviour.
• Lesson 6: Prompt Instruction Refinement
  Explains the theory of systematically refining prompt instructions by modifying components like Role, Task, Context, Examples and Output Format.
• Lesson 7: Applying Prompt Instruction Refinement with Python
  Provides hands-on practice, iteratively refining a prompt to transform a generic recipe analyser into a precise dietary consultant that produces structured JSON.
• Lesson 8: Tokens, Embeddings and Vector Search
  Covers the foundations of NLP for LLMs. Defines tokenisation, embeddings as semantic vectors and vector search (similarity search) as the basis for finding relevant information.
• Lesson 9: Implementing Tokens, Embeddings and Vector Search
  Hands-on practice with tokenisation. Implementing embedding generation and vector search to build a semantic search system from scratch.
• Lesson 10: Strategic Model Selection and Economics
  Learn the business trade-offs of model selection. Covers performance, cost, speed and control (TCO). Compares general-purpose (generation) vs. specialised (reasoning) models.
• Lesson 11: Applying Model Selection and Economics
  Apply model selection theory. Calculate Total Cost of Ownership (TCO), including error costs. Implement a hybrid model routing system to balance cost and quality.
• Lesson 12: Retrieval Augmented Generation (RAG) Workflow
  Introduces the RAG architecture. Compares naive vs. advanced modular RAG. Covers the data ingestion pipeline, focusing on data formats and intelligent chunking strategies.
• Lesson 13: Semantic Search with Vector Databases for RAG
  Explains semantic search and the role of vector databases. Covers indexing algorithms (HNSW) for speed and advanced retrieval techniques like HyDE and re-ranking.
• Lesson: 14 Prompt Engineering for RAG Synthesis
  Learn to write prompts for RAG. Covers grounding answers in context, handling conflicts, managing uncertainty and enforcing verifiability by generating inline citations.
• Lesson 15: Implementing RAG with Vector Databases
  Build a complete RAG system. Practice vector database operations in ChromaDB, including adding documents, applying metadata filters and implementing a retrieval and generation pipeline.
• Lesson 16: Evaluating RAG Systems
  Learn to evaluate RAG system quality. Introduces key metrics: Context Precision, Context Recall, Faithfulness and Answer Relevancy. Covers frameworks like RAGAS.
• Lesson 17: RAG Evaluation Implementation
  Implement a RAG evaluation pipeline using the RAGAS framework. Learn to calculate and interpret quality metrics to diagnose and improve your RAG system's performance.

Project: Build an end-to-end RAG chatbot. Ingest NASA mission data, build a vector search pipeline, generate answers and evaluate the system's quality.

Multimodal AI Applications

• Lesson 1: Introduction
  Introduction to the Multimodal AI Applications course and what you will learn.
• Lesson 2: Multimodal AI Fundamentals
  Discover multimodal AI fundamentals and technologies, including models and use cases that process and generate text, images, audio and video for richer, real-world applications.
• Lesson 3: Using Multimodal AI Technologies
  Explore practical applications of multimodal AI by using APIs and open-source models for image captioning and audio transcription, with hands-on exercises and secure credential handling.
• Lesson 4: Transformers and Multimodal Processing
  Explore how transformers unify text, images, audio and video through attention, embeddings and fusion strategies, powering state-of-the-art multimodal understanding and generation.
• Lesson 5: Multimodal AI Tooling
  Explore practical tools for building multimodal AI apps, compare commercial and open-source options and use Pydantic AI to create reliable, structured, vendor-agnostic workflows.
• Lesson 6: Introduction to Enterprise Visual Content Processing
  Explore enterprise visual content processing: core computer vision tasks, digital image representation and real-world applications for efficiency, safety and automation.
• Lesson 7: Vision Pre-processing Pipelines with HuggingFace
  Explore vision data pipelines using HuggingFace, from dataset loading to resizing and normalisation, with demos and hands-on exercises for effective image pre-processing.
• Lesson 8: Understanding Embeddings in Computer Vision Learn how embeddings convert images into compact vectors for efficient search, enable cross-modal tasks with models like CLIP and power large-scale, robust computer vision systems.
• Lesson 9: Image Search Using CLIP Embeddings
  Explore how to build text-to-image and image-to-image search using CLIP embeddings, combining theory, real-world demos, hands-on practice and solution walkthroughs.
• Lesson 10: Using Multimodal Model APIs for Vision
  Explore multimodal vision APIs: prompt design, parameter tuning, structured outputs, cost control, integration and best practices for robust, efficient image analysis.
• Lesson 11: Gemini Vision API Basics
  Explore Gemini Vision API basics by practising image moderation, learning to analyse images and implement moderation workflows using real-world examples and guided hands-on exercises.
• Lesson 12: Vision Transformer Models and Architectures
  Explore Vision Transformer models: core architecture, image tokenisation, self- and cross-attention and top models for segmentation, detection and enterprise use.
• Lesson 13: Using Vision Transformers
  Explore vision transformers with hands-on demos to extract image embeddings and perform object detection and segmentation using state-of-the-art models.
• Lesson 14: Vision-Language Models
  Learn how vision-language models align images and text for tasks like search, captioning and visual question answering, with a focus on enterprise deployment considerations.
• Lesson 15: Multimodal Vision Applications with CLIP
  Explore zero-shot image classification and auto-labelling for driving scenes using CLIP, enabling efficient, scalable multimodal vision applications.
• Lesson 16: Diffusion Models and Image Generation
  Explore how diffusion models generate images by reversing noise through iterative denoising, a key technique behind modern generative image models.
• Lesson 17: Introduction to Enterprise Audio Processing
  Discover enterprise audio processing, including core speech tasks, use cases and integration strategies for modern business environments.
• Lesson 18: Audio Data Representation
  Explore how audio is digitised for AI, including sample rate, bit depth, channels, formats and best practices for preprocessing and analysis.
• Lesson 19: Audio Processing with librosa
  Explore audio processing with librosa to load, resample, convert, analyse and visualise audio data through hands-on exercises.
• Lesson 20: Sound Retrieval and Classification
  Explore audio embeddings for efficient sound classification and retrieval using models like CLAP to enable semantic audio analysis at scale.
• Lesson 21: Sound Retrieval and Classification with CLAP
  Apply CLAP for sound retrieval, similarity and zero-shot classification to detect fan on and off states in real audio data.
• Lesson 22: Speech Processing
  Discover automatic speech recognition with Whisper, a robust multilingual model for transcription, translation and real-world speech processing.
• Lesson 23: Implementing Speech Processing with Whisper and Gemini
  Explore real-world speech transcription and translation using Whisper and Gemini, including multilingual support and alignment techniques.
• Lesson 24: Audio Intelligence
  Explore advances in audio intelligence, including multimodal systems, speech recognition, text-to-speech, ethics and enterprise controls.
• Lesson 25: Audio Sentiment Analysis with Gemini
  Explore audio sentiment and command analysis using Pydantic AI and Gemini to extract emotions and recognise spoken commands.
• Lesson 26: Audio Classification and Moderation
  Explore voice content moderation, including compliance, privacy, layered detection and operational excellence.
• Lesson 27: Building a Basic Voice Moderation System with Gemini
  Build a voice moderation system using Gemini to transcribe audio, detect personal data disclosures and flag policy violations.
• Lesson 28: Introduction to Enterprise Video Processing
  Discover how enterprise video AI addresses temporal complexity using efficient frame selection for understanding and moderation.
• Lesson 29: AI Models for Video Understanding
  Explore AI models for real-time detection, motion tracking and temporal understanding to enable scalable video analytics.
• Lesson 30: Implementing Object Recognition and Tracking
  Learn how to detect and track objects in videos, apply multi-object tracking and count items in practical scenarios.
• Lesson 31: Video Understanding and Search
  Explore methods for analysing and searching video using foundation models, balancing accuracy, cost and performance.
• Lesson 32: Video Understanding and Search with Gemini and Clip4Clip
  Explore automated video description, key moment detection and natural language video search using AI models and structured outputs.
• Lesson 33: Video Classification and Moderation
  Learn to classify and moderate video by modelling temporal patterns and combining automation with human oversight.
• Lesson 34: Video Classification and Moderation with Gemini
  Build automated systems for video classification and moderation using Gemini and Pydantic AI in real-world scenarios.
• Lesson 35: Video Generation
  Explore generative video AI tools and workflows that turn text, images or footage into dynamic video content.
• Lesson 36: Video Generation with Veo 3
  Generate marketing videos using Veo 3 with text-to-video and image-to-video workflows, understanding strengths and limitations.
• Lesson 37: Multimodal AI Deployment
  Explore deployment strategies for multimodal AI systems via unified APIs and orchestration approaches.
• Lesson 38: Implementation Tools and Serving Strategies
  Explore tools and strategies for implementing, serving and monitoring AI solutions from prototyping to production.
• Lesson 39: Using Gradio and Pydantic AI
  Build multimodal chatbots and analysis apps using Gradio and Pydantic AI, covering async programming and interface customisation.
• Lesson 40: Multimodal AI Performance Monitoring and Logging
  Learn to monitor and log multimodal AI systems, tracking performance, costs and failures across modalities.
• Lesson 41: Logging and Performance Monitoring with Gradio and Arize Phoenix
  Implement logging and performance monitoring for multimodal AI chatbots to enable robust analytics and debugging.
• Course Project: Evaluating Multimodal Applications
  Learn how to evaluate multimodal AI applications using user feedback, automated metrics and continuous monitoring.
• Lesson 43: Testing Multimodal Apps with Pydantic AI Evals
  Build robust testing frameworks for multimodal AI apps using structured outputs and semantic evaluation techniques.
• Lesson 44: Scaling Multimodal AI Architecture
  Learn strategies to scale multimodal AI systems, focusing on performance, reliability, cost and architectural trade-offs.

Project: Create an AI agent that simulates customer service scenarios and specialised monitoring agents that analyse text, images, video and audio communications.