|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/rag-llm-evaluation-ai-test/
课程评论：没有评论
课程名称：RAG-LLM评估与测试自动化入门 课程概述：当前，LLM（大语言模型）在各个行业中广泛应用，众多企业为了提升客户服务正构建定制的基于AI的RAG-LLM。工程师们如何对这些系统进行测试呢？不同于传统软件测试，AI系统需要特殊的评估方法。该课程从基础开始，讲解AI系统（LLM）的架构如何在幕后运作，然后深入探讨LLM评估指标。课程将教你如何有效使用RAGAS框架库，通过脚本示例评估LLM指标。你将学习如何使用Pytest断言检查指标基准得分，并设计强大的LLM测试/评估自动化框架。 课程学习内容： - 对大语言模型（LLM）的高层次概述 - 理解基于检索增强生成（RAG）架构的定制LLM的构建方式 - 常用的RAG基于LLM的评估基准/指标 - RAGAS评估框架的介绍，用于评估/测试LLM - 生成实用脚本以自动化并断言LLM的指标得分 - 使用RAGAS框架自动化单轮和多轮与LLM的交互场景 - 为评估LLM的指标生成测试数据 通过本课程，学员将能够创建RAGAS Pytest评估框架，以断言RAG-（定制）LLM的指标。同时，课程涵盖了七种常用的评估和测试LLM的指标理论，学员也能够将相同的方法应用到其他任何指标的评估中。 实践经验：课程提供RAG-LLM的实操机会，但在脚本阶段，学员需要拥有Open AI的基本订阅服务（最低10美元的信用额度即可）以访问API。 课程先决条件：需具备Python和PyTest基础知识，以理解框架。课程结束时有两个专门部分提供所需的Python和Pytest知识。此外，基本的API测试知识也是必需的。
LLMs are everywhere! Every business is building its own custom AI-based RAG-LLMs to improve customer service. But how are engineers testing them? Unlike traditional software testing, AI-based systems need a special methodology for evaluation.This course starts from the ground up, explaining the architecture of how AI systems (LLMs) work behind the scenes. Then, it dives deep into LLM evaluation metrics.This course shows you how to effectively use the RAGAS framework library to evaluate LLM metrics through scripted examples. This allows you to use Pytest assertions to check metric benchmark scores and design a robust LLM Test/evaluation automation framework.What will you learn from the course?High level overview on Large Language Models (LLM)Understand how Custom LLM's are built using Retrieval Augmented Generation (RAG) ArchitectureCommon Benchmarks/Metrics used in Evaluating RAG based LLM'sIntroduction to RAGAS Evaluation framework for evaluating/test LLM'sPractical Scripts generation to automate and assert the Metrics Score of LLM's.Automate Scenarios such as Single turn interactions and Multi turn interactions with LLM's using RAGAS FrameworkGenerate Test Data for evaluating the Metrics of LLM using RAGAS Framework.By end of the course, you will be able to create RAGAS Pytest Evaluation Framework to assert the Metrics of RAG- (Custom) LLM'sImportant Note:This course covers Top 7 Metrics which are commonly used to Evaluate and test the LLM's. Same logic can be applied to rest of any other metric evaluations.Handson Experience:Course provides Practice RAG -LLM for you for Handson, but at scripting phase, you need a basic subscription of Open AI to access their API's (Minimal 10$ credit will suffice)Course Prerequisites:Python, PyTest basics are required to understand the Framework.We have 2 dedicated sections at the end of this course which gives you necessary knowledge on Python & Pytest required to follow the course.Basic knowledge on API Testing.
