|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/data-cleaning-in-python/
课程评论：没有评论
**课程名称：** Python 数据清洗 **课程概述：** 本课程深入探讨了数据清洗在构建智能自动化系统中的关键作用。数据清洗作为数据预处理的重要环节，旨在提升数据的有效性、准确性、完整性、一致性和统一性，为构建可靠且能产生良好结果的机器学习模型奠定基础。课程强调，即使拥有再优秀的模型，如果数据质量不高，模型结果也无法信赖。 课程指出，初学者接触到的公开数据集通常已经过充分分析，可以直接用于模型训练。然而，现实世界中的数据往往充斥着各种问题，包括但不限于： * **缺失值：** 数据集中存在空白或未填充的条目。 * **噪声值 / 单变量异常值：** 数据中存在偏离正常范围的随机错误或极端值。 * **多变量异常值：** 在多个特征共同作用下，出现不符合常规模式的数据点。 * **数据重复：** 同一条记录在数据集中出现多次。 * **数据标准化与归一化：** 提升数据质量，使其符合统一的格式和尺度。 * **处理类别型特征：** 对非数值型数据进行适当的转换和处理。 课程强调，未经数据清洗的原始数据，无论数据来源如何（包括从多个在线源获取的用于构建应用程序的原始数据），都无法产生有价值的分析和应用。“垃圾进，垃圾出”原则在此尤为适用，即不解决数据问题，模型效率再高，结果也不可靠。 **课程内容结构：** 本课程将系统性地讲解数据清洗的常见问题及其解决方案。每个知识点都包含三个组成部分： 1. **理论解释：** 介绍相关概念的背景和意义。 2. **数学评估：** 通过数学方法分析和量化数据问题。 3. **代码实现：** 使用 Python 和 Jupyter Notebook 演示解决这些问题的实际代码。 课程的讲座内容以 `*.1.*` 和 `*.2.*` 的格式命名，其中第一个 `*` 代表章节号，第二个 `*` 代表该章节内的讲座序号。`*.1.*` 讲座侧重于理论和数学评估，而 `*.2.*` 讲座则专注于实际的 Python 代码实现。 **核心价值：** 通过本课程的学习，学员将掌握识别和解决数据中各种常见问题的能力，从而能够对业务数据进行有意义的分析，并构建更可靠、更有效的机器学习模型。
Data cleaning or Data cleansing is very important from the perspective of building intelligent automated systems. Data cleansing is a preprocessing step that improves the data validity, accuracy, completeness, consistency and uniformity. It is essential for building reliable machine learning models that can produce good results. Otherwise, no matter how good the model is, its results cannot be trusted. Beginners with machine learning starts working with the publicly available datasets that are thoroughly analyzed with such issues and are therefore, ready to be used for training models and getting good results. But it is far from how the data is, in real world. Common problems with the data may include missing values, noise values or univariate outliers, multivariate outliers, data duplication, improving the quality of data through standardizing and normalizing it, dealing with categorical features. The datasets that are in raw form and have all such issues cannot be benefited from, without knowing the data cleaning and preprocessing steps. The data directly acquired from multiple online sources, for building useful application, are even more exposed to such problems. Therefore, learning the data cleansing skills help users make useful analysis with their business data. Otherwise, the term 'garbage in garbage out' refers to the fact that without sorting out the issues in the data, no matter how efficient the model is, the results would be unreliable. In this course, we discuss the common problems with data, coming from different sources. We also discuss and implement how to resolve these issues handsomely. Each concept has three components that are theoretical explanation, mathematical evaluation and code. The lectures *.1.* refers to the theory and mathematical evaluation of a concept while the lectures *.2.* refers to the practical code of each concept. In *.1.*, the first (*) refers to the Section number, while the second (*) refers to the lecture number within a section. All the codes are written in Python using Jupyter Notebook.
