|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/big-data-and-machine-learning-part-1-extract-data-from-pdf/
课程评论:没有评论
**课程名称:** 建筑业大数据:从 PDF 中提取数据 **课程概述:** 本课程是为零编程基础的初学者设计的,旨在介绍 #大数据 和 #机器学习 概念,并使用 #Python 进行编程。我们将通过真实数据集,分步讲解大数据与机器学习的相关主要流程。本部分(第一部分)是整个课程的开篇,专注于数据收集与提取,特别是从 PDF 文档、图纸等各类 PDF 格式文件中提取数据。 **课程内容亮点:** * **Python 入门:** 学习安装 Python、Anaconda,以及选择和配置适合初学者的 Python IDE(如 VS Code)。 * **PDF 数据提取:** 掌握使用 Python 和 Tika OCR 技术将 PDF 文件转换为文本内容和元数据的方法,并学习处理文件迭代、字符串分割和列表清理的实用技巧。 * **正则表达式应用:** 深入学习 Python 中的正则表达式,包括模式匹配、日期格式提取、数组匹配以及如何构建正则表达式循环,为后续的数据清洗和结构化打下基础。 * **Python 基础:** 学习 Python 中的数组(列表)操作,包括数据添加、索引查找,以及如何创建和使用函数。 * **Pandas DataFrame:** 学习如何安装 Pandas,创建和操作强大的二维表格数据结构 DataFrame,包括列操作、合并列、转换数据类型以及将 DataFrame 保存为 CSV 文件。 * **数据可视化与 Kaggle:** 学习在 Kaggle 平台上创建账户,使用 Jupyter Notebook 进行实时编码和数据可视化。重点讲解如何使用 Matplotlib 和 Seaborn 库对数据进行图形化展示,以及如何处理 CSV 文件、转换日期格式、计算日期差、添加/删除列等 DataFrame 操作。 * **真实数据集实战:** 将有机会处理两个包含大量 PDF 文件的真实数据集,将 PDF 数据转换为文本和表格形式,并在 Kaggle 上进行可视化分析。 * **GitHub 代码管理:** 学习 GitHub 的基本概念和使用方法,包括如何上传文件、创建和同步仓库,以及使用 GitHub Desktop 来管理你的代码和项目。 **课程特色:** 本课程强调实践性,将带领学员从 Python 环境搭建到 Kaggle 平台的数据可视化,经历完整的数据提取及分析流程。课程作者将分享自己在学习过程中遇到的软件安装和库配置难题,并提供简洁有效的解决方案,帮助学员节省宝贵时间。课程还将解答与大数据和机器学习主题相关的基础疑问,提供更有针对性的学习指导。 **最终目标:** 通过学习本课程,学员将能够独立完成从 PDF 文档中提取数据、使用 Python 进行数据处理和可视化,并将代码托管到 GitHub 的全过程。
This course is intended to be an initiation to learn #BigData and #MachineLearning with #Python programming for absolute beginners that have no background in programming. In this course, we will step by step, using the example of real data, we will go through the main processes related to the topic "Big data and machine learning". Since the material turned out to be voluminous, I divided the course into five parts. ⇉ This part - the first part is devoted to the collection and extraction of data from documents. In this course, you will learn how to extract data from PDF documents, drawings and any other documents in PDF format. ⇉ We will work on real data. We will have two sets of data consisting of PDF files that we will transform to the text and to tabular form. We will visualize the received data on the Kaggle platform using python libraries, which will help us to depict our received data in a graphical format. ⇉ During the training process, we will install Python and such libraries as Pandas, seaborn, matplotlib and others. We will upload the received data to the Kaggle platform and here using the "Jupiter Notebook" we will visualize our data and at the end, we will upload our data to the GitHub platform.⚐ Topics covered in this course: Lecture 2. Python. Choosing python IDE. Anaconda. Install Python.How to convert a PDF to text?Python or Anaconda?What is the best Python IDE for beginners?How do I install VS Code?How do I install Python?How to run Python in VS Code?How does Python interpreter choose VS code?Lecture 3. 1st Dataset. PDF files. Tika OCR. Extracting content and metadata.How do I convert a PDF to TXT in Python?How can I iterate over files in a given directory?Install Apache Tika on Windows.How to split a string into a list?Remove blank strings from a list?Lecture 4. Regular Expression in Python. Pattern matching in Python.What is regular expression with example?How to match regular expression in Python?Debug a regular expression in Python?What is the regular expression for date format?How do you check if an array contains a regular expression?Create loop with regular expression.Lecture 5. Array und Function in Python. Add data to Array. Create function.How do you add a string to an array?How do you find the index of an element in a list?How can I extract the date from a string?How to declare and add items to an array in Python?How do you write a function in Python?Lecture 6. Pandas DataFrame. Two-dimensional size-mutable, tabular data structure.Install pandas on PythonHow do I create a pandas DataFrame?Reduce number of columns in a pandas DataFrameCombine column values into a list in a new columnHow to convert array into DataFrame in Python?How to change column names in pandas Dataframe?Save a Dataframe as CSV tableLecture 7. Kaggle. Jupiter Notebook. Create an account. Plotting with matplotlib and seaborn.Upload a file to kaggle kernelHow do you use kaggle dataset?Run Jupyter notebook using Kaggle kernelsConvert a CSV to dataframe in Python Jupyter NotebookHow to use the functions of Pandas Dataframe?Change the date format of a column in pandasHow do I convert a string to datetime Objects in Python?Calculate Difference Between Two Dates in Pandas DataframeHow do I delete a column in pandas DataFrame?Add columns in pandas DataFrame?How do you visualize a dataset?How do you plot a DataFrame in pandas?Lecture 8. 2nd Dataset. Task. Data from PDF. Getting data from PDF drawings.Independent Work TasksLearn to Code - on real data (16 PDF files to chart)A brief overview of the data in the taskLecture 9. 2nd Dataset. My solution.This is my solution.It may seem very simple and perhaps not the most effective.Lecture 10. GitHub. Desktop GitHub. Store and manage codeWhat is GitHub and how do you use it?What can I use GitHub for?How do I upload files to GitHub?Install GitHub DesktopHow to sync with a remote Git repository?Adding a repository from your local computer to GitHub⇛ This is a practical course where we will analyse the process of data extraction step-by-step. In this course, you will go through all the steps from installing python to data visualization on the Kaggle platform.When I got acquainted with the topic "Big Data and Machine Learning" myself, I often came across problems when installing software and errors while installing various libraries and tools for working with big data. ⇉ It took me a lot of time to find the right solutions and I would like to save this time for you. ⚐ To understand the topic, I had to look for a large number of questions for which I received non-targeted answers. In this course, you will find answers to basic questions that are related to the topic of Big Data and Machine Learning. Part 1: Extract Data from PDF.