txtai 教程系列 10----从文档中提取文本

柔光的暖阳◎ 2023-09-27 19:21 18阅读 0赞

安装依赖  
安装txtai和所有依赖项。由于本文使用的是可选管道，因此我们需要安装管道附加包。  
pip install txtai\[pipeline\]

## Get test data ##

wget -N https://github.com/neuml/txtai/releases/download/v3.5.0/tests.tar.gz  
tar -xvzf tests.tar.gz

## Install NLTK ##

import nltk  
nltk.download(‘punkt’)  
创建一个 Textractor 实例  
Textractor 实例是提取文本的主要入口点。此方法由 Apache Tika 提供支持，Apache Tika 是一个用 Java 编写的强大的文本提取库。Apache Tika支持大量文件格式：PDF、Word、Excel、HTML 等。Python Tika 包会自动安装 Tika 并启动用于读取提取数据的本地 REST API 实例。

注意：这需要在本地安装 Java。  
from txtai.pipeline import Textractor

## Create textractor model ##

textractor = Textractor()  
提取文本  
下面的示例显示了如何从文件中提取文本。  
textractor(“txtai/article.pdf”)  
Introducing txtai, an AI-powered search engine built on Transformers Add Natural Language Understanding to any application Search is the base of many applications. Once data starts to pile up, users want to be able to find it. It’s the foundation of the internet and an ever-growing challenge that is never solved or done. The field of Natural Language Processing (NLP) is rapidly evolving with a number of new developments. Large-scale general language models are an exciting new capability allowing us to add amazing functionality quickly with limited compute and people. Innovation continues with new models and advancements coming in at what seems a weekly basis. This article introduces txtai, an AI-powered search engine that enables Natural Language Understanding (NLU) based search in any application. Introducing txtai txtai b