Open topic with navigation

# Manipulate and Enrich Documents

CFS provides features to manipulate and enrich documents. This means that you can add additional information to the documents, and improve the quality of the information, before the documents are indexed into IDOL. For example, you can:

- Add additional fields to a document.
- Extract content from HTML pages, discarding irrelevant content such as headers, sidebars, advertisements, and scripts.
- Split long documents into multiple sections. This can improve performance when you query IDOL, because IDOL can return a specific part of a document in response to a query.
- Reject documents that do not contain content in a specific language.
- Perform Eduction on document fields. Eduction extracts entities from a document, and writes them to specific document fields. An entity can be a word, phrase, or block of information - for example an address or telephone number.
- Perform Optical Character Recognition (OCR), object detection, or face recognition on image files and add the results to the document.
- Extract speech from audio and video files, and add the transcription to the document content.

The simplest way to manipulate documents is to use the import tasks and index tasks that are included with CFS. For information about the tasks that are available, see Manipulate and Enrich Documents. You can configure these tasks by modifying configuration parameters in the CFS configuration file.

Import tasks can call other services, for example an IDOL Speech Server or Image Server. You can use an Image Server to perform Optical Character Recognition, logo detection, or face recognition on images. You can use an IDOL Speech Server to extract speech from audio and video files. This enables IDOL to use the data in images and speech for retrieval, clustering, and other operations.

CFS also supports Lua, an embedded scripting language. You can write Lua scripts to manipulate documents and define custom processing rules. For information about the Lua functions that are provided with CFS, refer to the Connector Framework Server Reference.

Related Topics

- Import Process
- Write and Run Lua Scripts

© 2016 Hewlett Packard Enterprise Development LP | Send documentation feedback to HPE Close We welcome your comments! To open the configured email client on this computer, open an email window. Otherwise, copy the information below to a web mail client, and send this email to AutonomyTPFeedback@hpe.com. Product: Topic Title: Help Topic: Feedback:

_HP_HTML5_bannerTitle.htm