Open topic with navigation

# Optimize the Language Identification Set

After you prepare the classifier list, you can run a language identification task. However, before you do so, HPE recommends that you optimize the language weights in the classifier list.

Optimization balances the language models. After training, some classifiers might be stronger than others owing to the properties of the training material and the languages themselves. The optimization process weights the language models so that weaker languages have increased recall, without compromising the precision for stronger language models.

The optimization process requires several audio files, each from a known language. It analyzes sections throughout each file, and determines whether each section was identified correctly. If a particular language is frequently not identified, IDOL Speech Server usually increases the weighting of the language classifier. Conversely, if IDOL Speech Server frequently identifies a particular language more often than it occurs, it usually reduces the weighting of the language classifier.

To optimize a language set

1. Gather the data to use. You need at least one audio file for each language represented in the classifier list. This audio data must not be the same data that you used to train the language classifiers.
2. Convert each audio file into a language identification feature file (see Create Language Identification Feature Files).

3. Create a list that contains the name of each feature file and a label that specifies the language that the file represents. Use the following format:

       LanguageName;LifFilename

   You can specify multiple files for each language. Either type the file names in a comma-separated list, or add multiple instances of a language. For example:

       English;english1.lif
       English;english2.lif
       French;french1.lif,french2.lif
       German;german.lif
       Italian;italian.lif

   For more information about IDOL Speech Server's list manager, see Create and Manage Lists.

4. If you intend to write the optimized weights to a list in the list manager, you must create the list before you specify it in the following step.

5. Run the optimization task. Send an `AddTask` action to IDOL Speech Server, and set the following parameters:

   `Type` | The task name. Set to `LangIdOptimize`.
   `DataList` | The list that specifies the feature files.
   `DataPath` | The path to the directory that contains the feature files listed in the `DataList`.
   `ClassList` | The list that specifies the classifiers (see Combine Classifiers into a Language Identification Set).
   `ClassPath` | The path to the directory that contains the language classifiers.
   `Out` | The classifier list to write the optimized weights to.

For example:

    http://localhost:13000/action=AddTask&Type=LangIdOptimize&DataList=ListManager/OptList&DataPath=C:/LangID/Data&ClassList=ListManager/ClassList&ClassPath=C:/LangID/class&Out=ListManager/OptClassSet

This action uses port `13000` to instruct IDOL Speech Server, which is located on the local machine, to test the feature files listed in the `OptList` list against the classifiers specified in the `ClassList` list and then generate an updated classifier list–`OptClassSet`.

This action returns a token. You can use the token to:

- Check the task status. See Check the Status of a Task.
- Retrieve the updated classifier file. See Find Recommended Language Resources to Use in a Task.

The `LangIdOptimize` action also generates a log file that contains information about the optimization process. See Review the Optimization Log File.

© 2013-2016 Hewlett Packard Enterprise Development LP | Send documentation feedback to HPE Close We welcome your comments! To open the configured email client on this computer, open an email window. Otherwise, copy the information below to a web mail client, and send this email to AutonomyTPFeedback@hpe.com. Product: Topic Title: Help Topic: Feedback:

_HP_HTML5_bannerTitle.htm